Scoliosis dynamic modeling method and system based on multi-modal data

Abstract

The invention provides a scoliosis dynamic modeling method and system based on multi-modal data, and the method comprises the steps: obtaining the spine scanning data of each scanning type of a user during the treatment period, and recognizing the trunk structure data of each trunk scale type of the user; based on the trunk structure data of each trunk scale type, constructing a trunk three-dimensional structure model of each trunk scale type corresponding to each collection time point during the treatment period, and based on the trunk three-dimensional structure model of each trunk scale type corresponding to each collection time point, through a four-dimensional modeling strategy, establishing a four-dimensional model of each trunk scale type corresponding to each collection time point; constructing a trunk four-dimensional dynamic model of each trunk scale type; and performing model fusion processing on the trunk four-dimensional dynamic model of each trunk scale type to obtain an initial scoliosis dynamic model of the user, and performing feature marking processing on the initial scoliosis dynamic model to obtain a scoliosis dynamic model of the user. By adopting the scheme, the identification accuracy of the dynamic change of the scoliosis of the patient can be improved.

Description

Technical Field

[0001] This invention relates to the field of four-dimensional modeling technology, and in particular to a method and system for dynamic modeling of scoliosis based on multimodal data. Background Technology

[0002] Scoliosis scanning is a technique that performs regional scanning of the spine in patients with scoliosis. Medical professionals often use the scan data to create a 3D model, generating a scoliosis model of the patient to effectively model the spinal structure and identify the degree of scoliosis. However, this method cannot effectively identify the recovery status and extent of scoliosis, resulting in poor dynamic recognition of scoliosis recovery. Therefore, improving the dynamic recognition of scoliosis is a current research focus.

[0003] Traditional dynamic scoliosis identification requires multiple scoliosis scans and separate identification of the 3D model constructed after each scan. However, a single model cannot identify dynamic changes. The dynamic changes of scoliosis can only be observed by medical staff with their naked eyes and subjectively analyzed, resulting in poor accuracy in identifying the dynamic changes of scoliosis in patients. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a method and system for dynamic modeling of scoliosis based on multimodal data, which addresses the shortcomings of the prior art.

[0005] The technical solution of this invention to solve the above-mentioned technical problems is as follows: A dynamic modeling method for scoliosis based on multimodal data, the method comprising:

[0006] Acquire spinal scan data of the user during treatment for each scan type, and based on the spinal scan data of each scan type, identify trunk structure data of the user for each trunk scale type;

[0007] Based on the trunk structure data of each trunk scale type, construct the three-dimensional trunk structure model of each trunk scale type corresponding to each acquisition time point during the treatment period, and construct the four-dimensional dynamic trunk model of each trunk scale type based on the three-dimensional trunk structure model of each trunk scale type corresponding to each acquisition time point through a four-dimensional modeling strategy.

[0008] The four-dimensional dynamic model of the trunk of each trunk scale type is fused to obtain the initial dynamic model of the user's scoliosis. The initial dynamic model of the scoliosis is then labeled with features to obtain the user's dynamic model of scoliosis.

[0009] Optionally, the step of identifying trunk structure data for each trunk scale type of the user based on spinal scan data of each of the aforementioned scan types includes:

[0010] For each scan type, the spinal scan data of that scan type is divided into sub-spine scan data corresponding to each acquisition time point;

[0011] Each sub-spine scan data is mapped into the standard trunk model of the scan type according to the data space acquisition location of each sub-spine scan data, so as to obtain the trunk mapping data corresponding to each sub-spine scan data.

[0012] Based on the trunk mapping data corresponding to each sub-spine scan data, a trunk mapping data distribution map corresponding to each sub-spine scan data is constructed, and the trunk mapping data distribution map corresponding to each sub-spine scan data is used as the trunk structure data corresponding to each sub-spine scan data.

[0013] Optionally, the three-dimensional torso structure model based on each torso scale type corresponding to each of the acquisition time points is used to construct a four-dimensional dynamic torso model for each torso scale type through a four-dimensional modeling strategy, including:

[0014] For each torso scale type, the three-dimensional torso structure models are sorted according to the time sequence of the acquisition time points corresponding to each three-dimensional torso structure model to obtain a model sequence, and the feature location points of each three-dimensional torso structure model are identified.

[0015] Based on the feature location points of each torso 3D structural model, feature point marking processing is performed on each torso 3D structural model to obtain a torso 3D marked model. Based on each torso 3D marked model, the change curve of each feature point is identified according to the model sequence.

[0016] Based on the change curves of each feature point and the three-dimensional marker models of each torso, a four-dimensional dynamic model of the torso of each torso scale type is constructed through a four-dimensional modeling program.

[0017] Optionally, the step of performing model fusion processing on the four-dimensional dynamic model of the trunk for each trunk scale type to obtain the user's initial scoliosis dynamic model includes:

[0018] Obtain the fusion coordination site of each four-dimensional dynamic model of the torso, and identify the position change curve of each fusion coordination site in each of the four-dimensional dynamic models of the torso;

[0019] Based on the position change curve of each fusion coordination point in each of the four-dimensional dynamic models of the trunk, the four-dimensional dynamic models of the trunk are projected onto the same spatial plane for model fusion processing through a four-dimensional model fusion program to obtain the user's initial scoliosis dynamic model.

[0020] Optionally, the step of performing feature labeling processing on the initial scoliosis dynamic model to obtain the user's scoliosis dynamic model includes:

[0021] The spinal change features and spinal structure features of the initial scoliosis dynamic model are extracted using a feature extraction network.

[0022] Identify the model location range corresponding to the spinal change features and the model structure range corresponding to the spinal structure features, and perform feature labeling processing on the initial scoliosis dynamic model based on the model location range corresponding to the initial spinal change features and the model structure range corresponding to the spinal structure features to obtain the user's scoliosis dynamic model.

[0023] Optionally, the method further includes:

[0024] Based on the model location range of the spinal change features in the dynamic model of scoliosis and the model structure range of the spinal structure features in the dynamic model of scoliosis, the data change distribution information of each spinal feature data type of the user is identified through the spinal feature data recognition network.

[0025] Based on the data change distribution information of each of the aforementioned spinal feature data types, the indicator change trends of each spinal indicator type of the user are identified through an indicator evaluation strategy.

[0026] Based on the trend of changes in various spinal indicators of the user and the dynamic model of scoliosis, a scoliosis recovery guidance report is generated for the user.

[0027] Furthermore, to address the aforementioned technical problems, this invention also provides a dynamic modeling system for scoliosis based on multimodal data, the system comprising:

[0028] The acquisition module is used to acquire spinal scan data of the user during treatment for each scan type, and to identify trunk structure data of the user for each trunk scale type based on the spinal scan data of each scan type.

[0029] The construction module is used to construct a three-dimensional trunk structure model corresponding to each trunk scale type based on the trunk structure data of each trunk scale type during the treatment period, and to construct a four-dimensional dynamic trunk model of each trunk scale type based on the three-dimensional trunk structure model of each trunk scale type corresponding to each of the acquisition time points through a four-dimensional modeling strategy.

[0030] The fusion module is used to perform model fusion processing on the four-dimensional dynamic model of the trunk of each trunk scale type to obtain the user's initial scoliosis dynamic model, and to perform feature labeling processing on the initial scoliosis dynamic model to obtain the user's scoliosis dynamic model.

[0031] Optionally, the acquisition module is specifically used for:

[0032] For each scan type, the spinal scan data of that scan type is divided into sub-spine scan data corresponding to each acquisition time point;

[0033] Each sub-spine scan data is mapped into the standard trunk model of the scan type according to the data space acquisition location of each sub-spine scan data, so as to obtain the trunk mapping data corresponding to each sub-spine scan data.

[0034] Based on the trunk mapping data corresponding to each sub-spine scan data, a trunk mapping data distribution map corresponding to each sub-spine scan data is constructed, and the trunk mapping data distribution map corresponding to each sub-spine scan data is used as the trunk structure data corresponding to each sub-spine scan data.

[0035] Optionally, the building module is specifically used for:

[0036] For each torso scale type, the three-dimensional torso structure models are sorted according to the time sequence of the acquisition time points corresponding to each three-dimensional torso structure model to obtain a model sequence, and the feature location points of each three-dimensional torso structure model are identified.

[0037] Based on the feature location points of each torso 3D structural model, feature point marking processing is performed on each torso 3D structural model to obtain a torso 3D marked model. Based on each torso 3D marked model, the change curve of each feature point is identified according to the model sequence.

[0038] Based on the change curves of each feature point and the three-dimensional marker models of each torso, a four-dimensional dynamic model of the torso of each torso scale type is constructed through a four-dimensional modeling program.

[0039] Optionally, the fusion module is specifically used for:

[0040] Obtain the fusion coordination site of each four-dimensional dynamic model of the torso, and identify the position change curve of each fusion coordination site in each of the four-dimensional dynamic models of the torso;

[0041] Based on the position change curve of each fusion coordination point in each of the four-dimensional dynamic models of the trunk, the four-dimensional dynamic models of the trunk are projected onto the same spatial plane for model fusion processing through a four-dimensional model fusion program to obtain the user's initial scoliosis dynamic model.

[0042] Optionally, the fusion module is specifically used for:

[0043] The spinal change features and spinal structure features of the initial scoliosis dynamic model are extracted using a feature extraction network.

[0044] Identify the model location range corresponding to the spinal change features and the model structure range corresponding to the spinal structure features, and perform feature labeling processing on the initial scoliosis dynamic model based on the model location range corresponding to the initial spinal change features and the model structure range corresponding to the spinal structure features to obtain the user's scoliosis dynamic model.

[0045] Optionally, the device may also include:

[0046] The first identification module is used to identify the data change distribution information of each spinal feature data type of the user based on the model position range of the spinal change features in the dynamic model of scoliosis and the model structure range of the spinal structure features in the dynamic model of scoliosis, through a spinal feature data identification network.

[0047] The second identification module is used to identify the indicator change trends of each spinal indicator type of the user based on the data change distribution information of each of the spinal feature data types and through an indicator evaluation strategy.

[0048] The generation module is used to generate a scoliosis recovery guidance report for the user based on the indicator change trends of each type of spinal indicator and the dynamic model of scoliosis.

[0049] Thirdly, this application provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the method described in any one of the first aspects.

[0050] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps of the method described in any one of the first aspects.

[0051] Fifthly, this application provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps of the method described in any one of the first aspects.

[0052] This invention provides a method and system for dynamic modeling of scoliosis based on multimodal data. The method includes: acquiring spinal scan data of various scan types from a user during treatment, and identifying trunk structure data of each trunk scale type based on the spinal scan data of each scan type; constructing a three-dimensional trunk structure model of each trunk scale type corresponding to each acquisition time point during treatment based on the trunk structure data of each trunk scale type, and constructing a four-dimensional dynamic model of the trunk of each trunk scale type based on the three-dimensional trunk structure model of each trunk scale type corresponding to each acquisition time point through a four-dimensional modeling strategy; performing model fusion processing on the four-dimensional dynamic model of the trunk of each trunk scale type to obtain an initial dynamic model of scoliosis for the user, and performing feature labeling processing on the initial dynamic model of scoliosis to obtain the dynamic model of scoliosis for the user. This solution, through multimodal data fusion, constructs an initial dynamic model of scoliosis based on different trunk scale types of the user, and builds a four-dimensional model in chronological order. Then, the model features are labeled to obtain the user's dynamic scoliosis model. This not only enables accurate, intuitive, and effective identification of the dynamic changes in the patient's spine, but also allows for multimodal and different trunk scale modeling to analyze the treatment effect and treatment problems of the user's scoliosis from different perspectives. It can accurately predict the mechanical response of bones and muscles under different correction schemes, upgrading the correction path planning from "experience-based speculation" to "data-driven" scientific decision-making. This solves the problem that traditional methods cannot dynamically adapt to the patient's growth and development, thus effectively improving the accuracy of identifying dynamic changes in the patient's scoliosis. Attached Figure Description

[0053] To more clearly illustrate the solutions in this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0054] Figure 1 This is a flowchart of the dynamic modeling method for scoliosis based on multimodal data provided in this embodiment of the invention;

[0055] Figure 2 This is a schematic diagram of the structure of the scoliosis dynamic modeling system based on multimodal data provided in an embodiment of the present invention;

[0056] Figure 3 An internal structural diagram of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0057] The scoliosis dynamic modeling method based on multimodal data provided in this invention is applied to a scoliosis dynamic modeling system based on multimodal data. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application. The terms "comprising" and "having," and any variations thereof, in the specification, claims, and accompanying drawings of this application are intended to cover non-exclusive inclusion. The terms "first," "second," etc., in the specification, claims, or accompanying drawings of this application are used to distinguish different objects, not to describe a particular order.

[0058] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0059] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0060] The scoliosis dynamic modeling method based on multimodal data provided in this application embodiment can be applied to a multimodal data-based scoliosis dynamic modeling application environment. This method can be applied to a terminal, a server, or a system including both a terminal and a server, and is implemented through interaction between the terminal and the server. The terminal can be, but is not limited to, various personal computers, laptops, etc. The terminal constructs an initial scoliosis dynamic model based on different trunk scale types of the user, and then performs four-dimensional model construction in chronological order by combining multimodal data fusion. Model feature labeling is then performed to obtain the user's scoliosis dynamic model. This not only effectively and accurately identifies the dynamic changes in the patient's spine, but also, through multimodal modeling of different trunk scale types, it can analyze the treatment effect and treatment problems of the user's scoliosis from different perspectives. This allows for accurate prediction of the mechanical response of bones and muscles under different correction schemes, upgrading correction path planning from "experience-based speculation" to "data-driven" scientific decision-making. This solves the problem that traditional methods cannot dynamically adapt to the patient's growth and development, thus effectively improving the accuracy of identifying dynamic changes in the patient's scoliosis.

[0061] In one embodiment, such as Figure 1 As shown, a dynamic modeling method for scoliosis based on multimodal data is provided. Taking the application of this method to a terminal as an example, the method includes the following steps:

[0062] Step S101: Obtain spinal scan data of each scan type during the user's treatment, and identify trunk structure data of each trunk scale type based on the spinal scan data of each scan type.

[0063] In this embodiment, the terminal uses different scanning devices to periodically scan the user's trunk during scoliosis correction treatment, obtaining spinal scan data of different scan types. These scan types include, but are not limited to, CT (Computed Tomography) scans for scanning the spinal skeletal structure, MRI (Magnetic Resonance Imaging) scans for scanning soft tissue structures, and surface infrared scans for scanning the user's body surface morphology. Then, based on the spinal scan data of each scan type, the terminal identifies different dimensions of the user's trunk structure data at various scales. These trunk scales include, but are not limited to, the spinal structure scale corresponding to the CT scan, the soft tissue structure scale corresponding to the MRI scan, and the body surface morphology scale corresponding to the surface infrared scan. The specific identification process will be explained in detail later.

[0064] Step S102: Based on the trunk structure data of each trunk scale type, construct a three-dimensional trunk structure model of each trunk scale type corresponding to each acquisition time point during the treatment period, and based on the three-dimensional trunk structure model of each trunk scale type corresponding to each acquisition time point, construct a four-dimensional dynamic trunk model of each trunk scale type through a four-dimensional modeling strategy.

[0065] In this embodiment, the terminal constructs a three-dimensional trunk structure model for each trunk scale type based on trunk structure data at each acquisition time point during treatment. Then, based on the three-dimensional trunk structure models at each acquisition time point, a four-dimensional dynamic trunk model for each trunk scale type is constructed using a four-dimensional modeling strategy. This four-dimensional modeling strategy involves dynamically processing the constructed three-dimensional trunk structure model in the time dimension to obtain the four-dimensional model. This strategy utilizes a multi-model dynamic fusion model animation modeling program based on AI (Artificial Intelligence) technology. The specific modeling process will be explained in detail later.

[0066] Step S103: Perform model fusion processing on the four-dimensional dynamic model of the trunk for each trunk scale type to obtain the user's initial scoliosis dynamic model, and perform feature labeling processing on the initial scoliosis dynamic model to obtain the user's scoliosis dynamic model.

[0067] In this embodiment, the terminal performs model fusion processing on the four-dimensional dynamic models of the torso at each torso scale type to obtain the user's initial scoliosis dynamic model. Then, it performs feature labeling processing on the initial scoliosis dynamic model to obtain the user's final scoliosis dynamic model. This feature labeling processing is used to label the spinal change features and spinal structural features in the initial scoliosis dynamic model, identifying the trend, direction, degree of change, and abnormalities in the spinal structure. The specific labeling process will be described in detail later.

[0068] Based on the above scheme, by combining multimodal data fusion, an initial dynamic model of scoliosis is constructed based on different trunk scale types of the user and in chronological order. Then, model feature labeling is performed to obtain the user's dynamic model of scoliosis. This not only enables accurate, intuitive and effective identification of the dynamic changes in the patient's spine, but also allows for multimodal and different trunk scale modeling to analyze the treatment effect of the user's scoliosis from different perspectives and to address treatment problems. This allows for accurate prediction of the mechanical response of bones and muscles under different correction schemes, upgrading the correction path planning from "experience-based speculation" to "data-driven" scientific decision-making. This solves the problem that traditional methods cannot dynamically adapt to the patient's growth and development, thus effectively improving the accuracy of identifying dynamic changes in the patient's scoliosis.

[0069] Optionally, based on the spinal scan data of each scan type, identify the trunk structure data of each trunk scale type of the user, including: for each scan type, splitting the spinal scan data of the scan type into sub-spine scan data corresponding to each acquisition time point; performing data mapping processing on each sub-spine scan data in the standard trunk model of the scan type according to the data space acquisition location of each sub-spine scan data to obtain the trunk mapping data corresponding to each sub-spine scan data; constructing a trunk mapping data distribution map corresponding to each sub-spine scan data based on the trunk mapping data corresponding to each sub-spine scan data, and using the trunk mapping data distribution map corresponding to each sub-spine scan data as the trunk structure data corresponding to each sub-spine scan data.

[0070] In this embodiment, for each scan type, the terminal breaks down the spinal scan data into sub-spine scan data corresponding to each acquisition time point. Then, the terminal performs data mapping processing on each sub-spine scan data according to its data space acquisition location within a standard trunk model for that scan type, obtaining trunk mapping data corresponding to each sub-spine scan data. This standard trunk model includes, but is not limited to, a standard spine model for CT scans, a standard soft tissue model for MRI scans, and a standard surface model for infrared scans. The model structures of these models are preset sample models on the terminal and are used only for data mapping.

[0071] Then, based on the trunk mapping data corresponding to each sub-spine scan data, the terminal constructs a trunk mapping data distribution map corresponding to each sub-spine scan data, and uses this distribution map as the trunk structure data corresponding to each sub-spine scan data. This mapping data distribution map represents the spatial distribution of the trunk mapping data within the model.

[0072] Based on the above scheme, after three-dimensional mapping of the model, a mapping data distribution map is constructed to identify the trunk structure data corresponding to each sub-spine scan data, thereby improving the accuracy and efficiency of trunk structure data identification.

[0073] Optionally, based on the three-dimensional torso structure models of each torso scale type corresponding to each acquisition time point, a four-dimensional dynamic model of the torso for each torso scale type is constructed using a four-dimensional modeling strategy. This includes: for each torso scale type, sorting each torso three-dimensional structure model according to the time sequence of the acquisition time points corresponding to each torso three-dimensional structure model to obtain a model sequence, and identifying each feature location point of each torso three-dimensional structure model; based on each feature location point of each torso three-dimensional structure model, performing feature point marking processing on each torso three-dimensional structure model to obtain a torso three-dimensional marked model, and based on each torso three-dimensional marked model, identifying the change curve of each feature point according to the model sequence; and based on the change curve of each feature point and each torso three-dimensional marked model, constructing a four-dimensional dynamic model of the torso for each torso scale type using a four-dimensional modeling program.

[0074] In this embodiment, for each torso scale type, the terminal sorts the three-dimensional torso structural models according to the chronological order of their corresponding acquisition time points to obtain a model sequence, and identifies the feature location points of each torso three-dimensional structural model. Specifically, the terminal selects the torso three-dimensional structural model corresponding to the earliest acquisition time point in the chronological order, identifies the corresponding three-dimensional feature data of the torso three-dimensional structural model through a three-dimensional feature recognition network, and uses the location points corresponding to the three-dimensional feature data as the feature location points of the torso three-dimensional structural model. Furthermore, in each torso three-dimensional structural model, the terminal adapts the location points corresponding to the feature location points of that torso three-dimensional structural model as the feature location points of that torso three-dimensional structural model. That is, the feature location points in each model determined by this scheme are the same location point whose features change over time, resulting in spatial displacement.

[0075] Then, based on the feature points of each 3D torso structure model, the terminal performs feature point marking processing on each torso 3D structure model to obtain a 3D marked torso model. Based on each 3D marked torso model, and following the model sequence, the terminal identifies the change curves of each feature point. These feature point change curves represent changes in the time axis as the vertical coordinate and the spatial displacement direction and distance as the planar coordinates. Finally, based on the feature point change curves and each 3D marked torso model, the terminal uses an AI-powered intelligent model animation generation program to construct a 4D dynamic torso model for each torso scale type.

[0076] Based on the above scheme, by identifying the change curves of feature points of each three-dimensional torso structure model, and then constructing a four-dimensional dynamic model of the torso for each torso scale type, the accuracy of the constructed torso dynamic model is improved.

[0077] Optionally, the four-dimensional dynamic model of the trunk of each trunk scale type is subjected to model fusion processing to obtain the user's initial scoliosis dynamic model, including: obtaining the fusion coordination point of each four-dimensional dynamic model of the trunk and identifying the position change curve of each fusion coordination point in each four-dimensional dynamic model of the trunk; based on the position change curve of each fusion coordination point in each four-dimensional dynamic model of the trunk, the four-dimensional dynamic models of the trunk are projected onto the same spatial plane for model fusion processing through a four-dimensional model fusion program to obtain the user's initial scoliosis dynamic model.

[0078] In this embodiment, the terminal acquires the fusion coordination point of each four-dimensional dynamic model of the torso and identifies the positional change curve of each fusion coordination point in each four-dimensional dynamic model of the torso. The fusion coordination point is a spatial location point preset by the operator, where the data characteristics of the torso structure data of different torso scale types are the same. For example, the location point corresponding to the vertex of the spine, which corresponds to the location point in the spinal structure model, soft tissue structure model, and body surface structure model. That is, the terminal presets the model fusion location points of four-dimensional dynamic models of the torso of different torso scale types, and then spatially fuses the four-dimensional dynamic models of the torso according to the overlap of location points, thus obtaining the user's initial scoliosis dynamic model.

[0079] Then, based on the position change curve of each fusion coordination point in each trunk four-dimensional dynamic model, the terminal projects each trunk four-dimensional dynamic model onto the same spatial plane for model fusion processing through a four-dimensional model fusion program to obtain the user's initial scoliosis dynamic model.

[0080] Based on the above scheme, the point change curve is identified by pre-set coordination fusion points, and then the model is fused, which ensures the fusion accuracy of the fused four-dimensional dynamic model.

[0081] Optionally, feature labeling processing is performed on the initial scoliosis dynamic model to obtain the user's scoliosis dynamic model, including: extracting the spinal change features and spinal structure features of the initial scoliosis dynamic model through a feature extraction network; identifying the model position range corresponding to the spinal change features and the model structure range corresponding to the spinal structure features; and performing feature labeling processing on the initial scoliosis dynamic model based on the model position range corresponding to the initial spinal change features and the model structure range corresponding to the spinal structure features to obtain the user's scoliosis dynamic model.

[0082] In this embodiment, the terminal extracts the spinal change features and spinal structure features of the initial scoliosis dynamic model through a feature extraction network. The feature extraction network is a deep learning convolutional neural network that extracts features from four-dimensional features. The spinal change features include, but are not limited to, features of spinal change direction, spinal change degree, and spinal change trend, while the spinal structure features include, but are not limited to, features of spinal curvature degree, spinal curvature point deviation, spinal curvature range, and spinal curvature point.

[0083] Identify the model location range corresponding to the spinal change features and the model structure range corresponding to the spinal structure features. Based on the initial model location range corresponding to the spinal change features and the model structure range corresponding to the spinal structure features, perform feature labeling processing on the initial scoliosis dynamic model to obtain the user's scoliosis dynamic model.

[0084] Based on the above scheme, by identifying and labeling the model features, medical staff can intuitively and visually identify the patient's scoliosis recovery status and scoliosis condition, thereby improving the auxiliary guidance effect for medical staff.

[0085] Optionally, the method further includes: identifying the data change distribution information of each spinal feature data type of the user through a spinal feature data recognition network based on the model location range of spinal change features in the dynamic model of scoliosis and the model structure range of spinal structure features in the dynamic model of scoliosis; identifying the indicator change trend of each spinal indicator type of the user through an indicator evaluation strategy based on the data change distribution information of each spinal feature data type of the user; and generating a scoliosis recovery guidance report for the user based on the indicator change trend of each spinal indicator type of the user and the dynamic model of scoliosis.

[0086] In this embodiment, the terminal, based on the model location range of spinal change features and the model structure range of spinal structural features in the dynamic scoliosis model, identifies the data change distribution information of various spinal feature data types of the user through a spinal feature data recognition network. These various spinal feature data types include, but are not limited to, scoliosis degree feature types, scoliosis trend feature types, scoliosis change amplitude feature types, scoliosis range change feature types, scoliosis degree change feature types, and scoliosis angle change feature types.

[0087] Then, based on the data change distribution information of each spinal feature data type, the terminal identifies the indicator change trends of each user's spinal indicator type through an indicator evaluation strategy. This indicator evaluation strategy includes the spinal feature data range corresponding to each indicator value of different spinal indicator types. The terminal identifies the feature value distribution information of each spinal indicator type through range adaptation, and based on the feature value distribution information of each spinal indicator type, identifies the indicator change trends of each spinal indicator type through a linear trend recognition algorithm. This linear trend recognition algorithm is a linear change trend recognition algorithm based on linear regression. The spinal indicator types include, but are not limited to, scoliosis grade indicators, scoliosis abnormality degree indicators, scoliosis recovery effect indicators, and scoliosis body shape impact degree indicators.

[0088] Finally, based on the trend of changes in various spinal indicators and the dynamic model of scoliosis, the terminal generates a scoliosis recovery guidance report for the user. Specifically, the terminal presets a guidance report template and fills it with the trend of changes in various spinal indicators and the dynamic model of scoliosis to obtain the user's scoliosis recovery guidance report.

[0089] Based on the above scheme, by performing feature recognition and feature trend analysis on the dynamic model of scoliosis constructed in this scheme, a scoliosis recovery guidance report for the user can be generated. This report can not only visually and intuitively display the user's scoliosis recovery status, but also provide feedback to medical staff on the changes and trends of the user's spinal characteristics from the perspective of data indicators, thereby improving the guidance effect for medical staff on scoliosis.

[0090] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0091] Based on the same inventive concept, this application also provides a multimodal data-based dynamic modeling system for implementing the above-mentioned multimodal data-based dynamic modeling method for scoliosis. The solution provided by this system is similar to the implementation described in the above method. Therefore, the specific limitations of one or more embodiments of the multimodal data-based dynamic modeling system for scoliosis provided below can be found in the limitations of the multimodal data-based dynamic modeling method for scoliosis described above, and will not be repeated here.

[0092] Further reference Figure 2 As a response to the above Figure 1 The present application provides an embodiment of a multimodal data-based dynamic modeling system for scoliosis, which includes an acquisition module 210, a construction module 220, and a fusion module 230.

[0093] The acquisition module is used to acquire spinal scan data of the user during treatment for each scan type, and to identify trunk structure data of the user for each trunk scale type based on the spinal scan data of each scan type.

[0094] The construction module is used to construct a three-dimensional trunk structure model corresponding to each trunk scale type based on the trunk structure data of each trunk scale type during the treatment period, and to construct a four-dimensional dynamic trunk model of each trunk scale type based on the three-dimensional trunk structure model of each trunk scale type corresponding to each of the acquisition time points through a four-dimensional modeling strategy.

[0095] The fusion module is used to perform model fusion processing on the four-dimensional dynamic model of the trunk of each trunk scale type to obtain the user's initial scoliosis dynamic model, and to perform feature labeling processing on the initial scoliosis dynamic model to obtain the user's scoliosis dynamic model.

[0096] Optionally, the acquisition module is specifically used for:

[0097] For each scan type, the spinal scan data of that scan type is divided into sub-spine scan data corresponding to each acquisition time point;

[0098] Each sub-spine scan data is mapped into the standard trunk model of the scan type according to the data space acquisition location of each sub-spine scan data, so as to obtain the trunk mapping data corresponding to each sub-spine scan data.

[0099] Based on the trunk mapping data corresponding to each sub-spine scan data, a trunk mapping data distribution map corresponding to each sub-spine scan data is constructed, and the trunk mapping data distribution map corresponding to each sub-spine scan data is used as the trunk structure data corresponding to each sub-spine scan data.

[0100] Optionally, the building module is specifically used for:

[0101] For each torso scale type, the three-dimensional torso structure models are sorted according to the time sequence of the acquisition time points corresponding to each three-dimensional torso structure model to obtain a model sequence, and the feature location points of each three-dimensional torso structure model are identified.

[0102] Based on the feature location points of each torso 3D structural model, feature point marking processing is performed on each torso 3D structural model to obtain a torso 3D marked model. Based on each torso 3D marked model, the change curve of each feature point is identified according to the model sequence.

[0103] Based on the change curves of each feature point and the three-dimensional marker models of each torso, a four-dimensional dynamic model of the torso of each torso scale type is constructed through a four-dimensional modeling program.

[0104] Optionally, the fusion module is specifically used for:

[0105] Obtain the fusion coordination site of each four-dimensional dynamic model of the torso, and identify the position change curve of each fusion coordination site in each of the four-dimensional dynamic models of the torso;

[0106] Based on the position change curve of each fusion coordination point in each of the four-dimensional dynamic models of the trunk, the four-dimensional dynamic models of the trunk are projected onto the same spatial plane for model fusion processing through a four-dimensional model fusion program to obtain the user's initial scoliosis dynamic model.

[0107] Optionally, the fusion module is specifically used for:

[0108] The spinal change features and spinal structure features of the initial scoliosis dynamic model are extracted using a feature extraction network.

[0109] Identify the model location range corresponding to the spinal change features and the model structure range corresponding to the spinal structure features, and perform feature labeling processing on the initial scoliosis dynamic model based on the model location range corresponding to the initial spinal change features and the model structure range corresponding to the spinal structure features to obtain the user's scoliosis dynamic model.

[0110] Optionally, the device may also include:

[0111] The first identification module is used to identify the data change distribution information of each spinal feature data type of the user based on the model position range of the spinal change features in the dynamic model of scoliosis and the model structure range of the spinal structure features in the dynamic model of scoliosis, through a spinal feature data identification network.

[0112] The second identification module is used to identify the indicator change trends of each spinal indicator type of the user based on the data change distribution information of each of the spinal feature data types and through an indicator evaluation strategy.

[0113] The generation module is used to generate a scoliosis recovery guidance report for the user based on the indicator change trends of each type of spinal indicator and the dynamic model of scoliosis.

[0114] The modules in the aforementioned multimodal data-based dynamic modeling system for scoliosis can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the computer device's memory as software, so that the processor can call and execute the corresponding operations of each module.

[0115] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 3 As shown, the computer device includes a processor, memory, communication interface, display screen, and input system connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements a dynamic modeling method for scoliosis based on multimodal data. The display screen can be an LCD screen or an e-ink screen. The input system can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0116] Those skilled in the art will understand that Figure 3The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0117] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method described in any one of the first aspects.

[0118] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in any one of the first aspects.

[0119] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the method described in any one of the first aspects.

[0120] It should be noted that the patient information (including but not limited to patient device information, patient personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the patient or fully authorized by all parties.

[0121] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0122] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0123] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A dynamic modeling method for scoliosis based on multimodal data, characterized in that, The method includes: Acquire spinal scan data of the user during treatment for each scan type, and based on the spinal scan data of each scan type, identify trunk structure data of the user for each trunk scale type; Based on the trunk structure data of each trunk scale type, construct the three-dimensional trunk structure model of each trunk scale type corresponding to each acquisition time point during the treatment period, and construct the four-dimensional dynamic trunk model of each trunk scale type based on the three-dimensional trunk structure model of each trunk scale type corresponding to each acquisition time point through a four-dimensional modeling strategy. The four-dimensional dynamic model of the trunk of each trunk scale type is fused to obtain the initial dynamic model of the user's scoliosis. The initial dynamic model of the scoliosis is then labeled with features to obtain the user's dynamic model of scoliosis.

2. The method according to claim 1, characterized in that, The process of identifying trunk structure data for each trunk scale type of the user based on spinal scan data of each of the aforementioned scan types includes: For each scan type, the spinal scan data of that scan type is divided into sub-spine scan data corresponding to each acquisition time point; Each sub-spine scan data is mapped into the standard trunk model of the scan type according to the data space acquisition location of each sub-spine scan data, so as to obtain the trunk mapping data corresponding to each sub-spine scan data. Based on the trunk mapping data corresponding to each sub-spine scan data, a trunk mapping data distribution map corresponding to each sub-spine scan data is constructed, and the trunk mapping data distribution map corresponding to each sub-spine scan data is used as the trunk structure data corresponding to each sub-spine scan data.

3. The method according to claim 1, characterized in that, The three-dimensional torso structure model based on each torso scale type corresponding to each of the aforementioned acquisition time points, through a four-dimensional modeling strategy, constructs a four-dimensional dynamic torso model for each torso scale type, including: For each torso scale type, the three-dimensional torso structure models are sorted according to the time sequence of the acquisition time points corresponding to each three-dimensional torso structure model to obtain a model sequence, and the feature location points of each three-dimensional torso structure model are identified. Based on the feature location points of each torso 3D structural model, feature point marking processing is performed on each torso 3D structural model to obtain a torso 3D marked model. Based on each torso 3D marked model, the change curve of each feature point is identified according to the model sequence. Based on the change curves of each feature point and the three-dimensional marker models of each torso, a four-dimensional dynamic model of the torso of each torso scale type is constructed through a four-dimensional modeling program.

4. The method according to claim 1, characterized in that, The process of fusing the four-dimensional dynamic models of the trunk for each trunk scale type to obtain the user's initial scoliosis dynamic model includes: Obtain the fusion coordination site of each four-dimensional dynamic model of the torso, and identify the position change curve of each fusion coordination site in each of the four-dimensional dynamic models of the torso; Based on the position change curve of each fusion coordination point in each of the four-dimensional dynamic models of the trunk, the four-dimensional dynamic models of the trunk are projected onto the same spatial plane for model fusion processing through a four-dimensional model fusion program to obtain the user's initial scoliosis dynamic model.

5. The method according to claim 1, characterized in that, The step of performing feature labeling processing on the initial scoliosis dynamic model to obtain the user's scoliosis dynamic model includes: The spinal change features and spinal structure features of the initial scoliosis dynamic model are extracted using a feature extraction network. Identify the model location range corresponding to the spinal change features and the model structure range corresponding to the spinal structure features, and perform feature labeling processing on the initial scoliosis dynamic model based on the model location range corresponding to the initial spinal change features and the model structure range corresponding to the spinal structure features to obtain the user's scoliosis dynamic model.

6. The method according to claim 5, characterized in that, The method further includes: Based on the model location range of the spinal change features in the dynamic model of scoliosis and the model structure range of the spinal structure features in the dynamic model of scoliosis, the data change distribution information of each spinal feature data type of the user is identified through the spinal feature data recognition network. Based on the data change distribution information of each of the aforementioned spinal feature data types, the indicator change trends of each spinal indicator type of the user are identified through an indicator evaluation strategy. Based on the trend of changes in various spinal indicators of the user and the dynamic model of scoliosis, a scoliosis recovery guidance report is generated for the user.

7. A dynamic modeling system for scoliosis based on multimodal data, characterized in that, The system includes: The acquisition module is used to acquire spinal scan data of the user during treatment for each scan type, and to identify trunk structure data of the user for each trunk scale type based on the spinal scan data of each scan type. The construction module is used to construct a three-dimensional trunk structure model corresponding to each trunk scale type based on the trunk structure data of each trunk scale type during the treatment period, and to construct a four-dimensional dynamic trunk model of each trunk scale type based on the three-dimensional trunk structure model of each trunk scale type corresponding to each of the acquisition time points through a four-dimensional modeling strategy. The fusion module is used to perform model fusion processing on the four-dimensional dynamic model of the trunk of each trunk scale type to obtain the user's initial scoliosis dynamic model, and to perform feature labeling processing on the initial scoliosis dynamic model to obtain the user's scoliosis dynamic model.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images