Spinal disease recognition system based on relationship perception and condition classification

Abstract

Embodiments of the present disclosure disclose a spine disease recognition system based on relationship perception and condition classification. A specific implementation of the system includes a data processing server, a feature extraction server, a disease recognition server, a report generation server, and a user interaction terminal, wherein: the data processing server is configured to perform preset standardization processing on spine disease patient data; the feature extraction server is configured to perform feature extraction and segmentation on standard image data; the disease recognition server is configured to perform spine disease recognition to obtain a spine disease recognition result; the report generation server is configured to perform multi-modal fusion and generate a structured spine disease recognition report; and the user interaction terminal is configured to display the image and the report and control at least one of a robot and a chemical dispensing device to perform a preset operation according to an electrical signal. The implementation achieves the effects of reducing the display waiting time of the system and reducing the redundant calculation cycle.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure relate to the field of computer technology, in particular to a spinal disease recognition system based on relationship perception and conditional classification. BACKGROUND

[0002] With the global population aging and the growing demand for precision medicine, the market for the diagnosis of spinal diseases is huge and growing. Against the background of the market of AI medical imaging system expected to expand significantly, there is a clear demand for the diagnosis speed and consistency of AI medical imaging system. Currently, when diagnosing spinal diseases, the commonly used methods are: radiologists use AI medical imaging systems (such as U-Net, visual transformer ViT, etc.) to assist in interpreting medical images (such as CT, MRI); and use computer-aided diagnosis (CADx) systems for auxiliary diagnosis.

[0003] However, when using the above systems to identify spinal diseases, the following technical problems often exist:

[0004] The system has a long display waiting time. Specifically, the system based on convolutional neural network (CNN, such as U-Net) is limited by the inherent receptive field, and when identifying and accurately segmenting lesions with various shapes and fuzzy boundaries, the performance will be significantly reduced, and the system needs to call the diagnosis model for verification multiple times. The display waiting time of the disease recognition result is long.

[0005] The system has redundant calculation cycles. Specifically, the system based on Transformer (such as visual transformer ViT) lacks a detailed understanding of the three-dimensional anatomical relationship of the spine and fails to fully utilize the topological structure information between the vertebral bodies. The existing computer-aided diagnosis (CADx) system lacks the ability to integrate different information for comprehensive differential diagnosis, and cannot automatically and seamlessly convert the quantitative analysis results of AI into high-quality diagnosis text. The spinal disease recognition report displayed by the user interaction terminal does not match the facts, and the system needs to be re-executed, causing redundant calculation cycles.

[0006] The above information disclosed in this BACKGROUND section is only for the purpose of enhancing the understanding of the background of the present inventive concepts, and therefore, it can contain information that does not form the prior art known to those of ordinary skill in the art in the country. SUMMARY

[0007] The summary section of the present disclosure is used to introduce the concepts in a brief manner, which will be described in detail in the following detailed description section. The summary section of the present disclosure is not intended to identify key or essential features of the claimed technical solutions, nor is it intended to limit the scope of the claimed technical solutions.

[0008] Some embodiments of the present disclosure propose a spine disease recognition system based on relationship perception and conditional classification to solve one or more of the technical problems mentioned in the background section.

[0009] Some embodiments of the present disclosure provide a spine disease recognition system based on relationship perception and conditional classification, which comprises a data processing server, a feature extraction server, a disease recognition server, a report generation server and a user interaction terminal connected in communication with each other, wherein: the data processing server is configured to perform preset standardization processing on spine disease patient data to generate standard image data and standard clinical data; the feature extraction server is configured to perform feature extraction and segmentation on the standard image data to generate a set of vertebral node differential feature vectors and a set of spine disease medical image segmentation masks; the disease recognition server is configured to generate a set of reconstructed images and a spine disease recognition result based on a set of spine conditions, the standard image data and the set of vertebral node differential feature vectors; and the report generation server is configured to perform multi-modal fusion on the standard clinical data, the set of vertebral node differential feature vectors and the spine disease recognition result received from the disease recognition server to obtain a multi-modal vector, and generate a structured spine disease recognition report based on the multi-modal vector.

[0010] The above various embodiments of the present disclosure have the following beneficial effects: through the relationship-aware and condition-based spinal disease recognition system of some embodiments of the present disclosure, by modeling the intervertebral biomechanical dependence and generating an anatomy-aware reconstruction conditioned on high-dimensional graph embedding, an interpretable decision mechanism is provided for large language models, the display latency of the system is reduced, and redundant computation cycles are reduced. Specifically, the reason for the long display latency of the disease recognition result of the system, the inconsistency between the spinal disease recognition report displayed by the user interaction terminal and the fact, the need for the system to re-execute, and the resulting redundant computation cycles are as follows: the long display latency of the system. Specifically, the system based on convolutional neural network (CNN, such as U-Net) is limited by the inherent receptive field, and when identifying and accurately segmenting lesions with various shapes and fuzzy boundaries, the performance will decrease significantly, and the system needs to call the diagnostic model for verification multiple times. The display latency of the disease recognition result is long. The system has redundant computation cycles. Specifically, the system based on Transformer (such as visual transformer ViT) lacks a fine understanding of the three-dimensional anatomical relationship of the spine and fails to fully utilize the topological structure information between the vertebral bodies. The existing computer-aided diagnosis (CADx) system lacks the ability to integrate different information for comprehensive differential diagnosis, and cannot automatically and seamlessly convert the quantitative analysis results of AI into high-quality diagnostic text. The spinal disease recognition report displayed by the user interaction terminal is inconsistent with the fact, and the system needs to be re-executed, resulting in redundant computation cycles. Based on this, the relationship-aware and condition-based spinal disease recognition system of some embodiments of the present disclosure includes a data processing server, a feature extraction server, a disease recognition server, a report generation server, and a user interaction terminal connected in communication, wherein: first, the above data processing server is used to perform preset standardization processing on the spinal disease patient data to generate standard image data and standard clinical data. Thus, the original medical images and clinical texts are standardized to generate image and clinical data in a unified format, providing high-quality input for downstream analysis. Second, the above feature extraction server is used to extract features and segment the above standard image data to generate a set of vertebral node differential feature vectors and a set of spinal disease medical image segmentation masks. Thus, the intervertebral differential features and segmentation masks are extracted by the graph transformer differential information interactor (GT-DII) to quantify the abnormalities of the spinal anatomy. Then, the above disease recognition server is used to generate a set of reconstructed images and spinal disease recognition results based on a set of spinal conditions, the above standard image data, and the above set of vertebral node differential feature vectors. Thus, the condition diffusion diagnosis network (CD-DN) is used to generate reconstructed images and calculate reconstruction errors to realize lesion type recognition and uncertainty quantification.Then, the report generation server is configured to perform multi-modal fusion on the standard clinical data, the set of vertebral node differential feature vectors, and the spinal disease recognition result received from the disease recognition server to obtain a multi-modal vector, and generate a structured spinal disease recognition report based on the multi-modal vector. In this way, multi-modal data is fused to generate a structured diagnostic report, and the LoRA method is used to fine-tune the form data set based on the artificially constructed instructions using the llama-factory framework. Then, the fine-tuned special large language model outputs operable clinical insights. Finally, the user interaction terminal is used to display the spinal disease medical image, the set of spinal disease medical image segmentation masks, and the structured spinal disease recognition report, and to control at least one of a preset robot and a preset chemical dispensing device to perform a preset operation according to an electrical signal. In this way, the image, the mask, and the report can be visualized, and the robot and the dispensing device can be controlled by the interactive buttons to perform precise treatment operations. This embodiment reduces the display waiting time of the system and reduces the effect of redundant calculation cycles. BRIEF DESCRIPTION OF DRAWINGS

[0011] The above and other features, advantages, and aspects of embodiments of the present disclosure will become more apparent by describing in detail some embodiments thereof with reference to the attached drawings. The same or similar components have the same or similar reference numbers throughout the drawings. It should be understood that the drawings are schematic and elements and components are not necessarily drawn to scale.

[0012] Figure 1 is a structural schematic diagram of some embodiments of a spinal disease recognition system based on relationship perception and conditional classification according to the present disclosure;

[0013] Figure 2 is a structural schematic diagram of some other embodiments of a spinal disease recognition system based on relationship perception and conditional classification according to the present disclosure;

[0014] Figure 3 is a front-end user interface diagram of a spinal disease recognition system based on relationship perception and conditional classification according to the present disclosure;

[0015] Figure 4 is an example training data structure diagram of a spinal disease recognition system based on relationship perception and conditional classification according to the present disclosure. DETAILED DESCRIPTION

[0016] Embodiments of the present disclosure will be described below in greater detail with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it is understood that the present disclosure can be embodied in various forms and should not be interpreted as being limited to the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure will be thoroughly and completely understood. It should be understood that the drawings and embodiments of the present disclosure are only for illustrative purposes and are not intended to limit the scope of protection of the present disclosure.

[0017] In addition, it should be further noted that only parts related to the present application are shown in the drawings for ease of description. The embodiments in the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0018] It should be noted that the concepts of "first", "second", etc. mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0019] It should be noted that the adjectives "one", "multiple" mentioned in the present disclosure are illustrative and not limiting, and those skilled in the art should understand that unless otherwise explicitly stated in the context, it should be understood as "one or more".

[0020] The names of the messages or information exchanged between the devices in the embodiments of the present disclosure are only for illustrative purposes, and are not used to limit the scope of the messages or information.

[0021] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

[0022] Please refer to Figure 1 , Figure 1 is a structural schematic diagram of some embodiments of the spinal disease recognition system based on relationship awareness and condition classification of the present disclosure. The spinal disease recognition system based on relationship awareness and condition classification includes a data processing server 101, a feature extraction server 102, a disease recognition server 103, a report generation server 104 and a user interaction terminal 105.

[0023] The data processing server 101 is a server for performing preset standardization processing on spinal disease patient data to generate standard image data and standard clinical data.

[0024] In some embodiments, the data processing server 101 described above can perform preset standardization processing on the spinal disease patient data containing spinal disease medical images and clinical text data to generate standard image data and standard clinical data. The data processing server 101 described above can include, but is not limited to, at least one of the following: a processor, a memory, a database, and a data integration special module (ETL engine, API gateway, etc.). The spinal disease medical images described above can include computed tomography (CT) or magnetic resonance imaging (MRI) images of the spine in DICOM format. The clinical text data described above can include structured text data (for example, JSON format data, including patient age, gender, chief complaint, past medical history, laboratory test values) and unstructured text data (for example, free-text clinical notes, pathology reports). The structure of specific spinal disease patient data can refer to Figure 4 , an example training data structure diagram of the spinal disease recognition system based on relationship perception and conditional classification according to the present disclosure.

[0025] In some optional implementations of some embodiments, the data processing server 101 described above can be configured to perform the following steps:

[0026] First, the computed tomography images and the magnetic resonance images are resampled to generate a set of spatially standardized spinal disease medical images.

[0027] In practice, first, the execution subject can use trilinear interpolation to resample the computed tomography (CT) images to obtain interpolated computed tomography images. Then, B-spline interpolation is used to resample the magnetic resonance (MRI) images to obtain interpolated magnetic resonance images. Finally, the interpolated computed tomography images and the interpolated magnetic resonance images are determined as the set of spatially standardized spinal disease medical images. The interpolated magnetic resonance images can include T1-weighted images (T1WI) and T2-weighted images (T2WI).

[0028] Second, the set of spatially standardized spinal disease medical images is normalized to generate standard image data.

[0029] In practice, first, the execution subject can normalize the interpolated computed tomography image in the spatially standardized spine disease medical image set using Z-score normalization to achieve grayscale value standardization, obtaining a normalized computed tomography image. Then, the T1 weighted image in the interpolated magnetic resonance image can be normalized using histogram matching, obtaining a normalized magnetic resonance image T1 weighted image. Second, the T2 weighted image in the interpolated magnetic resonance image is normalized using quantile normalization (QN), obtaining a normalized magnetic resonance image T2 weighted image. Finally, the normalized computed tomography image, the normalized magnetic resonance image T1 weighted image, and the normalized magnetic resonance image T2 weighted image are determined as standard image data.

[0030] In some other optional implementations of some embodiments, the data processing server 101 can be further configured to perform the following steps:

[0031] First, a parsing and encoding operation is performed on the structured text data to generate a structured clinical feature vector. In practice, the data processing server 101 can extract key fields (e.g., patient age, gender, chief complaint, past history, laboratory test values) of the structured text data to obtain a key field set, and one-hot encode the key field set to obtain a structured clinical feature vector.

[0032] Second, a data cleaning operation is performed on the unstructured text data to generate cleaned unstructured data. In practice, first, the data processing server 101 can remove special characters (e.g.,!, @, #, and %) in the unstructured text data based on a regular expression to obtain unstructured text data without special characters. Second, the data processing server 101 can use a pre-defined term mapping table to replace original terms in the unstructured text data without special characters with standardized terms, obtaining cleaned unstructured data.

[0033] Third, the key clinical entities are extracted from the cleaned unstructured data to generate a clinical entity vector. In practice, first, the data processing server 101 can use a biomedical pre-trained model (such as BioBERT) to generate a vector representation (such as a 768-dimensional vector) from the cleaned unstructured data. Second, a pre-trained named entity recognition (NER) model is used to extract key clinical entities (such as disease names, anatomical locations, and symptom descriptions) from the context-dependent vector representation to obtain a clinical entity vector. The pre-trained named entity recognition (NER) model can be a recognition model pre-trained based on de-identified medical corpora such as PubMed and MIMIC-III. For example, the pre-trained named entity recognition (NER) model can be BioBERT-Base.

[0034] Fourth, the structured clinical feature vector and the clinical entity vector are determined as standard clinical data.

[0035] The feature extraction server 102 is a server for feature extraction and segmentation of standard image data to generate a set of vertebral node differential feature vectors and a set of spinal disease medical image segmentation masks. The specific process can refer to Figure 2 , a structural schematic diagram of another embodiment of the spinal disease recognition system based on relationship perception and conditional classification according to the present disclosure.

[0036] In some embodiments, the feature extraction server 102 can perform feature extraction and segmentation on the standard image data received from the data processing server based on a pre-set three-dimensional convolutional neural network to generate a set of vertebral node differential feature vectors and a set of spinal disease medical image segmentation masks. The feature extraction server 102 can include but is not limited to at least one of a graphics processing unit (GPU) and a feature extraction engine. The pre-set three-dimensional convolutional neural network can be a three-dimensional convolutional neural network that performs three-dimensional feature extraction on all vertebral bodies in CT / MRI images and outputs high-precision three-dimensional segmentation masks. For example, the pre-set three-dimensional convolutional neural network can be a lightweight 3D U-Net. Here, the pre-set three-dimensional convolutional neural network includes an encoder and a decoder.

[0037] In some optional implementations of some embodiments, the feature extraction server 102 can be configured to perform the following steps:

[0038] First, based on the standard image data, a high-dimensional spinal disease feature map, a set of vertebral nodes, a set of vertebral node feature vectors, and a set of vertebral edge relationships are generated.

[0039] In practice, the feature extraction server 102 can input the standard image data into a preset three-dimensional convolutional neural network, and obtain a high-dimensional spinal disease feature map after a second downsampling layer in an encoder path of the preset three-dimensional convolutional neural network. The encoder of the preset three-dimensional convolutional neural network can include a convolution block 1 (3D convolution, batch normalization, and ReLU activation function), a downsampling 1 (max pooling), a convolution block 2 (3D convolution, batch normalization, and ReLU activation function), and a downsampling 2 (max pooling). The high-dimensional spinal disease feature map contains the positions of the spinal vertebrae, and each spatial position corresponds to a feature vector.

[0040] The feature extraction server 102 can determine the center point position of each vertebra in the spine corresponding to each vertebra in the spine according to the positions of all spinal vertebrae in the high-dimensional spinal disease feature map, as a vertebra node , to obtain a vertebra node set. , to obtain a vertebra node set.

[0041] The feature extraction server 102 can generate a vertebra node feature vector set and a vertebra edge relationship set based on the high-dimensional spinal disease feature map and the vertebra node set. First, the feature extraction server 102 can take the position of each vertebra node in the vertebra node set as a reference to extract a 3x3x3 neighborhood voxel feature cube, and compress the 3x3x3 neighborhood voxel feature cube into a vector using max pooling as a vertebra node feature vector to obtain a vertebra node feature vector set. Second, the feature extraction server 102 can establish an undirected edge of each vertebra node in the vertebra node set under the condition of connecting only adjacent vertebrae based on the spinal structure, and determine the coordinate difference vector of the two vertebra nodes in each undirected edge as a vertebra edge relationship to obtain a vertebra edge relationship set.

[0042] Second, based on the vertebra edge relationship set, the vertebra node feature vector set is mean aggregated to generate a neighborhood mean feature set. In practice, the feature extraction server 102 can mean aggregate at least one vertebra node feature vector corresponding to the neighborhood of the vertebra according to the following formula:

[0043] .

[0044] wherein, , represents different vertebrae, . represents the total number of vertebrae. represents the neighborhood of the vertebra (e.g., the neighborhood of vertebra L3 is vertebra L2 and vertebra L4). represents the neighborhood of the vertebra​ the centroid node feature vector of the vertebra. represent the centroid neighborhood mean features. As an example, the neighborhood mean features of the centroid node feature vector are , the neighborhood mean features of the centroid node feature vector are + ) / 2.

[0045] Thirdly, based on the above set of vertebra nodes and the above set of vertebra edge relationships, a graph structure is constructed, and based on the above graph structure, a set of vertebra attention weight vectors is generated. In practice, due to the bidirectional biomechanical relationship, the feature extraction server 102 can construct a graph based on the above set of vertebra nodes and the above set of vertebra edge relationships. The construction of the graph structure is based on the “vertebral center point” and “anatomical connectivity”, ensuring that the model understands long-range dependencies (such as spinal curvature). For example, the set of vertebra nodes , the vertebra edge relationship , and the graph structure .

[0046] The feature extraction server 102 can determine the attention weight of each vertebra node feature vector in the set of vertebra node feature vectors to the adjacent vertebra node feature vectors based on the graph attention mechanism (GAT) and the above graph structure, and obtain the set of vertebra attention weight vectors through the following formula:

[0047] .

[0048] .

[0049] wherein, represents the vertebra node feature vector of the vertebra. represents the vertebra node feature vector of the vertebra. represents the pre-trained shared linear transformation matrix. represents the pre-trained learnable attention vector. represents the transpose of the above pre-trained learnable attention vector. represents the vertebra edge relationship between the vertebra and the vertebra . represents the vertebra belonging to the neighborhood of the vertebra . represents the vertebra . represents the neighborhood of the vertebra a vector of attention weights of the vertebra. denotes a Leaky ReLU (Leaky Rectified Linear Unit) nonlinear activation function.

[0050] In the fourth step, a set of difference embedding vectors is generated based on the preset healthy vertebra prototype embedding vector, the set of vertebra node feature vectors and the set of neighborhood mean features. In practice, the feature extraction server 102 can generate the set of difference embedding vectors by the following formula :

[0051] .

[0052] .

[0053] wherein, denotes a difference embedding vector of the vertebra. denotes a vertebra node feature vector of the vertebra. denotes a preset healthy vertebra prototype embedding vector. denotes a set of difference embedding vectors. The preset healthy vertebra prototype embedding vector is a global “healthy vertebra prototype” embedding vector obtained by the feature extraction server 102 by aggregating the features of healthy vertebrae in all training samples in the training process. denotes a connection string. denotes a first preset multi-layer perceptron. The first preset multi-layer perceptron can be a small multi-layer perceptron, the input dimension of which can be 768 (the dimension of the vertebra node feature vector), and the output dimension of which can be 256. The first preset multi-layer perceptron can include a fully connected layer 1 and a fully connected layer 2. The output dimension of the fully connected layer 1 is 512, and the fully connected layer 1 is activated by a ReLU activation function. The output dimension of the fully connected layer 2 is 256, and the fully connected layer 2 is not activated. In the fifth step, a difference embedding gating function is constructed based on the set of difference embedding vectors. In practice, the feature extraction server 102 can construct the difference embedding gating function based on a second preset multi-layer perceptron and the set of difference embedding vectors. The second preset multi-layer perceptron can be a multi-layer perceptron with the same structure as the first preset multi-layer perceptron, except that the input dimension of the second preset multi-layer perceptron can be 256, and the output dimension of the second preset multi-layer perceptron can be 1. The output dimension of the fully connected layer 1 of the second preset multi-layer perceptron is 64, and the fully connected layer 1 is activated by a ReLU activation function. The output dimension of the fully connected layer 2 of the second preset multi-layer perceptron is 1, and the fully connected layer 2 is activated by a Sigmoid activation function. The difference embedding gating function

[0054] ​​​​is a difference embedding vector controlled gating function. The difference embedding gating function can be:

[0055] .

[0056] wherein, denotes a difference embedding vector of a vertebra . denotes a difference embedding gating function. denotes the second preset multi-layer perception. denotes a Sigmoid activation function.

[0057] In the sixth step, based on the set of vertebra node feature vectors, the set of vertebra attention weight vectors and the difference embedding gating function, a set of vertebra node difference feature vectors is determined. In practice, the feature extraction server 102 can determine the set of vertebra node difference feature vectors based on the set of vertebra node feature vectors, the set of vertebra attention weight vectors and the difference embedding gating function by the following formula:

[0058] .

[0059] wherein, denotes a vertebra node difference feature vector of a vertebra . denotes a vertebra attention weight vector between a vertebra and its neighborhood . denotes a vertebra node feature vector of a vertebra . denotes a vertebra node feature vector of a vertebra . denotes a difference embedding gating function of a vertebra . denotes an element-wise multiplication. The linear transformation matrix is used for the neighbor information channel and the self-difference information channel. denotes a pre-trained neighborhood information transformation matrix. denotes a pre-trained self-feature transformation matrix. denotes an activation function (e.g., an ELU activation function).

[0060] In the seventh step, the set of differential feature vectors of the vertebral nodes is decoded based on a preset decoder of a three-dimensional convolutional neural network to generate a set of segmentation masks of the medical image of the spinal disease. In practice, the feature extraction server 102 can input the set of differential feature vectors of the vertebral nodes into the decoder of the preset three-dimensional convolutional neural network, and obtain the set of segmentation masks of the medical image of the spinal disease after the output layer. The decoder of the preset three-dimensional convolutional neural network can include a feature remapping layer (1x1x1 convolution), an upsampling layer 1 (3D deconvolution), a skip connection layer (feature concatenation), an upsampling layer 2 (3D deconvolution), and an output layer (1x1x1 convolution and Sigmoid activation function).

[0061] The disease identification server 103 is a server for generating a set of reconstructed images and a spinal disease identification result based on the set of spinal conditions, the standard image data, and the set of differential feature vectors of the vertebral nodes.

[0062] In some embodiments, the disease identification server 103 can generate a set of reconstructed images based on the set of spinal conditions, a preset classifier, the standard image data received from the data processing server, and the set of differential feature vectors of the vertebral nodes received from the feature extraction server, and identify the spinal disease by determining a set of reconstruction errors of the set of reconstructed images to obtain the spinal disease identification result. The disease identification server 103 can include, but is not limited to, at least one of a processor, a memory, and a disease identification engine. The set of spinal conditions can include degenerative lesions, neoplastic lesions, fractures, and infections. The preset classifier can be a classifier for classifying the standard image data. For example, the preset classifier can be a conditional diffusion model based on 3D U-Net. The preset classifier can include an encoder (composed of multiple levels of down-sampling blocks, each down-sampling block containing a 3D ResNet block, a self-attention block, and a down-sampling operation), a bottleneck layer (located between the encoder and the decoder, usually composed of one or more 3D ResNet blocks), and a decoder (basically symmetrical to the encoder, composed of multiple up-sampling blocks, each up-sampling block containing an up-sampling operation, a feature concatenation operation, a 3D ResNet block, and a self-attention block).

[0063] In the process of adopting technical solutions to solve the above technical problems, the following technical problem two often accompanies: the traditional system uses simple category labels (such as "tumor" or "infection") as conditional input, and cannot encode complex anatomical relationships (such as the topological connection of intervertebral disc-vertebral body), the system cannot distinguish between image similarities but different anatomical mechanisms of lesions, resulting in the system needing to call the diagnosis model multiple times for verification, causing the display of the disease recognition result to wait for a long time. In view of the above technical problem two, the conventional solution is generally: based on traditional machine learning methods, based on U-Net, ResNet and other medical image classification models, and based on VerFormer (ViT with vertebral body perception attention) recognition method. And considering the shortcomings of the above methods, such as the lack of anatomical relationship modeling (local receptive field is difficult to capture global context and long-range dependency), anatomical structure flattening (3D spine is compressed into 2D tile sequence, losing the topological connection information of intervertebral disc-vertebral body), and low abnormal sensitivity (uniform attention mechanism cannot focus on the lesion area, such as early tumor signal), combined with the current technical status, the following solutions can be adopted:

[0064] In some optional implementations of some embodiments, the disease recognition server 103 described above can be configured to perform the following steps:

[0065] First, based on the preset time step, add Gaussian noise to the standard image data to generate a time step noisy image set. In practice, the disease recognition server 103 described above can add Gaussian noise to the standard image data based on the preset time step to generate a time step noisy image set through the following formula:

[0066] .

[0067] Wherein, represents the time step length Noisy image. represents the preset time step, . The preset time step can be a pre-set unit time length. represents the maximum length of the preset time step. represents the preset noise scheduling parameter (controls the noise intensity, decreases with the increase of ). represents the standard image. represents the standard Gaussian noise ( , represents a multivariate standard normal distribution, represents a unit covariance).

[0068] Secondly, a semantic embedding vector is created for each spinal disorder in the set of spinal disorders to generate a spinal disorder vector, resulting in a set of spinal disorder vectors. In practice, the disease recognition server 103 can map spinal disorders to fixed-length vectors through a learnable embedding layer (Embedding Layer) to create a semantic embedding vector for each spinal disorder in the set of spinal disorders. The learnable embedding layer (Embedding Layer) can be a pre-trained embedding layer that maps discrete class labels (such as words, IDs) to continuous low-dimensional dense vectors.

[0069] Thirdly, the set of vertebral node differential feature vectors is pooled to generate a high-dimensional feature set. In practice, the disease recognition server 103 can use global maximum pooling (GlobalMaxPool) to pool the set of vertebral node differential feature vectors to generate a high-dimensional feature set.

[0070] Fourthly, a joint conditional vector is generated based on the high-dimensional feature set and the set of spinal disorder vectors. In practice, the disease recognition server 103 can concatenate (e.g., using the concat function) the high-dimensional feature set and the set of spinal disorder vectors to generate a joint conditional vector.

[0071] Fifthly, a set of reconstructed images is generated based on a pre-set classifier, the set of time-step noisy images, the joint conditional vector, and the pre-set time step. In practice, the disease recognition server 103 can input the set of time-step noisy images, the joint conditional vector, and the pre-set time step into a pre-set classifier to perform denoising and obtain a set of reconstructed images.

[0072] Sixthly, the reconstruction error of each reconstructed image in the set of reconstructed images is determined to obtain a set of reconstruction errors. In practice, the disease recognition server 103 can quantify the reconstruction error of each reconstructed image in the set of reconstructed images and reflect the diagnosis matching degree through the error:

[0073] .

[0074] wherein, denotes a noisy image and a spinal disorder vector a joint conditional vector corresponding to the reconstructed image. denotes a standard image. denotes a reconstruction error.

[0075] Seventhly, a spinal disease recognition result is generated based on the set of reconstruction errors. In practice, the disease recognition server 103 can select a spinal disorder corresponding to the minimum value of the reconstruction error in the set of reconstruction errors as the spinal disease recognition result. ​ :

[0076] .

[0077] wherein, represents a spinal disease recognition result. The above spinal disease recognition result belongs to the above set of spinal disorder vectors. represents finding a spinal disorder that minimizes the reconstruction error .

[0078] The first step to the seventh step above is an invention point of an embodiment of the present disclosure, which solves the second technical problem that the traditional system uses simple category labels (such as “tumor” or “infection”) as condition input, cannot encode complex anatomical relationships (such as the topological connection of intervertebral disc-vertebral body), and the system cannot distinguish lesions with similar images but different anatomical mechanisms, resulting in the system needing to call the diagnostic model for verification multiple times, causing long display waiting time of disease recognition results. The reason why the system needs to call the diagnostic model for verification multiple times and causes long display waiting time of disease recognition results is as follows: the traditional model uses simple category labels (such as “tumor” or “infection”) as condition input, cannot encode complex anatomical relationships (such as the topological connection of intervertebral disc-vertebral body), and the model cannot distinguish lesions with similar images but different anatomical mechanisms. If the above factors are solved, the effect of reducing the display waiting time of disease recognition results can be achieved. In order to achieve this effect, first, based on a preset time step, Gaussian noise is added to the above standard image data to generate a time step noisy image set. Thus, through the controllable noise injection driven by the time step, the dynamic evolution process of the lesion in the image is simulated, providing standardized input for the diffusion model. Second, a semantic embedding vector is created for each spinal disorder in the above spinal disorder set to generate a spinal disorder vector, obtaining a spinal disorder vector set. Thus, the medical disorder name is converted into a machine-understandable semantic vector, and a mathematical representation space of the disorder is constructed. Third, the above set of vertebral node differential feature vectors is pooled to generate a high-dimensional feature set. Thus, the local features at the vertebral level are aggregated into global spinal representation, and key anatomical information is extracted for decision-making. Fourth, based on the above high-dimensional feature set and the above spinal disorder vector set, a joint condition vector is generated. Thus, the anatomical features and disorder semantics are fused to generate a joint condition signal that guides the reconstruction of the diffusion model. Fifth, based on the preset classifier, the above time step noisy image set, the above joint condition vector, and the above preset time step, a reconstructed image set is generated. Thus, under the anatomical-disorder joint condition constraint, inverse diffusion is performed to generate specific reconstruction images of disease hypotheses. Sixth, the reconstruction error of each reconstructed image in the above reconstructed image set is determined to obtain a reconstruction error set. Thus, the matching difference between different disease hypotheses and the original image is quantified to generate an objective decision basis. Seventh, based on the above reconstruction error set, a spinal disease recognition result is generated. Thus, the lesion type is determined based on the principle of minimizing reconstruction error, and a quantifiable diagnostic confidence is provided. Finally, in combination with the above report generation server 104 and the above user interaction terminal 105, the above spinal disease recognition result is fused with the above standard clinical data and the above set of vertebral node differential feature vectors in a multi-modal manner to obtain a spinal disease structured recognition report, and the above spinal disease structured recognition report is displayed, achieving the effect of reducing the display waiting time of disease recognition results.

[0079] The report generation server 104 is used to perform multimodal fusion of standard clinical data, vertebral node differential feature vector set and spinal disease identification results to obtain multimodal vectors, and generate a structured identification report of spinal diseases based on the multimodal vectors.

[0080] In some embodiments, the report generation server 104 can perform multimodal fusion on the standard clinical data received from the data processing server, the set of differential feature vectors of vertebral nodes received from the feature extraction server, and the spinal disease identification results received from the disease identification server to obtain a multimodal vector. Based on the multimodal vector, a parameter fine-tuning method, and a preset large language model, a structured identification report of spinal diseases can be generated. The report generation server 104 may include, but is not limited to, at least one of the following: a report generation engine.

[0081] In some alternative implementations of certain embodiments, the report generation server 104 described above can be configured to perform the following steps:

[0082] The first step is to concatenate the above set of differential feature vectors of vertebral nodes, the above results of spinal disease identification, and the above standard clinical data to obtain a multimodal vector.

[0083] In practice, firstly, the report generation server 104 can perform pooling operations on the aforementioned set of differential feature vectors of the vertebral nodes to obtain the global features of the vertebral nodes:

[0084] .

[0085] in, This represents the global features of the vertebral nodes. Indicates the vertebral body number. This indicates the total number of vertebrae. Represents the vertebral body The vertebral node difference feature vector.

[0086] Then, the global features of the vertebral nodes, the spinal disease identification results, and the standard clinical data are concatenated to obtain a multimodal vector.

[0087] The second step involves obtaining a large language model for spinal diseases based on the aforementioned low-rank adaptive method and the pre-defined large language model. In practice, firstly, the aforementioned report generation server 104 freezes all original parameters. Secondly, while maintaining the original weights from pre-training... Without changing the parameters, introduce two small, trainable low-rank matrices. and This reduces the number of parameters that need to be trained. Here, the model's weight updates are approximated as... , the forward propagation of the model becomes:

[0088] .

[0089] wherein, represents an output spinal disease structured recognition report. represents an input. represents pre-trained original weights. represents weight updates of the model. The rank is much smaller than and .

[0090] Then, the training target of the above pre-set large language model is set as:

[0091] .

[0092] wherein, the above spinal disease structured recognition report includes tokens. represents the token number of the above spinal disease structured recognition report. represents the token length of the above spinal disease structured recognition report. represents the token in the above spinal disease structured recognition report. represents all tokens before . represents trainable parameters of the low-rank matrix and . represents the above multi-modal vector. represents a log-likelihood probability. represents a cross-entropy loss function.

[0093] Finally, the above pre-set large language model is trained using the above multi-modal vector and low-rank adaptive method to obtain a spinal disease large language model. Wherein, the above pre-set large language model can be a large language model for outputting a medical recognition report according to input structured feature data and structured text data, for example, the above pre-set large language model can be Meta-Llama, Llama-3-70B-Instruct.

[0094] Thirdly, input the above multi-modal vector into the above spinal disease large language model to generate a spinal disease structured recognition report. Wherein, the above spinal disease structured recognition report can include: patient summary (e.g., 65-year-old male, chronic low back pain aggravated with left lower limb radiating pain), detailed findings (e.g., L4-L5 intervertebral disc posterior protrusion 5mm, compressing left L5 nerve root (confidence 94%)), differential remarks (based on uncertainty Value generation note. For example, Add “not rule out intraspinal tumor” when >0.15) and recommended treatment plan. Uncertainty The smaller the value, the more uncertain the characterization model is in distinguishing between the two options, i.e. the higher the uncertainty. The above uncertainty The values are as follows:

[0095] .

[0096] wherein, denotes the noisy image corresponding to the joint condition vector of the spinal disease identification result . denotes the standard image. denotes the suboptimal spinal disease identification result (corresponding to the reconstructed error corresponding to the second smallest value in the reconstructed error set ). denotes the noisy image corresponding to the joint condition vector of the suboptimal spinal disease identification result .

[0097] In some optional implementations of some embodiments, the above system further comprises:

[0098] The user interaction terminal 105 is a terminal for displaying the spinal disease medical image, the spinal disease medical image segmentation mask set and the spinal disease structured identification report, and controlling at least one of the preset robot and the preset chemical dispensing device to perform a preset operation according to the electrical signal. The specific user interaction terminal can refer to Figure 3 , the front-end user interface diagram of the spinal disease identification system based on relationship perception and conditional classification according to the present disclosure.

[0099] In some embodiments, the above user interaction terminal 105 can display the above spinal disease medical image, the above spinal disease medical image segmentation mask set received from the above feature extraction server and the above spinal disease structured identification report received from the above report generation server, and control at least one of the preset robot and the preset chemical dispensing device to perform a preset operation according to the received electrical signal. Wherein, the report generation server 104 can include but is not limited to at least one of the following: a display, a memory, a user input unit, a control interface.

[0100] In the process of adopting technical solutions to solve the above technical problems, the following technical problem three is often accompanied: in the case of high-precision operation demand (sub-millimeter tolerance scene) and emergency intervention timeliness (infection control, intraoperative massive bleeding, etc.), the user interaction terminal lacks a rule engine to convert diagnostic conclusions into device instructions, resulting in delayed system emergency response and decreased patient safety. To solve the above technical problem three, the conventional solution is generally: the user interaction terminal only displays the spinal disease lesion tissue image. The doctor needs to repeatedly switch between the image terminal (displaying the lesion) and the operating table (controlling the device). Considering the above method has the shortcoming of breaking the diagnosis and treatment process, and combining the current technical status, the following solution can be adopted:

[0101] In some optional implementations of some embodiments, the user interaction terminal 105 is configured to:

[0102] First, based on the alpha blending algorithm, the spinal disease medical image and the spinal disease medical image segmentation mask set are fused to obtain the spinal disease lesion tissue image. In practice, the user interaction terminal 105 can use the following alpha blending formula to superimpose the original image and the segmentation mask to highlight the lesion area:

[0103] .

[0104] .

[0105] Wherein, represents the spinal disease lesion tissue image. represents the spinal disease medical image. represents the spinal disease medical image transparency. represents the spinal disease medical image segmentation mask set. represents the red highlight lesion. represents element-wise multiplication.

[0106] Second, based on WebGL, the spinal disease lesion tissue image is projected to a three-dimensional coordinate system for three-dimensional display. In practice, the user interaction terminal 105 can use the medical image standard (RAS: Right, Anterior, Superior (right, front, up)) to establish a three-dimensional space coordinate system with the sacrum (S1 vertebral body) as the origin, and according to the geometric center coordinates of the vertebral body segmentation mask set, map each vertebral body to the three-dimensional space to preserve the true anatomical position relationship. Then, each voxel of the spinal disease lesion tissue image (including healthy tissue and lesion area) is converted into a three-dimensional grid vertex to construct a triangular facet structure and generate a vertex normal vector to simulate real light reflection effects to enhance the three-dimensional sense of the anatomical structure.

[0107] Thirdly, text display of the structured recognition report of the spinal disease is performed. In practice, the user interaction terminal 105 can perform text display of the structured recognition report of the spinal disease on the display.

[0108] Fourthly, interactive operation buttons are generated based on the structured recognition report of the spinal disease and a pre-defined treatment rule library. In practice, the user interaction terminal 105 can generate the interactive operation buttons based on the structured recognition report of the spinal disease and the pre-defined treatment rule library. Then the user interaction terminal 105 can display the interactive operation buttons on the display. The interactive operation buttons include a degenerative lesion treatment scheme, a fracture treatment scheme, a tumor lesion treatment scheme, and an infection treatment scheme. The pre-defined treatment rule library can be a rule library including a set of spinal disorders and corresponding treatment schemes.

[0109] Fifthly, in response to receiving a trigger signal of the interactive operation buttons, at least one of a pre-set robot and a pre-set chemical dispensing device is controlled to perform a pre-set operation according to the type of the interactive operation buttons, including the following sub-steps, wherein the pre-set robot includes a first pre-set robot and a second pre-set robot, and the pre-set chemical dispensing device includes a first pre-set chemical dispensing device and a second pre-set chemical dispensing device:

[0110] Sub-step one, in response to the interactive operation button representing the degenerative lesion treatment scheme, the lesion position is determined based on the set of medical image segmentation masks of the spinal disease, and the first pre-set robot is controlled to perform a vertebral body stabilization operation based on the lesion position. The first pre-set robot can be a robot capable of performing a vertebral body stabilization operation (such as pedicle screw implantation, percutaneous puncture resection). For example, the first pre-set robot can be Mazor X Stealth Edition (Medtronic).

[0111] Sub-step two, in response to the interactive operation button representing the fracture treatment scheme, a vertebroplasty puncture path is determined based on the set of medical image segmentation masks of the spinal disease, and a bone cement injection operation is performed by the second pre-set robot based on the vertebroplasty puncture path. In practice, the user interaction terminal 105 can determine the vertebroplasty puncture path according to the following formula:

[0112] .

[0113] wherein, The vertebroplasty puncture path is shown. The second preset robot can be a robot capable of performing bone cement injection operations according to the vertebroplasty puncture path. For example, the second preset robot can be a ROSA (Robot of Stereotactic Assistant) surgical robot.

[0114] Sub-step three, in response to the interactive operation button representing the tumor lesion treatment scheme, the target drug concentration is determined based on the above-mentioned structured identification report of spinal diseases, and the first preset chemical dispensing device is called to synthesize the sustained-release preparation based on the target drug concentration. In practice, the user interaction terminal 105 can determine the target drug concentration according to the following formula:

[0115] .

[0116] Wherein, The target drug concentration is represented by T, and the unit can be The first preset chemical dispensing device can be a device capable of synthesizing sustained-release preparations according to the target drug concentration. For example, the first preset chemical dispensing device can be a YuMi robot of ABB Company in Switzerland.

[0117] Sub-step four, in response to the interactive operation button representing the infection treatment scheme, the second preset chemical dispensing device is called to prepare antibiotic sustained-release microparticles based on the above-mentioned structured identification report of spinal diseases and the obtained pathogenic bacteria identification report. The obtained pathogenic bacteria identification report can include the type of pathogenic bacteria (such as pathogenic bacteria = “MRSA”). The second preset chemical dispensing device can be a device capable of preparing antibiotic sustained-release microparticles according to the type of pathogenic bacteria (such as vancomycin-PLGA microparticles synthesized according to MRSA drug resistance). For example, the second preset chemical dispensing device can be an MIT sustained-release microparticle robot.

[0118] The first step to the fifth step and its related content as one of the invention points of the embodiment of the present disclosure solves the technical problem three "in the case of high-precision operation demand (sub-millimeter tolerance scene) and emergency intervention timeliness (infection control, intraoperative massive bleeding, etc.), the user interaction terminal lacks a rule engine to convert diagnostic conclusions into device instructions, resulting in delayed system emergency response and decreased patient safety". The factors that lead to delayed system emergency response and decreased patient safety are often as follows: the interaction terminal lacks a rule engine to convert diagnostic conclusions into device instructions, and cannot generate executable operations. If the above factors are solved, the system emergency response is timely and the patient safety is increased. In order to achieve this effect, first, based on the alpha blending algorithm, the above spinal disease medical image and the above spinal disease medical image segmentation mask set are fused to obtain a spinal disease lesion tissue image. Thus, the lesion tissue region is highlighted for subsequent visualization. Second, based on WebGL, the above spinal disease lesion tissue image is projected to a three-dimensional coordinate system for three-dimensional display. Thus, the interactive and intuitive display of the spatial position and morphology of the lesion is realized. Third, the above spinal disease structured identification report is displayed as text. Thus, the above spinal disease structured identification report is clearly presented in the form of structured text, providing clear diagnostic basis and treatment recommendations for doctors. Fourth, based on the above spinal disease structured identification report and the pre-defined treatment rule library, an interactive operation button is generated. Thus, the diagnostic conclusion is converted into an executable treatment scheme option at one key. Fifth, in response to receiving a trigger electrical signal of the above interactive operation button, at least one of the pre-set robot and the pre-set chemical dispensing device is controlled to perform a pre-set operation according to the type of the above interactive operation button. Thus, the button instruction is received and the corresponding robot and dispensing device are driven to perform precise treatment operation, completing the steps from decision to execution. Finally, the effect of timely system emergency response and increased patient safety is realized.

[0119] The above various embodiments of the present disclosure have the following beneficial effects: through the relationship-aware and condition-based spinal disease recognition system of some embodiments of the present disclosure, by modeling the intervertebral biomechanical dependence and generating an anatomy-aware reconstruction conditioned on high-dimensional graph embedding, an interpretable decision mechanism is provided for large language models, the display latency of the system is reduced, and redundant computation cycles are reduced. Specifically, the reason for the long display latency of the disease recognition result of the system, the inconsistency between the spinal disease recognition report displayed by the user interaction terminal and the fact, the need for the system to re-execute, and the resulting redundant computation cycles are as follows: the long display latency of the system. Specifically, the system based on convolutional neural network (CNN, such as U-Net) is limited by the inherent receptive field, and when identifying and accurately segmenting lesions with various shapes and fuzzy boundaries, the performance will decrease significantly, and the system needs to call the diagnostic model for verification multiple times. The display latency of the disease recognition result is long. The system has redundant computation cycles. Specifically, the system based on Transformer (such as visual transformer ViT) lacks a fine understanding of the three-dimensional anatomical relationship of the spine and fails to fully utilize the topological structure information between the vertebral bodies. The existing computer-aided diagnosis (CADx) system lacks the ability to integrate different information for comprehensive differential diagnosis, and cannot automatically and seamlessly convert the quantitative analysis results of AI into high-quality diagnostic text. The spinal disease recognition report displayed by the user interaction terminal is inconsistent with the fact, and the system needs to be re-executed, resulting in redundant computation cycles. Based on this, the relationship-aware and condition-based spinal disease recognition system of some embodiments of the present disclosure includes a data processing server, a feature extraction server, a disease recognition server, a report generation server, and a user interaction terminal connected in communication, wherein: first, the above data processing server is used to perform preset standardization processing on the spinal disease patient data to generate standard image data and standard clinical data. Thus, the original medical images and clinical texts are standardized to generate image and clinical data in a unified format, providing high-quality input for downstream analysis. Second, the above feature extraction server is used to extract features and segment the above standard image data to generate a set of vertebral node differential feature vectors and a set of spinal disease medical image segmentation masks. Thus, the intervertebral differential features and segmentation masks are extracted by the graph transformer differential information interactor (GT-DII) to quantify the abnormalities of the spinal anatomy. Then, the above disease recognition server is used to generate a set of reconstructed images and spinal disease recognition results based on a set of spinal conditions, the above standard image data, and the above set of vertebral node differential feature vectors. Thus, the condition diffusion diagnosis network (CD-DN) is used to generate reconstructed images and calculate reconstruction errors to realize lesion type recognition and uncertainty quantification.Then, the report generation server is configured to perform multi-modal fusion on the standard clinical data, the set of vertebral node differential feature vectors, and the spinal disease recognition result received from the disease recognition server to obtain a multi-modal vector, and generate a structured spinal disease recognition report based on the multi-modal vector. In this way, multi-modal data is fused to generate a structured diagnosis report, and the LoRA method is used to fine-tune the form data set based on the artificially constructed instructions, and the llama-factory framework is used for fine-tuning; then, the fine-tuned special large language model outputs operable clinical insights. Finally, the user interaction terminal is used to display the spinal disease medical image, the set of spinal disease medical image segmentation masks, and the structured spinal disease recognition report, and to control at least one of a preset robot and a preset chemical dispensing device to perform a preset operation according to an electrical signal. In this way, the image, the mask, and the report can be visualized, and the robot and the dispensing device can be controlled by the interactive buttons to perform precise treatment operations. This embodiment reduces the display waiting time of the system and reduces the effect of redundant calculation cycles.

[0120] The above description is merely some preferred embodiments of the present disclosure and an explanation of the principles of the technology used. Those skilled in the art should understand that the scope of the application involved in the embodiments of the present disclosure is not limited to the technical solutions formed by the specific combinations of the above technical features, and should also cover other technical solutions formed by any combination of the above technical features or equivalent features without departing from the above inventive concept. For example, the above features can be replaced with similar technical features disclosed in the embodiments of the present disclosure (but not limited to) to form a technical solution.​

Claims

1. A spinal disease identification system based on relation perception and conditional classification, wherein, The spinal disease identification system includes: a data processing server, a feature extraction server, a disease identification server, and a report generation server that are interconnected. The data processing server is used to perform pre-defined standardization processing on spinal disease patient data to generate standard imaging data and standard clinical data. The feature extraction server is used to extract and segment features from the standard image data to generate a set of vertebral node differential feature vectors and a set of medical image segmentation masks for spinal diseases. The feature extraction server is configured as follows: Based on standard image data, a high-dimensional spinal disease feature map, a set of vertebral nodes, a set of vertebral node feature vectors, and a set of vertebral edge relationships are generated. Based on the set of vertebral edge relationships, the set of vertebral node feature vectors is aggregated by mean to generate a neighborhood mean feature set. Based on the set of vertebral nodes and the set of vertebral edge relationships, an undirected graph structure is constructed, and a set of vertebral attention weight vectors is generated according to the undirected graph structure. Based on the preset healthy vertebral body prototype embedding vector, the vertebral body node feature vector set, and the neighborhood mean feature set, a differential embedding vector set is generated; Based on the set of differential embedding vectors, a differential embedding gate function is constructed; Based on the set of vertebral node feature vectors, the set of vertebral attention weight vectors, and the differential embedding gating function, the set of vertebral node differential feature vectors is determined; Based on a pre-defined three-dimensional convolutional neural network decoder, the differential feature vector set of the vertebral nodes is decoded to generate a medical image segmentation mask set for spinal diseases. The disease identification server is used to generate a set of reconstructed images and spinal disease identification results based on the set of spinal diseases, the standard image data, and the set of differential feature vectors of vertebral nodes. The disease identification server is configured as follows: Based on a preset time step, Gaussian noise is added to the standard image data to generate a set of noisy images at the time step. A semantic embedding vector is created for each spinal disease in the set of spinal diseases to generate a spinal disease vector, resulting in a set of spinal disease vectors; Pooling is performed on the set of differential feature vectors of the vertebral nodes to generate a high-dimensional feature set; Based on the high-dimensional feature set and the spinal disease vector set, a joint condition vector is generated; Based on a preset classifier, the noisy image set at the specified time step, the joint conditional vector, and the preset time step, a set of reconstructed images is generated. The reconstruction error of each reconstructed image in the reconstructed image set is determined to obtain the reconstruction error set; Based on the reconstruction error set, spinal disease identification results are generated; The report generation server is used to perform multimodal fusion of the standard clinical data, the set of differential feature vectors of vertebral nodes and the spinal disease identification results to obtain multimodal vectors, and generate a structured identification report of spinal diseases based on the multimodal vectors.

2. The system according to claim 1, wherein, The system also includes: The user interaction terminal is used to display medical images of spinal diseases, the segmentation mask set of the medical images of spinal diseases, and the structured identification report of spinal diseases, and to control at least one of the preset robot and the preset chemical dispensing device to perform preset operations according to electrical signals.

3. The system according to claim 1, wherein, The spinal disease patient data includes: spinal disease medical imaging and clinical text data.

4. The system according to claim 3, wherein, The medical imaging of the spinal disease includes: computed tomography (CT) images and magnetic resonance imaging (MRI) images, and The data processing server is configured to: The computed tomography images and the magnetic resonance imaging images are resampled to generate a spatially standardized medical image set for spinal diseases; The spatially standardized medical image set of spinal diseases is normalized to generate standard image data.

5. The system according to claim 4, wherein, The clinical text data includes: structured text data and unstructured text data, and The data processing server is further configured to: The structured text data is parsed and encoded to generate structured clinical feature vectors; The unstructured text data is cleaned to generate cleaned unstructured data. Key entities are extracted from the cleaned unstructured data to generate clinical entity vectors; The structured clinical feature vector and the clinical entity vector are determined as standard clinical data.

6. The system according to claim 1, wherein, The report generation server is configured to: The vertebral node differential feature vector set, the spinal disease identification results, and the standard clinical data are concatenated to obtain a multimodal vector; A large language model for spinal diseases is generated based on a low-rank adaptive method and a pre-defined large language model. The multimodal vectors are input into the large language model of spinal diseases to generate a structured recognition report of spinal diseases.

Images