Image retrieval method for similar cases of adolescent scoliosis

Abstract

The present invention discloses a method for retrieving images of similar cases of adolescent scoliosis, belonging to the technical field of adolescent scoliosis image processing and case analysis. The method comprises: obtaining an AP-position spinal image, an L-position spinal image, a BL-position spinal image, and a BR-position spinal image of a patient; processing each spinal image using a pre-trained semantic segmentation model; extracting features of the AP-position vertebral contour, the L-position vertebral contour, the BL-position vertebral contour, and the BR-position vertebral contour; constructing a spinal feature vector using the AP-position vertebral features, the L-position vertebral features, the BL-position vertebral features, and the BR-position vertebral features; searching a database for the spinal feature vector with the closest spatial distance to the spinal feature vector, and using the case image corresponding to the closest spinal feature vector as the desired similar case image of adolescent scoliosis. The present invention is highly robust, applicable to special needs requiring high similarity between medical image samples, and has high accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of adolescent scoliosis image processing and case analysis, and in particular to a method for retrieving images of similar cases of adolescent scoliosis. Background Art

[0002] Adolescent idiopathic scoliosis (AIS) is a three-dimensional spinal deformity with an unknown etiology. According to epidemiological statistics, the incidence rate is as high as 0.5%-5.2%. The deformity can lead to cosmetic changes, psychological and social impairments, decreased marriage and childbearing rates, and increased suicide rates. In severe cases, it can cause chest compression, decreased lung capacity, and even disability, severely impacting adolescents' health. When the scoliosis exceeds 45 degrees, surgical treatment is available to correct the deformity through physical means and slow the progression of the spinal curvature. Spinal fusion is a surgical procedure for scoliosis, in which alloy rods are implanted to correct and stabilize the vertebrae with the most severe curvature.

[0003] Clinically, imaging analysis of AIS cases is essential. Severe cases typically undergo corrective surgery, while milder cases utilize treatments such as bracing to prevent or delay scoliosis progression. Both treatment options require imaging analysis and evaluation. Because the spinal curvature of AIS is the most significant factor influencing surgical and nonsurgical treatments and rehabilitation outcomes, identifying imaging data and treatment records of similar cases in historical medical records is of great clinical value for planning surgery, optimizing surgical procedures, and reducing surgical risks.

[0004] In order to find medical records with spinal structures similar to those of current AIS patients, clinical doctors need to search through the patient's imaging data one by one based on their personal memory. This is not only tedious but also time-consuming, making it an almost impossible task.

[0005] The similarity retrieval technology based on image key points in the existing technology is to retrieve images similar to the original image from a set of images, and the commonly used technology is image feature extraction and matching technology. The technical principle of image feature extraction and matching is to detect and describe the key points (feature points) in the image. Key points are points in the image that are unique and robust and can be used to describe the structure and content of the image. The feature extraction algorithm detects key points and calculates the corresponding feature description operators, while the feature matching algorithm determines the matching key points by comparing the feature description operators in different images. Feature description operators for calculating key points, such as the SIFT (Scale-Invariant Feature Transform) algorithm and the SURF (Speeded Up Robust Features) algorithm, etc. The above feature extraction algorithms are characterized by strong robustness to scale, rotation, translation, and illumination changes, and have a wide range of applications in image matching, object recognition, panoramic stitching and other fields, such as Figure 1 shown.

[0006] The disadvantages of keypoint-based image similarity retrieval are as follows:

[0007] 1. The computational complexity is high and the number of feature points is small;

[0008] Second, the number of key points extracted from different image objects is not fixed, and the benchmarks for similarity calculation are different, so the system is not very robust;

[0009] 3. In view of the special needs of high similarity between medical image samples, the accuracy of the above-mentioned general feature extraction and matching algorithms is not high. Summary of the Invention

[0010] The technical problem to be solved by the present invention is to provide a method for retrieving images of similar cases of adolescent scoliosis that is highly robust, suitable for the special needs of high similarity between medical image samples, and has high accuracy.

[0011] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0012] A method for retrieving images of similar cases of adolescent scoliosis, comprising:

[0013] Step S101: Acquire an AP spinal image, an L spinal image, a left flexion BL spinal image, and a right flexion BR spinal image of the patient;

[0014] Step S102: using a pre-trained semantic segmentation model to process each spinal image, and extracting the AP-position vertebral body contour, L-position vertebral body contour, BL-position vertebral body contour, and BR-position vertebral body contour respectively;

[0015] Step S103: extracting features of the AP, L, BL, and BR vertebral contours to obtain AP, L, BL, and BR vertebral features, respectively.

[0016] Step S104: constructing a spine feature vector using the AP-position vertebral features, the L-position vertebral features, the BL-position vertebral features, and the BR-position vertebral features;

[0017] Step S105: Search the database for the spinal feature vector that is closest in spatial distance to the spinal feature vector, and use the case image corresponding to the closest spinal feature vector as the required adolescent scoliosis similar case image, wherein the database stores adolescent scoliosis case images and their corresponding spinal feature vectors.

[0018] The present invention has the following beneficial effects:

[0019] The present invention relates to a method for retrieving images of similar cases of adolescent scoliosis. The method first performs semantic segmentation and feature extraction on the patient's AP, L, BL, and BR spinal images. The extracted features are then used to construct a spinal feature vector. Finally, the spinal feature vector with the closest spatial distance to the spinal feature vector is searched in a database to obtain the desired images of similar cases of adolescent scoliosis. The present invention extracts features from the AP, L, BL, and BR spinal images, and the features are highly representative. Therefore, the present invention is highly robust and applicable to special needs where the similarity between medical image samples is very high, with high accuracy. Furthermore, the present invention can accurately obtain diagnostic and treatment data for similar cases, assisting doctors in comparing case data and scientifically planning surgical plans. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 Schematic diagram of SIFT feature extraction and image similarity matching in the existing technology;

[0021] Figure 2 Schematic diagram of the process of the image retrieval method for similar cases of adolescent scoliosis of the present invention;

[0022] Figure 3 Schematic diagram of the principle of the image retrieval method for similar cases of adolescent scoliosis of the present invention;

[0023] Figure 4 Schematic diagram of the AP spinal image, L spinal image, BL spinal image and BR spinal image in the present invention;

[0024] Figure 5 Schematic diagram of the BL-position spinal image coordinate rotation in the present invention;

[0025] Figure 6 Schematic diagram of the BR-position spinal image coordinate rotation in the present invention;

[0026] Figure 7 Schematic diagram of the Manhattan distance principle in the present invention;

[0027] Figure 8 Schematic diagram of Manhattan distance calculation in the present invention;

[0028] Figure 9 Schematic diagram of the original image file directory structure in the present invention;

[0029] Figure 10 Schematic diagram of the effect of an embodiment of the present invention, wherein (a) is the query image q1, (b) is the R1 image, (c) is the R2 image, (d) is the R3 image, and (e) is the R4 image;

[0030] Figure 11 Schematic diagram of L-position vertebral feature extraction in Comparative Example 1 of the present invention. DETAILED DESCRIPTION

[0031] In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0032] To address the inefficiency of manual retrieval of images of similar AIS cases and the low accuracy of key-point-based image feature extraction and matching methods, this paper proposes a new image processing and similarity retrieval method, which shortens the retrieval time of thousands of images to just a few minutes, speeding up the retrieval of similar case images. At the same time, it accurately obtains diagnostic and treatment data of similar cases, assisting doctors in comparing case data and scientifically planning surgical plans.

[0033] The present invention provides a similar case image retrieval method for adolescent scoliosis (similarity retrieval method for AIS case images), such as Figure 2 、 Figure 3 Shown, including:

[0034] Step S101: Acquire an AP spinal image, an L spinal image, a left flexion BL spinal image, and a right flexion BR spinal image of the patient;

[0035] In this step, the image format of each spinal image can support digital image file formats such as JPEG, PNG, WebP and BMP.

[0036] Step S102: using a pre-trained semantic segmentation model to process each spinal image, and extracting the AP-position vertebral body contour, L-position vertebral body contour, BL-position vertebral body contour, and BR-position vertebral body contour respectively;

[0037] As an optional embodiment, the semantic segmentation model is a U-Net deep neural network model.

[0038] This invention uses the U-Net deep neural network to segment vertebrae from multiple X-ray images. U-Net is a neural network architecture for semantic segmentation, proposed by Olaf Ronneberger et al. in 2015. The name U-Net comes from its U-shaped network structure, which consists of a contracting path and an expanding path. The contracting path consists of multiple convolutional layers and pooling layers, each followed by a pooling layer. The purpose of the pooling layer is to reduce the image resolution, thereby reducing the number of network parameters and computational complexity. The output of the contracting path is the image feature map. The encoder consists of multiple convolutional layers and activation functions to extract image features. The output of the encoder is the encoded representation of the image. The expanding path consists of multiple upsampling layers and convolutional layers, each followed by a convolutional layer. The purpose of the upsampling layer is to increase the image resolution, thereby restoring the image to its original size. The output of the expanding path is the image segmentation result. The decoder consists of multiple convolutional layers and activation functions to restore the image to its original size and details. The output of the decoder is the final image segmentation result.

[0039] In the specific implementation, semantic segmentation technology is used to segment the AP, L, BL and BR of the preoperative image to obtain the precise coordinates of each spinal structure and each vertebra in the image. Figure 4 As shown, the present invention trains a model for each view, Model_AP, Model_L, Model_BL, and Model_BR. Model design and training are common knowledge in the field. The model design and training process in the present invention are briefly described as follows:

[0040] The model design process can be as follows: First, create a dataset for each view, splitting it into training and validation sets. Preprocess the training images into PNG files, resize them to a fixed size, and convert them all into NumPy arrays. Then, convert the arrays into tensors to ensure the correct mapping between the original images and the annotated labels. Define the U-Net model structure. Since X-ray images in this paper are grayscale images, set the number of channels to 1 and the number of classes to 1.

[0041] Table 1

[0042]

[0043] The training process of the model can be as follows: for each view, first calculate the loss value and update the model parameters through backpropagation; calculate the loss of the validation dataset and keep the loss value of each training stage; record the loss value and model performance data of each image batch.

[0044] After saving the model, one of the images can be captured and passed to the trained model, which returns a predicted image. The DICE coefficient is used to evaluate the performance of the model. The DICE coefficients for the four views in this invention are: AP 90.5%, L 86.2%, BL 93%, and BR 93%.

[0045] That is, as another optional embodiment, step S102 includes:

[0046] Step S1021: Processing the AP-position spinal image using a pre-trained AP-position vertebral semantic segmentation model (specifically, the aforementioned Model_AP) to extract the vertebral contours of the AP-position spinal image, including the cervical vertebra C7, thoracic vertebrae T1-T12, and lumbar vertebrae L1-L5, i.e., the AP-position vertebral contours, and simultaneously identifying each vertebra;

[0047] Step S1022: Processing the L-position spinal image using a pre-trained L-position vertebral semantic segmentation model (specifically, Model_L) to extract the vertebral contours of thoracic vertebrae T2-T7 in the L-position spinal image, i.e., the L-position vertebral contours, and simultaneously identifying each vertebra;

[0048] Step S1023: Use the pre-trained BL-position vertebral semantic segmentation model (specifically the above-mentioned Model_BL) and the BR-position vertebral semantic segmentation model (specifically the above-mentioned Model_BR) to process the BL-position spinal image and the BR-position spinal image respectively, extract the vertebral contours containing C7-L5 in the BL-position spinal image and the BR-position spinal image, that is, the BL-position vertebral contour and the BR-position vertebral contour, and identify each vertebra at the same time.

[0049] In this way, through the above steps S1021 - S1023 , the vertebral contours of 18 vertebrae at the AP position, the vertebral contours of 6 vertebrae at the L position, the vertebral contours of 18 vertebrae at the BL position, and the vertebral contours of 18 vertebrae at the BR position can be obtained.

[0050] It is understandable that the above steps S1021-S1023 are performed in no particular order.

[0051] In addition, it can be seen from the above steps S1021-S1023 that the present invention is based on structure-based image feature extraction, which is specifically described as follows:

[0052] Keypoint-based image features search for typical features from the local features of all image pixels. These features are often unrelated to the higher-level semantics of the image. Structure-based image features, on the other hand, target specific pathological regions. In AIS, they focus solely on the spine in X-ray images, excluding features from other image components. Semantic segmentation techniques are used to extract spinal features. This technique determines the precise location and relationship of each vertebra in the two-dimensional AP, L, left flexion BL, and BR images, thereby constructing the spatial structure of the spine in AIS patients. The structural data extracted from the AP and L views describes the three-dimensional structure of the spine in natural standing. The BL and BR represent the ability of the spine to deform under external forces and describe the structure of the spine in flexion and extension. These structural data are the most typical disease features of AIS and are unaffected by features of other body parts or background in the X-ray image. Therefore, structure-based image features better represent the high-level semantic characteristics of the spine than keypoint-based image features, laying the foundation for image comparison and retrieval.

[0053] Step S103: extracting features of the AP, L, BL, and BR vertebral contours to obtain AP, L, BL, and BR vertebral features, respectively.

[0054] This step is to extract features from the segmented vertebral image.

[0055] As an optional embodiment, step S103 includes:

[0056] Step S1031: Calculate the centroid coordinates of each vertebra according to the AP-position vertebral contour, and use the x-axis coordinates of the centroid coordinates of each vertebra as the AP-position vertebral feature;

[0057] In this step, the calculation method of the center of mass coordinates is:

[0058] .

[0059] Preferably, the center of mass coordinates of lumbar vertebra L5 is used as the origin in the AP vertebral contour. Because L5 is the last segment of the lower lumbar spine, it has distinct morphological features, is easily identifiable, and is relatively stable and not easily affected by posture changes. Therefore, using its center of mass as the origin can help more accurately locate other vertebrae and related structures.

[0060] Step S1032: Based on the BL position vertebral body contour and the BR position vertebral body contour, calculate the center of mass coordinates of each vertebra when they are rotated until the line connecting C7 and L5 (center of mass) is vertical, and use the x-axis coordinates of the center of mass coordinates of each vertebra in the BL position vertebral body contour as the BL position vertebral body feature, and use the x-axis coordinates of the center of mass coordinates of each vertebra in the BR position vertebral body contour as the BR position vertebral body feature.

[0061] Preferably, the vertebral body contour at the BL position and the vertebral body contour at the BR position both use the centroid coordinates of the lumbar vertebra L5 as the origin.

[0062] In this step, if Figure 5 、 Figure 6 As shown in the figure, because the coordinates of vertebrae located away from the median perpendicular line of the sacrum in the BL and BR images are artificially altered and magnified, these coordinate values ​​significantly affect the similarity matching results. Therefore, coordinate translation and rotation methods are used to correct these errors. The method uses the coordinates of the L5 vertebral centroid as the center, rotating the BL image by θ degrees clockwise and the BR image by θ degrees counterclockwise. The angle θ is the angle between the line connecting C7 and L5 and the y-axis.

[0063] In specific implementation, the coordinates of the lumbar vertebra L5 center of mass can be first translated to the origin (0, 0). Assuming the coordinates of the L5 center of mass are (x0, y0), the translation formula is:

[0064] x' = x - x0

[0065] y' = y –y0

[0066] The coordinates of the translated vertebral center of mass are then rotated clockwise or counterclockwise by θ degrees around the L5 center of mass (which has become the coordinate origin at this time). The formula for calculating the coordinates of each vertebral center of mass after rotation is:

[0067] Rotate θ counterclockwise:

[0068]

[0069]

[0070] Rotate θ clockwise:

[0071]

[0072]

[0073] Thus, through steps S1031-S1032, the AP, BL, and BR vertebral features are obtained, namely, the x-axis coordinates of the centroid coordinates of each AP, BL, and BR vertebra. The present invention is directed to adolescent scoliosis. Therefore, the x-axis coordinates of the centroid coordinates of each vertebra represent the distance the spine deviates from the midline, reflecting the disease condition. Therefore, only the x-axis coordinates need to be used as the AP vertebral features, eliminating the need to calculate the y-axis coordinates. This reduces the amount of computation and improves the accuracy of similarity calculations.

[0074] To improve the accuracy of case image retrieval, preferably, step S103 further includes (specifically, may include after step S1032):

[0075] Step S10321: performing standardization processing on the AP position vertebral features, BL position vertebral features, and BR position vertebral features.

[0076] Since the X-ray image size is affected by factors such as the patient's age, height, and shooting distance, in order to eliminate the influence of the above factors on the measurement results, the vertebral centroid coordinates are standardized. The standardization process includes:

[0077] Step A1: Calculate the average vertebral width ;

[0078] Step A2: Divide the AP position vertebral body feature, BL position vertebral body feature and BR position vertebral body feature by the average value of the vertebral body width respectively to obtain the final AP position vertebral body feature, BL position vertebral body feature and BR position vertebral body feature.

[0079] In specific implementation, the coordinates of the centroid of each vertebra (horizontally) divided by , get the new coordinate value, namely:

[0080]

[0081] So, after After normalization, the horizontal coordinate of the vertebral centroid can be regarded as the deviation from the width of one vertebra, that is, the deviation from the distance of several vertebrae.

[0082] As another optional embodiment, step S103 includes:

[0083] Step S1033: Calculate the Cobb angle according to the L-position vertebral contour to obtain the L-position vertebral feature.

[0084] In this step, since the lateral spinal image in the present invention is a closed continuous contour, the Cobb angle value is used as the feature of the thoracic sagittal L-view image. The calculation of the Cobb angle is common knowledge in the art and will not be repeated here.

[0085] In this way, through the above steps S1031-S1032, the x-axis coordinates of the center of mass coordinates of each of the 18 vertebrae in the AP position, the x-axis coordinates of the center of mass coordinates of each of the 18 vertebrae in the BL position, and the x-axis coordinates of the center of mass coordinates of each of the 18 vertebrae in the BR position can be obtained. After the standardization processing of step S10321, the AP position vertebral features, BL position vertebral features and BR position vertebral features are obtained; the Cobb angle value obtained through the above step S1033 is used as the L position vertebral feature.

[0086] It is understandable that the above steps S1031-S1033 are performed in no particular order.

[0087] Step S104: constructing a spine feature vector using the AP-position vertebral features, the L-position vertebral features, the BL-position vertebral features, and the BR-position vertebral features;

[0088] In this step, the spinal feature vector can be composed of the following dimensions: AP (x-axis coordinate of the center of mass coordinates of 18 standardized vertebrae), L (1 Cobb angle value), BL (x-axis coordinate of the center of mass coordinates of 18 standardized vertebrae), and BR (x-axis coordinate of the center of mass coordinates of 18 standardized vertebrae).

[0089] The x-axis coordinates of the standardized AP, BL, and BR vertebral centroids are collected. The Cobb angle is used to replace the L angle in the left sagittal plane of the thoracic spine. A one-dimensional vector, called an embedding, is constructed sequentially and expressed as a fixed-length array of 55. This vector is named AIS55 in this paper. It is shown below:

[0090] .

[0091] As an optional embodiment, step S104 includes:

[0092] The spinal feature vector is weighted, and the weighted processing includes: performing bitwise multiplication of a preset weight vector and the spinal feature vector to obtain a final spinal feature vector, wherein T2 to T5 in the AP position vertebral contour, BL position vertebral contour, and BR position vertebral contour are divided into the upper thoracic segment (PT), T6 to T11 as the main thoracic segment (MT), T12 to L1 as the thoracolumbar segment (TL), and L2 to L5 as the lumbar segment (L). For the preset weights of the AP position vertebral features, BL position vertebral features, and BR position vertebral features in the preset weight vector, the weight of the segment where the main curve (i.e., the scoliosis with the largest Cobb angle) is located is set to twice the weights of other segments.

[0093] In this step, the weight vector is designed based on clinical experience. , the weight vector The result of bitwise multiplication (Hadamard product) with AIS55 is a weighted vector. Through weighting, the precision (Presion) and recall (Recall) of AIS55 feature retrieval are higher, resulting in excellent model performance. The weight setting rules are as follows:

[0094] Increase the weight of all vertebral dimensions for the primary curve (i.e., the curve with the largest Cobb angle). Increasing the offset of the vertebral body where the primary curve is located can highlight its features and suppress noise in other dimensions. In this example, the weight of the primary curve is set to twice that of other dimensions.

[0095] In specific implementation, as shown in Table 2, when the main curve is located in the upper thoracic segment (T2~T5), the weight ratio of the PT:MT:TL:L dimension interval on the weight vector is 2:1:1:1; when the main curve is located in the main thoracic segment (T6~T11), the weight ratio of the PT:MT:TL:L dimension interval on the weight vector is 1:2:1:1; when the main curve is located in the thoracolumbar segment (T12~L1), the weight ratio of the PT:MT:TL:L dimension interval on the weight vector is 1:1:2:1; when the main curve is located in the lumbar segment (L2~L5), the weight ratio of the PT:MT:TL:L dimension interval on the weight vector is 1:1:1:2.

[0096] Table 2 AIS main bend position and weight ratio

[0097]

[0098] Preferably, the preset weight of the L-position vertebral feature in the preset weight vector is set to be the same as the weight of the segment where the main curve is located.

[0099] At this point, the extracted spinal features are subjected to calculation of the vertebral center of mass, coordinate rotation, normalization, vector splicing, and weight correction to obtain features describing the patient's spinal structure, which are represented by a high-dimensional vector, namely the spinal feature vector.

[0100] Thus, through a weighted operation, a preset weight vector is used to adjust the contribution of each component of the feature vector (AP, L, BL, BR) in calculating the Manhattan distance. This vector is determined by empirical rules. Multiplying the preset weight vector by the AIS55 feature vector bitwise essentially transforms the feature vector spatially, which helps improve the model's precision and recall, resulting in higher precision and recall scores for AIS55 feature retrieval, and thus excellent model performance.

[0101] Step S105: Search the database for the spinal feature vector that is closest in spatial distance to the spinal feature vector, and use the case image corresponding to the closest spinal feature vector as the required adolescent scoliosis similar case image, wherein the database stores adolescent scoliosis case images and their corresponding spinal feature vectors.

[0102] In this step, the database is searched for the spinal eigenvector that is closest in spatial distance to the spinal eigenvector. The searched spinal eigenvector can be one or the first n spinal eigenvectors. The database stores images of adolescent scoliosis cases and their corresponding spinal eigenvectors. The spinal eigenvectors are generated in the same manner as in steps S101-S104 above and will not be further described here.

[0103] As an optional embodiment, in step S105, the spatial distance is calculated using a Manhattan distance calculation method.

[0104] The present invention uses the standardized Manhattan distance to measure the similarity of AIS structural features. Since the extracted features are represented as vectors, in order to compare the "size" of two image features, it is usually converted to calculating the distance between the two vectors in space. There are many methods to calculate the distance between two vectors, such as Euclidean distance, cosine distance, Manhattan distance, etc. Among them, Euclidean distance is suitable for measuring the distance between low-dimensional vectors, and the size of each dimension has a greater impact on the distance. As the data dimension increases, the recognition of Euclidean distance gradually decreases. Cosine distance only focuses on the direction of the vector and is not affected by the length of the vector, which is applicable to data of different scales. A major disadvantage of cosine similarity is that it does not consider the size of the vector, only its direction. Manhattan distance, also known as taxi distance or city block distance, refers to the distance between two vectors if they can only move at right angles. The movement of diagonals is not involved in the calculation of distance. Manhattan distance retains the differences between the relative dimensions of high-dimensional vectors and distinguishes them through a summation operation. Figure 7The red line in the middle represents the Manhattan distance, while the green line represents the Euclidean distance, which is the straight-line distance. Manhattan distance only involves addition and subtraction algebraic operations, and is much faster than the Euclidean distance, which requires square root operations.

[0105] The Manhattan distance formula is:

[0106]

[0107] like Figure 8 As shown, in one embodiment of the present invention, the vector with the highest similarity to vector v1 is retrieved, and the Manhattan distances of v2, v3, v4 and v1 are calculated respectively:

[0108] D v2v1 = |-2 – 3| + |0 – (-1)| + |3 – 6| = 9

[0109] D v3v1 = |-3– 3| + |0 – (-1)| + |1 – 6| = 12

[0110] D v4v1 = |-2– 3| + |6 – (-1)| + |1 – 6| = 17

[0111] From this we can see that the Manhattan distance between vectors v2 and v1 is the shortest, that is, the similarity between v2 and v1 is the highest.

[0112] In specific implementation, the basic principle of vector similarity search is to compare vectors in a multidimensional vector space and find one or more vectors that are closest to the query vector. The vector query SQL example code is as follows:

[0113] SELECT code, embedding, embedding <+> %s AS DISTANCE

[0114] FROM items

[0115] WHERE embedding <+> %s < %s

[0116] ORDER BY embedding <+> %s

[0117] LIMIT %s

[0118] In the above statement, <+> indicates calculating the Manhattan distance of the vector, and LIMIT %s indicates obtaining the first n records most similar to the query vector from the query results.

[0119] As another optional embodiment, in step S105, the database is a vector database.

[0120] X-ray images of adolescent scoliosis cases include preoperative views (AP, L, BL, BR), and may also include postoperative views (AP and L), three-month postoperative views (AP and L), six-month postoperative views (AP and L), and one-year postoperative views (AP and L). Original X-ray image files are organized and managed in the operating system file directory format. The directory structure can be as follows: Figure 9 As shown in the figure, by entering the case number, you can access each image data through the file storage path.

[0121] In today's big data era, traditional relational database technology faces challenges when dealing with large-scale structured data, such as the curse of dimensionality and high-dimensional similarity search. Vector databases effectively address these issues. Focusing on vector management, vector databases are specifically designed for storing and processing vector data, offering high scalability and efficient similarity search capabilities.

[0122] To add an AIS55 vector to the vector database: First, establish a correspondence between the AIS and the image file. This involves obtaining the directory structure and storage path of the original AP, L, BL, and BR images (original X-ray image files) from the previous step. This directory structure and storage path, along with attributes such as the case name, creation time, and modification time, are collectively referred to as the vector's metadata. Next, use an SQL insert statement to write the AIS55 vector and its corresponding metadata into the vector database. The SQL example code is as follows:

[0123] 1. Create a table (aisvectors) of type vector to store embeddings.

[0124] CREATE TABLE aisvectors (

[0125] id bigserial PRIMARY KEY,

[0126] code text,

[0127] imagepath text,

[0128] embedding vector(55) );

[0130] 2. Insert vector data into the aisvectors table.

[0131] INSERT INTO

[0132] aisvectors (code, imagepath, embedding)

[0133] VALUES

[0134] ('B7212', ' / localpath / dir1', '[0.23, 0.31, …, 0.01, 0.0003]'),

[0135] ('B7107', ' / localpath / dir2', '[0.07, 0.11, …, 0.34, 0.20]'),

[0136] ('B7456', ' / localpath / dir3', '[0.29, 0.39, …, 0.004, 0.008]'),

[0137] ('B6919', ' / localpath / dir4', '[0.33, 0.42, …, 0.002, 0.005]'),

[0138] If the case record in the AIS55 vector database is updated, the following SQL update statement is used. The example is as follows:

[0139] UPDATE

[0140] aisvectors (code, imagepath, embedding)

[0141] VALUES

[0142] ('B7212', ' / localpath / dir1', '[0.23, 0.31, …, 0.01, 0.0003]')

[0143] As another optional embodiment, in step S105, the database uses the IVFFLAT algorithm to create an index.

[0144] In practice, to accelerate vector data retrieval, we use the IVFFLAT algorithm to build an index. IVFFLAT is an approximate nearest neighbor search algorithm based on an inverted index. It divides the vector space into several partitions, each containing a number of vectors, and creates an inverted index to quickly find vectors similar to a given vector.

[0145] As another optional embodiment, in step S105, the vector search uses a WHERE query clause, and the threshold function of the WHERE query clause is:

[0146]

[0147] Among them, x=1, 2, 3, 4, 5, 6 correspond to Lenke type 1, type 2, type 3, type 4, type 5, and type 6 respectively.

[0148] In practice, the vector search algorithm described above requires a threshold value in the WHERE query clause to measure the Manhattan distance between two vectors. This retrieval returns only vectors that meet the query criteria. To improve the recall rate of the search, the present invention employs the aforementioned threshold function. This function overcomes the drawbacks of fixed thresholds by outputting a variable threshold value based on the input variable value.

[0149] In addition, the present invention can also develop visualization functions, allowing users to translate, rotate, and scale any two similar AIS case images, calculate the overlap of spinal structures, etc., to analyze the structural characteristics of AIS and plan surgical plans.

[0150] In summary, the image retrieval method for similar cases of adolescent scoliosis of the present invention has the following beneficial effects:

[0151] (1) This paper uses artificial intelligence technology to extract spinal structure data from adolescent idiopathic scoliosis (AIS) images and constructs an AIS55 case vector database. This vector database can be used to efficiently retrieve similar cases, providing an intelligent decision-making basis for accurate diagnosis and preoperative planning.

[0152] (2) The present invention can discover excellent surgical examples from AIS medical imaging cases, providing a convenient tool for the promotion of medical surgical experience.

[0153] like Figure 10 As shown, in one embodiment of the present invention, the query parameters may be as follows:

[0154] 1. Query vector:

[0155] q1= [0.002753874,0.08376014,0.052543916,0.15146308,0.36657998,0.555181, 0.84130853,1.1364688,1.4142865,1.5030645,1.4915406,1.331669,1.06556 57,0.7525067,0.4054391,0.06487643,-0.14535125,0,1.674,3.585704,4.285397,4.9850903,5.6847835,5.4760036,5.267224,4.745274,4.8757615,4.927 9566,4.5886893,4.3016167,3.8840573,3.33601,2.631378,1.9528435,1.3004065,0.12601973,0.34373495,0.21275893,0.91245204,1.6121451,1.5077552 ,1.820925,2.3950696,3.0214093,3.8043337,4.5350633,5.3179874,5.8660345,6.361887,6.7533493,7.118714,7.431884,7.7450533,8.032125,8.423588]

[0156] 2. Weight vector:

[0157] w1=[0.5,0.5, 0.5,0.5, 0.5,0.5, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.5,0.5,0.5,0.5, 0.5,0.5,1.0, 0.5,0.5, 0.5,0.5, 0.5,0.5, 1.0, 1.0, 1.0, 1.0, 1.0,1.0, 0.5,0.5, 0.5,0.5, 0.5,0.5, 0.5,0.5, 0.5,0.5, 0.5,0.5, 1.0, 1.0, 1.0,1.0, 1.0, 1.0, 0.5,0.5, 0.5,0.5, 0.5,0.5]

[0158] 3. Query threshold: 100

[0159] 4. Number of returned vectors: 4

[0160] R1=[-0.0015377061,-0.16789855,-0.22794075,-0.13026895,0.13832852,0.38695467,0.7192331, 0.9843457,1.1241019,2.2482295,1.0540901,0.6933651,0.03504756,-1.0325185,-0.29104722,-0.50563806,0.3356561,0,0.306,3.7662315,4.4550543,5.1438766,5.8326993,5.388633,5.1221933,5.211006,5.4478416,4.7077312,5.0037756,4.9149623,4.559709,3.8788075,3.0794883,2.2209601,1.3624319,0.592717,-0.14739348,-1.7738026,-1.08498,-0.39615756,0.136722,0.9360413, 2.0018003,2.8307242,3.7484612,4.4885716,5.1398687,5.791166,6.0576057,6.4424634,6.797716,7.1529694,7.715453,8.307542,8.988443]

[0161] R2=[0.5756511,0.28294164,0.13623881,0.03617143,0.054748163,0.26149622,0.6072004,0.9627429,1.2955827,1.4924101,1.5151664,1.3509853,1.2115327,0.8348367,0.5179045,0.18361463,0.018390767,0,2.41,5.8042502,6.5590916,6.681953,7.9105644,7.787703,7.41912,7.142682,6.589807,5.9140706,5.1461883,4.2247295,3.3647015,2.6275346,1.8596524,1.2453467,0.72318673,0.29317266,-0.25970253,-1.4365697,-0.68172824,0.07311316,0.4416966,1.0867177,1.8853152,2.306774,3.5132256,4.311823,4.9568443,5.386858,5.694011,4.9397335,6.09331,6.3083167,6.861192,7.4140673,8.120519]

[0162] R3=[0.64829594,0.5916941,0.450728,0.35900488,0.3518008,0.50350165,0.5719312,0.9518457,1.259861,1.6003858,1.842969,1.8759364,1.9131333,1.7573828,1.1728317,0.6468923,0.18295741,0,0.478,3.9703898,4.623998,5.277607,5.8663483,6.2097807,6.847584,7.043831,6.528682,5.91541,5.277607,4.639804,4.002001,3.3887284,2.9471724,2.4320238,2.0149987,1.6960973,1.303603,0.3887084,1.042317,1.6959255,2.349534,3.011868,3.8213873,4.287474,4.581845,4.72903,4.8271537,4.9007463,4.974339,4.974339,5.146055,5.415895,5.7838583,6.249945,6.6669703]

[0163] R4=[0.49172747,0.612714,0.58938897,0.51645577,0.58721375,0.6520544, 0.86601764,0.945901,0.9666554,0.98678386,0.96670496,1.138679,1.2983 042,1.2372142,1.0280125,0.72896177,0.31975934,0,3.556,7.61211,8.237241,8.862372,9.487503,10.112634,10.737764,11.362895,10.739458,10.08 8916,9.492584,8.869147,8.24571,7.6764855,7.025943,6.4025054,5.7790685,5.2369494,4.6948304,-1.6141273,-0.98899645,-0.3638656,0.26126522 ,0.5052188,1.1015499,1.589457,2.0773642,2.6194832,2.9989667,3.37845,3.866357,4.2729464,4.7879596,5.275867,5.763774,6.2516813,6.766694]

[0164] 5. Calculate the Manhattan distance between the nearest neighbor vector and the query vector:

[0165] Dist_R1 = 17.25

[0166] Dist_R2 = 26.04

[0167] Dist_R3 = 28.00

[0168] Dist_R4 = 87.40

[0169] According to the above calculation results, the image most similar to the query image q1 is the image corresponding to the R1 vector.

[0170] In addition, to verify the accuracy of the image retrieval of the present invention, the following comparative experiments were conducted:

[0171] Example 1: Using the above method of the present invention;

[0172] Comparative Example 1: The same as Example 1, except that the L-position vertebral feature does not use the Cobb angle, but instead uses the center line to fit and collect the x-axis coordinates of 6 equidistant coordinate points on the center line (such as Figure 11 shown);

[0173] Comparative Example 2: Same as Example 1, except that the vertebral contours at the BL position and the vertebral contours at the BR position are not rotated. Instead, the original features of the vertebral contours are sampled, and the x-axis coordinates of the centroid coordinates of each vertebra are directly used as the vertebral features at the BL position and the vertebral features at the BR position.

[0174] Comparative Example 3: The same as Example 1, except that only the AP and L spinal images are used, and the BL and BR spinal images are not used;

[0175] Comparative Example 4: The same as Example 1, except that only the AP-position spine image, the BL-position spine image, and the BR-position spine image are used, and the L-position spine image is not used;

[0176] Comparative Example 5: The same as Example 1, except that similarity retrieval without weight vector processing is used.

[0177] The above Example 1 and Comparative Examples 1-5 were compared on 89 test samples. The results are shown in Table 3:

[0178] Table 3 Comparative test results

[0179]

[0180] As can be seen from the above, the average search precision (mAP@5) of the first five search values ​​in Example 1 is improved by 6.4% compared with that in Comparative Example 1; the average search precision (mAP@5) of the first five search values ​​in Example 1 is improved by 12.6% compared with that in Comparative Example 2; the average search precision (mAP@5) of the first five search values ​​in Example 1 is improved by 20.3% compared with that in Comparative Example 3; the average search precision (mAP@5) of the first five search values ​​in Example 1 is improved by 5.5% compared with that in Comparative Example 4; and the average search precision (mAP@5) of the first five search values ​​in Example 1 is improved by 10.1% compared with that in Comparative Example 5. This shows that the image retrieval accuracy of the present invention is relatively high.

[0181] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A method for retrieving images of similar cases of adolescent scoliosis, characterized in that: include: Step S101: Acquire an AP spinal image, an L spinal image, a left flexion BL spinal image, and a right flexion BR spinal image of the patient; Step S102: using a pre-trained semantic segmentation model to process each spinal image, and extracting the AP-position vertebral body contour, L-position vertebral body contour, BL-position vertebral body contour, and BR-position vertebral body contour respectively; Step S103: extracting features of the AP, L, BL, and BR vertebral contours to obtain AP, L, BL, and BR vertebral features, respectively. Step S104: constructing a spine feature vector using the AP-position vertebral features, the L-position vertebral features, the BL-position vertebral features, and the BR-position vertebral features; Step S105: searching a database for a spinal eigenvector having a spatial distance closest to the spinal eigenvector, and using the case image corresponding to the closest spinal eigenvector as the desired adolescent scoliosis similar case image, wherein the database stores adolescent scoliosis case images and their corresponding spinal eigenvectors; Wherein, in step S102, the semantic segmentation model is a U-Net deep neural network model; The step S102 includes: The AP-position spinal image is processed using a pre-trained AP-position vertebral semantic segmentation model to extract the vertebral contours of the AP-position spinal image, including the cervical vertebra C7, thoracic vertebrae T1-T12, and lumbar vertebrae L1-L5, i.e., the AP-position vertebral contours, and each vertebra is identified; Processing the L-position spinal image using a pre-trained L-position vertebral semantic segmentation model to extract the vertebral contours of thoracic vertebrae T2-T7 in the L-position spinal image, i.e., the L-position vertebral contours, and simultaneously identifying each vertebra; Using a pre-trained BL-position vertebral semantic segmentation model and a pre-trained BR-position vertebral semantic segmentation model, the BL-position spinal image and the BR-position spinal image are processed respectively to extract the vertebral contours containing C7-L5 in the BL-position spinal image and the BR-position spinal image, i.e., the BL-position vertebral contour and the pre-trained BR-position vertebral contour, and simultaneously identify each vertebra; Wherein, the step S103 includes: Calculating the centroid coordinates of each vertebra according to the AP-position vertebral contour, and using the x-axis coordinates of the centroid coordinates of each vertebra as the AP-position vertebral feature; Calculate the centroid coordinates of each vertebra when the BL and BR vertebral contours are rotated to a point where the line connecting C7 and L5 is vertical, using the x-axis coordinates of the centroid coordinates of each vertebra in the BL vertebral contour as the BL vertebral feature, and using the x-axis coordinates of the centroid coordinates of each vertebra in the BR vertebral contour as the BR vertebral feature; Wherein, the step S103 further includes: The AP position vertebral features, the BL position vertebral features, and the BR position vertebral features are standardized, and the standardization includes: The mean value of vertebral body width was calculated; Dividing the AP position vertebral body feature, BL position vertebral body feature, and BR position vertebral body feature by the average value of the vertebral body width respectively to obtain the final AP position vertebral body feature, BL position vertebral body feature, and BR position vertebral body feature; Wherein, the step S104 includes: Performing weighted processing on the spinal column feature vector, the weighted processing comprising: Multiplying the preset weight vector by the spinal feature vector bitwise to obtain a final spinal feature vector, wherein the AP vertebral contour, BL vertebral contour, and BR vertebral contour are divided into T2-T5 as the upper thoracic segment, T6-T11 as the main thoracic segment, T12-L1 as the thoracolumbar segment, and L2-L5 as the lumbar segment. For the preset weights of the AP vertebral features, BL vertebral features, and BR vertebral features in the preset weight vector, the weight of the segment where the main curve is located is set to twice the weights of other segments; In step S105, the vector search uses a WHERE query clause, and the threshold function of the WHERE query clause is: ； Among them, x=1, 2, 3, 4, 5, 6 correspond to Lenke type 1, type 2, type 3, type 4, type 5, and type 6 respectively.

2. The method according to claim 1, characterized in that The AP position vertebral body contour, the BL position vertebral body contour, and the BR position vertebral body contour all use the centroid coordinates of the lumbar vertebra L5 as the origin.

3. The method according to claim 1, characterized in that The step S103 includes: The Cobb angle is calculated according to the L-position vertebral body contour to obtain the L-position vertebral body feature.

4. The method according to claim 1, wherein The preset weight of the L-position vertebral feature in the preset weight vector is set to be the same as the weight of the segment where the main curve is located.

5. The method according to claim 1, wherein In step S105, the spatial distance is calculated using the Manhattan distance calculation method; And / or, the database is a vector database; And / or, the database is indexed using the IVFFLAT algorithm.

Images