A method and device for identifying scoliosis based on symmetry perception of natural images

Abstract

The present invention discloses a scoliosis recognition method and device based on symmetry perception of natural images. The original RGB image of the human back is first horizontally flipped, and the symmetry feature is obtained through a feature extraction network; then, through a symmetry feature matching module, the symmetry feature is firstly fused to obtain a fusion feature, and then the symmetry perception feature is obtained by using an attention mechanism, and finally, the symmetry matching feature is obtained through a splicing convolution operation; finally, the multi-category classification task is converted into an ordinal regression task, and the scoliosis severity level and fine-grained angle range are simultaneously predicted through an ordinal regression module. The present invention adopts an end-to-end deep learning framework to learn scoliosis grading and angle estimation, uses a visual attention network based on an attention mechanism to learn the symmetry features of scoliosis images, and converts the ordinal regression problem into multiple binary classification sub-problems, realizing the construction of a scoliosis severity grading and angle joint estimation system.

Description

Technical Field

[0001] The invention relates to a scoliosis recognition method and device based on natural image symmetry perception, belonging to computer vision technology. Background Art

[0002] Scoliosis is a very serious spinal disease in humans, especially in adolescents. Early screening of adolescent idiopathic scoliosis provides an opportunity for timely treatment and helps reduce the damage caused by scoliosis. However, traditional scoliosis screening methods usually rely on radiographic imaging such as X-ray images and specialized measurement tools, and can only be performed by professional doctors or reputable medical institutions. Due to the low positive value, radiographic examinations are usually unnecessary. In addition, due to the complex etiology and diverse types of scoliosis, the decision to perform surgery cannot be simply based on the patient's age or the degree of spinal curvature. Factors such as the rate of progression of the deformity, the patient's skeletal maturity, and the degree of influence of the deformity on posture should all be taken into account. Therefore, the treatment process of scoliosis usually requires long-term monitoring and multiple measurements. In this case, the traditional scoliosis screening method is highly professional, costly, and time-consuming, which is not conducive to large-scale dissemination and promotion.

[0003] Scoliosis detection methods based on image deep learning can be roughly divided into three categories. The first category estimates the severity of scoliosis directly from X-ray images. For example, Fraiwan et al. used a deep transfer learning method to diagnose spondylolisthesis and scoliosis through X-ray images without any measurement. The second category of methods first detects or segments the vertebrae, and then calculates or uses a regression algorithm to obtain the Cobb angle based on the position of the vertebrae. For example, Lin et al. designed a framework called Seg4Reg, which includes two deep neural networks focusing on segmentation and regression respectively. Based on the results generated by the segmentation model, the regression network predicts the Cobb angle directly from the segmentation mask. The third category of methods attempts to find accurate landmarks of the vertical body as an alternative to the segmentation algorithm, such as the S2VR algorithm proposed by Sun et al. - by considering the explicit dependencies between multiple outputs, the accuracy of the Cobb angle is improved. However, these methods still require the use of X-ray images, which makes it difficult to avoid the risk of unnecessary radiation exposure to patients. Summary of the invention

[0004] Purpose of the invention: In order to reduce the radiation risk caused by the use of X-ray examinations so that patients, especially adolescent patients, can detect scoliosis in time and take treatment measures, the present invention provides a scoliosis recognition method and device based on natural image symmetry perception. The scoliosis classification and angle estimation can be achieved only through the RGB image of the patient's back, while ensuring high-accuracy recognition, it also has strong robustness.

[0005] Technical solution: To achieve the above purpose, the technical solution adopted by the present invention is:

[0006] A scoliosis recognition method based on natural image symmetry perception comprises the following steps:

[0007] S01: Collect RGB images of the human back;

[0008] S02: Preprocess the RGB image to obtain the original RGB image, horizontally flip the original RGB image, and obtain symmetry features through a feature extraction network;

[0009] S03: Through the symmetry feature matching module, the symmetry features are first fused to obtain fusion features, and then the attention mechanism is used to obtain symmetry perception features. Finally, the symmetry matching features are obtained through the splicing convolution operation.

[0010] S04: Through the ordinal regression module, the scoliosis severity level and fine-grained angle range are predicted simultaneously; since the scoliosis severity level and the fine-grained angle range have a natural sequential relationship, the recognition of the scoliosis severity level based on the symmetric matching features and the classification of the fine-grained angle range are regarded as ordinal regression problems, that is, the multi-category classification task is converted into an ordinal regression task.

[0011] Specifically, in step S02, since the back of a healthy human body is symmetrical, the original RGB image is horizontally flipped, and the original RGB image and the horizontally flipped original RGB image are input into a feature extraction network with a visual attention network as the backbone network for feature extraction, and feature F and feature F are obtained respectively. f , F and F f It is a symmetric feature.

[0012] Specifically, in step S03, the process of acquiring the symmetric matching feature is as follows:

[0013] First, for F and F f Perform feature fusion to obtain fusion feature F c , the feature fusion process is expressed as:

[0014]

[0015] Among them: cat represents the feature connection operation, represents a convolution with a 1×1 kernel, denotes convolution with a 3×3 kernel, BN denotes batch normalization, and σ denotes the activation function of the rectified linear unit;

[0016] Then, the features of the input image are enhanced through the attention mechanism and the dependency of the original RGB image and the horizontally flipped RGB image is modeled. The attention mechanism is expressed as:

[0017]

[0018] Where: Q, K and V represent the query, key and value in the attention mechanism respectively, d k represents the channel dimension; let F be Q, and F c As K and V, calculate the attention value Attention(Q, K, V), and then get the perceptual feature F′; f As Q, F c As K and V, calculate the attention value Attention(Q, K, V), and then get the perceptual feature F f ′; F′ and F f ′ is the symmetry perception feature;

[0019] Finally, for F′ and F f ′ performs concatenated convolution operation to obtain symmetric matching features, expressed as:

[0020]

[0021] Among them, F ir Symmetric matching feature.

[0022] Specifically, in step S04, assuming that there are K severity levels of scoliosis or fine-grained scoliosis angle ranges, identifying the severity levels of scoliosis or fine-grained scoliosis angle ranges is to classify the symmetric matching features into K categories; the K-category classification task is converted into an ordinal regression task with K ranks, that is, converted into K-1 binary classification sub-problems.

[0023] Specifically, in step S04, the K-category classification task is converted into an ordinal regression task with K ranks, that is, converted into K-1 binary classification sub-problems; the ordinal regression task with K ranks includes K-1 binary classifiers, and the kth binary classifier is used to predict whether the rank of the input sample is greater than k, that is, to predict whether the input sample belongs to category k, k∈{1, 2, ..., K-1}; a matrix Y of size (K-1)×2 is used to represent the true probability value vector of the input sample, Y k is the true probability value of the kth binary classifier;

[0024]

[0025] Where: r represents the true label of the input sample scoliosis severity level or fine-grained angle range;

[0026] The cross entropy loss based on the Softmax function is used to calculate the estimation loss L of the scoliosis severity level or fine-grained angle range respectively. severity :

[0027]

[0028] in: Indicates the predicted probability value that the input sample belongs to category k;

[0029] Input sample scoliosis severity level or fine-grained angle range prediction label for:

[0030]

[0031] The estimated loss in scoliosis severity is denoted as L general , the estimated loss of the fine-grained angle range is recorded as L fine , calculate the total loss L as:

[0032] L=λ general L general +λ fine L fine

[0033] Where: general and λ fine L general and L fine The weight of

[0034] The binary classifier is trained in a supervised manner using the total loss L.

[0035] Specifically, in step S01, an RGB image of the back of a human body is first collected through a mobile client, and then uploaded to a backend server through a FastAPI component; steps S02 to S04 are executed in the backend server; a FastAPI application instance library is created in the backend server, and an endpoint named / upload_file is defined in the FastAPI application instance library to receive the RGB image uploaded by the mobile client; the RGB image is preprocessed using a function in an image processing library PIL, the RGB image is first adjusted to a set size, then converted into a tensor form, and finally a batch dimension is added to obtain the original RGB image; the backend server analyzes the original RGB image to obtain a predicted value of the scoliosis severity level and a predicted value of the scoliosis angle, and then the predicted result is fed back to the mobile client through the FastAPI component.

[0036] Specifically, the mobile client collects an RGB image of the human back by taking a photo or selecting a picture, and displays the predicted value of the severity level of scoliosis and the predicted value of the scoliosis angle on the mobile client, and displays popular science knowledge about scoliosis.

[0037] A scoliosis recognition device based on natural image symmetry perception, comprising a scoliosis image acquisition unit, an image preprocessing unit, a symmetry feature matching unit, a scoliosis severity level and scoliosis angle estimation unit, and a result display unit;

[0038] The scoliosis image acquisition unit is used to collect RGB images of the back of the human body;

[0039] The image preprocessing unit first adjusts the RGB image to a set size, then converts it into a tensor form, and finally adds a batch dimension to obtain the original RGB image;

[0040] The symmetry feature matching unit extracts the symmetry features of the original RGB image, first performs feature fusion on the symmetry features to obtain fusion features, then uses an attention mechanism to obtain symmetry perception features, and finally obtains symmetry matching features through a concatenated convolution operation;

[0041] The scoliosis severity level and scoliosis angle estimation unit uses an ordinal regression model to perform ordinal regression judgment on symmetric matching features to predict the scoliosis severity level and fine-grained angle range;

[0042] The result display unit is used to display the severity level of scoliosis and the scoliosis angle.

[0043] Specifically, the scoliosis image acquisition unit and result display unit are set in the mobile client and stored as a WeChat applet; the image preprocessing unit, symmetry feature matching unit, scoliosis severity level and scoliosis angle estimation unit are set in the back-end server.

[0044] Specifically, the back-end server uses the FastAPI component to build APIs, and uses the APIs to process the communication between the back-end server and the mobile client. When building the APIs, the key libraries imported into the back-end server include libraries for running ASGI applications, libraries for creating Web applications, PyTorch libraries / torch libraries, image processing libraries PIL and torchvision; at the same time, a FastAPI application instance library is created in the back-end server, and an endpoint named / upload_file is defined in the FastAPI application instance library to receive the RGB image uploaded by the mobile client and save it to the specified path; the RGB image is preprocessed using the function in the image processing library PIL, first adjusting the RGB image to a set size, then converting it into a tensor form, and finally adding a batch dimension to obtain the original RGB image; the back-end server analyzes the original RGB image to obtain the predicted value of the scoliosis severity level and the predicted value of the scoliosis angle, and then feeds the predicted result back to the mobile client through the FastAPI component, and displays the predicted result through the WeChat applet interface.

[0045] Beneficial effects: The scoliosis recognition method and device based on symmetry perception of natural images provided by the present invention have the following advantages over the prior art: 1. The attention mechanism is introduced to enhance the features of the input image, and the dependency relationship between the input images is modeled to obtain the symmetry perception features of the image; 2. In the ordinal regression module, multiple binary classifiers are used to process the sequential relationship between different categories; 3. Patients can directly upload RGB images of the back through the WeChat applet for scoliosis detection, which is convenient for large-scale scoliosis screening to help patients detect scoliosis as early as possible and intervene in time. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 Schematic diagram of the implementation process of the method of the present invention;

[0047] Figure 2 It is a structural diagram of the symmetric feature matching module;

[0048] Figure 3 It is a structural diagram of the ordinal regression module;

[0049] Figure 4 Schematic diagram of the structure of a neural network based on symmetry perception of natural images. DETAILED DESCRIPTION

[0050] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0051] The present invention provides a scoliosis recognition method based on symmetry perception of natural images. Through the symmetry feature matching module and the ordinal regression module, convenient, fast and highly accurate scoliosis screening is achieved. It is recommended to be displayed in the form of a WeChat applet. In the symmetry feature matching module, the symmetry relationship of the human back is captured by processing the original RGB image and its horizontally flipped RGB image, thereby extracting the symmetry features of the spine image. In the ordinal regression module, the multi-category classification problem is converted into a series of binary classification sub-problems, thereby improving the accuracy of scoliosis severity prediction. Figure 1 The figure shows the implementation flow chart of the method of this case, and each specific step is explained below.

[0052] S01: Upload a photo of your back via WeChat Mini Program

[0053] In the user interface of the WeChat applet on the mobile client, RGB images of the human back are collected by taking photos or selecting pictures, and uploaded to the backend server through the FastAPI component.

[0054] S02: Obtaining symmetry features based on RGB images

[0055] The RGB image is preprocessed to obtain the original RGB image. Figure 2 As shown in the figure, since the back of a healthy human body is symmetrical, the original RGB image is horizontally flipped, and the original RGB image and the horizontally flipped original RGB image are input into the feature extraction network with the visual attention network as the backbone network for feature extraction, and feature F and feature F are obtained respectively. f , F and F f It is a symmetric feature.

[0056] S03: Obtain symmetric matching features

[0057] After obtaining the original RGB image and its horizontally flipped original RGB image, in order to enhance this symmetric relationship, such as Figure 2 As shown in Figure 1, symmetric matching features are obtained through symmetric feature matching modules, attention mechanisms, and concatenated convolution operations. The process of obtaining symmetric matching features is as follows:

[0058] First, use the symmetric feature matching module to match F and F f Perform feature fusion to obtain fusion feature F c , the feature fusion process is expressed as:

[0059]

[0060] Among them: cat represents the feature connection operation, represents a convolution with a 1×1 kernel, denotes a convolution with a 3×3 kernel, BN denotes batch normalization, and σ denotes the activation function of a rectified linear unit.

[0061] Then, the attention mechanism is used to enhance the features of the input image and model the dependencies between the original RGB image and the horizontally flipped RGB image. The core idea of ​​the attention mechanism is to improve the effect of feature extraction by focusing on the key parts of the image. The attention mechanism is expressed as:

[0062]

[0063] Where: Q, K and V represent the query, key and value in the attention mechanism respectively, d k represents the channel dimension; let F be Q, and F c As K and V, calculate the attention value Attention(Q, K, V), and then get the perceptual feature F′; f As Q, F c As K and V, calculate the attention value Attention(Q, K, V), and then get the perceptual feature F f ′; F′ and F f ′ is a symmetric perception feature.

[0064] Finally, for F′ and F f ′ performs concatenated convolution operation to obtain symmetric matching features, expressed as:

[0065]

[0066] Among them, F ir Symmetric matching feature.

[0067] S04: Obtain prediction results through the ordinal regression module

[0068] Since the severity level of scoliosis and the fine-grained angle range have a natural order relationship, the identification of the severity level of scoliosis based on the symmetric matching features and the classification of the fine-grained angle range are regarded as ordinal regression problems, that is, the multi-class classification task is converted into an ordinal regression task. Figure 3 The ordinal regression module shown simultaneously predicts scoliosis severity grade and fine-grained angle range.

[0069] In the ordinal regression model, assuming that there are K scoliosis severity levels or fine-grained scoliosis angle ranges, identifying the scoliosis severity levels or fine-grained scoliosis angle ranges is to classify the symmetrical matching features into K categories; the K-category classification task is converted into an ordinal regression task with K ranks, that is, converted into K-1 binary classification sub-problems.

[0070] The ordinal regression task with K ranks includes K-1 binary classifiers, and the kth binary classifier is used to predict whether the rank of the input sample is greater than k, that is, to predict whether the input sample belongs to category k, k∈{1, 2, ..., K-1}; a matrix Y of size (K-1)×2 is used to represent the true probability value vector of the input sample, Y k is the true probability value of the kth binary classifier, expressed as:

[0071]

[0072] Where r represents the true label of the input sample scoliosis severity level or fine-grained angle range.

[0073] In this example, after processing by the ordinal regression module, 4 scoliosis severity levels and 10 fine-grained angle ranges can be generated; the 4 scoliosis severity levels are: normal (average Cobb angle is 6.48°), mild (average Cobb angle is 17.91°), moderate (average Cobb angle is 27.13°), severe (average Cobb angle is 75.71°); the 10 angle categories are 0°~4°, 5°~9°, 10°~14°, 15°~19°, 20°~24, 25°~29°, 30°~34°, 35°~39°, 40°~44°, 45°~49°.

[0074] The cross entropy loss based on the Softmax function is used to calculate the estimation loss L of the scoliosis severity level or fine-grained angle range respectively. severity :

[0075]

[0076] in: Indicates the predicted probability value that the input sample belongs to category k;

[0077] Input sample scoliosis severity level or fine-grained angle range prediction label for:

[0078]

[0079] The estimated loss in scoliosis severity is denoted as L general , the estimated loss of the fine-grained angle range is recorded as L fine , calculate the total loss L as:

[0080] L=λ general L general +λ fine L fine

[0081] Where: general and λfine L general and L fine The weight of

[0082] The binary classifier is trained in a supervised manner using the total loss L.

[0083] S05: Display the prediction results on the display interface of the WeChat applet

[0084] The backend server sends the predicted value of the scoliosis severity level and the predicted value of the scoliosis angle to the WeChat applet through the FastAPI component, and displays it through the display interface of the WeChat applet, while also displaying popular science knowledge about scoliosis.

[0085] In this example, in step S01, an RGB image of the back of the human body is first collected through the mobile client, and then uploaded to the back-end server through the FastAPI component; steps S02 to S04 are executed in the back-end server; a FastAPI application instance library is created in the back-end server, and an endpoint named / upload_file is defined in the FastAPI application instance library to receive the RGB image uploaded by the mobile client; the RGB image is preprocessed using the function in the image processing library PIL, first adjusting the RGB image to 224×224 pixels, then converting it into a tensor form, and finally adding a batch dimension to obtain the original RGB image; the back-end server analyzes the original RGB image to obtain the predicted value of the scoliosis severity level and the predicted value of the scoliosis angle, and then feeds the predicted result back to the mobile client through the FastAPI component.

[0086] A device for implementing the above-mentioned scoliosis identification method based on natural image symmetry perception, mainly includes a scoliosis image acquisition unit and a result display unit arranged on a mobile client, and an image preprocessing unit, a symmetry feature matching unit, a scoliosis severity level and a scoliosis angle estimation unit arranged on a back-end server; wherein the scoliosis image acquisition unit and the result display unit of the mobile client are stored as a WeChat applet. The back-end server uses FastAPI components to build APIs, and uses APIs to process the communication between the back-end server and the mobile client. When building APIs, the key libraries imported into the back-end server include libraries for running ASGI applications, libraries for creating Web applications, PyTorch libraries / torch libraries, image processing libraries PIL and torchvision; at the same time, a FastAPI application instance library is created in the back-end server, and an endpoint named / upload_file is defined in the FastAPI application instance library to receive RGB images uploaded by the mobile client and save them to the specified path; the RGB image is preprocessed using the function in the image processing library PIL, first adjusting the RGB image to 224×224 pixels, then converting it into a tensor form, and finally adding a batch dimension to obtain the original RGB image; the back-end server analyzes the original RGB image to obtain the predicted value of the scoliosis severity level and the predicted value of the scoliosis angle, and then feeds the predicted result back to the mobile client through the FastAPI component, and displays the predicted result through the WeChat applet interface.

[0087] The scoliosis image acquisition unit is used to collect RGB images of the back of the human body.

[0088] The image preprocessing unit first adjusts the RGB image to a set size, then converts it into a tensor form, and finally adds a batch dimension to obtain the original RGB image.

[0089] The symmetry feature matching unit extracts the symmetry features of the original RGB image, first performs feature fusion on the symmetry features to obtain fusion features, then uses an attention mechanism to obtain symmetry perception features, and finally obtains symmetry matching features through a splicing convolution operation.

[0090] The scoliosis severity level and scoliosis angle estimation unit uses an ordinal regression model to perform ordinal regression judgment on symmetric matching features to predict the scoliosis severity level and fine-grained angle range.

[0091] The result display unit is used to display the severity level of scoliosis and the scoliosis angle.

[0092] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the above embodiments do not limit the present invention in any form, and any technical solution obtained by equivalent replacement or equivalent transformation falls within the protection scope of the present invention.

Claims

1. A scoliosis recognition method based on natural image symmetry perception, characterized in that: The steps include: S01: Collect RGB images of the human back; S02: Preprocess the RGB image to obtain the original RGB image, flip the original RGB image horizontally, input the original RGB image and the horizontally flipped original RGB image into the feature extraction network with the visual attention network as the backbone network for feature extraction, and obtain feature F and feature F respectively. f , F and F f It is a symmetric feature; S03: Through the symmetry feature matching module, the symmetry features are first fused to obtain fusion features, and then the attention mechanism is used to obtain symmetry perception features. Finally, the symmetry matching features are obtained through the splicing convolution operation. The process of obtaining symmetric matching features is as follows: First, for F and F f Perform feature fusion to obtain fusion feature F c , the feature fusion process is expressed as: Among them: cat represents the feature connection operation, represents a convolution with a 1×1 kernel, denotes convolution with a 3×3 kernel, BN denotes batch normalization, and σ denotes the activation function of the rectified linear unit; Then, the features of the input image are enhanced through the attention mechanism and the dependency of the original RGB image and the horizontally flipped RGB image is modeled. The attention mechanism is expressed as: Where: Q, K and V represent the query, key and value in the attention mechanism respectively, d k represents the channel dimension; let F be Q, and F c As K and V, calculate the attention value Attention(Q,K,V), and then get the perceptual feature F'; f As Q, F c As K and V, calculate the attention value Attention(Q,K,V), and then get the perceptual feature F f ';F' and F f ' is a symmetric perception feature; Finally, for F' and F f 'Perform concatenated convolution operation to obtain symmetric matching features, expressed as: Among them, F ir It is a symmetric matching feature; S04: Use the ordinal regression module to simultaneously predict the severity level of scoliosis and the fine-grained angle range; assuming that there are K severity levels of scoliosis or fine-grained angle ranges of scoliosis, identifying the severity level of scoliosis or the fine-grained angle range of scoliosis is to classify the symmetric matching features into K categories; convert the K-category classification task into an ordinal regression task with K ranks, that is, convert it into K-1 binary classification sub-problems; The ordinal regression task with K ranks includes K-1 binary classifiers. The kth binary classifier is used to predict whether the rank of the input sample is greater than k, that is, to predict whether the input sample belongs to category k, k∈{1,2,…,K-1}; a matrix Y of size (K-1)×2 is used to represent the true probability value vector of the input sample, Y k is the true probability value of the kth binary classifier; Where: r represents the true label of the input sample scoliosis severity level or fine-grained angle range; The cross entropy loss based on the Softmax function is used to calculate the estimation loss L of the scoliosis severity level or fine-grained angle range respectively. severity : in: Indicates the predicted probability value that the input sample belongs to category k; Input sample scoliosis severity level or fine-grained angle range prediction label for: The estimated loss in scoliosis severity is denoted as L general , the estimated loss of the fine-grained angle range is recorded as L fine , calculate the total loss L as: L=λ general L general +λ fine L fine Where: general and λ fine L general and L fine The weight of The binary classifier is trained in a supervised manner using the total loss L.

2. The method for identifying scoliosis based on natural image symmetry perception according to claim 1, characterized in that: In the step S01, an RGB image of the back of a human body is first collected through a mobile client, and then uploaded to a backend server through a FastAPI component; steps S02 to S04 are executed in the backend server; a FastAPI application instance library is created in the backend server, and an endpoint named / upload_file is defined in the FastAPI application instance library to receive the RGB image uploaded by the mobile client; the RGB image is preprocessed using a function in an image processing library PIL, the RGB image is first adjusted to a set size, then converted into a tensor form, and finally a batch dimension is added to obtain the original RGB image; the backend server analyzes the original RGB image to obtain a predicted value of the scoliosis severity level and a predicted value of the scoliosis angle, and then the predicted result is fed back to the mobile client through the FastAPI component.

3. The method for identifying scoliosis based on natural image symmetry perception according to claim 2, characterized in that: The mobile client collects an RGB image of the human back by taking a photo or selecting a picture, and displays a predicted value of the severity level of scoliosis and a predicted value of the scoliosis angle on the mobile client, and displays popular science knowledge about scoliosis.

4. A scoliosis recognition device based on natural image symmetry perception, characterized in that: Used to implement the scoliosis identification method based on natural image symmetry perception as described in claim 1, comprising a scoliosis image acquisition unit, an image preprocessing unit, a symmetry feature matching unit, a scoliosis severity level and scoliosis angle estimation unit, and a result display unit; The scoliosis image acquisition unit is used to collect RGB images of the back of the human body; The image preprocessing unit first adjusts the RGB image to a set size, then converts it into a tensor form, and finally adds a batch dimension to obtain the original RGB image; The symmetry feature matching unit horizontally flips the original RGB image, inputs the original RGB image and the horizontally flipped original RGB image into a feature extraction network with a visual attention network as a backbone network to extract features, obtains symmetry features, first performs feature fusion on the symmetry features to obtain fusion features, then uses an attention mechanism to obtain symmetry perception features, and finally obtains symmetry matching features through a splicing convolution operation; The scoliosis severity level and scoliosis angle estimation unit uses an ordinal regression model to perform ordinal regression judgment on symmetric matching features to predict the scoliosis severity level and fine-grained angle range; The result display unit is used to display the severity level of scoliosis and the scoliosis angle.

5. The scoliosis identification device based on natural image symmetry perception according to claim 4, characterized in that: The scoliosis image acquisition unit and result display unit are arranged on a mobile client and stored as a WeChat applet; the image preprocessing unit, symmetry feature matching unit, scoliosis severity level and scoliosis angle estimation unit are arranged in a back-end server.

6. The scoliosis identification device based on natural image symmetry perception according to claim 5, characterized in that: The backend server uses FastAPI components to build APIs, and uses APIs to process the communication between the backend server and the mobile client. When building APIs, the key libraries imported into the backend server include libraries for running ASGI applications, libraries for creating Web applications, PyTorch libraries / torch libraries, image processing libraries PIL and torchvision; at the same time, a FastAPI application instance library is created in the backend server, and an endpoint named / upload_file is defined in the FastAPI application instance library to receive RGB images uploaded by the mobile client and save them to a specified path; The RGB image is preprocessed using the functions in the image processing library PIL. The RGB image is first adjusted to a set size, then converted into a tensor form, and finally a batch dimension is added to obtain the original RGB image. The backend server analyzes the original RGB image to obtain the predicted values ​​of the scoliosis severity level and the scoliosis angle, and then the predicted results are fed back to the mobile client through the FastAPI component, and the predicted results are displayed through the WeChat applet interface.

Images