Scoliosis grading detection method and device based on multi-input neural network

Abstract

The invention relates to a scoliosis grading detection method and device based on a multi-input neural network, and belongs to the technical field of spine malformation intelligent detection. A piezoresistive sensor array and a TOF camera are used for collecting plantar pressure data and back three-dimensional point cloud data, the plantar pressure data are converted into a plantar pressure distribution image, the three-dimensional point cloud data are converted into a back depth image, and the plantar pressure distribution image and the back depth image serve as two inputs of a multi-input network. According to the method, plantar pressure image input is added on the basis of a conventional scoliosis detection method based on deep learning, and two kinds of data are utilized to train a comprehensive classification model to classify the severity of scoliosis. The method is fully automatic, the labor cost is greatly saved, and the classification result integrating the two factors is more reliable.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent detection of spinal deformities, and relates to a method and device for grading scoliosis detection based on a multi-input neural network. Background Art

[0002] Scoliosis, this complex three-dimensional spinal deformity problem, poses a severe challenge to human health. It not only affects the physical beauty of patients, but may also trigger a series of serious health problems, such as back pain, difficulty breathing, limited cardiopulmonary function, and even lead to disability, seriously affecting the quality of life of patients. Therefore, early and accurate diagnosis of scoliosis is crucial for taking timely treatment measures and controlling the development of the disease.

[0003] Currently, the general survey methods for spinal deformities mainly rely on two traditional means: manual detection and X-ray film detection. However, both of these methods have their limitations.

[0004] Manual detection mainly relies on the experience and subjective judgment of doctors. During the screening process, different doctors may draw different conclusions about the spinal condition of the same patient due to differences in experience, knowledge level, and observation angle. This subjectivity not only reduces the accuracy of the screening, but may also lead to missed diagnosis or misdiagnosis, delaying the treatment opportunity for patients. In addition, the efficiency of manual detection is relatively low and it is difficult to meet the needs of large-scale screening.

[0005] In contrast, although X-ray film detection has high accuracy, its cost is high and it causes certain radiation damage to the human body. For patients who need to be screened repeatedly, frequent X-ray exposure may increase health risks. Therefore, X-ray film detection is not suitable as a routine general survey method.

[0006] In view of the limitations of manual detection and X-ray film detection, there is an urgent need for a non-invasive, fast and efficient scoliosis detection method. Summary of the Invention

[0007] In view of this, the purpose of the present invention is to provide a method and device for grading scoliosis detection based on a multi-input neural network, which respectively use a piezoresistive sensor array and a TOF camera to collect plantar pressure data and back three-dimensional point cloud data, and then convert both data into image forms, input them respectively into two branches in the network for feature extraction, and classify the severity after feature fusion.

[0008] To achieve the above purpose, the present invention provides the following technical solutions:

[0009] On the one hand, a method for grading scoliosis detection based on a multi-input neural network is proposed, and the method includes the following steps:

[0010] S1. Collect the plantar pressure data of the human body through a piezoresistive sensor array, and collect the three-dimensional point cloud data of the human back through a TOF camera;

[0011] S2. Convert the plantar pressure data of the human body and the three-dimensional point cloud data of the back into corresponding plantar pressure distribution images and back depth images respectively;

[0012] S3. Input the obtained plantar pressure distribution image and back depth image into a multi-input neural network. Among them, the plantar pressure distribution image and the back depth image respectively pass through multiple 2D convolutional layers to obtain corresponding features;

[0013] S4. Stitch the obtained plantar pressure distribution feature map and back depth feature map to obtain a combined feature;

[0014] S5. Input the combined feature into the Softmax function for classification to obtain the scoliosis grading detection result.

[0015] Furthermore, in step S1, place the piezoresistive sensor array on the sole of the person to be detected, and the TOF camera faces the back of the person to be detected. Both the piezoresistive sensor array and the TOF camera are connected to the computer, and direct the person to be detected to move on the piezoresistive sensor array according to a preset motion paradigm.

[0016] Furthermore, in step S2, the process of converting the plantar pressure data of the human body into a plantar pressure distribution image is as follows:

[0017] Suppose the collected plantar pressure data of the human body is a one-dimensional array of length N, then the collected plantar pressure data of the human body is expressed as: A = [a1, a2, …, a n , …, a N , where a n is the value of the nth sensor;

[0018] Arrange the obtained one-dimensional array according to the positions of each sensor in the sensor array to obtain the plantar pressure distribution image. The plantar pressure distribution image is expressed as B, where each element b ij in B is expressed as:

[0019]

[0020] In the formula, i ∈ [1, N1], j ∈ [1, N2], i and j respectively represent the elements in the ith row and jth column of the plantar pressure distribution image B, and N1 and N2 respectively represent the number of sensors arranged horizontally and vertically in the piezoresistive sensor array, and N1 × N2 = N.

[0021] Furthermore, in step S2, the process of converting the three-dimensional point cloud data of the human back into a back depth image is as follows:

[0022] The points in the three-dimensional point cloud data of the human back include their abscissa, ordinate, and depth coordinate in space. The points in the point cloud data should be set in the xy plane, and z is used as the depth value;

[0023] Let the point set in the three-dimensional point cloud data be P, then the coordinates of the t-th point are p t =(x t ,y t ,z t ). The points in the point set are normalized to the size of the converted back depth image. The calculation formula is as follows:

[0024]

[0025]

[0026] In the formula, are the abscissa and ordinate of the t-th point in the back depth image, represents the depth value of the t-th point in the back depth image, x min ,x max are the minimum and maximum abscissas of all points in the point set P, y min ,y max are the minimum and maximum ordinates of all points in the point set P, z min ,z max are the minimum and maximum depth coordinates of all points in the point set P, and L×H represents the pixel value of the converted back depth image, where L represents the width and H represents the height.

[0027] Furthermore, in step S3, features are extracted from the plantar pressure distribution image and the human back depth image respectively through multiple 2D convolutional layers. Among them, the convolutional kernel size of the plantar pressure part is 3*3, and the convolutional kernel size of the back depth image part is 5*5.

[0028] The calculation formula for the size of the feature map after extraction is as follows:

[0029]

[0030] Among them, w' and h' are the sizes of the feature map after extraction, w and h are the sizes of the original image, p represents the number of zero-padding layers, k represents the convolutional kernel size, and s represents the stride.

[0031] Further, in step S4, after extracting features through the convolutional layer, the two types of features are concatenated. Let the plantar pressure distribution feature map be w1×h1×m, and the depth image feature map be w2×h2×n. Since the feature size of the plantar pressure distribution image is smaller than that of the depth image feature, the plantar pressure feature is first upsampled, adjusted to the same size, and then concatenated. Let the combined feature after concatenation be F, then F is expressed as:

[0032] F = w1×h1×(m + n)

[0033] In the formula, m and n respectively represent the sizes of the two types of feature maps;

[0034] Further, in step S5, the combined feature F is input into the Softmax function for a four-class classification, and the results are divided into normal, mild, moderate, and severe. Among them, in the fully connected layer FC, let the output size be D, then the calculation process is:

[0035] y = W·x + b

[0036] Where x is the input feature vector, W is the weight matrix, b is the bias vector, and y is the output of the fully connected layer;

[0037] For the classification task, it is further output through the linear layer Linear, and the calculation of the linear layer is:

[0038] y' = W′·y + b'

[0039] Where y' is the output of the linear layer, W′ is the weight matrix, and b′ is the bias vector; the output classification results respectively represent the situations of normal, mild, moderate, and severe scoliosis;

[0040] Then the Softmax layer converts the output of the linear layer into a probability distribution, and the formula is:

[0041]

[0042] Let the prediction result be res, then:

[0043] res = argmax(p)

[0044] Where the above formula means searching for the maximum probability in the output probability set, and the classification where the maximum value in the probability set is located is the disease level.

[0045] On the other hand, a scoliosis grading detection device based on a multi-input neural network is also proposed. The device includes a computer, a piezoresistive sensor array, and a TOF camera. Among them,

[0046] The computer is respectively connected to a piezoresistive sensor array and a TOF camera. The piezoresistive sensor array is used to obtain the plantar pressure data of the human body, and the TOF camera is used to obtain the three-dimensional point cloud data of the human back, and the obtained data is transmitted to the computer;

[0047] The computer is used to execute the aforementioned scoliosis grading detection method based on a multi-input neural network and output the detection result.

[0048] The beneficial effects of the present invention are as follows:

[0049] The present invention significantly improves the accuracy and efficiency of diagnosis through multi-modal data fusion, bringing various beneficial technical effects. First, through the dual-modal data acquisition of the piezoresistive sensor and the TOF camera, this technology can simultaneously capture the differences in human biomechanical loads and spinal deformations, overcoming the limitations of a single data source such as X-ray films. The feature complementarity enables the classification accuracy of the model after fusing the plantar pressure image and the back depth image to be improved compared to single-modal, providing more comprehensive and accurate information for clinical diagnosis.

[0050] The present invention has the advantages of non-invasiveness and high safety, completely avoiding the ion radiation risks of traditional X-rays and CTs, and is suitable for long-term follow-up and high-frequency screening scenarios for children and adolescents. The non-invasive operation enables the subject to only stand and complete standardized movements, with no contact throughout the process and extremely short detection time, greatly improving the comfort and acceptance of the subject. At the same time, the efficient and automated processing flow enables the realization of data standard conversion and end-to-end model architecture, with a processing efficiency of up to 50 samples per minute, greatly improving the diagnosis efficiency.

[0051] The present invention adopts strategies such as sparse convolution kernels and refinement convolution kernels for problems such as uneven back point cloud density and noise interference of plantar pressure sensors, enhancing the model's local feature capture ability and anti-noise ability. The cross-body compatibility enables the model to maintain stable classification performance among subjects of different heights and weights. Finally, the accurate grading and clinical practicability enable this technology to achieve accurate grading of normal, mild, moderate, and severe based on the probability output of Softmax, providing strong support for clinical diagnosis and treatment.

[0052] Other advantages, objectives, and features of the present invention will be described to some extent in the subsequent specification, and to some extent, will be obvious to those skilled in the art based on the study of the following text, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following specification. Brief Description of the Drawings

[0053] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be described in detail with reference to the accompanying drawings, where:

[0054] Figure 1 It is a schematic flowchart of a scoliosis grading detection method based on a multi-input neural network according to an embodiment of the present invention;

[0055] Figure 2 It is a schematic diagram of a detection network of a scoliosis grading detection method based on a multi-input neural network according to an embodiment of the present invention;

[0056] Figure 3 It is a schematic structural diagram of a scoliosis grading detection device based on a multi-input neural network according to an embodiment of the present invention. Specific Embodiments

[0057] The following uses specific specific examples to illustrate the embodiments of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the drawings provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner. Without conflict, the following embodiments and the features in the embodiments can be combined with each other.

[0058] Among them, the drawings are only for illustrative purposes, showing only schematic diagrams, not physical diagrams, and should not be construed as a limitation of the present invention; in order to better illustrate the embodiments of the present invention, some components in the drawings will be omitted, enlarged, or reduced, which do not represent the dimensions of the actual product; for those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.

[0059] In the drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components; in the description of the present invention, it should be understood that if there are terms such as "upper", "lower", "left", "right", "front", "rear", etc. indicating the orientation or positional relationship, they are based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, the terms describing the positional relationship in the drawings are only for illustrative purposes and should not be construed as a limitation of the present invention. For those of ordinary skill in the art, the specific meanings of the above terms can be understood according to specific circumstances.

[0060] Please refer to Figures 1 to 3 , which is a scoliosis grading detection method and device based on a multi-input neural network.

[0061] Example 1

[0062] This example provides a detailed implementation process of a scoliosis grading detection method based on a multi-input neural network, as Figure 1 shown, which specifically includes the following steps:

[0063] S1. Collect human plantar pressure data through a piezoresistive sensor array, and collect human back three-dimensional point cloud data through a TOF camera;

[0064] S2. Convert the human plantar pressure data and the back three-dimensional point cloud data into corresponding plantar pressure distribution images and back depth images respectively;

[0065] S3. Input the obtained plantar pressure distribution image and back depth image into a multi-input neural network. Among them, the plantar pressure distribution image and the back depth image respectively pass through multiple 2D convolutional layers to obtain corresponding features;

[0066] S4. Stitch the obtained plantar pressure distribution feature map and back depth feature map to obtain a combined feature;

[0067] S5. Input the combined feature into the Softmax function for classification to obtain the scoliosis grading detection result.

[0068] In step S1 of this example, place the piezoresistive sensor array on the plantar surface of the person to be detected, and the TOF camera face the back of the person to be detected. Both the piezoresistive sensor array and the TOF camera are connected to a computer, and direct the person to be detected to move on the piezoresistive sensor array according to a preset motion paradigm.

[0069] In step S2 of this example, the process of converting the human plantar pressure data into a plantar pressure distribution image is as follows:

[0070] Suppose the human plantar pressure data collected is a one-dimensional array of length N, then the human plantar pressure data collected is expressed as: A = [a1, a2,..., a n ,..., a N , where a n is the value of the nth sensor;

[0071] Arrange the obtained one-dimensional array according to the position of each sensor in the sensor array to obtain the plantar pressure distribution image. The plantar pressure distribution image is expressed as B, where each element b ij in B is expressed as:

[0072]

[0073] Where \(i\in[1,N1]\), \(j\in[1,N2[\), \(i\) and \(j\) respectively represent the elements in the \(i\)-th row and \(j\)-th column of the plantar pressure distribution image \(B\), \(N1\) and \(N2\) respectively represent the number of sensors arranged horizontally and vertically in the piezoresistive sensor array, and \(N1\times N2 = N\).

[0074] In this embodiment, the collected data is a one-dimensional array with a length of 1008, and the image obtained according to the sensor arrangement in the sensor array used for collection is 36*28 pixels; let the original data be \(A=[a1,a2,\cdots,a 1008 \), where \(a i is the value of the \(i\)-th sensor. Let the converted image be \(B\), where the elements of \(B\) are \(b ij The calculation method is as follows:

[0075] b ij =a (i-1)×36+j

[0076] where the value range of \(i\) is from 1 to 36, and the value range of \(j\) is from 1 to 28.

[0077] In step S2 of this embodiment, the process of converting the three-dimensional point cloud data of the human back into a back depth image is as follows:

[0078] The points in the three-dimensional point cloud data of the human back include their abscissa, ordinate, and depth coordinate in space. The points in the point cloud data should be set on the \(xy\) plane, and \(z\) is used as the depth value;

[0079] Let the point set in the three-dimensional point cloud data be \(P\), then the coordinates of the \(t\)-th point are \(p t =(x t ,y t ,z t ). Normalize the points in the point set to the size of the converted back depth image. The calculation formula is as follows:

[0080]

[0081] In the formula, are the abscissa and ordinate of the \(t\)-th point in the back depth image, represents the depth value of the \(t\)-th point in the back depth image, \(x min ,x max are the minimum and maximum abscissas of all points in the point set \(P\), \(y min ,y max are the minimum and maximum ordinates of all points in the point set \(P\), \(z min ,z max are the minimum and maximum depth coordinates of all points in the point set \(P\), \(L\times H\) represents the pixel value of the converted back depth image, where \(L\) represents the width and \(H\) represents the height.

[0082] In this embodiment, the points in the three-dimensional point cloud data include their abscissa, ordinate, and depth coordinate in space. For ease of analysis, they are converted into a depth image of 640*576 pixels.

[0083] In step S3 of this embodiment, Figure 2 The structural schematic diagram of the scoliosis grading detection network based on the multi-input neural network in this embodiment is shown. Among them, for the plantar pressure distribution image and the human back depth image, features are extracted through the 2D convolutional layer three times respectively. Among them, the convolutional kernel size of the plantar pressure part is 3*3, and the convolutional kernel size of the back depth image part is 5*5.

[0084] The calculation formula for the size of the feature map after extraction is as follows:

[0085]

[0086] Where w' and h' are the sizes of the feature maps after extraction, w and h are the sizes of the original images, p represents the number of zero-padding layers, k represents the convolutional kernel size, and s represents the stride.

[0087] In step S4 of this embodiment, after the features are extracted through the convolutional layer, the two types of features are concatenated. Let the plantar pressure distribution feature map be w1×h1×m, and the depth image feature map be w2×h2×n. Since the feature size of the plantar pressure distribution image is smaller than that of the depth image, the plantar pressure features are first upsampled, adjusted to the same size and then concatenated. Let the combined feature after concatenation be F, then F is expressed as:

[0088] F = w1×h1×(m + n)

[0089] In the formula, m and n represent the sizes of the two types of feature maps respectively.

[0090] In step S5 of this embodiment, the combined feature F is input into the Softmax function for a four-classification, and the results are divided into normal, mild, moderate, and severe. Among them, in the fully connected layer FC, let the output size be D, then the calculation process is:

[0091] y = W·x + b

[0092] Where x is the input feature vector, W is the weight matrix of the fully connected layer, b is the bias vector of the fully connected layer, and y is the output of the fully connected layer;

[0093] For the four-classification task, it is further output through the linear layer Linear, and the output size is 4. Then the calculation of the linear layer is:

[0094] y' = W′·y + b'

[0095] where y' is the output of the linear layer, W' is the weight matrix of the linear layer, and b′ is the bias vector of the linear layer; in this embodiment, the four classification results of the output respectively represent the conditions of normal, mild, moderate, and severe scoliosis.

[0096] Then the Softmax layer converts the output of the linear layer into a probability distribution, and the formula is:

[0097]

[0098] Let the prediction result be res, then:

[0099] res = argmax(p)

[0100] The classification where the maximum value among the four probabilities lies is the disease level.

[0101] Embodiment 2

[0102] Based on Embodiment 1, this embodiment proposes a scoliosis grading detection device based on a multi-input neural network, as Figure 3 shown, which includes: a computer, a piezoresistive sensor array, and a TOF camera. Among them, the computer is respectively connected to the piezoresistive sensor array and the TOF camera. The piezoresistive sensor array is used to obtain the plantar pressure data of the human body, and the TOF camera is used to obtain the three-dimensional point cloud data of the human back and transmit the obtained data to the computer;

[0103] The computer is used to execute the aforementioned scoliosis grading detection method based on a multi-input neural network and output the detection result.

[0104] In summary, the present invention respectively uses a piezoresistive sensor array and a TOF camera to collect the plantar pressure data and the three-dimensional point cloud data of the back, then preprocesses the plantar pressure data and the three-dimensional point cloud data respectively, and then converts the plantar pressure data into a plantar pressure distribution image and the three-dimensional point cloud data into a back depth image, which are used as the two inputs of the multi-input network. The present invention adds the input of the plantar pressure image on the basis of the conventional scoliosis detection method based on deep learning, and uses the two types of data to train a comprehensive classification model to classify the severity of scoliosis. This method is also a non-invasive detection method, and takes into account the factor of abnormal plantar pressure of scoliosis patients compared with other detection methods. At the same time, this method is completely automated, greatly saving labor costs, and the classification results considering the two factors are more reliable.

[0105] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the purpose and scope of the present technical solution, and they should all be covered within the scope of the claims of the present invention.

Claims

1. A scoliosis grading detection method based on a multi-input neural network, characterized in that: The method comprises the following steps: S1, collect human foot pressure data through a piezoresistive sensor array, and collect three-dimensional point cloud data of the back of the human body through a TOF camera; S2, converting the plantar pressure data and the back three-dimensional point cloud data of the human body into corresponding plantar pressure distribution images and back depth images respectively; S3, inputting the obtained plantar pressure distribution image and back depth image into a multi-input neural network, wherein the plantar pressure distribution image and the back depth image are respectively subjected to multiple 2D convolution layers to obtain corresponding features; S4, splicing the obtained plantar pressure distribution feature map and back depth feature map to obtain a merged feature; S5. Input the combined features into the Softmax function for classification to obtain the scoliosis grading detection result.

2. The method for scoliosis grading detection based on a multi-input neural network according to claim 1, characterized in that: In step S1, the piezoresistive sensor array is placed on the sole of the person to be detected, and the TOF camera is facing the back of the person to be detected. The piezoresistive sensor array and the TOF camera are both connected to a computer to instruct the person to be detected to move on the piezoresistive sensor array according to a preset action paradigm.

3. The method for scoliosis grading detection based on a multi-input neural network according to claim 1, characterized in that: In step S2, the process of converting the human plantar pressure data into a plantar pressure distribution image is as follows: Assuming that the collected human plantar pressure data is a one-dimensional array with a length of N, the collected human plantar pressure data is expressed as: A = [a1, a2, ..., a n ,…,a N ], where a n is the value of the nth sensor; The acquired one-dimensional array is arranged according to the position of each sensor in the sensor array to obtain a plantar pressure distribution image, which is represented by B, where each element b in B ij It is expressed as: Wherein, i∈[1,N1],j∈[1,N2], i,j represent the elements in the i-th row and j-th column in the plantar pressure distribution image B, N1,N2 represent the number of sensors arranged horizontally and the number of sensors arranged vertically of the piezoresistive sensor array, and N1×N2=N.

4. The method for scoliosis grading detection based on a multi-input neural network according to claim 1, characterized in that: In step S2, the process of converting the three-dimensional point cloud data of the back of the human body into a back depth image is as follows: The points in the 3D point cloud data of the back of the human body contain their horizontal coordinates, vertical coordinates, and depth coordinates in space. The points in the point cloud data should be set in the xy plane, and z should be used as the depth value; Assume that the point set in the three-dimensional point cloud data is P, then the coordinates of the tth point are p t =(x t ,y t ,z t ), normalize the points in the point set to the size of the converted back depth image, and the calculation formula is as follows: In the formula, are the horizontal and vertical coordinates of the t-th point in the back depth image, represents the depth value of the t-th point in the back depth image, x min ,x max is the minimum and maximum value of the horizontal coordinates of all points in the point set P, y min ,y max is the minimum and maximum value of the ordinate of all points in the point set P, z min ,z max are the minimum and maximum depth coordinates of all points in the point set P, L×H represents the pixel value of the transformed back depth image, where L represents the width and H represents the height.

5. The method for scoliosis grading detection based on a multi-input neural network according to claim 1, characterized in that: In step S3, the plantar pressure distribution image and the human back depth image are respectively subjected to multiple 2D convolution layers to extract features, wherein the convolution kernel size of the plantar pressure part is 3*3, and the convolution kernel size of the back depth image part is 5*5. The calculation formula for the feature map size after extraction is as follows: Where w' and h' are the sizes of the extracted feature maps, w and h are the sizes of the original images, p represents the number of zero-padding layers, k represents the convolution kernel size, and s represents the step size.

6. The method for scoliosis grading detection based on a multi-input neural network according to claim 1, characterized in that: In step S4, after the features are extracted by the convolution layer, the two features are spliced. The plantar pressure distribution feature map is w1×h1×m, and the depth image feature map is w2×h2×n. Since the feature size of the plantar pressure distribution image is smaller than the feature size of the depth image, the plantar pressure feature is first upsampled and adjusted to a uniform size before splicing. The combined feature after splicing is F, then F is expressed as: F=w1×h1×(m+n) In the formula, m and n represent the sizes of the two feature maps respectively.

7. The method for scoliosis grading detection based on a multi-input neural network according to claim 1, characterized in that: In step S5, the merged feature F is input into the Softmax function for a four-class classification, and the result is divided into normal, mild, moderate and severe. In the fully connected layer FC, the output size is set to D, and the calculation process is: y=W·x+b Where x is the input feature vector, W is the weight matrix, b is the bias vector, and y is the output of the fully connected layer; For classification tasks, the linear layer is further output, and the calculation of the linear layer is: y'=W'·y+b' Where y' is the output of the linear layer, W' is the weight matrix, and b' is the bias vector; the output classification results represent normal, mild, moderate, and severe scoliosis conditions respectively; Then the Softmax layer converts the linear layer output into a probability distribution as follows: Assume the prediction result is res, then: res = argmax(p) The above formula indicates searching for the maximum probability in the output probability set, and the category where the maximum value in the probability set is located is the disease level.

8. A scoliosis grading detection device based on a multi-input neural network, characterized in that: The device comprises a computer, a piezoresistive sensor array, and a TOF camera, wherein: The computer is connected to the piezoresistive sensor array and the TOF camera respectively. The piezoresistive sensor array is used to obtain the pressure data of the human foot, and the TOF camera is used to obtain the three-dimensional point cloud data of the back of the human body, and transmit the obtained data to the computer; The computer is used to execute the scoliosis grading detection method based on a multi-input neural network as described in any one of the preceding claims 1-7, and output the detection result.

Images