Scoliosis screening method and system based on cloud computing

Abstract

According to the scoliosis screening method and system based on cloud computing, a depth camera is used for collecting a human body back image, an edge control host carries out preprocessing to obtain a depth image, and a continuous depth value area is taken as a human body back key area; converting the acquired depth data of the back area of the human body into spatial three-dimensional coordinate points, and gathering the three-dimensional coordinate points to form point cloud data; the point cloud data is sent to a cloud computing center, and the cloud computing center calculates the ATR angle of each vertebral section of the spine according to the point cloud data of the back of the human body; and the cloud computing center sends the computed ATR angle data of each vertebral section position back to the edge control host for scoliosis screening. The method provided by the invention not only reduces the dependence on the performance of local equipment, but also improves the efficiency and accuracy of data processing through the centralized processing of the cloud, and reduces the overall cost at the same time.

Description

Technical Field

[0001] The present invention belongs to the field of data processing technology, relates to vertebral angle detection for scoliosis, and is a scoliosis screening method and system based on cloud computing. Background Art

[0002] Scoliosis refers to the lateral curvature of the spine in addition to the normal forward and backward curvature. This condition may cause asymmetric body posture and may affect cardiopulmonary function in severe cases. Scoliosis screening is a method that helps doctors detect potential scoliosis in a large population. The purpose of screening is to detect scoliosis early so that timely treatment can be given to prevent the condition from worsening. Scoliosis screening is usually performed on children and adolescents, especially around puberty, because this period is the fastest growing stage of the spine and the period when scoliosis develops fastest.

[0003] Traditional scoliosis screening devices usually collect the subject's body shape data through image acquisition devices to generate point cloud images. These devices process this data locally, analyze the point cloud through built-in algorithms, and identify the morphology and position of the spine for further judgment by doctors. The screening process includes comparing the natural curvature of the patient's spine with a standard model and detecting any abnormal lateral curvature. If the curvature angle is detected to exceed a certain threshold, it is generally considered a sign of scoliosis. This screening method has the following problems:

[0004] First, point cloud data processing and scoliosis screening algorithms have high demands on computing resources, which requires that screening devices must be equipped with high-performance hardware, thereby increasing the cost of the equipment. This is a limiting factor for large-scale screening activities because it limits the popularity and application of screening technology. Secondly, once the screening equipment is sold, the update and maintenance of the algorithm program becomes complicated. This makes it difficult for the equipment to be updated to the latest screening algorithm in a timely manner, affecting the accuracy and reliability of the screening results. Finally, due to the untimely update of the algorithm, the screening results on each device may differ, which is not conducive to unifying the screening standards and improving the screening quality.

[0005] Cloud computing is an emerging data computing model this year, but data transmission efficiency is still one of the problems to be solved. For scoliosis screening, how to quickly transmit a large amount of basic data collected on-site to cloud computing and get accurate results back remains to be studied. In addition, the existing point cloud analysis and calculation methods are still insufficient in how to quickly and accurately calculate and analyze the spinal status from a large amount of point cloud data. Due to individual differences during detection, the existing ATR angle calculation method still has some errors. Summary of the invention

[0006] The technical problem to be solved by the present invention is that the existing scoliosis screening scheme relies on local equipment when detecting spinal data, has high requirements on the equipment, and it is difficult to perform large-scale screening for scoliosis.

[0007] The technical solution of the present invention is: a scoliosis screening method based on cloud computing, comprising the following steps:

[0008] Step 1: Use a depth camera to collect images of the back of the human body, and the edge control host continuously captures the depth image data for filtering preprocessing;

[0009] Step 2: The edge control host obtains an area of ​​the depth image with a depth value range of 100 mm to 2000 mm as a continuous depth value in the preprocessed depth image, and uses the area as a key area of ​​the back of the human body;

[0010] Step 3: The edge control host converts the acquired depth data of the back area of ​​the human body, and converts the depth image data into three-dimensional coordinate points in space according to the focal length and optical center parameters of the depth camera. The three-dimensional coordinate points are aggregated to form point cloud data.

[0011] Step 4, sending the point cloud data to the cloud computing center, which calculates the ATR angle of each vertebral segment of the spine based on the point cloud data of the human back;

[0012] Step 5: The cloud computing center sends the calculated ATR angle data of each vertebral position back to the edge control host for scoliosis screening.

[0013] The present invention also provides a scoliosis screening system based on cloud computing, which is used to implement the above-mentioned scoliosis screening method, including:

[0014] A data acquisition module, used for acquiring depth image data of the back of a human body;

[0015] A data analysis module, used for acquiring a depth image within a preset depth value range as a continuous depth value area, and taking the area as a key area of ​​the back of the human body;

[0016] A data conversion module, used for converting the depth data of the back area of ​​the human body into point cloud data of the back of the human body;

[0017] The calculation module is used to calculate the ATR angle of each vertebral segment of the spine based on the point cloud data of the human back.

[0018] The cloud computing center in the present invention can be equipped with CPU computing resources with strong computing power, and can use multi-core CPUs for parallel computing, significantly reducing computing time. Through cloud computing, resource sharing of point cloud data calculations can be achieved, reducing costs. The algorithm is concentrated in the cloud, which is convenient for algorithm upgrade, optimization, and deployment. At the same time, centralized management in the cloud also facilitates unified updating and maintenance of the algorithm, ensuring that all devices can use the latest screening algorithm and improving the accuracy and consistency of screening results. When using cloud computing, the present invention reduces the amount of data to be transmitted by preprocessing the on-site detection data, while retaining effective computing data for later calculations, and obtains accurate angle information in the cloud through the designed ATR angle calculation method, which is effectively used for scoliosis screening. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 Flow chart of the method of the present invention.

[0020] Figure 2 This is an example diagram of a mask image of a key area of ​​the back of a human body in the method of the present invention.

[0021] Figure 3 Schematic diagram of the system of the present invention. DETAILED DESCRIPTION

[0022] The present invention provides a scoliosis screening method based on cloud computing, such as Figure 1 As shown, the following steps are included.

[0023] Step 1: Using the image of the human back captured by the depth camera, the edge control host continuously captures the depth image data and pre-processes it.

[0024] (1) The depth camera is a 3D depth camera, which is electrically connected to the edge control host. The Adam's Forward Bend Test (Adam test) is a simple and practical examination method for scoliosis in clinical practice. The depth camera collects the back image of the human body in the Adam's posture, and transmits the back image to the edge control host to obtain a H*W depth image, wherein H represents the height of the image, that is, the number of pixels in the vertical direction, and W represents the width of the image, that is, the number of pixels in the horizontal direction. Each pixel point of the depth image has a depth value, and the depth value corresponds to the distance from the pixel point in the camera field of view to the camera sensor plane.

[0025] (2) The edge control host continuously acquires 10 to 20 frames of depth image data and performs average filtering on them to obtain the image data frame D after the first step of filtering. Then, the second step of filtering is performed to calculate the depth average value d'(x, y) at each pixel in the image data frame D, which is obtained by the following formula:

[0026]

[0027] Where d(x,y) is the depth value of the position (x,y) in the image data frame D. And so on; n×m is the window area, and the depth average is calculated by defining a window range. m and n are the side lengths of the window with (x, y) as the center point, in pixels, and m and n are empirically set to 3 to 7; i represents the index of the window in the x-axis direction, ranging from 1 to n; j represents the index of the window in the y-axis direction, ranging from 1 to m. The depth average is used as the depth value after filtering at the pixel position to obtain the preprocessed depth image.

[0028] By performing two-step filtering on the depth image data, the noise in the depth image can be reduced, the depth image can be made smoother and more stable, and the stability and accuracy of the data can be improved.

[0029] Step 2: The edge control host obtains a depth image with a depth value range of 100 mm to 2000 mm as a continuous depth value area, and uses the area as a key area of ​​the back of the human body. The method includes:

[0030] (1) The depth image is offset by n pixels, where n is in the range of 1 to 10. In this embodiment, the offset is 2 pixels. The depth image after offset is subtracted from the depth image before offset to generate a difference image of the depth image. The value of each pixel in the difference image represents the depth difference of the corresponding position in the two images. The difference image is binarized:

[0031] Check the value of each pixel in the difference image. If the pixel value is less than or equal to 10, set the pixel value to 0. If it is greater than 10, set it to 255.

[0032] (2) In OpenCV, the center point filling algorithm function floodFill is used to fill the processed difference image, and the images before and after filling are again differentially operated to obtain a mask image. The calculation formula for the differential operation is:

[0033] F(p,q)=|I before (p,q)-I after (p,q)|

[0034] Among them, I before (p,q) is the pixel value of the image at position (p,q) before filling, I after (p,q) is the pixel value of the padded image at (p,q), and F(p,q) is the pixel value of the difference image at position (p,q).

[0035] The mask image is based on the comparison of the pixel value of the differential image with a set threshold. The pixel values ​​less than or equal to the threshold are set to 0, and the pixel values ​​greater than the threshold are set to 255. The calculation formula is:

[0036]

[0037] Where M(p,q) is the pixel value of the mask image at position (p,q), and T is the set threshold. Here T is usually 10, which means that only when the pixel value of the difference image is greater than 10, the corresponding pixel value in the mask image is set to 255, otherwise it is 0.

[0038] (3) The effective area on the mask image, that is, the area composed of pixels with a pixel value of 255, is defined as an area with continuous depth values ​​as the back area of ​​the human body, such as Figure 2 shown.

[0039] Step 3: The edge control host converts the acquired depth data of the back area of ​​the human body. According to the current camera focal length and optical center parameters, the two-dimensional depth image data is converted into three-dimensional coordinate points. The three-dimensional coordinate points are combined to form point cloud data. The formula is:

[0040]

[0041] Z=Z

[0042] Among them, (u,v) is the pixel coordinate in the depth image, Z is the corresponding depth value, (cx,cy) is the optical center coordinate, and (fx,fy) is the focal length.

[0043] Step 4: Send the point cloud data to the cloud computing center. The cloud computing center restores the point cloud data to obtain the point cloud data of the human back, and calculates the ATR angle of each vertebral segment of the spine based on the point cloud data of the human back.

[0044] The edge control host packages and compresses the point cloud data of the human back area obtained after preprocessing into a compressed file; the edge control host sends the compressed file to the cloud computing center through the Internet.

[0045] The cloud computing center slices the point cloud data according to the number of vertebrae to obtain the start and end indexes of each vertebrae. Each vertebrae is a thoracic vertebra and a lumbar vertebra. From top to bottom, there are 12 thoracic vertebrae and 5 lumbar vertebrae.

[0046] In the three-dimensional coordinate points of the point cloud data, the x-axis direction corresponds to the left and right direction of the back, the y-axis direction corresponds to the front and back direction of the back when bending over, and the z-axis direction corresponds to the depth direction. When the depth camera collects the image of the back of the human body, the human body is in a bent over state, and the camera collects the back image directly above the human body. Therefore, the z-axis also represents the depth of the back at the pixel position from the camera plane. The ATR angle is obtained by calculating the point cloud data as follows.

[0047] Perform mean filtering in the x-axis direction: for each point cloud data and its n points on the left and right sides in the x-axis direction, n is 2 to 5. In this embodiment, 3 points are taken, that is, the z coordinate values ​​of the 2n+1 points in total are included in the mean calculation, and the calculation result is used as the new z coordinate value of the point.

[0048] Mean filtering in the y-axis direction: through the y-value start and end index of each vertebral segment, the back data between the indexes are obtained, which is the slice data of the current vertebral segment, and then the slice data of the current vertebral segment is processed in the y-axis direction to obtain a one-dimensional slice data. 17 vertebrae get 17 slices.

[0049] Mean filtering is used to eliminate the impact of individual screening differences on the accuracy of calculation results.

[0050] Perform curve fitting of the function on the one-dimensional slice data to obtain the function y=f(x), and analyze the bending point of the curve through the function;

[0051] The center point of the spine is analyzed through the bending point, and the coordinates of the center point of the spine at the vertebral segment are obtained.

[0052] Through the coordinates of the spine center point of the vertebra, remove the points on the vertebral slice within the 22mm horizontal distance on the left and right sides. The remaining points on the slice after removal are taken as set M. The length of the scoliosis measuring ruler is 22mm, and the corresponding value is taken here. Select different points A and B from the set M at random. Point A is higher than point B. The height here also corresponds to the height in the physical world, that is, the physical height of point A is higher than point B, because the depth camera takes the image directly above the human body in the bent state. If described by the depth value of the depth camera, the depth value (z value) of point A is less than the depth value of point B. Calculate the function of the straight line l where the two points are located. Use the straight line function to judge all points in M ​​to see if there are points higher than the straight line l. If a point C higher than the straight line l is found, use point C to update and replace the original lower point B, and recalculate the function of the straight line l. Until all points in the set M are below the straight line l.

[0053] The coordinates of point A and point B obtained by the latest update are (x1, y, z1) and (x2, y, z2), respectively, and the ATR angle of the current vertebra is calculated using the following formula:

[0054]

[0055] Among them, (x1, y, z1) are the coordinates of point A, and (x2, y, z2) are the coordinates of point B.

[0056] Step 5: The cloud computing center packages and compresses the calculated ATR angle data of each vertebral position into a file and sends it back to the edge control host. The edge control host is connected to the display terminal and presents the results on the display terminal to check the ATR angle of each vertebra. The ATR angle range of the first-level scoliosis vertebra is 5 to 7°; the ATR angle range of the second-level scoliosis vertebra is 7 to 10°; the ATR angle range of the third-level scoliosis vertebra is above 10°. This angle can help doctors evaluate the scoliosis of the spine and decide whether further diagnosis is needed.

[0057] The present invention also provides a scoliosis screening system based on cloud computing, such as Figure 3 As shown, the method for implementing the above-mentioned scoliosis screening includes:

[0058] A data acquisition module, used for acquiring depth image data of the back of a human body;

[0059] A data analysis module, used for acquiring a depth image within a preset depth value range as a continuous depth value area, and taking the area as a key area of ​​the back of the human body;

[0060] A data conversion module, used for converting the depth data of the back area of ​​the human body into point cloud data of the back of the human body;

[0061] The calculation module is used to calculate the ATR angle of each vertebral segment of the spine based on the point cloud data of the human back.

[0062] The computer program code for executing the above-mentioned scoliosis screening method can be written in one or more programming languages ​​or a combination thereof, the data acquisition module is a depth camera, the data analysis module and the data conversion module are implemented in the edge control host, and the edge control host includes a processing device, such as a central processing unit, a graphics processing unit, etc., which can perform various appropriate actions and processes according to the computer program stored in the computer-readable storage medium, and the computer-readable storage medium can be included in the above-mentioned edge control host; or it can exist separately without being assembled into the edge control host. The computing module is a cloud server, and can communicate with the edge control host using any currently known or future developed network protocol such as HTTP (HyperText TransferProtocol).

[0063] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

Claims

1. A scoliosis screening method based on cloud computing, characterized in that The following steps are involved: Step 1: Use a depth camera to collect images of the back of the human body, and the edge control host continuously captures the depth image data for filtering preprocessing; Step 2: The edge control host obtains an area of ​​the depth image with a depth value range of 100 mm to 2000 mm as a continuous depth value in the preprocessed depth image, and uses the area as a key area of ​​the back of the human body; Step 3: The edge control host converts the acquired depth data of the back area of ​​the human body, and converts the depth image data into three-dimensional coordinate points in space according to the focal length and optical center parameters of the depth camera. The three-dimensional coordinate points are aggregated to form point cloud data. Step 4, sending the point cloud data to the cloud computing center, which calculates the ATR angle of each vertebral segment of the spine based on the point cloud data of the human back; Step 5: The cloud computing center sends the calculated ATR angle data of each vertebral position back to the edge control host for scoliosis screening.

2. A scoliosis screening method based on cloud computing according to claim 1, The feature is that in step 1, the depth camera collects the image of the back of the human body in Adam's posture, the edge control host obtains the depth image data based on the collected image, continuously obtains 10 to 20 frames of depth image data, performs average filtering, and obtains the image data frame D after the first step of filtering, and then performs the second step of filtering to calculate the depth average value d'(x, y) at each pixel point in the image data frame D, which is obtained by the following formula: in, d(x,y) is the depth value of the pixel (x,y) position in the image data frame D; n×m is the window area, m and n are the side lengths of the window centered at (x,y), in pixels, and the value range of m and n is 3 to 7; i represents the index of the window in the x-axis direction, ranging from 1 to n; j represents the index of the window in the y-axis direction, ranging from 1 to m; The depth average is used as the filtered depth value of the pixel position to obtain a preprocessed depth image.

3. The method for scoliosis screening based on cloud computing according to claim 1, characterized in that Step 2 specifically includes: 1) The depth image is offset by n pixels, where n ranges from 1 to 10. The depth image after offset is subtracted from the depth image before offset to generate a difference image of the depth image. The value of each pixel in the difference image represents the depth difference of the corresponding position in the two images. The difference image is binarized, that is, the value of each pixel in the difference image is checked. If the value of the pixel is less than or equal to 10, the value of the pixel is set to 0. If it is greater than 10, it is set to 255. 2) Use OpenCV's center point filling algorithm function floodFill to fill the processed difference image, and perform a difference operation on the image before and after filling to obtain a mask image. The calculation formula for the difference operation is: F(p,q)=|I before (p,q)-I after (p,q)| Among them, I before (p,q) is the pixel value of the image at position (p,q) before filling, I after (p,q) is the pixel value of the padded image at (p,q), and F(p,q) is the pixel value at position (p,q) after the difference operation; The mask image is obtained by comparing the pixel value obtained by the difference operation with a set threshold. The pixel values ​​less than or equal to the threshold are set to 0, and the pixel values ​​greater than the threshold are set to 255. The calculation formula is: Where M(p,q) is the pixel value of the mask image at position (p,q), and T is the set threshold; 3) The effective area on the mask image, that is, the area composed of pixels with a pixel value of 255, is defined as an area with continuous depth values ​​as the back area of ​​the human body.

4. The scoliosis screening method based on cloud computing according to claim 1, wherein in step 3, the two-dimensional depth image data is converted into three-dimensional coordinate points specifically as follows: Z=Z in, (u,v) is the pixel coordinate in the depth image, Z is the corresponding depth value, (X,Y,Z) is the three-dimensional coordinate point, (cx,cy) is the optical center coordinate, and (fx,fy) is the focal length.

5. The scoliosis screening method based on cloud computing according to claim 1, It is characterized in that in step 4, the cloud computing center calculates the ATR angle of each vertebral segment of the spine, and the cloud computing center slices the point cloud data according to the number of vertebral segments to obtain the start and end index of each vertebral segment. The vertebral segments include all vertebral segments of the thoracic spine and the lumbar spine. The calculation of the ATR angle includes: In the three-dimensional coordinate points of the point cloud data, the x-axis direction corresponds to the left and right direction of the back, the y-axis direction corresponds to the front and back direction of the back in the bent state, and the z-axis direction corresponds to the depth direction; Perform mean filtering in the x-axis direction: For each point cloud data, the z coordinate values ​​of the n points on the left and right sides of the x-axis direction, where n is 2 to 5, i.e., the z coordinate values ​​of the 2n+1 points in total are calculated, and the calculated result is used as the new z coordinate value of the point; Mean filtering in the y-axis direction: through the y-value start and end indexes of each vertebral segment, the back data between the indexes is obtained as the data of the current vertebral segment slice, and then the data of the current vertebral segment slice is processed by averaging in the y-axis direction to obtain a one-dimensional slice data; Perform curve fitting of the function on the one-dimensional slice data to obtain the function y=f(x), and analyze the bending point of the curve through the function; The center point of the spine is analyzed through the bending point, and the coordinates of the center point of the spine at the vertebral segment are obtained; Through the coordinates of the spine center point of the vertebral segment, remove the points on the vertebral segment slice within the horizontal distance of 22mm on the left and right sides. After removing, the remaining points on the slice are taken as a set M. From them, different points A and B are randomly selected. Point A is higher than point B, that is, the depth value of point A is less than point B. The function of the straight line l where the two points are located is calculated. The straight line function is used to judge all points in M ​​to see if there is a point higher than the straight line l. If a point C higher than the straight line l is found, point C is used to replace point B, and the function of the straight line l is recalculated until all points in the set M are below the straight line l. The coordinates of point A and the final updated point B are (x1, y, z1) and (x2, y, z2) respectively. The ATR angle of the current vertebra is calculated using the following formula:

6. A scoliosis screening system based on cloud computing, characterized in that A method for screening for scoliosis according to any one of claims 1 to 5, comprising: A data acquisition module, used for acquiring depth image data of the back of a human body; A data analysis module, used for acquiring a depth image within a preset depth value range as a continuous depth value area, and taking the area as a key area of ​​the back of the human body; A data conversion module, used for converting the depth data of the back area of ​​the human body into point cloud data of the back of the human body; The calculation module is used to calculate the ATR angle of each vertebral segment of the spine based on the point cloud data of the human back.

Images