Improved spinal scoliosis cobb angle automatic measurement method and system based on YOLOv8n-obb

Abstract

This invention discloses an automatic measurement method and system for the Cobb angle of scoliosis based on an improved YOLOv8n-obb model. The method includes acquiring a spinal image (containing several vertebrae), inputting the spinal image into a pre-trained deep learning model based on an improved YOLOv8n-obb model, and outputting the positions of several vertebrae. The deep learning model includes a backbone network, a cervical network, and a head network. The backbone network is a multi-scale convolutional module combining C2f, DCNv2, FADC, and SPPF, used to acquire multi-scale features of the spinal image. The cervical network is a feature fusion module combining DSS, an attention layer, and a sampling layer, used to perform feature fusion on the multi-scale features. The head network includes multiple detection heads, used to perform rotational target detection based on the multi-scale features fused to obtain the positions of several vertebrae, and then obtaining the Cobb angle based on these vertebrae positions. This invention uses a deep learning model improved by YOLOv8n-obb to perform rotational target detection on the vertebrae positions, thereby predicting the Cobb angle of scoliosis.

Description

Technical Field

[0001] This invention relates to the field of intelligent diagnosis and treatment technology, specifically to an automatic measurement method and system for the Cobb angle of scoliosis based on YOLOv8n-obb. Background Technology

[0002] Scoliosis is a common spinal deformity characterized by a lateral curvature of the spine when viewed from the front or side, typically forming a "C" or "S" shape. It can occur at any age, but is most common in childhood and adolescence, particularly during puberty. The Cobb angle, proposed in 1948 by American orthopedic surgeon John Robert Cobb, is a measure of the severity of scoliosis and has been widely used in clinical practice and research, considered the "gold standard" for scoliosis diagnosis. The Cobb angle is determined by measuring the angle between the most tilted upper and lower vertebrae on an X-ray. Traditionally, marking and measuring the Cobb angle in clinical diagnosis requires manual measurement by physicians. However, manual measurement has low repeatability and accuracy, especially in postoperative Cobb angle measurements, where reliability drops significantly. Furthermore, manual measurement relies heavily on the physician's subjective judgment, leading to inconsistencies in later results. Additionally, manual measurement is more time-consuming than automated measurement, resulting in lower efficiency in clinical practice. Therefore, an accurate and automated prediction technology for the Cobb angle of scoliosis is of vital clinical significance. Summary of the Invention

[0003] To address the problems existing in the prior art, this invention provides an automatic measurement method and system for the Cobb angle of scoliosis based on the improved YOLOv8n-obb. Based on the improved YOLOv8n-obb deep learning model, the method performs rotational target detection on the vertebral position, thereby predicting the Cobb angle of scoliosis with higher prediction accuracy.

[0004] Other features and advantages of this application will become apparent from the following detailed description, or may be learned in part from practice of this application.

[0005] According to a first aspect of this application, an automatic measurement method for the Cobb angle of scoliosis based on YOLOv8n-obb is provided, comprising: Acquire a spinal image, the spinal image including several vertebrae; The spinal image is input into a pre-trained deep learning model based on improved YOLOv8n-obb, which outputs the positions of several vertebrae. The deep learning model includes a backbone network, a cervical network, and a head network. The backbone network is a multi-scale convolutional module combining C2f, DCNv2, FADC, and SPPF, used to acquire multi-scale features of the spinal image. The cervical network is a feature fusion module combining DSS, attention layer, and sampling layer, used to perform feature fusion on the multi-scale features. The head network includes multiple detection heads, used to perform rotational target detection based on the multi-scale features fused to obtain the positions of several vertebrae. The pre-trained deep learning model based on improved YOLOv8n-obb is trained using several spinal images used for pre-training. The Cobb angle is obtained based on the positions of several vertebrae.

[0006] In some embodiments of this application, based on the foregoing scheme, the multi-scale convolutional module includes a first convolutional layer, a second convolutional layer combining a first C2f, a third convolutional layer combining a second C2f, a fourth convolutional layer combining a third C2f and DCNv2, a fifth convolutional layer combining a fourth C2f and FADC, and an SPPF layer. The second convolutional layer combining the first C2f outputs a first feature map, the third convolutional layer combining the second C2f outputs a second feature map, the fourth convolutional layer combining the third C2f and DCNv2 outputs a third feature map, the fifth convolutional layer combining the fourth C2f and FADC outputs a fourth feature map, and the SPPF layer outputs a fifth feature map.

[0007] In some embodiments of this application, based on the foregoing scheme, the fourth convolutional layer combining the third C2f and DCNv2 includes a first input unit, a first convolutional unit, a first feature splitting unit, a multi-layer DCNv2 bottleneck structure, a first splicing unit, a second convolutional unit, and a first output unit; The second feature map is input into the first input unit and processed sequentially by the first convolutional unit, the first feature splitting unit, and the multi-layer DCNv2 bottleneck structure. The feature maps obtained from the first convolutional unit, the first feature splitting unit, and the multi-layer DCNv2 bottleneck structure are then fused in the first concatenation unit. The fused feature map is then processed by the second convolutional unit, and finally output by the first output unit to obtain the third feature map. The multi-layer DCNv2 bottleneck structure includes multiple DCNv2 bottleneck structures stacked sequentially, and the DCNv2 bottleneck structure includes a third convolutional unit and a DCNv2 unit stacked sequentially. The DCNv2 unit is a deformable convolution, which changes the shape of the convolution kernel by changing the sampling results.

[0008] In some embodiments of this application, based on the foregoing scheme, the fifth convolutional layer combining the fourth C2f and FADC includes a second input unit, a third convolutional unit, a second feature splitting unit, a multi-layer FADC bottleneck structure, a second splicing unit, a fourth convolutional unit, and a second output unit. The third feature map is input to the second input unit and processed sequentially by the third convolutional unit, the second feature splitting unit, and the multi-layer FADC bottleneck structure. The feature maps obtained from the third convolutional unit, the second feature splitting unit, and the multi-layer FADC bottleneck structure are then fused in the second concatenation unit. The fused feature map is then processed by the fourth convolutional unit, and finally output by the second output unit to obtain the fourth feature map. The multi-layer FADC bottleneck structure includes multiple FADC bottleneck structures stacked sequentially, and each FADC bottleneck structure includes a fourth convolutional unit and an adaptive dilated convolutional unit stacked sequentially. The adaptive dilated convolutional unit dynamically adjusts the dilation rate in space by adaptively selecting the dilation rate, adaptive kernel, and frequency.

[0009] In some embodiments of this application, based on the foregoing scheme, the feature fusion module includes a DSS layer, an attention layer, a sixth convolutional layer, a seventh convolutional layer, an eighth convolutional layer, a ninth convolutional layer, a tenth convolutional layer, an eleventh convolutional layer, a fifth C2f, a sixth C2f, a seventh C2f, an eighth C2f, a first sampling layer, a second sampling layer, a first splicing layer, and a second splicing layer. The first feature map is input into the sixth convolutional layer for processing; the second feature map, the feature map obtained after inputting the second feature map into the eighth convolutional layer, the feature map obtained after inputting the fifth feature map into the tenth convolutional layer, and the third feature map are fused in the second sampling layer, then processed by the sixth C2f layer, and then input into the seventh convolutional layer for processing; the feature map obtained after processing the sixth convolutional layer, the second feature map, and the feature map obtained after processing the seventh convolutional layer are fused in the first sampling layer, and then processed by the fifth C2f layer; the fourth feature map is input into the DSS layer, and then fused with the feature map obtained after processing the fifth C2f layer in the attention layer. The feature map obtained after the fusion in the attention layer is the sixth feature map. The feature map obtained after the fifth C2f processing is input into the ninth convolutional layer, and then fused with the feature map obtained after the seventh convolutional layer in the first splicing layer. After being processed by the seventh C2f, the feature map obtained by the seventh C2f processing is the seventh feature map. The feature map obtained after the seventh C2f processing is input into the eleventh convolutional layer for processing, and the fifth feature map is input into the tenth convolutional layer for processing. The feature map obtained from the eleventh convolutional layer and the feature map obtained from the tenth convolutional layer are fused in the second splicing layer, and then processed by the eighth C2f to obtain the eighth feature map.

[0010] In some embodiments of this application, based on the foregoing scheme, the head network includes a first detection head, a second detection head, and a third detection head; The sixth feature map is input into the first detection head to obtain the first prediction box of the vertebral position; The seventh feature map is input into the second detection head to obtain a second prediction box for the vertebral position; The eighth feature map is input into the third detection head to obtain the third prediction box of the vertebral position; The first, second, and third prediction boxes for the vertebral position are merged to obtain the final detection box for the vertebral position.

[0011] In some embodiments of this application, based on the foregoing scheme, obtaining the Cobb angle based on the positions of several vertebrae includes: Obtain the slope of the final detection box at each vertebra position. If the coordinates of the boundary corners of the final detection box coincide, reset the slope to infinity. Sort all the slopes to generate a slope list; Based on the slope list, calculate the angle between each pair of slopes. The angles that are greater than or equal to the sorting threshold are Cobb angles. If the index vertebrae of two angles overlap and the number of overlapping indices exceeds 1, the angle with the smaller span is discarded and the larger angle of the next position is taken. If the angle is an obtuse angle exceeding 90 degrees, the supplementary angle of the angle is used to replace the angle.

[0012] According to a second aspect of this application, an automatic measurement system for the Cobb angle of scoliosis based on the improved YOLOv8n-obb is provided, comprising: The first acquisition module is used to acquire a spinal image, the spinal image including a number of vertebrae; The second acquisition module is used to input the spinal image into a pre-trained deep learning model based on YOLOv8n-obb, and output several vertebral positions. The deep learning model includes a backbone network, a cervical network, and a head network. The backbone network is a multi-scale convolutional module combining C2f, DCNv2, FADC, and SPPF, and is used to acquire multi-scale features of the spinal image. The cervical network is a feature fusion module combining DSS, attention layer, and sampling layer, and is used to perform feature fusion on the multi-scale features. The head network includes multiple detection heads, and is used to perform rotational target detection based on the multi-scale features fused to acquire several vertebral positions. The pre-trained deep learning model based on YOLOv8n-obb is trained using several spinal images used for pre-training. The third acquisition module is used to obtain the Cobb angle based on the positions of several vertebrae.

[0013] According to a third aspect of this application, a computer-readable storage medium is provided that stores a computer program thereon, the computer program including executable instructions that, when executed by a processor, implement the method described above.

[0014] According to a fourth aspect of this application, an electronic device is provided, comprising: One or more processors; A memory for storing executable instructions of the processor, which, when executed by the one or more processors, cause the one or more processors to implement the method described above.

[0015] The beneficial effects of this application are as follows: This application provides an automatic measurement method and system for the Cobb angle of scoliosis based on improved YOLOv8n-obb. It establishes a deep learning model based on improved YOLOv8n-obb. In this model, the backbone network is a multi-scale convolutional module combining C2f, DCNv2, FADC, and SPPF, while the cervical network is a feature fusion module combining DSS, attention layers, and sampling layers. This enhances the ability to fuse feature maps at different scales, thereby improving the ability to locate and identify vertebral positions in rotated target images. This application processes spinal images using a pre-trained deep learning model based on improved YOLOv8n-obb to obtain several vertebral positions, and then obtains the Cobb angle based on these vertebral positions, achieving high accuracy in Cobb angle measurement.

[0016] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and do not limit this application. Attached Figure Description

[0017] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and are intended to explain the invention, but do not constitute an undue limitation thereof. In the drawings: Figure 1 This is a flowchart of an automatic measurement method for the Cobb angle of scoliosis based on YOLOv8n-obb, according to the present invention. Figure 2 This is a schematic diagram illustrating the pre-training of the deep learning model of this invention; Figure 3 This is a schematic diagram of the deep learning model of the present invention; Figure 4 This is a schematic diagram of the fourth convolutional layer of the present invention, which combines the third C2f and DCNv2; Figure 5 This is a schematic diagram of the bottleneck structure of the DCNv2 of the present invention; Figure 6 This is a schematic diagram of the fifth convolutional layer of the present invention, which combines the fourth C2f and FADC; Figure 7 This is a schematic diagram of the FADC bottleneck structure of the present invention; Figure 8 This is a schematic diagram of the feature map processing of the present invention; Figure 9 This is a schematic diagram of an automatic measurement system for the Cobb angle of scoliosis based on YOLOv8n-obb, according to the present invention. Figure 10 This is a schematic diagram of an electronic device according to the present invention. Detailed Implementation

[0018] To make the objectives, features, and advantages of this invention more apparent and understandable, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0019] It should be understood that the terms "comprising" and other similar expressions in the specification, claims, and accompanying drawings of this invention are intended to cover a non-exclusive inclusion, such as a process, method, system, or apparatus that includes a series of steps or layers and is not limited to the listed steps or layers. Furthermore, "first" and "second" are used to distinguish different objects and are not intended to describe a specific order.

[0020] According to the first aspect of this application, Figure 1As shown in the figure, this embodiment provides an automatic measurement method for the Cobb angle of scoliosis based on YOLOv8n-obb, which includes: Step S101: Obtain a spinal image, the spinal image including several vertebrae.

[0021] In some embodiments of this example, acquiring the spinal image includes: Obtain a full-view anteroposterior X-ray image of the human spine; Cropping of full-view anteroposterior X-ray images of the human spine; The cropped full-view X-ray image of the human spine is normalized to obtain the spinal image.

[0022] Step S102: Input the spinal image into a pre-trained deep learning model based on YOLOv8n-obb, which outputs several vertebral positions. The deep learning model includes a backbone network, a cervical network, and a head network. The backbone network is a multi-scale convolutional module combining C2f, DCNv2, FADC, and SPPF, used to acquire multi-scale features of the spinal image. The cervical network is a feature fusion module combining DSS, attention layer, and sampling layer, used to perform feature fusion on the multi-scale features. The head network includes multiple detection heads, used to perform rotational target detection based on the multi-scale features fused to obtain several vertebral positions. The pre-trained deep learning model based on YOLOv8n-obb is trained using several spinal images used for pre-training.

[0023] In some embodiments of this example, private data (full-view anteroposterior X-ray images of the human spine from hospitals) and public data (full-view anteroposterior X-ray images of the human spine selected by professional doctors) are collected together to form a number of spinal images for pre-training. A dataset is established from these spinal images for pre-training, and the dataset preprocessing process is as follows: Figure 2 As shown, since the resolution of a complete anteroposterior (AP) X-ray of the human spine is generally around 512x1024, and the resolution of each image varies, it is not conducive to inputting it into a deep learning model for training. Therefore, cropping the anteroposterior X-ray image of the human spine is necessary. This method also helps to conceal the patient's personal information and protect their privacy. After cropping the complete spine, the cropped anteroposterior X-ray image of the human spine is normalized to a resolution of 512x1024, thus obtaining the spinal image.

[0024] like Figure 2As shown in the flowchart, the dataset is divided into three parts in a 6:2:2 ratio: training set, validation set, and test set. The training set is used to train the input deep learning model, the validation set is used to evaluate and validate the training results during the training process, and the test set is responsible for evaluating the deep learning model's vertebral position localization metrics and the accuracy of Cobb angle calculation.

[0025] In some implementations of this embodiment, such as Figure 3 As shown, in the backbone network, the multi-scale convolutional module includes a first convolutional layer, a second convolutional layer combining the first C2f, a third convolutional layer combining the second C2f, a fourth convolutional layer combining the third C2f and DCNv2, a fifth convolutional layer combining the fourth C2f and FADC, and an SPPF layer. The second convolutional layer combining the first C2f outputs a first feature map, the third convolutional layer combining the second C2f outputs a second feature map, the fourth convolutional layer combining the third C2f and DCNv2 outputs a third feature map, the fifth convolutional layer combining the fourth C2f and FADC outputs a fourth feature map, and the SPPF layer outputs a fifth feature map.

[0026] In this embodiment, compared to YOLOv5, YOLOv8 replaces the C3 layer with C2f, and further improves upon C2f to form a fourth convolutional layer combining the third C2f and DCNv2, and a fifth convolutional layer combining the fourth C2f and FADC. Both the fourth convolutional layer combining the third C2f and DCNv2, and the fifth convolutional layer combining the fourth C2f and FADC, consist of convolutional modules and multiple bottleneck modules.

[0027] In some implementations of this embodiment, such as Figure 3 As shown, the feature fusion module includes a DSS layer, an attention layer, a sixth convolutional layer, a seventh convolutional layer, an eighth convolutional layer, a ninth convolutional layer, a tenth convolutional layer, an eleventh convolutional layer, a fifth C2f layer, a sixth C2f layer, a seventh C2f layer, an eighth C2f layer, a first sampling layer, a second sampling layer, a first splicing layer, and a second splicing layer. The first feature map is input into the sixth convolutional layer for processing; the second feature map, the feature map obtained after inputting the second feature map into the eighth convolutional layer, the feature map obtained after inputting the fifth feature map into the tenth convolutional layer, and the third feature map are fused in the second sampling layer, then processed by the sixth C2f layer, and then input into the seventh convolutional layer for processing; the feature map obtained after processing the sixth convolutional layer, the second feature map, and the feature map obtained after processing the seventh convolutional layer are fused in the first sampling layer, and then processed by the fifth C2f layer; the fourth feature map is input into the DSS layer, and then fused with the feature map obtained after processing the fifth C2f layer in the attention layer. The feature map obtained after the fusion in the attention layer is the sixth feature map. The feature map obtained after the fifth C2f processing is input into the ninth convolutional layer, and then fused with the feature map obtained after the seventh convolutional layer in the first splicing layer. After being processed by the seventh C2f, the feature map obtained by the seventh C2f processing is the seventh feature map. The feature map obtained after the seventh C2f processing is input into the eleventh convolutional layer for processing, and the fifth feature map is input into the tenth convolutional layer for processing. The feature map obtained from the eleventh convolutional layer and the feature map obtained from the tenth convolutional layer are fused in the second splicing layer, and then processed by the eighth C2f to obtain the eighth feature map.

[0028] Therefore, for the Neck part, a new Dynamic Scale Sequence Feature Fusion (DSS) module is adopted to enhance the ability to locate and recognize features of rotated target images, replace part of the stitching layer (Contact), and enhance the model's ability to fuse feature maps of different scales.

[0029] Furthermore, the backbone network in this embodiment adopts the Cross-Stage Partial (CSP) concept, that is, the first feature map is input into the neck network and directly passed across stages. The first feature map is then processed to obtain the second, third, fourth and fifth feature maps. The second, third, fourth and fifth feature maps are then input into the neck network and fused with the first feature map processed by the neck network. This helps to reduce computational consumption and save computational costs. At the same time, it also enhances the computational power of CNN to a certain extent.

[0030] In this embodiment, the structure of the convolutional layer Conv is a CBS structure, namely Convolution-Batch Normalization-SiLU.

[0031] In some implementations of this embodiment, such as Figure 4 As shown, the fourth convolutional layer combining the third C2f and DCNv2 includes a first input unit, a first convolutional unit, a first feature splitting unit, a multi-layer DCNv2 bottleneck structure, a first splicing unit, a second convolutional unit, and a first output unit. The second feature map is input into the first input unit and processed sequentially by the first convolutional unit, the first feature splitting unit, and the multi-layer DCNv2 bottleneck structure. The feature maps obtained after processing by the first convolutional unit, the first feature splitting unit, and the multi-layer DCNv2 bottleneck structure are fused in the first concatenation unit. The feature map obtained after fusion by the first concatenation unit is processed by the second convolutional unit and output by the first output unit to obtain the third feature map.

[0032] In some embodiments of this example, the multi-layer DCNv2 bottleneck structure includes multiple DCNv2 bottleneck structures stacked sequentially, such as... Figure 5 As shown, the DCNv2 bottleneck structure includes a third convolutional unit and a DCNv2 unit stacked sequentially.

[0033] In some embodiments of this example, the DCNv2 unit is a deformable convolutional DCN. The deformable convolutional DCN is not a simple rectangle, but rather its kernel shape should adaptively change according to the target being detected (vertebral position), thus enhancing the detection of targets at the vertebral position. However, this deformable convolution method does not directly change the shape of the convolutional kernel, but rather modifies the sampling result. This indirect method achieves a certain degree of modification to the shape of the convolutional kernel. In the DCNv2 unit, It is a set of pixels being sampled, which can be used right The sampling results were expanded and thus altered, with a weighting coefficient added to each sampling point. Further changes to the sampling results It is in A decimal between.

[0034] (1) in, For learnable offsets, For the predefined base sampling offset, These are the sampling points of the feature map obtained after processing by the DCNv2 unit. It is a set of pixels that are sampled. The weights of the convolution kernel for the DCNv2 unit. The shape of the convolution kernel for the DCNv2 unit. This is the function for the sampling results.

[0035] In some implementations of this embodiment, such as Figure 6As shown, the fifth convolutional layer combining the fourth C2f and FADC includes a second input unit, a third convolutional unit, a second feature splitting unit, a multi-layer FADC bottleneck structure, a second splicing unit, a fourth convolutional unit, and a second output unit. The third feature map is input into the second input unit and processed sequentially by the third convolution unit, the second feature splitting unit, and the multi-layer FADC bottleneck structure. The feature maps obtained after processing by the third convolution unit, the second feature splitting unit, and the multi-layer FADC bottleneck structure are fused in the second concatenation unit. The feature map obtained after fusion by the second concatenation unit is processed by the fourth convolution unit and output by the second output unit to obtain the fourth feature map.

[0036] In some embodiments of this example, the multilayer FADC bottleneck structure includes multiple FADC bottleneck structures stacked sequentially, such as... Figure 7 As shown, the FADC bottleneck structure includes a fourth convolutional unit and an adaptive dilated convolutional unit stacked sequentially.

[0037] In some embodiments of this example, the adaptive dilated convolution unit can expand the receptive field by inserting gaps between consecutive elements, and dynamically adjusts the dilation rate in space based on local frequency by introducing Frequency Adaptive Dilated Convolution (FADC). In vertebral detection and localization, due to scoliosis, the shape of the vertebrae can be deformed in various ways, including bending, expansion, and compression. Therefore, expanding the receptive field of the convolution plays a role in vertebral localization. The adaptive dilated convolution unit in the FADC bottleneck structure includes three strategies: Adaptive Dilation Rate (AdaDR), Adaptive Kernel (AdaKern), and Frequency Selection (FreqSelect).

[0038] Compared to ordinary dilated convolution, the FADC bottleneck structure introduces an adaptive dilation rate to balance the dilation rate, enabling optimal dilation for each pixel. The calculation formula is as follows: (2) in, This indicates the kernel size of the adaptive dilated convolution unit. The weight parameters represent the kernel weights of the adaptive dilated convolutional unit. This represents the feature map obtained after processing the second feature splitting unit of the input. Offset The corresponding pixel value, For adaptive expansion rate, the parameter can be set as follows: The convolutional layers are used for prediction.

[0039] The adaptive hole rate strategy described above optimizes both effective bandwidth and receptive field. Next, an adaptive kernel is introduced to dynamically adjust high-frequency and low-frequency components. and These represent the dynamic weights of the high-frequency and low-frequency component channels, respectively. In the bottleneck structure of the FADC, this is predicted through global pooling and convolution. These two parameters are adaptively and dynamically adjusted based on the contextual information of the input image. This strategy enhances the model's ability to capture details such as vertebral shape and size to some extent.

[0040] (3) in, and These represent the dynamic weights of the high-frequency component channel and the low-frequency component channel, respectively. The weight parameters of the convolution kernel of the adaptively dilated convolutional unit after adaptive kernel adjustment are represented. These represent the weighting parameters for the high-frequency component channel and the low-frequency component channel, respectively.

[0041] Frequency selection aims to enhance the receiving field by utilizing the high-frequency and low-frequency components in the balanced characteristic representation. The calculation formula is as follows: (4) (5) in, It is an operation of the inverse fast Fourier transform. It is a binary mask designed to extract relevant frequencies. For Fourier transform, It is a characteristic of frequency balance after frequency selective learning. For adaptive weighting coefficients, For a predefined total number of frequency bands, Frequency point Distance to the frequency center , All are frequency thresholds.

[0042] In traditional FPN fusion mechanisms, feature map processing often only upsamples smaller feature maps, frequently neglecting larger ones. For example... Figure 8As shown, in this embodiment, some splicing layers in the neck network are replaced with sampling layers. For large-sized feature maps, they need to be processed by convolutional layers, their channel count adjusted, and then downsampled. For small-sized feature maps, they need to be processed by convolutional layers, their channel count adjusted, and then upsampled. In this way, the third, fourth, and fifth feature maps will be converted to the same size, thus facilitating splicing.

[0043] (6) in, These are the fifth feature map, the fourth feature map, and the third feature map, respectively.

[0044] In the neck network, a DSS layer is added. After convolution of the images input from the fourth and fifth feature maps, the images are fed into the upsampler Dysample. Dysample, as an ultra-lightweight and efficient upsampler, can effectively reduce the computational resource consumption of the DSS layer and has lower inference latency, memory usage, and parameter count. Subsequently, images of the same size from the third, fourth, and fifth feature maps are stacked together and then subjected to 3D convolution (3DConv). By combining features at multiple scales in this way, it is highly targeted to handle different size variations and rotation angles of the vertebrae.

[0045] The calculation formula for the feature map obtained after DSS layer processing is: (7) (8) in, The width is obtained after processing with the DSS layer. and height Feature map, It is a proportional parameter that controls the standard deviation of the two-dimensional Gaussian filter in convolution. Width and height Gaussian weights of the feature map, Width and height The semantic information of the feature map.

[0046] In some implementations of this embodiment, such as Figure 3 As shown, the head network includes several detection heads (OBBhead), which are a first detection head, a second detection head, and a third detection head; The sixth feature map is input into the first detection head to obtain the first prediction box of the vertebral position; The seventh feature map is input into the second detection head to obtain a second prediction box for the vertebral position; The eighth feature map is input to the third detection head to obtain the third prediction box of the vertebral position; The first, second, and third prediction boxes for the vertebral position are merged to obtain the final detection box for the vertebral position.

[0047] Therefore, the rotational target detection method is used to first detect and locate the position of the vertebrae so that the corresponding Cobb angle can be calculated later.

[0048] Step S103: Obtain the Cobb angle based on the positions of several vertebrae.

[0049] In some embodiments of this example, the identification of vertebrae and the localization of the vertex coordinates of each vertebra are accomplished using YOLOv8's rotated target detection, specifically the OBB head. After rotated target detection, various formats of bounding box information can be obtained; here, the format of locating the bounding box by its four corner points is selected. These coordinate points are rearranged in a certain order to obtain the coordinates of the four corner points.

[0050] After obtaining the above coordinates, a certain number of abnormal location detection boxes need to be removed. First, during the preprocessing of the full-frontal X-ray images of the human spine, all full-frontal X-ray images of the human spine have been cropped, retaining only the spinal region. Therefore, most of the abnormal noise caused by other features to target detection can be eliminated. In addition, based on the cropping preprocessing, a certain confidence threshold is set. That is, when the confidence of a detection box is lower than the threshold, the detection box is discarded.

[0051] Using the above method, relatively accurate vertebral rotation target detection results of the entire spine can be obtained. Subsequently, it is necessary to analyze the corner points on the same side based on each corner point. The coordinates are reordered from top to bottom to ensure that there is no interference between corner coordinates. After determining the coordinates of each set of corner points, the slope of the upper and lower boundary corners of each final detection box is calculated. After sorting, a slope list is generated. Additionally, since the final detection box is a rectangle, and its upper and lower boundaries are always parallel, only the slope of the upper boundary needs to be calculated.

[0052] (9) After obtaining the list of slopes, the entire list is traversed, the angle between each pair of slopes is calculated, and the index of the angle between each pair of slopes is recorded, generating a list of angles. Angles with a value greater than or equal to a sorting threshold are designated as Cobb angles, calculated using the following formula: (10) in, For a pair of slopes.

[0053] In addition, when obtaining the Cobb angle, three rules need to be followed to exclude outliers: (1) If the indexes of the two included angles overlap and the number of overlapping indices exceeds 1, the included angle with the smaller span is discarded and the larger angle of the next larger span is taken in order; (2) If the larger included angle is an obtuse angle exceeding 90 degrees, the supplementary angle of the included angle is added to the included angle list and the included angles are compared; (3) If the coordinates of the boundary corner points overlap, that is, the denominator in the slope calculation formula is 0, the slope is directly reset to infinity to avoid anomalies when calculating the included angle.

[0054] In some embodiments of this example, the Cobb angle is evaluated using an accuracy rate. Recall rate and mean precision Assess the accuracy of Cobb angles.

[0055] (11) (12) (13) in, For a real example, As a false positive example, For false negatives, For the first The average precision of the class This represents the number of categories in the object detection task.

[0056] To evaluate the accuracy of deep learning models, a series of recognized metrics were used. These included Precision (P), Recall (R), mAP50, and mAP50-95. The Precision-Recall Curve (PR Curve) describes the relationship between recall and precision, with recall on the x-axis and precision on the y-axis. Average Precision (AP) calculates the area under the PR Curve, aiming to balance the reflection of P and R metrics to some extent. Mean Average Precision (mAP) calculates the average precision based on AP, providing a more comprehensive reflection of model performance. Furthermore, mAP50 refers to the mAP calculated with an intersection greater than the joint (IoU) threshold of 0.50, which is more valuable for objects with obvious features that are relatively easy to detect. mAP50-95 represents the average mAP calculated across different IoU values ​​with the same traversal length between 0.50 and 0.95, offering a more meaningful reference for the overall model performance.

[0057] To investigate the performance of deep learning models in vertebral rotation target detection before predicting the Cobb angle of scoliosis, evaluation metrics were trained and calculated for the YOLOv8n-obb model with different modules added, and ablation experiments were conducted. The deep learning model based on the improved YOLOv8n-obb model consistently performed well in the ablation experiments. Compared to the baseline YOLOv8n-obb model, the model maintained an approximately 10% advantage in mAP50 score and a more than 13% advantage in mAP50-95 score. In terms of model size and parameter count, the deep learning model was only 2% larger than the baseline, indicating that the deep learning model achieved better recognition rates with a relatively smaller scale. Furthermore, the loss of the deep learning model decreased less over the same training time. Therefore, the deep learning model demonstrated strong competitiveness in the vertebral recognition and detection task for scoliosis.

[0058] To more comprehensively evaluate the model's effectiveness in predicting the Cobb angle of scoliosis, the predicted Cobb angle was compared with the actual Cobb angle annotated by a professional physician. To measure this difference, Mean Absolute Error (MAE) and symmetric mean absolute percentage error (SMAPE) were introduced to evaluate the predicted and actual values ​​of the Cobb angle on each spinal X-ray image. The actual calculation formula is as follows: (14) (15) in, Represented as predicted values, therefore, Similarly, the actual value is ,have For the three principal Cobb angles, superior Cobb angle, and inferior Cobb angle along a spine, the SMAPE value can be calculated separately. However, for each X-ray image, there may not actually be three Cobb angles; the actual value and the true value in the denominator cannot both be 0. Therefore, when this occurs, the SMAPE value for that angle is set to 0. Finally, the SMAPE values ​​from all images are averaged, which provides a reliable evaluation of the model's Cobb angle prediction capability.

[0059] In the Cobb angle prediction experiment, all the above models were evaluated and tested. MAE1, MAE2, and MAE3 represent the absolute average errors of the main Cobb angle, upper Cobb angle, and lower Cobb angle, respectively; MAE_Avg represents the average MAE of the three angles; and SMAPE represents the SMAPE of each Cobb angle relative to the standard value in the entire Cobb angle test set. The deep learning model outperformed the baseline in all metrics, indicating that the deep learning model is quite competitive in Cobb angle prediction.

[0060] According to the second aspect of this application, such as Figure 9 As shown in the figure, this embodiment provides an automatic measurement system for the Cobb angle of scoliosis based on YOLOv8n-obb, which includes: The first acquisition module is used to acquire a spinal image, the spinal image including a number of vertebrae; The second acquisition module is used to input the spinal image into a pre-trained deep learning model based on YOLOv8n-obb, and output several vertebral positions. The deep learning model includes a backbone network, a cervical network, and a head network. The backbone network is a multi-scale convolutional module combining C2f, DCNv2, FADC, and SPPF, and is used to acquire multi-scale features of the spinal image. The cervical network is a feature fusion module combining DSS, attention layer, and sampling layer, and is used to perform feature fusion on the multi-scale features. The head network includes multiple detection heads, and is used to perform rotational target detection based on the multi-scale features fused to acquire several vertebral positions. The pre-trained deep learning model based on YOLOv8n-obb is trained using several spinal images used for pre-training. The third acquisition module is used to obtain the Cobb angle based on the positions of several vertebrae.

[0061] Specifically, this embodiment corresponds one-to-one with the above method embodiments. The functions of each module have been described in detail in the corresponding method embodiments, so they will not be repeated here.

[0062] According to a third aspect of this application, this embodiment provides a computer-readable storage medium having a computer program stored thereon, the computer program including executable instructions that, when executed by a processor, implement the method described above.

[0063] The present invention can implement all or part of the processes in the above methods, or it can be accomplished by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or system capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0064] According to the fourth aspect of this application, such as Figure 10 As shown, an electronic device is provided, comprising: One or more processors; Memory is used to store executable instructions for the processor, which, when executed by one or more processors, cause one or more processors to implement the methods described above.

[0065] Electronic devices are manifested in the form of general-purpose computing devices. Components of an electronic device may include, but are not limited to: at least one processor, at least one memory, and a bus connecting different system components (including memory and processor).

[0066] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the computer system, connecting all parts of the computer system through various interfaces and lines.

[0067] Memory can be used to store computer programs and / or modules. The processor implements various functions of the computer system by running or executing the computer programs and / or modules stored in the memory, and by accessing data stored in the memory. Memory can mainly include a program storage area and a data storage area. The program storage area can store the operating system and at least one application program required for a function (e.g., sound playback, image playback, etc.); the data storage area can store data created based on the use of the mobile phone (e.g., audio data, video data, etc.). Furthermore, memory can include high-speed random access memory, and can also include non-volatile memory, such as hard disks, RAM, plug-in hard disks, SmartMedia Cards (SMC), Secure Digital (SD) cards, Flash Cards, at least one disk storage device, flash memory device, or other volatile solid-state storage devices.

[0068] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, servers, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage and memory) containing computer-usable program code.

[0069] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), servers, and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A system that specifies functions in one or more boxes.

[0070] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including an instruction set implemented in a process. Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0071] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0072] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0073] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0074] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An automatic measurement method for the Cobb angle of scoliosis based on YOLOv8n-obb, characterized in that, include: Acquire a spinal image, the spinal image including several vertebrae; The spinal image is input into a pre-trained deep learning model based on improved YOLOv8n-obb, which outputs the positions of several vertebrae. The deep learning model includes a backbone network, a cervical network, and a head network. The backbone network is a multi-scale convolutional module combining C2f, DCNv2, FADC, and SPPF, used to acquire multi-scale features of the spinal image. The cervical network is a feature fusion module combining DSS, attention layer, and sampling layer, used to perform feature fusion on the multi-scale features. The head network includes multiple detection heads, used to perform rotational target detection based on the multi-scale features fused to obtain the positions of several vertebrae. The pre-trained deep learning model based on improved YOLOv8n-obb is trained using several spinal images used for pre-training. The Cobb angle is obtained based on the positions of several vertebrae.

2. The automatic measurement method for the Cobb angle of scoliosis based on YOLOv8n-obb as described in claim 1, characterized in that: The multi-scale convolutional module includes a first convolutional layer, a second convolutional layer combining a first C2f, a third convolutional layer combining a second C2f, a fourth convolutional layer combining a third C2f and DCNv2, a fifth convolutional layer combining a fourth C2f and FADC, and an SPPF layer. The second convolutional layer combining the first C2f outputs a first feature map, the third convolutional layer combining the second C2f outputs a second feature map, the fourth convolutional layer combining the third C2f and DCNv2 outputs a third feature map, the fifth convolutional layer combining the fourth C2f and FADC outputs a fourth feature map, and the SPPF layer outputs a fifth feature map.

3. The automatic measurement method for the Cobb angle of scoliosis based on YOLOv8n-obb as described in claim 2, characterized in that: The fourth convolutional layer combining the third C2f and DCNv2 includes a first input unit, a first convolutional unit, a first feature splitting unit, a multi-layer DCNv2 bottleneck structure, a first splicing unit, a second convolutional unit, and a first output unit. The second feature map is input into the first input unit and processed sequentially by the first convolutional unit, the first feature splitting unit, and the multi-layer DCNv2 bottleneck structure. The feature maps obtained from the first convolutional unit, the first feature splitting unit, and the multi-layer DCNv2 bottleneck structure are then fused in the first concatenation unit. The fused feature map is then processed by the second convolutional unit, and finally output by the first output unit to obtain the third feature map. The multi-layer DCNv2 bottleneck structure includes multiple DCNv2 bottleneck structures stacked sequentially, and the DCNv2 bottleneck structure includes a third convolutional unit and a DCNv2 unit stacked sequentially. The DCNv2 unit is a deformable convolution, which changes the shape of the convolution kernel by changing the sampling results.

4. The automatic measurement method for the Cobb angle of scoliosis based on YOLOv8n-obb as described in claim 2, characterized in that: The fifth convolutional layer, which combines the fourth C2f and FADC, includes a second input unit, a third convolutional unit, a second feature splitting unit, a multi-layer FADC bottleneck structure, a second splicing unit, a fourth convolutional unit, and a second output unit. The third feature map is input to the second input unit and processed sequentially by the third convolutional unit, the second feature splitting unit, and the multi-layer FADC bottleneck structure. The feature maps obtained from the third convolutional unit, the second feature splitting unit, and the multi-layer FADC bottleneck structure are then fused in the second concatenation unit. The fused feature map is then processed by the fourth convolutional unit, and finally output by the second output unit to obtain the fourth feature map. The multi-layer FADC bottleneck structure includes multiple FADC bottleneck structures stacked sequentially, and each FADC bottleneck structure includes a fourth convolutional unit and an adaptive dilated convolutional unit stacked sequentially. The adaptive dilated convolutional unit dynamically adjusts the dilation rate in space by adaptively selecting the dilation rate, adaptive kernel, and frequency.

5. The automatic measurement method for the Cobb angle of scoliosis based on YOLOv8n-obb as described in claim 2, characterized in that: The feature fusion module includes a DSS layer, an attention layer, a sixth convolutional layer, a seventh convolutional layer, an eighth convolutional layer, a ninth convolutional layer, a tenth convolutional layer, an eleventh convolutional layer, a fifth C2f layer, a sixth C2f layer, a seventh C2f layer, an eighth C2f layer, a first sampling layer, a second sampling layer, a first splicing layer, and a second splicing layer. The first feature map is input into the sixth convolutional layer for processing; The second feature map is input into the eighth convolutional layer to obtain the feature map, the fifth feature map is input into the tenth convolutional layer to obtain the feature map, and the third feature map are fused in the second sampling layer, processed by the sixth C2f, and then input into the seventh convolutional layer for further processing. The feature map obtained after processing by the sixth convolutional layer, the second feature map, and the feature map obtained after processing by the seventh convolutional layer are fused in the first sampling layer and then processed by the fifth C2f. After the fourth feature map is input into the DSS layer, it is fused with the feature map obtained after the fifth C2f processing in the attention layer. The feature map obtained after the fusion in the attention layer is the sixth feature map. The feature map obtained after the fifth C2f processing is input into the ninth convolutional layer, and then fused with the feature map obtained after the seventh convolutional layer in the first splicing layer. After being processed by the seventh C2f, the feature map obtained by the seventh C2f processing is the seventh feature map. The feature map obtained after the seventh C2f processing is input into the eleventh convolutional layer for processing, and the fifth feature map is input into the tenth convolutional layer for processing. The feature map obtained from the eleventh convolutional layer and the feature map obtained from the tenth convolutional layer are fused in the second splicing layer, and then processed by the eighth C2f to obtain the eighth feature map.

6. The automatic measurement method for the Cobb angle of scoliosis based on YOLOv8n-obb as described in claim 5, characterized in that: The head network includes a first detection head, a second detection head, and a third detection head; The sixth feature map is input into the first detection head to obtain the first prediction box of the vertebral position; The seventh feature map is input into the second detection head to obtain a second prediction box for the vertebral position; The eighth feature map is input into the third detection head to obtain the third prediction box of the vertebral position; The first, second, and third prediction boxes for the vertebral position are merged to obtain the final detection box for the vertebral position.

7. The automatic measurement method for the Cobb angle of scoliosis based on YOLOv8n-obb as described in claim 6, characterized in that, The process of obtaining the Cobb angle based on several vertebral positions includes: Obtain the slope of the final detection box at each vertebra position. If the coordinates of the boundary corners of the final detection box coincide, reset the slope to infinity. Sort all the slopes to generate a slope list; Based on the slope list, calculate the angle between each pair of slopes. The angles that are greater than or equal to the sorting threshold are Cobb angles. If the index vertebrae of two angles overlap and the number of overlapping indices exceeds 1, the angle with the smaller span is discarded and the larger angle of the next position is taken. If the angle is an obtuse angle exceeding 90 degrees, the supplementary angle of the angle is used to replace the angle.

8. An automatic measurement system for the Cobb angle of scoliosis based on YOLOv8n-obb, characterized in that, include: The first acquisition module is used to acquire a spinal image, the spinal image including a number of vertebrae; The second acquisition module is used to input the spinal image into a pre-trained deep learning model based on YOLOv8n-obb, and output several vertebral positions. The deep learning model includes a backbone network, a cervical network, and a head network. The backbone network is a multi-scale convolutional module combining C2f, DCNv2, FADC, and SPPF, and is used to acquire multi-scale features of the spinal image. The cervical network is a feature fusion module combining DSS, attention layer, and sampling layer, and is used to perform feature fusion on the multi-scale features. The head network includes multiple detection heads, and is used to perform rotational target detection based on the multi-scale features fused to acquire several vertebral positions. The pre-trained deep learning model based on YOLOv8n-obb is trained using several spinal images used for pre-training. The third acquisition module is used to obtain the Cobb angle based on the positions of several vertebrae.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, The computer program includes executable instructions that, when executed by a processor, implement the method of any one of claims 1-7.

10. An electronic device, characterized in that, include: One or more processors; A memory for storing executable instructions of the processor, which, when executed by the one or more processors, cause the one or more processors to perform the method according to any one of claims 1-7.

Images