Spinal corner detection methods, devices and electronic devices

Abstract

This application relates to a method, apparatus, and electronic device for detecting spinal corner points. The method includes: acquiring a spinal image to be detected and a preset spinal template database, wherein the spinal template database includes several sets of first spinal templates; identifying a group of annotation points in the spinal image to be detected, wherein the group of annotation points includes multiple marker points pre-annotated on the spinal image to be detected; determining a second spinal template matching the spinal image to be detected based on the group of annotation points and the several sets of first spinal templates; and mapping the spinal corner points of the second spinal template to the spinal image to be detected according to a preset mapping rule, thereby determining the spinal corner points of the spinal image to be detected. This application solves the technical problem that existing corner detection technologies may fail to detect spinal corner points due to exposure metering effects.

Description

Technical Field

[0001] This application relates to the field of medical image processing, and in particular to a method, apparatus, and electronic device for detecting spinal corner points. Background Technology

[0002] Scoliosis is a common spinal disorder, and its severity is mostly assessed by the angle of scoliosis, which is often measured clinically using the Cobb angle method. Currently, whether Cobb angle measurement is performed manually or using automated algorithms, it is necessary to detect the spinal angle points before calculating the Cobb angle. Therefore, spinal angle point detection on the digital X-ray images of the examinee is essential.

[0003] Currently, most methods for detecting spine corners in X-ray images (whether deep learning or traditional methods) are based on feature point extraction followed by identification. Deep learning methods often utilize convolutional neural networks (such as U-Net and ResNet) to extract heatmaps, and then regress the spine center and its four corners to achieve corner extraction. Other methods typically employ statistical models or segment the spine to obtain an initial region, then use multiple feature detection operators to extract features within this defined region, and finally use machine learning algorithms for classification.

[0004] However, in real-world scenarios, due to the influence of exposure measurement, traditional methods for feature point or spine corner detection may fail to detect feature points or spine corners, leading to detection failure.

[0005] There is currently no effective solution to the technical problem in related technologies where the spine corner point may be undetectable due to the influence of exposure metering. Summary of the Invention

[0006] This embodiment provides a method, apparatus, electronic device, and storage medium for detecting spinal corner points, in order to solve the technical problem in related technologies that spinal corner points may not be detected due to the influence of exposure measurement.

[0007] Firstly, this embodiment provides a method for detecting spinal corner points, the detection method comprising:

[0008] The spine image to be detected and a preset spine template database are obtained, wherein the spine template database includes several sets of first spine templates;

[0009] Identify a group of marker points in the spinal image to be detected, wherein the group of marker points includes multiple marker points pre-marked on the spinal image to be detected;

[0010] Based on the set of marker points and the plurality of sets of first spinal templates, a second spinal template that matches the spinal image to be detected is determined;

[0011] According to the preset mapping rules, the spinal corner points of the second spinal template are mapped to the spinal image to be detected, so as to determine the spinal corner points of the spinal image to be detected.

[0012] In some embodiments, determining a second spinal template that matches the spinal image to be detected based on the set of marker points and the plurality of first spinal templates includes:

[0013] Based on the set of annotation points, determine the multiple sets of first spine templates that have the highest similarity to the spine image to be detected from the multiple sets of first spine templates;

[0014] The multiple sets of first spine templates are processed according to preset processing rules, and a second spine template matching the spine image to be detected is obtained based on the processing results.

[0015] In some embodiments, determining the plurality of first spine templates with the highest similarity to the spine image to be detected from the plurality of first spine templates based on the set of annotation points includes:

[0016] The image of the spine to be detected is segmented according to a preset image segmentation rule to obtain the detected spinal segment, wherein the detected spinal segment is the spinal segment of the image of the spine to be detected, and the marker point is the spinal corner point located at the endpoint of the detected spinal segment.

[0017] Determine several first transformation matrices between the group of annotation points and the first matching point group of the several groups of first spinal templates, wherein the first matching point group includes multiple spinal corner points in the first spinal template that correspond to the positions of the multiple annotation points;

[0018] Based on the aforementioned first transformation matrices, the spinal segments of the aforementioned groups of first spinal templates are respectively converted into several control spinal segments;

[0019] Based on the detected spinal segment and the several control spinal segments, a plurality of first spinal templates with the highest similarity to the spinal image to be detected are determined.

[0020] In some embodiments, determining the multiple sets of first spinal templates with the highest similarity to the spinal image to be detected, based on the detected spinal segment and the plurality of control spinal segments, includes:

[0021] The number of difference pixels between the several control spinal segments and the detected spinal segment is counted respectively, and the control spinal segments with the smallest number of difference pixels are determined based on the statistical results.

[0022] Multiple sets of first spinal templates corresponding to the multiple control spinal segments are identified as the multiple sets of first spinal templates with the highest similarity to the spinal image to be detected.

[0023] In some embodiments, determining the multiple sets of first spinal templates with the highest similarity to the spinal image to be detected, based on the detected spinal segment and the plurality of control spinal segments, includes:

[0024] Calculate the L2 norm of the difference between the detected spinal segment and the several control spinal segments respectively, and determine the several control spinal segments with the smallest L2 norm based on the calculation results;

[0025] Multiple sets of first spinal templates corresponding to the multiple control spinal segments are identified as the multiple sets of first spinal templates with the highest similarity to the spinal image to be detected.

[0026] In some embodiments, processing the multiple sets of first spinal templates according to preset processing rules, and obtaining a second spinal template matching the spinal image to be detected based on the processing results, includes:

[0027] The average value of the spinal corner points of the multiple sets of first spinal templates is calculated according to a preset processing algorithm, and the spinal corner points of the second spinal template are determined based on the processing result.

[0028] The spinal segment of the second spinal template is determined based on the spinal corner points of the second spinal template.

[0029] In some embodiments, mapping the spinal corner points of the second spinal template to the spinal image to be detected according to a preset mapping rule, in order to determine the spinal corner points of the spinal image to be detected, includes:

[0030] The second spinal template is registered to the spinal image to be detected using a preset registration algorithm, and the deformation field is determined based on the registration process.

[0031] The spinal corner points of the second spinal template are mapped onto the spinal image to be detected based on the deformation field, and the spinal corner points of the spinal image to be detected are determined based on the mapping result.

[0032] In some embodiments, registering the second spinal template to the spinal image to be detected using a preset registration algorithm, and determining the deformation field based on the registration process includes:

[0033] A second transformation matrix is ​​determined between the second matching point group of the second spine template and the annotation point group of the spine image to be detected, wherein the second matching point group includes multiple spine points in the second spine template corresponding to the positions of the multiple annotation points;

[0034] The spinal segments of the second spinal template are converted into registered spinal segments according to the second transformation matrix;

[0035] The registered spinal segment is registered to the spinal segment of the spinal image to be detected using a non-rigid registration algorithm;

[0036] The deformation field is determined based on the registration process.

[0037] Secondly, this embodiment provides a spinal corner detection device, the detection device comprising:

[0038] The data acquisition module is used to acquire the spinal image to be detected and a preset spinal template database, wherein the spinal template database includes several sets of first spinal templates.

[0039] A corner recognition module is used to identify a group of marked points in the spinal image to be detected, wherein the group of marked points includes multiple pre-marked points on the spinal image to be detected;

[0040] The template determination module is used to determine a second spine template that matches the spine image to be detected based on the group of marker points and the plurality of groups of first spine templates;

[0041] The corner mapping module is used to map the spine corners of the second spine template to the spine image to be detected according to a preset mapping rule, so as to determine the spine corners of the spine image to be detected.

[0042] Thirdly, this embodiment provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the spinal corner detection method described in the first aspect above.

[0043] Fourthly, this embodiment provides a storage medium storing a computer program that, when executed by a processor, implements the spinal corner detection method described in the first aspect above.

[0044] Compared with related technologies, the spinal corner detection method provided in this embodiment uses pre-annotated spinal corners to match and determine the closest second spinal template from a database containing existing spinal template data. The spinal corners of the second spinal template are then converted and mapped according to certain rules to be used as the spinal corners of the spinal image to be detected. This solves the technical problem that existing corner detection technologies may not be able to detect spinal corners due to exposure metering, and achieves the technical effect of accurately and stably extracting spinal corners from spinal images.

[0045] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description

[0046] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0047] Figure 1 This is a hardware structure block diagram of the terminal of the spinal corner detection method in this embodiment.

[0048] Figure 2 This is a flowchart of the spinal corner detection method in this embodiment.

[0049] Figure 3 This is the image of the spine to be detected in this embodiment.

[0050] Figure 4 This is a schematic diagram of the segmentation results of the spine image to be detected in this embodiment.

[0051] Figure 5 This is an image of the thoracic vertebral segment mapped with spinal angles in this embodiment.

[0052] Figure 6 This is an image of a lumbar vertebral segment mapped with spinal angle points in this embodiment.

[0053] Figure 7 This is a schematic diagram showing the positions of the superior and inferior endplates of the vertebral segment in this embodiment.

[0054] Figure 8 This is a flowchart of the spinal corner detection method according to a preferred embodiment.

[0055] Figure 9 This is a schematic diagram of the manual marking of the lumbar vertebral segment corners in this preferred embodiment.

[0056] Figure 10 This is a schematic diagram illustrating the effect of corner mapping in a lateral lumbar spine image of this preferred embodiment.

[0057] Figure 11 This is a schematic diagram of the manual marking of the thoracic vertebral segment corners in this preferred embodiment.

[0058] Figure 12 This is a structural block diagram of the spinal corner detection device in this embodiment. Detailed Implementation

[0059] To better understand the purpose, technical solution, and advantages of this application, the application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0060] Unless otherwise defined, the technical or scientific terms used in this application shall have the general meaning as understood by one of ordinary skill in the art to which this application pertains. Words such as “a,” “an,” “an,” “the,” “the,” and “these,” used in this application, do not indicate quantitative limitation and may be singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps or modules (units) is not limited to the listed steps or modules (units) but may include steps or modules (units) not listed, or may include other steps or modules (units) inherent to such processes, methods, products, or devices. The terms “connected,” “linked,” and “coupled,” used in this application, are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. The term “multiple” used in this application refers to two or more. The "and / or" operator describes the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: A alone, A and B simultaneously, and B alone. Typically, the character " / " indicates that the objects before and after it are in an "or" relationship. The terms "first," "second," and "third," etc., used in this application are merely for distinguishing similar objects and do not represent a specific ordering of the objects.

[0061] The method embodiments provided in this example can be executed on a terminal, computer, or similar computing device. For example, it can run on a terminal. Figure 1 This is a hardware structure block diagram of the terminal of the spine corner detection method in this embodiment. For example... Figure 1 As shown, a terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 and a memory 104 for storing data are also included. The processor 102 may be, but is not limited to, a microprocessor (MCU) or a programmable logic device (FPGA). The terminal may also include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that… Figure 1The structure shown is for illustrative purposes only and does not limit the structure of the terminal described above. For example, the terminal may also include components that are larger than... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown are illustrated.

[0062] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the spinal corner detection method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0063] The transmission device 106 is used to receive or send data via a network. This network includes a wireless network provided by the terminal's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 can be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0064] This embodiment provides a method for detecting spinal corner points. Figure 2 This is a flowchart of the spinal corner detection method in this embodiment, as follows: Figure 2 As shown, the process includes the following steps:

[0065] Step S210: Obtain the spinal image to be detected and a preset spinal template database, wherein the spinal template database includes several sets of first spinal templates.

[0066] Specifically, the detection equipment first acquires an image of the spine to be detected. This image is an X-ray image of the human body, thus displaying the human spine. The equipment also acquires a pre-set database containing several sets of first spine templates. Each set of templates contains a mask of a specific spinal segment and multiple corresponding spinal corner points. It's important to note that a spinal segment is a section of the human spine, such as the lumbar or thoracic vertebrae. Correspondingly, in segmenting the spinal image, the segmentation result is the target spinal segment mask, which represents the segmented target spinal segment in the image. The spinal segment mask is then represented as a specific-shaped spinal segment graphic. Each spinal segment is divided into multiple vertebral segments, and the vertices of each segment constitute the spinal corner points of that segment. Specifically, refer to... Figure 3 and Figure 4 .in, Figure 3 This is the image of the spine to be detected in this embodiment. Figure 4 This is a schematic diagram of the segmentation results of the spine image to be detected in this embodiment. Figure 4 The segmentation result shown is a mask graphic of the lumbar vertebral segment in the spinal image to be detected. For example, a spinal template database is constructed by collecting existing spinal data; for instance, various shapes of spinal segment masks and corresponding spinal corner point data can be collected from the network.

[0067] Step S220: Identify a group of labeled points in the spinal image to be detected, wherein the group of labeled points includes multiple labeled points pre-annotated on the spinal image to be detected.

[0068] Specifically, after acquiring the image of the spine to be detected, the detection device identifies a group of marker points in the spine image. This group of marker points consists of multiple marker points pre-annotated on the spine image. For example, these marker points are some spinal corner points manually marked on the spine image, so the group of marker points reflects the position of some spinal corner points of the corresponding spinal segment; of course, feature point detection algorithms can also be used to identify some spinal corner points as marker points.

[0069] Step S230: Based on the set of marker points and several sets of first spine templates, determine a second spine template that matches the spine image to be detected.

[0070] Specifically, the detection device uses the set of markers in the spinal image to be detected as the matching basis, matches a first spinal template similar to the spinal image to be detected from the spinal template database, and finally determines a second spinal template that matches the spinal image to be detected. It is important to note that the matching second spinal template refers to the spinal template whose spinal segment mask is most similar to the spinal segment mask in the spinal image to be detected within a limited range of spinal template data. Therefore, the second spinal template can be the first spinal template in the database that best matches the spinal image to be detected, or it can be a spinal template in the database that has been transformed to better match the spinal image to be detected.

[0071] Step S240: According to the preset mapping rules, the spine corner points of the second spine template are mapped to the spine image to be detected, so as to determine the spine corner points of the spine image to be detected.

[0072] Specifically, after determining the second spinal template, the detection device maps the spinal corner points in the second spinal template to the spinal image to be detected using a preset mapping rule, and uses the mapped spinal corner points as the spinal corner points of the spinal image to be detected. For example, Figure 5 This is an image of the thoracic vertebral segment mapped with spinal angle points in this embodiment. Figure 6 This is an image of the lumbar vertebral segment mapped with spinal angle points in this embodiment. For example... Figure 5 and Figure 6 As shown, the dots in the spine image are the spine corner points. The spine corner point detection method in this embodiment can effectively extract the spine corner points from the spine image to be detected.

[0073] Through the above steps, all spinal corner points in a spinal image can be detected. Specifically, some spinal corner points are pre-marked in the spinal image to be detected, and a spinal template database is pre-established by collecting existing spinal corner point data. After acquiring the spinal image to be detected and the spinal template database, the detection terminal matches a relatively close first spinal template from the database based on the group of marker points on the spinal image to be detected, and finally determines the second spinal template that best matches the spinal image to be detected. Finally, the spinal corner points of the second spinal template are mapped to the spinal image to be detected according to a preset mapping rule, and used as the spinal corner points of the spinal image to be detected. The difference between this method and common spinal corner point detection methods in the prior art is that the prior art generally extracts feature points directly from the spinal image through deep learning or traditional methods. However, the extraction of these feature points is affected by the image exposure, which may result in the failure to detect the feature points and thus lead to task failure. In this embodiment, the spinal corner points of the spine image to be detected are not directly extracted from the spine image. Instead, by matching the pre-annotated spinal corner points from a database containing existing spinal template data, the closest second spinal template is determined. The spinal corner points of the second spinal template are then converted and mapped according to certain rules to serve as the spinal corner points of the spine image to be detected. Therefore, the detection concept of the spinal corner point detection method in this embodiment differs from that in existing technologies. Since the spinal corner points are not directly extracted from the spine image, they are not affected by the exposure of the spine image. Thus, this embodiment proposes a novel spinal corner point detection method, solving the technical problem that existing corner point detection technologies may fail to detect spinal corner points due to exposure measurement effects, achieving accurate and stable extraction of spinal corner points from the spine image. It should be noted that the accuracy of the spinal corner point detection method in this embodiment is mainly affected by the number of spinal templates in the database. When the number of spinal templates is small, the accuracy of the spinal corner point detection method is low; however, as the number of spinal templates in the database increases, the accuracy of the spinal corner point detection method increases.

[0074] In some embodiments, the step of determining a second spinal template that matches the spinal image to be detected, based on a set of marker points and several sets of first spinal templates, specifically includes:

[0075] Based on the annotation point group, the first spine template with the highest similarity to the spine image to be detected is determined from several groups of first spine templates;

[0076] Multiple sets of first spine templates are processed according to preset processing rules, and a second spine template matching the spine image to be detected is obtained based on the processing results.

[0077] Specifically, the detection device first determines, based on the set of marker points, multiple sets of first spine templates with the highest similarity to the spine image to be detected from the database. Similarity refers to the closeness between the spinal segment mask in the spine template and the spinal segment mask in the spine image to be detected. After determining the multiple sets of first spine templates with the highest similarity, the detection device processes these templates using preset processing rules to obtain the second spine template most similar to the spine image to be detected. For example, the preset processing rule could be to select the first spine template most similar to the spine image to be detected from the multiple sets of first spine templates as the second spine template, or it could be to calculate the second spine template by averaging the multiple sets of first spine templates.

[0078] Furthermore, in some embodiments, the step of determining the multiple sets of first spine templates with the highest similarity to the spine image to be detected from a plurality of sets of first spine templates based on annotation point groups specifically includes:

[0079] The spinal image to be detected is segmented according to a preset image segmentation rule to obtain the detected spinal segment, wherein the detected spinal segment is the spinal segment of the spinal image to be detected, and the marker point is the spinal corner point located at the endpoint of the detected spinal segment.

[0080] Determine several first transformation matrices between the group of annotation points and the first matching point group of several groups of first spine templates, wherein the first matching point group includes multiple spine corner points in the first spine template that correspond to the positions of multiple annotation points;

[0081] Based on several first transformation matrices, the spinal segments of several groups of first spinal templates are converted into several control spinal segments respectively;

[0082] Based on the detected spinal segment and several control spinal segments, multiple sets of first spinal templates with the highest similarity to the spinal image to be detected are identified.

[0083] Specifically, this embodiment provides a process for matching multiple sets of first spine templates from a database that are closest to the spine image to be detected.

[0084] First, the spinal segment to be detected is segmented from the spinal image to be detected according to preset image segmentation rules. A mask image for the detected spinal segment is then generated. For example, if the spinal image to be detected includes the complete spine, but only the lumbar segment needs to be detected, this step involves segmenting and extracting the lumbar segment from the spinal image to be detected and generating a lumbar segment mask. In this example, the lumbar segment is the detected spinal segment. The pre-annotated points are the four spinal corners located at the endpoints of the detected spinal segment. Specifically, a spinal segment consists of multiple vertebral segments, and each vertebral segment has a superior endplate and a inferior endplate. See the specific reference... Figure 7 , Figure 7This is a schematic diagram showing the positions of the superior and inferior endplates of the vertebral segment in this embodiment. Label 1 indicates the superior endplate of the vertebral segment, and label 2 indicates the inferior endplate. The four spinal corner points in this embodiment refer to the two corner points corresponding to the ends of the superior endplate of the first vertebral segment in the spinal segment detected in the spinal image, and the two corner points corresponding to the ends of the superior endplate of the first vertebral segment in the spinal segment below the spinal segment detected.

[0085] The marker point group is the set of these four spinal corner points. In the actual calculation process, the marker point group can be a vector composed of the coordinates of the four spinal corner points. The four spinal corner points in the spinal template corresponding to the marker point positions are denoted as matching points. The coordinates of the four matching points can be set to form a matching point group, which is also a vector. Therefore, the first transformation matrix between the marker point group and the first matching point group can be calculated. Specifically, the first transformation matrix between the marker point group and several first matching point groups is calculated separately. The number of first transformation matrices is the same as that of the first spinal template and they correspond to each other. For example, the marker point group can be denoted as P = (p1, p2, p3, p4), where p1, p2, p3, and p4 are the coordinates of the four spinal corner points, respectively. The first matching point group can be denoted as P' i = (p1, p2, p3, p4), where i = 1, 2, ..., n, representing the first matching point groups of different first spinal templates. Then P and P' i The relationship between them satisfies P = A` i *P` i , where A` i For P and P' i The first transformation matrix between them is such that the first transformation matrix A`1 corresponds to the first matching point group P`1, and the first transformation matrix A`2 corresponds to the first matching point group P`2. Therefore, the first transformation matrix between the marked point group and each first matching point group can be calculated using the above relationship.

[0086] Then, the spinal segment masks in the first spinal template are graphically transformed using a first transformation matrix. Specifically, each first spinal template contains corresponding spinal segment masks and first matching point groups. Each first matching point group has a corresponding first transformation matrix. Therefore, there is a correspondence between the first transformation matrix and the spinal segment masks. The corresponding spinal segment masks are transformed using the first transformation matrix to obtain new spinal segment masks. This new spinal segment mask is designated as the control spinal segment mask. Then, the mask differences between the detected spinal segment and the control spinal segment are compared to determine the multiple control spinal segment masks that are closest in shape to the detected spinal segment mask. Finally, the multiple first spinal templates to which these multiple control spinal segment masks belong are selected to determine the multiple sets of first spinal templates with the highest similarity to the spinal image to be detected.

[0087] Therefore, this embodiment provides a specific first spine template matching process based on marker point groups. This process involves matching multiple sets of first spine templates with the highest similarity to the spine image to be detected from a database.

[0088] Furthermore, in some specific embodiments, the step of determining multiple sets of first spinal templates with the highest similarity to the spinal image to be detected, based on the detected spinal segment and several control spinal segments, specifically includes:

[0089] The number of difference pixels between several control spinal segments and the detected spinal segments was counted, and the control spinal segments with the smallest number of difference pixels were determined based on the statistical results.

[0090] Multiple sets of first spinal templates corresponding to multiple control spinal segments were identified as the sets of first spinal templates with the highest similarity to the spinal image to be detected.

[0091] Specifically, this embodiment provides a process for calculating the mask similarity between the detected spinal segment and the control spinal segment, that is, directly counting the number of pixels that are inconsistent between the control spinal segment mask image and the detected spinal segment mask image.

[0092] For example, in one statistical approach, since the four spinal corner points in the marked point group correspond perfectly to the four spinal corner points in the first matching point group, these spinal corner points can be used as references. The detection spinal segment mask and the control spinal segment mask are overlapped. Because their shapes are not perfectly identical, some parts of the masks do not overlap. The number of pixels in the non-overlapping parts can then be counted; this number represents the number of differing pixels. The smaller the number of differing pixels, the more similar the mask shapes are between the detection spinal segment and the control spinal segment. Therefore, in this way, multiple control spinal segments most similar to the detection spinal segment can be identified, and ultimately, multiple sets of first spinal templates with the highest similarity to the image of the spine to be detected can be determined.

[0093] Furthermore, in some other specific embodiments, the step of determining multiple sets of first spinal templates with the highest similarity to the spinal image to be detected, based on the detected spinal segment and several control spinal segments, specifically includes:

[0094] Calculate the L2 norm of the difference between the detected spinal segment and several control spinal segments, and determine the control spinal segments with the smallest L2 norm based on the calculation results;

[0095] Multiple sets of first spinal templates corresponding to multiple control spinal segments were identified as the sets of first spinal templates with the highest similarity to the spinal image to be detected.

[0096] Specifically, this embodiment provides another procedure for calculating the mask similarity between the detected spinal segment and the control spinal segment, namely, calculating the L2 norm of the difference between the detected spinal segment mask and the control spinal segment mask.

[0097] For example, the detected spinal segment mask is denoted as 'a', and the spinal segment mask in the first spinal template is denoted as 'a''. i The second norm is denoted as V` i , where i = 1, 2, ..., n, correspond to different control spinal masks. V` i The formula for calculating V` is: i =||aA` i *a` i ||, where A` i *a` i This is the control spinal segment mask. Through the aforementioned V` i The calculation formula can calculate the L2 norm between the detected spinal segment mask and different control spinal segment masks. The L2 norm reflects the degree of difference between the detected spinal segment mask and the control spinal segment mask. The smaller the L2 norm, the smaller the difference between the two. Therefore, the L2 norm can be used to determine the multiple control spinal segment masks that are most similar to the detected spinal segment mask, and finally determine the multiple sets of first spinal templates that have the highest similarity to the spinal image to be detected.

[0098] In some embodiments, the step of processing multiple sets of first spinal templates according to preset processing rules, and obtaining a second spinal template that matches the spinal image to be detected based on the processing results specifically includes:

[0099] The average value of the spinal corner points of multiple sets of first spinal templates is calculated according to the preset processing algorithm, and the spinal corner points of the second spinal template are determined based on the processing results.

[0100] The spinal segment of the second spinal template is determined based on the spinal corner points of the second spinal template.

[0101] Specifically, this embodiment provides a specific processing rule. By processing multiple sets of first spine templates using this rule, a second spine template can be obtained. The spine template consists of two parts: spine corner points and spine segments. Therefore, in this processing rule, multiple sets of spine corner point data from multiple sets of second spine templates are first obtained. These spine corner point data are then averaged to obtain a single set of spine corner point data. This set of spine corner points serves as the spine corner points of the second spine template, and the shape enclosed by these spine corner points is used as the mask shape for the spine segments of the second spine template. Thus, the second spine template can finally be obtained. There are various methods for averaging the spine corner point data of the first spine template. For example, one method is to use the K-means algorithm. Specifically, the spine corner points of the N sets of second spine templates are divided into two categories. The cluster center of the category with the largest number of spine corner points is selected as the optimal set of spine corner points, and this set of spine corner points is used as the spine corner points of the second spine template. Another approach is to use the PCA algorithm. Specifically, principal component analysis is performed on the spinal corner points of N sets of first spinal templates, and the optimal set of spinal corner points is obtained based on the analysis results. This set of spinal corner points is then used as the spinal corner points of the second spinal template. Besides these two methods, the average value of the spinal corner points in multiple sets of first spinal templates can also be calculated directly. By adding the spinal corner points at corresponding positions in different sets of spinal corner points and averaging the results, an optimal set of spinal corner points can be obtained, which is then used as the spinal corner points of the second spinal template.

[0102] In some embodiments, the step of mapping the spinal corner points of the second spinal template to the spinal image to be detected according to a preset mapping rule, in order to determine the spinal corner points of the spinal image to be detected, specifically includes:

[0103] The second spinal template is registered to the spinal image to be detected using a preset registration algorithm, and the deformation field is determined based on the registration process.

[0104] The spinal corners of the second spinal template are mapped to the spinal image to be detected based on the deformation field, and the spinal corners of the spinal image to be detected are determined based on the mapping results.

[0105] Specifically, this embodiment provides a specific mapping rule. According to this rule, the spinal corner points of the second spinal template can be mapped to the spinal image to be detected, serving as the spinal corner points of the spinal image to be detected. The specific preset registration method includes rigid and non-rigid mates. Through rigid and non-rigid mates, the second spinal template is registered to the spinal image to be detected. The registration process is a conversion process between the two, reflecting the conversion relationship between them, which is the deformation field. Correspondingly, the deformation field can realize the conversion between the spinal corner points of the second spinal template and the spinal corner points of the spinal image to be detected. That is, the deformation field reflects the mapping relationship between the two sets of spinal corner points. Therefore, the spinal corner points of the second spinal template can be mapped to the spinal image to be detected based on the deformation field, and the mapped spinal corner points are the spinal corner points of the spinal image to be detected.

[0106] Furthermore, in some specific embodiments, the step of registering the second spinal template to the spinal image to be detected using a preset registration algorithm, and determining the deformation field based on the registration process specifically includes:

[0107] A second transformation matrix is ​​determined between a second set of matching points in a second spine template and a set of labeled points in a spine image to be detected, wherein the second set of matching points includes multiple spine points in the second spine template that correspond to multiple labeled point positions;

[0108] The spinal segments of the second spinal template are converted into registered spinal segments according to the second transformation matrix;

[0109] The non-rigid registration algorithm is used to register the registered spinal segments to the spinal segments in the spinal image to be detected.

[0110] The deformation field is determined based on the registration process.

[0111] Specifically, this embodiment further provides a specific registration step. First, rigid registration is performed. In the rigid registration process, the second matching point group in the second spinal template is determined first, and then the second transformation matrix between the second matching point group and the labeled point group is solved. This solution process is the same as the solution process for the first transformation matrix in the above embodiment.

[0112] Specifically, the marked point group can be denoted as P = (p1, p2, p3, p4), the second matching point group can be denoted as P` = (p1, p2, p3, p4), and the second transformation matrix is ​​denoted as A`. The relationship between P and P` satisfies P = A` * P`. The second transformation matrix A` can be calculated using the above formula. The specific solution process is as follows:

[0113] First, calculate the center point uA of P and the center point uB of P`;

[0114] Then construct the covariance matrix.

[0115] Perform singular value decomposition (SVD) on matrix H: [U, S, V] = svd(H). Obtain matrices U and V according to the SVD decomposition formula, and calculate the parameter R based on matrices U and V, where parameter R = VU. T ;

[0116] Let parameter t = -R*uA + uB, and calculate parameter t according to the aforementioned formula;

[0117] Finally, the second transformation matrix A' is obtained, where A' = [R, t].

[0118] Then, the spinal segment mask of the second spinal template is denoted as a`, and the registration spinal segment mask is denoted as a``. The registration spinal segment mask a`` = A`*a`. The registration spinal segment mask a`` can be obtained through this calculation formula, and the rigid registration is completed.

[0119] Since there is still a certain degree of difference between the registered spinal segment mask a` and the detected spinal segment mask a, it is necessary to continue non-rigid registration to register the registered spinal segment mask a` to the detected spinal segment mask a. The non-rigid registration algorithm can be the Demons algorithm (Symmetric Demons, Finite Element Registration, LDDMM, etc.). The registration process will be illustrated using the Demons algorithm as an example:

[0120] First, establish the model: u` = arg min(E D (Is(φ), It)+λE R (u));

[0121] Where Is represents the registration spinal segment mask a`` pattern, and It represents the detection spinal segment mask a pattern;

[0122] Φ is the deformation field, where φ = I + u(x), I is the identity matrix, and u(x) is the displacement field, which can also be abbreviated as u, representing the displacement at position x.

[0123] E D It is a similarity measure (such as mean squared error), E R It is the regularization term of the displacement field. arg min f(x) represents the value of the independent variable x that minimizes the objective function f(x). Therefore, u(x)' is the function E D (Is(φ), It)+λE R The value of the displacement field u(x) when (u) takes its minimum value.

[0124] Solving the above equation yields the final deformation field φ between the registered spinal segment mask a`` and the detected spinal segment mask a. This deformation field reflects the transformation relationship between the registered spinal segment mask a`` and the detected spinal segment mask a.

[0125] Finally, the transformation relationships determined in rigid registration and non-rigid registration are combined into a transformation field. This transformation field essentially reflects the transformation relationship between the spinal segment mask a' of the second spinal template and the detection spinal segment mask a. Finally, this transformation field is used to convert the spinal corner points of the second spinal template into the spinal corner points of the spinal image to be detected.

[0126] The technical solutions in this application are described and illustrated below through specific preferred embodiments.

[0127] Figure 8 This is a flowchart of the spinal corner detection method according to a preferred embodiment. Figure 8 As shown, the process includes the following steps:

[0128] Step S310: Establish a template library for the spine based on existing spinal corner point data.

[0129] Specifically, we collect existing spinal corner point datasets from the internet and establish a spinal corner point database and its corresponding spinal segmentation mask.

[0130] Step S320: Segment the image to be tested to obtain the spinal segment mask.

[0131] Specifically, for any image to be tested, deep learning or traditional methods are used to segment the image to obtain the spinal segments in the image, and then the corresponding spinal segment mask is generated.

[0132] Step S330: Based on the segmentation results, select the optimal N sets of template corner points from the template library.

[0133] Specifically, the selection process will be explained using the lumbar spine as an example:

[0134] First, manually mark the corner points of the L1 and S1 superior endplates on the image of the lumbar vertebrae to be tested. The set of corner points is P1 = {P1, P2, P3, P4}, and determine the lumbar vertebrae mask1; refer to the following for details. Figure 9 , Figure 9 This is a schematic diagram of the manual marking of the lumbar vertebral segment corners in this preferred embodiment.

[0135] Then, for any set of lumbar spine template corner points in the template library (including 22 corner points from L1 to L5 and the upper endplate of S1), find the coordinates P' = {P1, P2, P3, P4} of the upper endplate of L1 and S1 and their corresponding lumbar spine segments mask2. Use the SVD algorithm to solve the transformation matrix Ai in equation (1), where i = 1, 2, ..., n. n is equivalent to the number of each set of template corner points;

[0136] P1=Ai*P' (1)

[0137] For each transformation matrix Ai, transform its corresponding template mask2 to the space where mask1 is located, and take its L2 norm (or count the number of pixels that are inconsistent between mask1 and the transformed mask2), as shown in equation (2), and save it to V={}. Find the N groups with the smallest energy value difference from V={V1,V2,…Vn}, and find the corresponding N groups of lumbar spine templates according to the index number.

[0138] Vi=||mask1-Ai*mask2|| (2)

[0139] Where ||·|| is the L2 norm expression, which reflects the similarity between the lumbar spine segmentation result mask1 of the image under test and the transformed lumbar spine segmentation result mask2 in the template library.

[0140] Step S340: Use the kmeans algorithm (PCA algorithm, ASM algorithm) to calculate the mean of N sets of template corner points to obtain the optimal template corner points.

[0141] Specifically, for the K-means algorithm, let k=2, divide the N groups into k=2 classes, and take the cluster center of the class with the largest number of clusters as the optimal template corner point; or for the PCA algorithm, perform principal component analysis on the N groups of template corner points to obtain the optimal template corner point; or directly calculate the average value of the N groups of template corner points to obtain the optimal template corner point.

[0142] Step S350: The optimal template corner points are registered to the space where the corner point set P1 is located through rigid and non-rigid registration to obtain the deformation field.

[0143] Specifically, for rigid registration, the process begins by using equation (3) to transform the corner points of the optimal template, P = {P1, P2, P3, P4}, to the space where P1 is located, thus obtaining a preliminary transformation matrix A. Then, the transformation matrix A is used to transform the mask 3 enclosed by the optimal template, resulting in the mask, as shown in equation (4).

[0144] P1=A*P (3)

[0145] MASK = A * mask3 (4)

[0146] The specific solution process for matrix A is as follows:

[0147] 1. Calculate the center point uA of P and the center point uB of P1;

[0148] 2. Establish the covariance matrix

[0149] 3. Perform SVD decomposition on H [U, S, V] = svd(H) to obtain matrices U and V, thereby obtaining the parameter R = VU. T ;

[0150] 4. Solve for t: t = -R*uA + uB;

[0151] 5. Finally, we obtain matrix A, where A = [R, t].

[0152] For non-rigid registration, the MASK is registered onto mask1 using the Demons (Symmetric Demons, Finite Element Registration, LDDMM, etc.) non-rigid registration algorithm to obtain the (rigid and non-rigid) deformation fields.

[0153] Taking the Demons algorithm as an example, the model is built as follows:

[0154] u'=arg min(E D (Is(φ), It)+λE R (u))

[0155] Where Is represents the MASK image, It represents the mask1 image, φ = I + u(x), I is the identity matrix, u is the displacement field, and E D It is a similarity measure, such as mean squared error, E R It is the regularization term of the displacement field, and u` is the value of the displacement field u when the function takes its minimum value.

[0156] The model is solved iteratively until it converges.

[0157] Step S360: Map the corner points of the optimal template onto the image to be tested using the deformation field.

[0158] Specifically, the optimal template corner points are mapped onto the test image using the deformation field obtained in step S250, resulting in the final spine corner points. (Specific exhibition details follow.) Figure 10 , Figure 10 This is a schematic diagram of the effect of corner point mapping of a lateral lumbar spine image in this preferred embodiment. The white dots shown in the figure are the corner points of the lumbar spine segments obtained through mapping.

[0159] The above preferred embodiments describe the lumbar vertebral angle point extraction steps. Correspondingly, thoracic vertebral angle point extraction follows the same steps, requiring manual marking of the four angle points of the T1 and L1 superior endplates, such as... Figure 11 As shown. Figure 11 This is a schematic diagram of the manual marking of the thoracic vertebral segment corners in this preferred embodiment.

[0160] It should be noted that the spinal angle database may not cover all possible situations, so some data may not perform well in testing. Therefore, manual intervention is required to correct the position of some angles. At the same time, the spinal angles after manual intervention can be added to the spinal angle database as new template data to provide the necessary detection accuracy for subsequent angle detection.

[0161] This embodiment also provides a spinal corner detection device, which is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. The terms "module," "unit," "subunit," etc., used below refer to combinations of software and / or hardware that perform a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0162] Figure 12 This is a structural block diagram of the spinal corner detection device in this embodiment, as shown below. Figure 12 As shown, the device includes:

[0163] The data acquisition module 1210 is used to acquire the spinal image to be detected and the preset spinal template database, wherein the spinal template database includes several sets of first spinal templates.

[0164] The corner recognition module 1220 is used to identify a group of labeled points in a spinal image to be detected, wherein the group of labeled points includes multiple pre-labeled points on the spinal image to be detected.

[0165] The template determination module 1230 is used to determine a second spine template that matches the spine image to be detected based on a group of marker points and several groups of first spine templates.

[0166] The corner mapping module 1240 is used to map the spine corners of the second spine template to the spine image to be detected according to a preset mapping rule, so as to determine the spine corners of the spine image to be detected.

[0167] By combining the aforementioned modules, all spinal corner points in a spinal image can be detected. Unlike existing spinal corner detection devices, which typically extract feature points directly from the spinal image using deep learning or traditional methods, this method is susceptible to image exposure issues and may fail to detect feature points, leading to task failure. In this embodiment, the spinal corner points are not directly extracted from the spinal image. Instead, pre-annotated spinal corner points are matched against a database containing existing spinal template data to determine the closest second spinal template. The spinal corner points of this second template are then mapped and transformed according to certain rules to represent the spinal corner points in the image to be detected. Furthermore, the detection concept of this spinal corner detection device differs from existing technologies. Since it does not directly extract spinal corner points from the spinal image, it is not affected by the image's exposure. Therefore, this embodiment proposes a novel spinal corner detection device.

[0168] It should be noted that the working principle and process of the spinal corner detection device in this embodiment have been specifically described in the above method embodiments. Therefore, the spinal corner detection device in this embodiment can be understood by referring to the spinal corner detection method in the above embodiments.

[0169] It should be noted that the above modules can be functional modules or program modules, and can be implemented through software or hardware. For modules implemented through hardware, the above modules can reside in the same processor; or the above modules can be located in different processors in any combination.

[0170] This embodiment also provides an electronic device including a memory and a processor, the memory storing a computer program and the processor being configured to run the computer program to perform the steps in any of the above method embodiments.

[0171] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0172] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0173] S1, acquire the spinal image to be detected and the preset spinal template database, wherein the spinal template database includes several sets of first spinal templates.

[0174] S2, identify a set of markers in the spinal image to be detected, wherein the set of markers includes multiple markers pre-annotated on the spinal image to be detected.

[0175] S3, based on the set of marker points and several sets of first spinal templates, determines a second spinal template that matches the spinal image to be detected.

[0176] S4. According to the preset mapping rules, the spinal corner points of the second spinal template are mapped to the spinal image to be detected, so as to determine the spinal corner points of the spinal image to be detected.

[0177] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated in this embodiment.

[0178] Furthermore, in conjunction with the spinal corner detection method provided in the above embodiments, this embodiment can also provide a storage medium for implementation. The storage medium stores a computer program; when executed by a processor, the computer program implements any of the spinal corner detection methods described in the above embodiments.

[0179] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0180] It should be understood that the specific embodiments described herein are merely illustrative of the application and not intended to limit it. All other embodiments derived by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0181] Obviously, the accompanying drawings are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar situations based on these drawings without any creative effort. Furthermore, it is understood that although the work done in this development process may be complex and lengthy, for those skilled in the art, certain design, manufacturing, or production modifications made based on the technical content disclosed in this application are merely conventional technical means and should not be considered as insufficient disclosure of this application.

[0182] The term "embodiment" in this application refers to a specific feature, structure, or characteristic described in connection with an embodiment that may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily imply the same embodiment, nor does it imply that it is mutually exclusive with or independent of other embodiments. It will be clearly or implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0183] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of patent protection. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the appended claims.

Claims

1. A method of detecting a spinous process corner point, the method comprising: The detection method includes: The spine image to be detected and a preset spine template database are obtained, wherein the spine template database includes several sets of first spine templates; Identify a group of marker points in the spinal image to be detected, wherein the group of marker points includes multiple marker points pre-marked on the spinal image to be detected; Based on the set of annotation points and the plurality of first spinal templates, a second spinal template matching the spinal image to be detected is determined, comprising: Based on the set of annotation points, determine the multiple sets of first spine templates that have the highest similarity to the spine image to be detected from the multiple sets of first spine templates; The multiple sets of first spinal templates are processed according to preset processing rules, and a second spinal template matching the spinal image to be detected is obtained based on the processing results. According to the preset mapping rules, the spinal corner points of the second spinal template are mapped to the spinal image to be detected, so as to determine the spinal corner points of the spinal image to be detected.

2. The spinous process corner detection method of claim 1, wherein, The step of determining, based on the set of annotation points, the multiple sets of first spine templates with the highest similarity to the spine image to be detected from the plurality of sets of first spine templates includes: The image of the spine to be detected is segmented according to a preset image segmentation rule to obtain the detected spinal segment, wherein the detected spinal segment is the spinal segment of the image of the spine to be detected, and the marker point is the spinal corner point located at the endpoint of the detected spinal segment. Determine several first transformation matrices between the group of annotation points and the first matching point group of the several groups of first spinal templates, wherein the first matching point group includes multiple spinal corner points in the first spinal template that correspond to the positions of the multiple annotation points; Based on the aforementioned first transformation matrices, the spinal segments of the aforementioned groups of first spinal templates are respectively converted into several control spinal segments; Based on the detected spinal segment and the several control spinal segments, a plurality of first spinal templates with the highest similarity to the spinal image to be detected are determined.

3. The spinous process corner detection method of claim 2, wherein, The determination of multiple sets of first spinal templates with the highest similarity to the spinal image to be detected, based on the detected spinal segment and the plurality of control spinal segments, includes: The number of difference pixels between the several control spinal segments and the detected spinal segment is counted respectively, and the control spinal segments with the smallest number of difference pixels are determined based on the statistical results. Multiple sets of first spinal templates corresponding to the multiple control spinal segments are identified as the multiple sets of first spinal templates with the highest similarity to the spinal image to be detected.

4. The spinal corner detection method according to claim 2, characterized in that, The determination of multiple sets of first spinal templates with the highest similarity to the spinal image to be detected, based on the detected spinal segment and the plurality of control spinal segments, includes: Calculate the L2 norm of the difference between the detected spinal segment and the several control spinal segments respectively, and determine the several control spinal segments with the smallest L2 norm based on the calculation results; Multiple sets of first spinal templates corresponding to the multiple control spinal segments are identified as the multiple sets of first spinal templates with the highest similarity to the spinal image to be detected.

5. The spinal corner detection method according to any one of claims 1 to 4, characterized in that, The step of processing the multiple sets of first spinal templates according to preset processing rules, and obtaining a second spinal template that matches the spinal image to be detected based on the processing results, includes: The average value of the spinal corner points of the multiple sets of first spinal templates is calculated according to a preset processing algorithm, and the spinal corner points of the second spinal template are determined based on the processing result. The spinal segment of the second spinal template is determined based on the spinal corner points of the second spinal template.

6. The spinal corner detection method according to any one of claims 1 to 4, characterized in that, The step of mapping the spinal corner points of the second spinal template to the spinal image to be detected according to a preset mapping rule, so as to determine the spinal corner points of the spinal image to be detected, includes: The second spinal template is registered to the spinal image to be detected using a preset registration algorithm, and the deformation field is determined based on the registration process. The spinal corner points of the second spinal template are mapped onto the spinal image to be detected based on the deformation field, and the spinal corner points of the spinal image to be detected are determined based on the mapping result.

7. The spinal corner detection method according to claim 6, characterized in that, The step of registering the second spinal template to the spinal image to be detected using a preset registration algorithm, and determining the deformation field based on the registration process, includes: A second transformation matrix is ​​determined between the second matching point group of the second spine template and the annotation point group of the spine image to be detected, wherein the second matching point group includes multiple spine points in the second spine template corresponding to the positions of the multiple annotation points; The spinal segments of the second spinal template are converted into registered spinal segments according to the second transformation matrix; The registered spinal segment is registered to the spinal segment of the spinal image to be detected using a non-rigid registration algorithm; The deformation field is determined based on the registration process.

8. A spinal corner detection device, characterized in that, The detection device includes: The data acquisition module is used to acquire the spinal image to be detected and a preset spinal template database, wherein the spinal template database includes several sets of first spinal templates. A corner recognition module is used to identify a group of marked points in the spinal image to be detected, wherein the group of marked points includes multiple pre-marked points on the spinal image to be detected; A template determination module is used to determine a second spine template that matches the spine image to be detected based on the group of annotation points and the plurality of first spine templates, comprising: Based on the set of annotation points, determine the multiple sets of first spine templates that have the highest similarity to the spine image to be detected from the multiple sets of first spine templates; The multiple sets of first spinal templates are processed according to preset processing rules, and a second spinal template matching the spinal image to be detected is obtained based on the processing results. The corner mapping module is used to map the spine corners of the second spine template to the spine image to be detected according to a preset mapping rule, so as to determine the spine corners of the spine image to be detected.

9. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to perform the spinal corner detection method according to any one of claims 1 to 7.

Images