Spinal line generation method, spinal line generation device and electronic equipment

Abstract

The invention is suitable for the technical field of computer vision, and provides a spine line generation method, a spine line generation device and electronic equipment. The spine line generation method comprises the following steps: acquiring point cloud data of a central area of the back of a human body; for any target point, based on each point in a specific area corresponding to the target point, determining an asymmetry index of the target point, the asymmetry index of the target point representing asymmetry of the specific area; and generating a spinal line of the human body based on the asymmetry index of the target point. According to the method, the spinal line can be accurately and efficiently generated, and the method is suitable for large-scale scoliosis screening.

Description

Technical Field

[0001] This application belongs to the field of computer vision technology, and particularly relates to a spinal line generation method, a spinal line generation device, and an electronic device. Background Art

[0002] Idiopathic scoliosis is relatively common among teenagers, especially in girls aged 10 to the peak growth period and the end of skeletal maturity. This disease is defined as the Cobb angle of spinal lateral curvature reaching or exceeding 10 degrees without congenital or neuromuscular abnormalities. Although most teenage patients have no obvious symptoms, the condition may cause rib deformities and limited respiratory function, and some patients may also be troubled by appearance problems and psychological pressure. When the Cobb angle reaches 20 degrees, brace treatment is usually recommended, and when the angle exceeds 45 degrees, surgical intervention may be required.

[0003] In scoliosis screening, the current main methods include the Adams forward bend test combined with a spinal screening tool for preliminary evaluation, and then the spinal line is determined through X-ray images to measure the Cobb angle. However, this process needs to be operated by professional medical staff, which is time-consuming and not suitable for large-scale screening. At the same time, X-ray examination also brings certain radiation risks. Some studies have tried to introduce the moiré fringe imaging method, but due to its insufficient accuracy, subsequent X-ray confirmation is still required. Summary of the Invention

[0004] The embodiments of this application provide a spinal line generation method, a spinal line generation device, and an electronic device, which can accurately and efficiently generate the spinal line and are applicable to large-scale scoliosis screening.

[0005] In a first aspect, the embodiments of this application provide a spinal line generation method, and the spinal line generation method includes:

[0006] Obtain the point cloud data of the back central region of the human body;

[0007] For any target point, based on each point in the specific region corresponding to the target point, determine the asymmetry index of the target point. The target point is any point in the point cloud data of the back central region. The specific region includes the target point, N first adjacent points located on the left side of the target point, and N second adjacent points located on the right side of the target point. N is an integer greater than zero. The N first adjacent points, the N second adjacent points, and the target point are on the same target line. The target line is a line parallel to the shoulders of the human body in the back central region. The asymmetry index of the target point characterizes the asymmetry of the specific region;

[0008] Generate the spinal line of the human body based on the asymmetry index of the target point.

[0009] In the embodiments of the present application, by obtaining the point cloud data of the central region of the human back, and for any point (i.e., the target point) in the point cloud data, based on each point in the specific region corresponding to the target point, the asymmetry index of the target point is determined. The asymmetry index of the target point characterizes the asymmetry of the specific region. The asymmetry of the specific region in the central region of the human back can reflect the distortion of the back, and the distortion of the back can reflect the scoliosis condition of the back. Therefore, the present application can accurately and efficiently generate the spinal line of the human body based on the asymmetry index of the target point with less operation by professional medical staff, and is applicable to large-scale screening of scoliosis.

[0010] In a second aspect, an embodiment of the present application provides a spinal line generation device, which includes:

[0011] A point cloud acquisition module, configured to acquire the point cloud data of the central region of the human back;

[0012] An index determination module, configured to determine the asymmetry index of any target point based on each point in the specific region corresponding to the target point. The target point is any point in the point cloud data of the central region of the back. The specific region includes the target point, N first adjacent points located on the left side of the target point, and N second adjacent points located on the right side of the target point. N is an integer greater than zero. The N first adjacent points, the N second adjacent points, and the target point are on the same target line. The target line is a line parallel to the shoulders of the human body in the central region of the back. The asymmetry index of the target point characterizes the asymmetry of the specific region;

[0013] A spinal line generation module, configured to generate the spinal line of the human body based on the asymmetry index of the target point.

[0014] In a third aspect, an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the electronic device implements the method according to any one of the first aspects described above.

[0015] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a computer, the method according to the first aspect described above is implemented.

[0016] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program. When the computer program is run, the method according to any one of the first aspects described above is executed.

[0017] It can be understood that for the beneficial effects of the second to fifth aspects above, reference can be made to the relevant descriptions in the first aspect above, and details are not repeated here. Brief Description of the Drawings

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0019] Figure 1 is a schematic flowchart of the spinal line generation method provided by the embodiment of the present application;

[0020] Figure 2 is an example diagram of the back central region;

[0021] Figure 3-1 is an example diagram of the subject bending forward;

[0022] Figure 3-2 is an example diagram of the subject standing with their back to the data acquisition device;

[0023] Figure 4 is a schematic flowchart of the processing method for the point cloud data of the back region provided by the embodiment of the present application;

[0024] Figure 5 is an example diagram of the first adjacent point and the second adjacent point;

[0025] Figure 6 is an example diagram of the extreme point set;

[0026] Figure 7 is a schematic structural diagram of the spinal line generation device provided by the embodiment of the present application;

[0027] Figure 8 is a schematic structural diagram of the electronic device provided by the embodiment of the present application. Detailed Embodiments

[0028] In the following description, specific details such as specific system structures and technologies are presented for the purpose of illustration rather than limitation, so as to thoroughly understand the embodiments of the present application. However, those skilled in the art should clearly understand that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from interfering with the description of the present application.

[0029] It should be understood that, as used in the specification of this application and the appended claims, the term "comprising" indicates the presence of the described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or their groups.

[0030] It should also be understood that the term "and / or" as used in the specification of this application and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0031] As used in the specification of this application and the appended claims, the term "if" can be interpreted as "when" or "once" or "in response to determining" or "in response to detecting" depending on the context. Similarly, the phrases "if determined" or "if [the described condition or event] is detected" can be interpreted as meaning "once determined" or "in response to determining" or "once [the described condition or event] is detected" or "in response to detecting [the described condition or event]" depending on the context.

[0032] In addition, in the description of the specification of this application and the appended claims, the terms "first", "second", "third", etc. are only used for distinguishing descriptions and cannot be understood as indicating or implying relative importance.

[0033] Reference to "one embodiment" or "some embodiments" or the like described in the specification of this application means that a specific feature, structure or characteristic described in connection with that embodiment is included in one or more embodiments of this application. Thus, statements such as "in one embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments", etc. that appear in different places in this specification do not necessarily all refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "comprising", "including", "having" and their variants all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0034] The spine line generation method provided by the embodiments of this application can be applied to electronic devices such as tablet computers, laptop computers, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), etc. The embodiments of this application do not impose any restrictions on the specific types of electronic devices.

[0035] Please refer to Figure 1 , Figure 1The flowchart of the spinal cord line generation method provided by the embodiments of the present application is shown. By way of example and not limitation, the method includes the following steps:

[0036] Step 101: Obtain the point cloud data of the central region of the human back.

[0037] Among them, the above-mentioned central region of the back may refer to the region including the spinal cord line and the regions on both the left and right sides of the spinal cord line. The spinal cord line can also be referred to as the mid-spinal cord line.

[0038] Based on the point cloud data of the central region of the back, the present embodiment generates the spinal cord line, which can focus on the core region of the back when generating the spinal cord line, eliminate the influence of unnecessary data outside the shoulders, and can be used in the screening process of adolescent idiopathic scoliosis, reducing the workload of physicians during screening and providing high-quality reports and data for subsequent diagnosis.

[0039] In a possible implementation manner, the above step 101 may include:

[0040] Obtain the depth image and point cloud data of the human back;

[0041] Based on the depth image of the back, determine the left shoulder key point, right shoulder key point, neck key point, and waist key point of the human body;

[0042] Based on the left shoulder key point, right shoulder key point, neck key point, and waist key point, determine the point cloud data of the central region of the back from the point cloud data of the back.

[0043] Based on the depth image of the back obtained in real time, the electronic device can determine the position information of key points such as the left shoulder key point, right shoulder key point, neck key point, and waist key point of the human body in the depth image, and can map these key points to the three-dimensional coordinates in the point cloud data of the human back obtained in real time, so as to accurately locate these key points in three-dimensional space. Based on the position information of these key points in three-dimensional space, the boundary of the back region in the point cloud data can be better defined, and thus the point cloud data of the central region of the back can be determined from the point cloud data of the back. As Figure 2 shown is an example diagram of the central region of the back, Figure 2 the four blue dots in it are the four key points, namely the left shoulder key point, right shoulder key point, neck key point, and waist key point, Figure 2 the two blue lines in it are respectively the connection line between the left shoulder key point and the right shoulder key point, and the connection line between the neck key point and the waist key point. Based on these two connection lines, the central region of the back is determined, Figure 2 the region within the red rectangular frame in it is the central region of the back.

[0044] To map these key points to the three-dimensional coordinates in the point cloud data of the human back, it is necessary to align the depth image of the back and the point cloud data. Through the alignment process, it can be ensured that the space of the depth image and the point cloud data corresponds one by one, that is, each pixel in the depth image has a corresponding three-dimensional point in the point cloud data.

[0045] In one embodiment, the depth image and point cloud data of the human back can be collected by a data acquisition device, and the collected depth image and point cloud data are sent to an electronic device, and the electronic device generates a spinal line based on the received data.

[0046] In another embodiment, the depth image, point cloud data, and RGB image of the human back can also be collected by a data acquisition device, and the collected depth image, point cloud data, and RGB image are sent to an electronic device, and the electronic device generates a spinal line based on the received data.

[0047] As an example rather than a limitation, in large-scale scoliosis screening, Azure Kinect DK can be used as the data acquisition device. Azure Kinect DK is a depth camera that can measure the distance between an object and the camera through the Time of Flight (TOF) principle to generate a depth image. In addition, Azure Kinect DK can also collect RGB images and generate point cloud data. Azure Kinect DK is fixed at a fixed position and connected to a mobile computer (i.e., the electronic device). The screening process of the subjects adopts a standardized acquisition process: the subject turns his back to Azure Kinect DK, and Azure Kinect DK collects and saves the point cloud data, RGB image, depth image, etc. of the human back, and sends the collected data to the mobile computer. In the standardized acquisition process, the subject can bend forward as shown in Figure 3-1 (in this case, Data Acquisition Device 1 is used for data acquisition) or stand with his back to the data acquisition device as shown in Figure 3-2 (in this case, Data Acquisition Device 2 is used for data acquisition) to ensure that the point cloud data collected by the data acquisition device is mainly concentrated in the back area.

[0048] The RGB image is used to record the color image information of the back area. Through such data, the texture and color characteristics of the back can be intuitively analyzed. In this embodiment, after generating the spinal line, it can be displayed through the RGB image for the doctor to view.

[0049] The depth image provides the depth information of the back area, that is, the distance from each pixel to the camera.

[0050] The point cloud data of the back represents the three-dimensional coordinate information of the back, and the three-dimensional structure of the back is formed by a collection of a large number of points.

[0051] It is possible to process the RGB image, depth image, and point cloud data of the back collected in real time through the processing method of the point cloud data of the back region as Figure 4 shown, so as to obtain the point cloud data of the central region of the back.

[0052] As Figure 4 shown, the above processing process mainly includes the following five steps.

[0053] Step 1: Perform preprocessing such as cropping, denoising, and filtering on the RGB image and depth image to ensure the accuracy and reliability of the data.

[0054] The cropping operation is used to remove unnecessary background information and focus the image on the back region. According to the relative position of the camera and the standardized posture of the examinee, the interference regions in the image (mainly the surrounding environment or other non-back parts) can be removed, and only the back region to be analyzed is retained.

[0055] Due to environmental light or other interferences, the collected images may be affected by noise. The denoising operation can reduce the influence of random noise through smoothing processing, improve the accuracy of the image data, and use average smoothing or median filtering to remove the image data that does not conform to the actual characteristics of the back.

[0056] The filtering operation can further smooth the image data, remove fine interference information, and make the image smoother. By weakening the fine changes in the image, this operation can ensure the accuracy of subsequent alignment and key point detection.

[0057] Step 2: Registration and fusion.

[0058] When collecting data, there are slight perspective differences between the collected RGB image and depth image. In order to eliminate this difference, data registration is required. The purpose of registration is to ensure that the RGB image and depth image are aligned in the same coordinate system, that is, each RGB pixel point has a corresponding depth value.

[0059] In actual operation, data registration mainly analyzes the similarity between the RGB image and depth image, and gradually adjusts the positions of the two images until the corresponding relationship between their points reaches the optimal. After completing the registration, the RGB image can be fused with the depth image to obtain a three-dimensional data structure containing color and depth values (i.e., RGB-D data). This fused data tightly combines RGB information and depth information, providing a comprehensive three-dimensional image for subsequent analysis.

[0060] Step 3: Data alignment.

[0061] RGB-D data mainly provides color and depth information, while point cloud data is the three-dimensional structure of the back region. Through alignment operations, the spatial coordinates of these two types of data can be ensured to correspond one by one (i.e., mapping RGB-D data and point cloud data into the same coordinate system). After alignment, the color information and depth information of the back will be closely combined with the spatial positions of the three-dimensional points. Each pixel of the RGB-D data has a corresponding three-dimensional point in the point cloud data, providing high-precision data for subsequent analysis of back features.

[0062] Step Four, detect and identify key points.

[0063] In the fused RGB-D data and point cloud data, it is necessary to further detect and identify the key points of the back, such as the center positions of the shoulders, neck, and spine. Identifying these key points is very important for accurately extracting the central region of the back.

[0064] The key points of parts such as the shoulders and neck can be determined based on the depth image. Through the position information of these key points in the RGB image, these key points can be further mapped to the three-dimensional coordinates in the point cloud data, thereby accurately positioning these key points in three-dimensional space. By detecting the key points, the boundaries of the back region can be better defined, ensuring that subsequent analysis focuses on the core region of the back.

[0065] Step Five, crop the central region of the back.

[0066] Based on the detected key points, the regions in the point cloud data that are irrelevant to the back can be removed, and only the central region of the back is retained. The goal of this cropping operation is to further focus on the core region of the back, remove unnecessary data outside the shoulders, and ensure that the data is concentrated on the part that needs to be analyzed.

[0067] During the cropping process, using key points such as the left shoulder key point, right shoulder key point, and spine as a reference, a three-dimensional rectangle or other suitable region is defined, and only the point cloud data within this region is retained. After cropping, the resulting three-dimensional data only contains the point cloud information of the central region of the back, facilitating subsequent analysis and screening.

[0068] The above data processing method unifies multiple data sources into the same coordinate system through a standardized acquisition method, precise registration, and fusion technology, enabling the three-dimensional point cloud data of the back to be clearly and accurately presented. This processing method not only ensures the real-time nature of the data but also provides solid data support for back abnormality screening and spine analysis.

[0069] Of course, it can be understood that the data acquisition devices in the above two embodiments can also be integrated into an electronic device, that is, the electronic device itself performs data acquisition.

[0070] Step 102: For any target point, based on each point within the specific region corresponding to the target point, determine the asymmetry index of the target point.

[0071] Among them, the target point is any point in the point cloud data of the back center region. The specific region includes the target point, N first adjacent points located to the left of the target point, and N second adjacent points located to the right of the target point. N is an integer greater than zero. The N first adjacent points, the N second adjacent points, and the target point are on the same target line. The target line is a line parallel to the human shoulders within the back center region. The asymmetry index of the target point characterizes the asymmetry of the specific region.

[0072] The number of the first adjacent points is the same as that of the second adjacent points. The N first adjacent points are the N points on the target line where the target point is located, adjacent to the left of the target point and close to it. The N second adjacent points are the N points on the target line where the target point is located, adjacent to the right of the target point and close to it. As Figure 5 shown is an example diagram of the first adjacent points and the second adjacent points. Figure 5 The wavy line in Figure 5 is the target line. Both the p point and the i point in Figure 5 are extreme points. The p point and the i point are on the same target line. The three points to the left of the i point on this target line are the first adjacent points, and the three points to the right of the i point on this target line are the second adjacent points. On the basis of using the coordinate axes in

[0073] to represent the position information of the point cloud data, the connection line between the left shoulder key point and the right shoulder key point and the point with the same Y coordinate as these two key points can be used to represent the shoulders.

[0074] In a possible implementation, before performing Step 102, the number N of the first adjacent points and the second adjacent points can also be determined based on the position information of the target point in the back center region, so as to achieve a balance between the calculation efficiency and accuracy of the asymmetry index. That is, by flexibly adjusting the number N, the best compromise point in the calculation process of the asymmetry index can be found, so as to achieve a balance between the calculation efficiency and accuracy.

[0075] The correspondence between different position information and different quantities N can be preset (the smaller the quantity N corresponding to the position information closer to the edge of the back center area, and the larger the quantity N corresponding to the position information closer to the center of the back center area). Based on the position information of the target point in the back center area, the corresponding quantity N is found from this correspondence; alternatively, the back center area can also be divided into an edge area and a center area. Based on the position information of the target point in the back center area, it can be determined whether the target point is located in the edge area or the center area. If it is located in the edge area, the quantity N is set to a smaller value, and if it is located in the center area, the quantity N can be set to a larger value.

[0076] Before performing step 102, the point cloud data of the back center area obtained in step 101 can be processed first. By way of example and not limitation, this processing includes: performing 2D Delaunay fitting on the point cloud data of the back center area with a tolerance of 0.005 to generate a smooth triangular mesh to denoise the point cloud data of the back center area; data downsampling: applying a reduction factor of 0.1 to downsample the denoised point cloud data of the back center area to reduce the data density while maintaining the surface characteristics; surface smoothing processing: performing 10,000 iterations of surface smoothing on the downsampled point cloud data of the back center area with a relaxation factor of 0.0001 and enabling boundary smoothing (for example, using Gaussian filtering to smooth the boundary area) to reduce noise and ensure the smoothness and stability of the data. After performing the above processing on the point cloud data of the back center area, an effective calculation range can also be established for each point according to the distribution of the surface height of the back center area (for example, a height range is preset in advance, and points exceeding this height range are invalid points and no asymmetry index calculation is performed), which can avoid inaccurate calculation results in the surface edge or areas with extremely small curvature.

[0077] In a possible implementation manner, the above step 102 may include:

[0078] Determine the principal curvature of each point;

[0079] Based on the principal curvature of each point, determine the target parameter of each point, where the target parameter is at least one of mean curvature, Gaussian curvature, principal curvature difference, second-order flatness, and third-order flatness;

[0080] Based on the target parameter of each point, determine the asymmetry index of the target point.

[0081] Mean curvature is an "extrinsic" measure that characterizes the local curvature of a surface within the surrounding space. On the other hand, Gaussian curvature is an "intrinsic" measure of curvature that depends only on how distances are measured on the surface itself and is independent of its embedding in space. Simply put, any non-stretching transformation of the surface does not change its Gaussian curvature. For people with scoliosis, changes in the curvature of the spine can lead to rib protrusions, uneven shoulders, and asymmetrical back muscles. These deviations affect the curvature of the backplate, and this change can be used to quantify the degree of back asymmetry. Second-order flatness and third-order flatness are used to describe the high-order curvature characteristics of surface geometry and are commonly used to characterize the smoothness and symmetry of surface morphology. Based on objective parameters such as mean curvature, Gaussian curvature, principal curvature difference, second-order flatness, and third-order flatness, a more robust and detailed assessment of surface asymmetry can be made, providing a valuable reference for a deep understanding of surface shape characteristics.

[0082] Mean curvature, Gaussian curvature, second-order flatness, and third-order flatness are all high-order curvatures. Based on the high-order curvatures of the point cloud in this embodiment, the spinal line can be accurately and efficiently generated, providing a feasible and effective technical solution for large-scale screening of scoliosis.

[0083] The electronic device can generate the covariance matrix of each point based on the nearest neighbor points of each point (for example, determining the nearest distance through the Euclidean distance, and selecting the points with the Euclidean distance within the radius r as the nearest neighbor points, and the radius r can be preset according to actual needs). The covariance matrix reflects the degree of dispersion of each point and its nearest neighbor points on different axes. By calculating the eigenvalues and eigenvectors of the covariance matrix, the principal curvature of each point can be obtained. Based on the principal curvature, the Gaussian curvature, mean curvature, and principal curvature difference can be calculated. Based on the Gaussian curvature and mean curvature, the second-order flatness and third-order flatness can be calculated. By way of example and not limitation, each of the above points is point A in the point cloud data. Based on the principal curvature of point A, the mean curvature, Gaussian curvature, principal curvature difference, second-order flatness, and third-order flatness of point A can be determined.

[0084] Among them, the principal curvature of each point includes the maximum principal curvature and the minimum principal curvature of the point. The difference between the maximum principal curvature and the minimum principal curvature is the principal curvature difference. The average of the maximum principal curvature and the minimum principal curvature is the mean curvature.

[0085] The calculation formula for the second-order flatness is as follows:

[0086] PHI1 = (3H 2 - K)

[0087] Wherein, PHI1 represents the second-order flatness, H represents the mean curvature, and K represents the Gaussian curvature.

[0088] The second-order flatness combines the information of the mean curvature and the Gaussian curvature and is used to describe the second-order deviation of the surface smoothness. In flat regions (such as a plane), both the mean curvature and the Gaussian curvature tend to 0. Therefore, the second-order flatness is also close to 0. In the case of a minimal surface (i.e., the Gaussian curvature is negative, such as a saddle surface), the value of the second-order flatness will be more affected. The second-order flatness can be used to measure the smoothness of the surface. A larger value may indicate that the surface has strong bending characteristics.

[0089] The calculation formula for the third-order flatness is as follows:

[0090] PHI2 = (H 2 - K)(5H 2 - K)

[0091] where PHI2 represents the third-order flatness.

[0092] (H 2 - K) is an important geometric quantity that describes the local curvature distribution. (5H 2 - K) further enhances the curvature distribution, enabling the third-order flatness to reflect more complex morphological information. The third-order flatness enhances the sensitivity to surface changes through higher-order partial derivatives and is applicable to detecting local complexities of the surface, such as protrusions, depressions, and mutation points. Compared with the second-order flatness, the third-order flatness can capture more complex geometric changes and is applicable to surface asymmetry analysis or feature point detection.

[0093] In a possible implementation, determining the asymmetry index of the target point based on the target parameters of each point includes:

[0094] Determining the asymmetry index corresponding to each target parameter based on the target parameters of each point;

[0095] Determining the asymmetry index of the target point based on the asymmetry indices corresponding to each target parameter.

[0096] For any target parameter, the asymmetry index corresponding to the target parameter is the asymmetry index determined based on the target parameter of all points within the corresponding specific region.

[0097] The weights of each target parameter can be preset, and the asymmetry indices corresponding to each target parameter are weighted and summed based on the weights of each target parameter to obtain the asymmetry index of the target point. By way of example and not limitation, the target parameters are the mean curvature, the Gaussian curvature, the principal curvature difference, the second-order flatness, and the third-order flatness, and the weights of these five target parameters are equal, all being 0.2. Then, based on the weights of these five target parameters, the asymmetry indices corresponding to these five target parameters can be weighted and summed, that is, the average value of the asymmetry indices corresponding to these five target parameters is the asymmetry index of the target point.

[0098] In a possible implementation, based on the target parameters of each point, determine the asymmetry index corresponding to each target parameter, including:

[0099] For any target parameter, calculate the absolute value of the difference between the target parameters of all adjacent points within a specific region;

[0100] Based on the absolute value of the difference between the target parameters of all adjacent points, determine the asymmetry index corresponding to the target parameter.

[0101] The calculation formula for the asymmetry index corresponding to any target parameter is as follows:

[0102]

[0103] where i represents the abscissa of the target point; i - 1 represents the abscissa of the point adjacent to the target point on the left side of the target point, i - N represents the point adjacent to the point with abscissa i - N + 1 on the left side of the target point, and the point adjacent to the target point on the left side of the target point and the point adjacent to the point with abscissa i - N + 1 on the left side of the target point are both first adjacent points; i + 1 represents the abscissa of the point adjacent to the target point on the right side of the target point, i + N represents the point adjacent to the point with abscissa i + N - 1 on the right side of the target point, and the point adjacent to the target point on the right side of the target point and the point adjacent to the point with abscissa i + N - 1 on the right side of the target point are both second adjacent points; f k represents the target parameter of the point with abscissa k; f k+1 represents the target parameter of the point with abscissa k + 1.

[0104] Step 103, generate the spine line of the human body based on the asymmetry index of the target point.

[0105] Since the asymmetry index of the target point characterizes the asymmetry of a specific region, and the asymmetry of a specific region within the central region of the human back can reflect the distortion of the back, and the distortion of the back can reflect the scoliosis condition of the back, therefore, this application can accurately and efficiently generate the spine line of the human body based on the asymmetry index of the target point with less operation by professional medical staff, and is applicable to large-scale screening of scoliosis.

[0106] In a possible implementation, before executing step 103, it may further include:

[0107] If there are outliers in the asymmetry index of the target point, then based on the asymmetry index of the adjacent points of the target point corresponding to the outlier, determine the target asymmetry index of the target point corresponding to the outlier;

[0108] Update the asymmetry index of the target point corresponding to the outlier to the target asymmetry index.

[0109] After determining the asymmetry index of any point (i.e., the target point) in the point cloud data within the back center region through step 102, all the asymmetry indices can be cleaned to ensure the reliability and stability of the data used for spine line generation subsequently. Cleaning all the asymmetry indices includes: identifying and processing the outliers in all the asymmetry indices to avoid interference with further analysis. Specifically, it includes: first removing the NaN values in all the asymmetry indices; then calculating the first quartile, the third quartile, and the interquartile range (IQR), and accordingly determining the upper and lower boundaries for identifying outliers. Based on the upper and lower boundaries of the outliers, it is determined whether there are outliers among the non-NaN values of all the asymmetry indices; if there are outliers, the adjacent points of the target point corresponding to the outliers (i.e., the points adjacent to the target point corresponding to the outliers within the back center region) are determined, and based on the asymmetry indices of these adjacent points, the target asymmetry index of the target point corresponding to the outliers is determined (for example, taking the average of the asymmetry indices of these adjacent points as the target asymmetry index), and the target asymmetry index is used to replace the outliers, so as to reduce the impact caused by the outliers while maintaining the smoothness and consistency of the data. Of course, the outliers can also be directly set to NaN values to further reduce the outlier points that may have a negative impact on the analysis results.

[0110] In a possible implementation manner, the above step 103 may include:

[0111] For any target line, based on the asymmetry index of the target points on the target line, the extreme points on the target line are determined;

[0112] Based on the adjacent points among the extreme points on each target line, at least one extreme point set is formed;

[0113] The extreme point set with the longest length in the at least one extreme point set is fitted to obtain the spine line of the human body.

[0114] Before the extreme points on the target line, the asymmetry index of the target points on each target line can also be smoothed first, and on this basis, the extreme points (including local maxima or local minima) are detected. These extreme points represent some significant features on the surface. Based on the detected extreme points, a connected two-dimensional grid can be created, with 0 representing non-extreme points and 1 representing extreme points, to represent the connection relationship between the extreme points. By analyzing the set of connection points, especially the set with the longest length, the extreme points of this set can be fitted to obtain the most significant midline on the surface. This midline represents the main shape feature of the surface and can be used as the spinous process line of the back spine to analyze whether there is scoliosis on the left and right sides. As Figure 6The figure shown is an example diagram of the set of extreme points. Figure 6 The red dots in it are all extreme points. Adjacent extreme points form a set of extreme points. From Figure 6 it can be seen that there are multiple sets of extreme points. The set of extreme points near the center of the back center area has the longest length. By fitting the set of extreme points with the longest length, a spinal line can be generated.

[0115] In this embodiment, to evaluate the reliability of the fitted midline, point cloud data within the back center area of 161 adolescent idiopathic scoliosis (AIS) patients can be used for spinal line fitting. The Cobb angles of these patients range from 2.7 degrees to 39.4 degrees. The average execution time of the electronic device from inputting the original point cloud to outputting the final reconstructed spinal line is less than three seconds. Subsequently, these fitted spinal lines are systematically compared with the spinal lines extracted from the corresponding X-ray images. Specifically, two key methods are adopted to measure the similarity between the spinal lines fitted in this embodiment and the spinal lines extracted from the corresponding X-ray images: Fourier descriptors and curvature feature analysis. The similarity between the curves can be captured from different angles and the errors brought by various factors can be reduced. The experimental results are shown in Table 1 below, indicating that the spinal lines generated by fitting in this embodiment can well match the spinal lines extracted from the X-ray images under the two key methods and show a strong correlation.

[0116] Table 1

[0117]

[0118] Based on frequency domain analysis, Fourier descriptors can effectively capture the overall morphological features of the spine and have strong anti-noise properties. Its high correlation (0.946) indicates that this method can stably match the spinal midline and is suitable for situations where the morphological features are relatively regular. At the same time, its low variance (0.00516) and low standard deviation (0.009) mean that this method has little volatility and stronger stability among different individuals.

[0119] The curvature feature method directly depends on the geometric information of the spinal surface and can well capture local morphological changes. Its correlation reaches 0.903, indicating that this method can accurately fit the spinal line to a certain extent, especially more advantageous in dealing with situations where local features are more complex. It shows that there is a high correlation between the fitted spinal line and the spinal line in the X-ray image.

[0120] In the embodiments of the present application, by acquiring the point cloud data of the central region of the human back, and for any point (i.e., the target point) in the point cloud data, based on each point in the specific region corresponding to the target point, the asymmetry index of the target point is determined. The asymmetry index of the target point characterizes the asymmetry of the specific region. The asymmetry of the specific region in the central region of the human back can reflect the distortion of the back, and the distortion of the back can reflect the scoliosis situation of the back. Therefore, the present application can accurately and efficiently generate the spinal line of the human body based on the asymmetry index of the target point with less operation by professional medical staff, and is applicable to large-scale screening of scoliosis.

[0121] It should be understood that the magnitudes of the sequence numbers of the steps in the above embodiments do not mean the order of execution. The execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

[0122] Corresponding to the spinal line generation method described in the above embodiments, Figure 7 The structural schematic diagram of the spinal line generation device provided by the embodiments of the present application is shown. For the convenience of description, only the parts related to the embodiments of the present application are shown.

[0123] Referring to Figure 7 , the spinal line generation device includes:

[0124] A point cloud acquisition module 701, configured to acquire the point cloud data of the central region of the human back;

[0125] An index determination module 702, configured to determine the asymmetry index of any target point based on each point in the specific region corresponding to the target point. The target point is any point in the point cloud data of the central region of the back. The specific region includes the target point, N first adjacent points on the left side of the target point, and N second adjacent points on the right side of the target point. N is an integer greater than zero. The N first adjacent points, the N second adjacent points, and the target point are on the same target line. The target line is a line parallel to the shoulders of the human body in the central region of the back. The asymmetry index of the target point characterizes the asymmetry of the specific region;

[0126] A spinal line generation module 703, configured to generate the spinal line of the human body based on the asymmetry index of the target point.

[0127] Optionally, the above index determination module 702 includes:

[0128] A curvature determination unit, configured to determine the principal curvature of each point;

[0129] A parameter determination unit, configured to determine a target parameter for each point based on the principal curvature of each point, where the target parameter is at least one of mean curvature, Gaussian curvature, principal curvature difference, second-order smoothness, and third-order smoothness;

[0130] An index determination unit, configured to determine an asymmetry index of the target point based on the target parameter of each point.

[0131] Optionally, the above index determination unit includes:

[0132] A first determination subunit, configured to determine an asymmetry index corresponding to each target parameter based on the target parameter of each point;

[0133] A second determination subunit, configured to determine the asymmetry index of the target point based on the asymmetry indices corresponding to each target parameter.

[0134] Optionally, the above first determination subunit is specifically configured to:

[0135] For any one of the target parameters, calculate the absolute value of the difference between the target parameters of all adjacent points within the specific region;

[0136] Based on the absolute value of the difference between the target parameters of all adjacent points, determine the asymmetry index corresponding to the target parameter.

[0137] Optionally, the above spine line generation module 703 is specifically configured to:

[0138] For any one of the target lines, determine the extreme points on the target line based on the asymmetry index of the target points on the target line;

[0139] Based on the adjacent points among the extreme points on each target line, form at least one extreme point set;

[0140] Fit the extreme point set with the longest length among at least one of the extreme point sets to obtain the spine line of the human body.

[0141] Optionally, the above spine line generation device further includes:

[0142] A quantity determination module, configured to determine the quantities N of the first adjacent points and the second adjacent points based on the position information of the target points in the back center region.

[0143] Optionally, the above spine line generation device further includes:

[0144] A target determination module, configured to, if there is an outlier in the asymmetry indexes of the target points, determine the target asymmetry index of the target point corresponding to the outlier based on the asymmetry indexes of the adjacent points of the target point corresponding to the outlier;

[0145] An index update module, configured to update the asymmetry index of the target point corresponding to the outlier to the target asymmetry index.

[0146] Optionally, the above point cloud acquisition module 701 is specifically configured to:

[0147] Acquire the depth image and point cloud data of the back of the human body;

[0148] Based on the depth image of the back, determine the left shoulder key point, right shoulder key point, neck key point, and waist key point of the human body;

[0149] Based on the left shoulder key point, the right shoulder key point, the neck key point, and the waist key point, determine the point cloud data of the central area of the back from the point cloud data of the back.

[0150] It should be noted that, for the information interaction, execution process, etc. between the above devices / units, since they are based on the same concept as the method embodiments of the present application, their specific functions and the technical effects brought thereby can be specifically referred to the method embodiment part, and will not be elaborated here.

[0151] Figure 8 This is a schematic structural diagram of an electronic device provided by an embodiment of the present application. As Figure 8 shown, the electronic device 8 of this embodiment includes: at least one processor 80 ( Figure 8 only one is shown in the figure), a memory 81, and a computer program 82 stored in the memory 81 and executable on the at least one processor 80. When the processor 80 executes the computer program 82, the steps in any of the above method embodiments are implemented.

[0152] The electronic device may include, but is not limited to, a processor 80 and a memory 81. Those skilled in the art can understand that Figure 8 this is only an example of the electronic device 8, and does not constitute a limitation to the electronic device 8. It may include more or fewer components than those shown in the figure, or combine certain components, or different components. For example, it may also include input / output devices, network access devices, etc.

[0153] The so-called processor 80 may be a Central Processing Unit (CPU), and the processor 80 may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0154] In some embodiments, the memory 81 may be an internal storage unit of the electronic device 8, such as the hard disk or memory of the electronic device 8. In other embodiments, the memory 81 may also be an external storage device of the electronic device 8, such as a plug-in hard disk equipped on the electronic device 8, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. Further, the memory 81 may also include both the internal storage unit and the external storage device of the electronic device 8. The memory 81 is used to store an operating system, application programs, a BootLoader, data, and other programs, such as the program code of the computer program, etc. The memory 81 may also be used to temporarily store data that has been output or will be output.

[0155] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the above division of each functional unit and module is used as an example. In actual applications, the above functions can be allocated to different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit. In addition, the specific names of each functional unit and module are only for the convenience of mutual distinction and do not limit the protection scope of this application. The specific working process of the units and modules in the above system can refer to the corresponding process in the foregoing method embodiment and will not be described herein again.

[0156] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, to implement all or part of the processes in the above-described embodiment methods of this application, a computer program can be used to instruct relevant hardware to complete. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above-described method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file, or some intermediate form, etc. The computer-readable medium can at least include: any entity or device that can carry the computer program code to the device / electronic device, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium. For example, a USB flash drive, a portable hard disk, a magnetic disk, or an optical disc, etc. In some jurisdictions, according to legislation and patent practice, the computer-readable medium cannot be an electrical carrier signal and a telecommunication signal.

[0157] In the above embodiments, the descriptions of the various embodiments have their own focuses. For the parts not detailed or recorded in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0158] Those of ordinary skill in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0159] In the embodiments provided in this application, it should be understood that the disclosed device / electronic device and method can be implemented in other ways. For example, the device / electronic device embodiments described above are only illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection to each other can be through some interfaces. The indirect coupling or communication connection of the device or unit can be in an electrical, mechanical, or other form.

[0160] The unit described as a separation component may or may not be physically separated. The component displayed as a unit may or may not be a physical unit, that is, it may be located in one place or distributed across multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0161] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included within the protection scope of the present application.

Claims

1. A method for generating a spine line, characterized in that: The spine line generation method comprises: Obtain point cloud data of the central area of ​​the back of the human body; For any target point, based on each point in a specific area corresponding to the target point, determine an asymmetry index of the target point, the target point is any point in the point cloud data of the central area of ​​the back, the specific area includes the target point, N first adjacent points located on the left side of the target point, and N second adjacent points located on the right side of the target point, N is an integer greater than zero, the N first adjacent points, the N second adjacent points, and the target point are on the same target line, the target line is a line in the central area of ​​the back parallel to the shoulders of the human body, and the asymmetry index of the target point represents the asymmetry of the specific area; Based on the asymmetry index of the target point, a spinal column line of the human body is generated.

2. The method for generating a spine line according to claim 1, characterized in that: The determining of the asymmetry index of the target point based on each point in the specific area corresponding to the target point includes: determining the principal curvature of each of the points; Based on the principal curvature of each point, determining a target parameter of each point, wherein the target parameter is at least one of a mean curvature, a Gaussian curvature, a principal curvature difference, a second-order smoothness, and a third-order smoothness; Based on the target parameter of each point, an asymmetry index of the target point is determined.

3. The method for generating a spine line according to claim 2, characterized in that: Determining an asymmetry index of the target point based on the target parameter of each point includes: Based on the target parameter of each point, determining an asymmetry index corresponding to each target parameter; Based on the asymmetry index corresponding to each of the target parameters, the asymmetry index of the target point is determined.

4. The method for generating a spine line according to claim 3, characterized in that: The determining, based on the target parameter of each point, an asymmetry index corresponding to each target parameter includes: For any of the target parameters, calculating the absolute value of the difference between all adjacent points in the specific area; Based on the absolute value of the difference of the target parameter between all the adjacent points, an asymmetry index corresponding to the target parameter is determined.

5. The method for generating a spine line according to any one of claims 1 to 4, characterized in that: The step of generating the spinal column line of the human body based on the asymmetry index of the target point comprises: For any of the target lines, based on the asymmetry index of the target point on the target line, determining the extreme point on the target line; Based on adjacent points among the extreme value points on each of the target lines, at least one extreme value point set is formed; Fitting is performed on the extreme point set with the longest length in at least one of the extreme point sets to obtain the spinal column line of the human body.

6. The method for generating a spine line according to any one of claims 1 to 4, characterized in that: Before determining the asymmetry index of the target point based on each point in the specific area corresponding to the target point, the method further includes: Based on the position information of the target point in the central area of ​​the back, the number N of the first adjacent points and the second adjacent points is determined.

7. The method for generating a spine line according to any one of claims 1 to 4, characterized in that: Before generating the spine line of the human body based on the asymmetry index of the target point, the method further includes: If there is an outlier in the asymmetry index of the target point, determining the target asymmetry index of the target point corresponding to the outlier based on the asymmetry indexes of the adjacent points of the target point corresponding to the outlier; The asymmetry index of the target point corresponding to the outlier is updated to the target asymmetry index.

8. The method for generating a spine line according to any one of claims 1 to 4, characterized in that: The step of obtaining point cloud data of the central area of ​​the back of a human body includes: Acquire a depth image and point cloud data of the back of the human body; Based on the depth image of the back, determining the left shoulder key point, the right shoulder key point, the neck key point and the waist key point of the human body; Based on the left shoulder key point, the right shoulder key point, the neck key point and the waist key point, the point cloud data of the back center area is determined from the point cloud data of the back.

9. A spinal line generating device, characterized in that: The spinal line generating device comprises: A point cloud acquisition module is used to acquire point cloud data of the back center area of ​​a human body; An index determination module, for determining, for any target point, an asymmetry index of the target point based on each point in a specific area corresponding to the target point, the target point being any point in the point cloud data of the central area of ​​the back, the specific area comprising the target point, N first adjacent points located on the left side of the target point, and N second adjacent points located on the right side of the target point, N being an integer greater than zero, the N first adjacent points, the N second adjacent points, and the target point are on the same target line, the target line being a line in the central area of ​​the back parallel to the shoulders of the human body, and the asymmetry index of the target point characterizes the asymmetry of the specific area; The spine line generation module is used to generate the spine line of the human body based on the asymmetry index of the target point.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the electronic device implements the method according to any one of claims 1 to 8.

Images