Spine line generation method, spine line generation device, and electronic device

Abstract

The application is suitable for the technical field of computer vision, and provides a spine line generation method, a spine line generation device and electronic equipment. The spine line generation method comprises: acquiring point cloud data of a back center area of a human body; for any target point, determining an asymmetry index of the target point based on each point in a specific area corresponding to the target point, the asymmetry index of the target point representing the asymmetry of the specific area; and generating a spine line of the human body based on the asymmetry index of the target point. The spine line can be accurately and efficiently generated by the application, and the application is suitable for large-scale screening of scoliosis.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of computer vision, and particularly relates to a spine line generation method, a spine line generation device and an electronic device. BACKGROUND

[0002] Idiopathic scoliosis is common in adolescents, especially in females aged 10 to the peak of growth and the end of skeletal maturity. The condition is defined as a lateral curvature of the spine with a Cobb angle of 10 degrees or more without congenital or neuromuscular abnormalities. Although most adolescent patients have no obvious symptoms, the disease may cause rib deformity and limited respiratory function, and some patients may also be troubled by appearance problems and psychological pressure. When the Cobb angle reaches 20 degrees, it is usually recommended to use a brace for treatment, and when the angle exceeds 45 degrees, surgical intervention may be required.

[0003] In the screening of scoliosis, the current main method includes preliminary evaluation by Adams forward bending test combined with spine screening tool, and then determining the spine line through X-ray image and measuring the Cobb angle. However, this process needs to be operated by professional medical personnel, is time-consuming and not suitable for large-scale screening, and X-ray examination also brings a certain radiation risk. Some studies attempt to introduce the Moire fringe imaging method, but due to its insufficient accuracy, it still needs subsequent X-ray confirmation. SUMMARY

[0004] The embodiments of the present application provide a spine line generation method, a spine line generation device and an electronic device, which can accurately and efficiently generate a spine line and are suitable for large-scale screening of scoliosis.

[0005] In a first aspect, the embodiments of the present application provide a spine line generation method, which comprises:

[0006] obtaining point cloud data of a central area of the back of a human body;

[0007] For any target point, determining an asymmetry index of the target point based on each point in a specific region corresponding to the target point, the target point being any point in the point cloud data of the central area of the back, the specific region including the target point, N first neighboring points located on the left side of the target point and N second neighboring points located on the right side of the target point, N being an integer greater than zero, the N first neighboring points, the N second neighboring points and the target point being on a same target line, the target line being a line in the central area of the back parallel to the shoulder of the human body, the asymmetry index of the target point representing the asymmetry of the specific region;

[0008] generating a spine line of the human body based on the asymmetry index of the target point.

[0009] In the embodiments of the present application, the point cloud data of the central region of the back of the human body is obtained, and for any point (i.e., a target point) in the point cloud data, the asymmetry index of the target point is determined based on each point in the specific region corresponding to the target point. The asymmetry index of the target point represents the asymmetry of the specific region. The asymmetry of the specific region in the central region of the back of the human body can reflect the distortion of the back. The distortion of the back can reflect the lateral curvature of the back. Therefore, the present application can accurately and efficiently generate the spine line of the human body based on the asymmetry index of the target point under the condition of reducing the operation of professional medical personnel, and is suitable for large-scale screening of spinal scoliosis.

[0010] In a second aspect, the embodiments of the present application provide a spine line generation device, which comprises:

[0011] a point cloud obtaining module, configured to obtain point cloud data of a central region of a back of a human body;

[0012] an index determining module, configured to, for any target point, determine an asymmetry index of the target point based on each point in a specific region corresponding to the target point, the target point being any point in the point cloud data of the central region of the back of the human body, the specific region comprising the target point, N first neighboring points located on the left side of the target point, and N second neighboring points located on the right side of the target point, N being an integer greater than zero, the N first neighboring points, the N second neighboring points, and the target point being on a same target line, the target line being a line in the central region of the back of the human body parallel to the shoulder of the human body, the asymmetry index of the target point representing the asymmetry of the specific region;

[0013] a spine line generating module, configured to generate a spine line of the human body based on the asymmetry index of the target point.

[0014] In a third aspect, the embodiments of the present application provide an electronic device, comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor. When the processor executes the computer program, the electronic device implements the method of any one of the first aspect.

[0015] In a fourth aspect, the embodiments of the present application provide a computer readable storage medium, which stores a computer program. When the computer program is executed by a computer, the method of the first aspect is implemented.

[0016] In a fifth aspect, the embodiments of the present application provide a computer program product, comprising a computer program. When the computer program is executed, the method of any one of the first aspect is executed.

[0017] It can be understood that the beneficial effects of the above-mentioned second aspect to the fifth aspect can be referred to the related description in the above-mentioned first aspect, which will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, the drawings in the following description only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0019] Figure 1 is a flowchart of the spine line generation method provided by the embodiments of the present application;

[0020] Figure 2 is an example diagram of the center area of the back;

[0021] Figure 3-1 is an example diagram of the subject bending forward;

[0022] Figure 3-2 is an example diagram of the subject standing with his back to the data acquisition device;

[0023] Figure 4 is a flowchart of the processing method of the point cloud data of the back area provided by the embodiments of the present application;

[0024] Figure 5 is an example diagram of the first adjacent point and the second adjacent point;

[0025] Figure 6 is an example diagram of the extreme point set;

[0026] Figure 7 is a structural diagram of the spine line generation device provided by the embodiments of the present application;

[0027] Figure 8 is a structural diagram of the electronic device provided by the embodiments of the present application. DETAILED DESCRIPTION

[0028] In the following description, specific details are set forth in order to provide a thorough understanding of the embodiments of the present application. However, persons skilled in the art should understand that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed description of well-known systems, devices, circuits and methods is omitted so as not to obscure the description of the present application with unnecessary details.

[0029] It should be understood that the word “comprise” or variations such as “comprises” or “comprising”, when used in this specification and in the accompanying claims, specify the presence of stated features, integers, steps, operations, elements, components and / or groups but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0030] It should also be understood that the term “and / or” when used in this specification and in the following claims is to be interpreted as “one or the other or both” and / or “any combination of the items in the list”. It should be understood that the terms “a” and “an” as used in this specification and in the following claims indicate “one or more”.

[0031] As used in this specification and in the claims, the terms “if” and “when” can be interpreted to mean “upon” or “in response to a determination” or “in response to a detection” depending on the context. Similarly, the phrase “if it is determined” or “if [a described condition or event] is detected” can be interpreted to mean “upon a determination” or “in response to a determination” or “upon detecting [a described condition or event]” or “in response to detecting [a described condition or event]”, depending on the context.

[0032] In addition, the terms “first”, “second”, “third”, etc. as used in the description of the application and the following claims are only used to distinguish descriptions and cannot be understood as indicating or implying relative importance.

[0033] Reference in the specification to “one embodiment” or “some embodiments” etc. means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the application. The appearances of the phrases “in one embodiment”, “in some embodiments”, “in other embodiments”, “in additional embodiments” etc. in various places in the specification are not necessarily all referring to the same embodiment, although they can. The terms “comprise”, “comprises”, “comprising”, “include”, “includes”, “including” and the like are synonymous with “containing” or “comprising” and are used in the sense of “including but not limited to”, unless otherwise expressly specified.

[0034] The spine line generation method provided by the embodiments of the present application can be applied to electronic devices such as tablet computers, notebook computers, ultra-mobile personal computers (UMPC), netbooks, personal digital assistants (PDA), etc. The embodiments of the present application do not make any limitation on the specific type of electronic device.

[0035] Please refer to Figure 1 , Figure 1A flowchart of a spine line generation method provided by an embodiment of the present application is shown, which includes the following steps as an example but is not limited thereto:

[0036] In step 101, point cloud data of a central region of a back of a human body is obtained.

[0037] The central region of the back can refer to a region including a spine line and regions on both sides of the spine line. The spine line can also be referred to as a spine midline.

[0038] The embodiment generates the spine line based on the point cloud data of the central region of the back, can focus on the core region of the back when generating the spine line, eliminate the influence of unnecessary data other than the shoulders, and can be used in the process of screening adolescent scoliosis, reduce the workload of a doctor during screening, and provide high-quality reports and data for subsequent diagnosis.

[0039] In a possible implementation, the step 101 can include the following steps.

[0040] Obtaining a depth image and point cloud data of a back of a human body;

[0041] Determining a left shoulder key point, a right shoulder key point, a neck key point, and a waist key point of the human body based on the depth image of the back;

[0042] Determining point cloud data of a central region of the back from the point cloud data of the back based on the left shoulder key point, the right shoulder key point, the neck key point, and the waist key point.

[0043] Based on the depth image of the back obtained in real time, the electronic device can determine position information of key points such as the left shoulder key point, the right shoulder key point, the neck key point, and the waist key point of the human body in the depth image, can map these key points to three-dimensional coordinates in the point cloud data of the back of the human body obtained in real time, thereby accurately positioning these key points in a three-dimensional space, and based on the position information of the key points in the three-dimensional space, can better define the boundary of the back region in the point cloud data, thereby determining the point cloud data of the central region of the back from the point cloud data of the back. Figure 2 An example diagram of the central region of the back is shown, Figure 2 The four blue dots in the diagram are four key points of the left shoulder key point, the right shoulder key point, the neck key point, and the waist key point, Figure 2 The two blue lines in the diagram are a line connecting the left shoulder key point and the right shoulder key point and a line connecting the neck key point and the waist key point, and the central region of the back is determined based on the two lines, Figure 2 The region in the red rectangular frame in the diagram is the central region of the back.

[0044] In order to map these key points into three-dimensional coordinates in the point cloud data of the back of the human body, the depth image and the point cloud data of the back need to be aligned. Through the alignment, the one-to-one correspondence of the depth image and the point cloud data in space can be ensured, that is, each pixel in the depth image has a corresponding three-dimensional point in the point cloud data.

[0045] In an embodiment, the depth image and the point cloud data of the back of the human body can be collected by the data collection device, and the collected depth image and point cloud data are sent to the electronic device. The electronic device generates the spine line based on the received data.

[0046] In another embodiment, the depth image, point cloud data and RGB image of the back of the human body can also be collected by the data collection device, and the collected depth image, point cloud data and RGB image are sent to the electronic device. The electronic device generates the spine line based on the received data.

[0047] By way of example and not limitation, in the large-scale screening of scoliosis, the Azure Kinect DK, a depth camera capable of measuring the distance between the object and the camera through the Time of Flight (TOF) principle to generate a depth image, can be used as the data collection device. In addition, the Azure Kinect DK can also collect RGB images and generate point cloud data. The Azure Kinect DK is fixed at a fixed position and connected to a mobile computer (i.e., an electronic device). The screening process of the subject uses a standardized collection process: the subject faces away from the Azure Kinect DK, and the Azure Kinect DK collects and saves the point cloud data, RGB images, depth images, etc. of the back of the human body, and sends the collected data to the mobile computer. In the standardized collection process, the subject can bend forward as shown in Figure 3-1 or stand facing away from the data collection device as shown in Figure 3-2 to ensure that the point cloud data collected by the data collection device is mainly concentrated in the back region.

[0048] The RGB image is used to record the color image information of the back region. Through such data, the texture and color features of the back can be intuitively analyzed. In this embodiment, after the spine line is generated, the RGB image can be displayed for the convenience of the physician to view.

[0049] The depth image provides the depth information of the back region, that is, the distance of each pixel to the camera.

[0050] The point cloud data of the back represents the three-dimensional coordinate information of the back, and forms a three-dimensional structure of the back through a large number of point sets.

[0051] The point cloud data of the back region can be processed by the processing method of the point cloud data of the back region as shown in Figure 4 The RGB image, the depth image and the point cloud data of the back collected in real time are processed to obtain the point cloud data of the central region of the back.

[0052] As shown in Figure 4 The above processing process mainly includes the following five steps.

[0053] Step one, the RGB image and the depth image are preprocessed such as cropping, denoising and filtering to ensure the accuracy and reliability of the data.

[0054] The cropping operation is used to remove unnecessary background information and focus the image on the back region. According to the relative position of the camera and the standardized posture of the examinee, the interference area in the image (mainly the surrounding environment or other non-back parts) can be removed, and only the back region that needs to be analyzed is retained.

[0055] Due to environmental light or other interference, the collected image may be affected by noise. The denoising operation can reduce the influence of random noise through smoothing processing, improve the accuracy of image data, and use average smoothing or median filtering to remove image data that does not conform to the actual characteristics of the back.

[0056] The filtering operation can further smooth the image data and remove small interference information to make the image smoother. This operation can ensure the accuracy of subsequent alignment and key point detection by weakening the small changes in the image.

[0057] Step two, registration and fusion.

[0058] When collecting data, there is a small angle difference between the collected RGB image and the depth image. In order to eliminate this difference, data registration is needed. The purpose of registration is to ensure that the RGB image and the depth image are aligned in the same coordinate system, that is, each RGB pixel point has a corresponding depth value.

[0059] In actual operation, data registration mainly analyzes the similarity between the RGB image and the depth image, and gradually adjusts the position of the two images until the corresponding relationship between their points reaches the optimum. After completing the registration, the RGB image and the depth image can be fused to obtain a three-dimensional data structure containing color and depth values (i.e. RGB-D data). This fused data closely combines RGB information and depth information, providing a comprehensive three-dimensional image for subsequent analysis.

[0060] Step three, data alignment.

[0061] The RGB-D data mainly provides color and depth information, while the point cloud data is the three-dimensional structure of the back region. Through the alignment operation, the spatial coordinates of the two types of data can be ensured to be one-to-one correspondence (i.e., mapping the RGB-D data and the point cloud data to the same coordinate system). After alignment, the back information and depth information will be closely combined with the spatial position of the three-dimensional point. Each pixel of the RGB-D data has a corresponding three-dimensional point in the point cloud data, which provides high-precision data for subsequent analysis of the features of the back.

[0062] Step four, detecting and identifying key points.

[0063] In the fused RGB-D data and point cloud data, the key points of the back, such as the shoulder, neck and center of the spine, need to be further detected and identified. Identifying these key points is very important for accurately extracting the central region of the back.

[0064] The key points of the shoulder, neck and other parts can be determined based on the depth image. Through the position information of these key points in the RGB image, these key points can be further mapped to the three-dimensional coordinates in the point cloud data, so as to accurately locate these key points in the three-dimensional space. By detecting the key points, the boundary of the back region can be better defined, and subsequent analysis can be focused on the core region of the back.

[0065] Step five, cropping the central region of the back.

[0066] According to the detected key points, the regions of the point cloud data unrelated to the back can be removed, and only the central region of the back is retained. The goal of this cropping operation is to further focus on the core region of the back and remove unnecessary data outside the shoulder, ensuring that the data is concentrated on the part that needs to be analyzed.

[0067] In the cropping process, the left shoulder key point, the right shoulder key point, the spine key point and other key points are used as reference to define a three-dimensional rectangle or other suitable region, and only the point cloud data within this region is retained. After cropping, the obtained three-dimensional data only contains the point cloud information of the central region of the back, which is convenient for subsequent analysis and screening.

[0068] The above data processing method unifies various data sources to the same coordinate system through standardized acquisition method, accurate registration and fusion technology, so that the three-dimensional point cloud data of the back is clearly and accurately presented. This processing method not only ensures the real-time performance of the data, but also provides solid data support for back abnormal screening and spine analysis.

[0069] Of course, it can be understood that the data acquisition device in the above two embodiments can also be integrated in the electronic device, i.e., the electronic device itself performs data acquisition.

[0070] Step 102: For any target point, determine the asymmetry index of the target point based on each point in the specific region corresponding to the target point.

[0071] The target point is any point in the point cloud data of the central region of the back. The specific region includes the target point, N first neighboring points to the left of the target point, and N second neighboring points to the right of the target point. N is an integer greater than zero. The N first neighboring points, N second neighboring points, and the target point are on the same target line. The target line is a line parallel to the shoulders of the human body within the central region of the back. The asymmetry index of the target point characterizes the asymmetry of the specific region.

[0072] The number of first adjacent points and second adjacent points are the same. The N first adjacent points are the N points on the target line that are to the left of the target point and immediately adjacent to it. The N second adjacent points are the N points on the target line that are to the right of the target point and immediately adjacent to it. For example... Figure 5 The image shown is an example diagram of the first and second adjacent points. Figure 5 The wavy line in the image represents the target line. Figure 5 Points p and i in the equation are both extreme points, and they lie on the same target line. The three points to the left of point i on this target line are considered the first adjacent points, and the three points to the right of point i on this target line are considered the second adjacent points. When using... Figure 5 Based on the positional information of point cloud data represented by the mid-axis, the shoulder can be represented by a line connecting the left and right shoulder keypoints and the points with the same Y-coordinate as these two keypoints.

[0073] The aforementioned specific area is the target point, and the area formed by the N first adjacent points to the left of the target point and the N second adjacent points to the right of the target point.

[0074] In one possible implementation, before executing step 102, the number N of the first and second adjacent points can be determined based on the position information of the target point in the central region of the back, in order to achieve a balance between the calculation efficiency and accuracy of the asymmetry index. That is, by flexibly adjusting the number N, the optimal compromise point in the calculation process of the asymmetry index can be found, thereby achieving a balance between calculation efficiency and accuracy.

[0075] The correspondence between the different position information and the different number N (the position information closer to the edge of the back central region corresponds to a smaller number N, and the position information closer to the center of the back central region corresponds to a larger number N) can be set in advance, and the corresponding number N can be found from the correspondence based on the position information of the target point in the back central region; or the back central region can be divided into an edge region and a center region, and based on the position information of the target point in the back central region, it can be determined whether the target point is located in the edge region or the center region. If it is located in the edge region, the number N is set to a smaller value, and if it is located in the center region, the number N can be set to a larger value.

[0076] Before step 102 is performed, the point cloud data of the back central region obtained in step 101 can be processed. As an example but not limitation, the processing includes: performing 2D Delaunay fitting on the point cloud data of the back central region, with a tolerance of 0.005, to generate a smooth triangular mesh to denoise the point cloud data of the back central region; data downsampling: applying a reduction factor of 0.1 to downsample the denoised point cloud data of the back central region to reduce data density while maintaining surface characteristics; surface smoothing processing: performing 10,000 iterations of surface smoothing on the downsampled point cloud data of the back central region with a relaxation factor of 0.0001 and enabling boundary smoothing (e.g., using Gaussian filtering for smoothing processing on the boundary region), reducing noise and ensuring data smoothness and stability. After the above processing of the point cloud data of the back central region, an effective calculation range can also be established for each point according to the distribution of the surface height of the back central region (e.g., a height range is set in advance, and points exceeding this height range are invalid points and are not calculated for the asymmetry index). This can avoid inaccurate calculation results in areas with small surface curvature or small curvature.

[0077] In one possible implementation, step 102 can include:

[0078] determining the principal curvature of each point;

[0079] determining the target parameter of each point based on the principal curvature of each point, the target parameter being at least one of the average curvature, the Gaussian curvature, the principal curvature difference, the second-order smoothness, and the third-order smoothness;

[0080] determining the asymmetry index of the target point based on the target parameter of each point.

[0081] The mean curvature is an "extrinsic" measure that characterizes the local bending of a surface in the ambient space. On the other hand, the Gaussian curvature is an "intrinsic" measure of curvature that depends only on how distances are measured on the surface itself, and not on its embedding in space. Simply put, any non-stretching transformation of a surface does not change its Gaussian curvature. For a person with scoliosis, a change in the curvature of the spine will cause the ribs to protrude, the shoulders to be uneven, and the back muscles to be asymmetric. These deviations will affect the curvature of the back plate, and this change can be used to quantify the degree of back asymmetry. The second-order smoothness and the third-order smoothness are used to describe the high-order curvature characteristics of the surface geometry, and are usually used to characterize the smoothness and symmetry of the surface morphology. Based on the target parameters such as mean curvature, Gaussian curvature, principal curvature difference, second-order smoothness and third-order smoothness, the surface asymmetry can be more robustly and meticulously evaluated, providing valuable reference for in-depth understanding of the surface shape characteristics.

[0082] The mean curvature, Gaussian curvature, second-order smoothness and third-order smoothness are all high-order curvatures. Based on the high-order curvature of the point cloud, the spine line can be accurately and efficiently generated in the embodiment, providing a feasible and effective technical solution for large-scale screening of scoliosis.

[0083] The electronic device can generate a covariance matrix of each point based on the nearest neighbor points of each point (for example, the nearest distance is determined by the Euclidean distance, and the points within a radius r of the Euclidean distance are selected as the nearest neighbor points, and the radius r can be pre-set according to actual needs). The covariance matrix reflects the dispersion of each point and the nearest neighbor points in different axes. By calculating the eigenvalues and eigenvectors of the covariance matrix, the principal curvatures of each point can be obtained. Based on the principal curvatures, the Gaussian curvature, the mean curvature and the principal curvature difference can be calculated, and based on the Gaussian curvature and the mean curvature, the second-order smoothness and the third-order smoothness can be calculated. As an example but not limited to, each point described above is point A in the point cloud data. Based on the principal curvatures of point A, the mean curvature, the Gaussian curvature, the principal curvature difference, the second-order smoothness and the third-order smoothness of point A can be determined.

[0084] wherein the principal curvatures of each point include the maximum principal curvature and the minimum principal curvature of the point. The difference between the maximum principal curvature and the minimum principal curvature is the principal curvature difference. The average of the maximum principal curvature and the minimum principal curvature is the mean curvature.

[0085] The calculation formula of the second-order smoothness is as follows:

[0086] PHI1 = (3H 2 -K)

[0087] wherein PHI1 represents the second-order smoothness, H represents the mean curvature, and K represents the Gaussian curvature.

[0088] The second-order smoothness combines the information of mean curvature and Gaussian curvature to describe the second-order deviation of the surface smoothness. In the flat area (such as a plane), both the mean curvature and the Gaussian curvature tend to 0, and thus the second-order smoothness is close to 0. In the case of a minimal surface (i.e., the Gaussian curvature is negative, such as a saddle surface), the value of the second-order smoothness is more affected. The second-order smoothness can be used to measure the smoothness of the surface, and a larger value may indicate that the surface has stronger bending features.

[0089] The third-order smoothness is calculated as follows:

[0090] PHI2 = (H 2 -K)(5H 2 -K)

[0091] where PHI2 represents the third-order smoothness.

[0092] (H 2 -K) is an important geometric quantity that describes the local curvature distribution. (5H 2 -K) further enhances the curvature distribution, so that the third-order smoothness can reflect more complex morphological information. The third-order smoothness enhances the sensitivity to the surface change through higher-order partial derivatives, and is suitable for detecting the local complexity of the surface, such as protrusions, depressions, and abrupt points. Compared with the second-order smoothness, the third-order smoothness can capture more complex geometric changes and is suitable for surface asymmetry analysis or feature point detection.

[0093] In one possible implementation, the asymmetry index of the target point is determined based on the target parameters of each point, including:

[0094] The asymmetry index corresponding to each target parameter is determined based on the target parameters of each point.

[0095] The asymmetry index of the target point is determined based on the asymmetry index corresponding to each target parameter.

[0096] For any target parameter, the asymmetry index corresponding to the target parameter is determined based on the target parameter of all points in the corresponding specific area.

[0097] The weights of the target parameters can be preset, and the asymmetry indices corresponding to the target parameters are weighted and summed based on the weights of the target parameters to obtain the asymmetry index of the target point. As an example but not limitation, the target parameters are the mean curvature, the Gaussian curvature, the principal curvature difference, the second-order smoothness, and the third-order smoothness, and the weights of the five target parameters are equal, each being 0.2. Then, based on the weights of the five target parameters, the asymmetry indices corresponding to the five target parameters can be weighted and summed, that is, the average of the asymmetry indices corresponding to the five target parameters is the asymmetry index of the target point.

[0098] In a possible implementation, the asymmetry index corresponding to each target parameter is determined based on the target parameter of each point, comprising:

[0099] For any target parameter, the absolute value of the difference of the target parameter between all adjacent points in a specific region is calculated;

[0100] Based on the absolute value of the difference of the target parameter between all adjacent points, the asymmetry index corresponding to the target parameter is determined.

[0101] The calculation formula of the asymmetry index corresponding to any target parameter is as follows:

[0102]

[0103] Wherein, i represents the horizontal coordinate of the target point; i-1 represents the horizontal coordinate of the point adjacent to the target point on the left side of the target point, i-N represents the point adjacent to the target point on the left side and the point with horizontal coordinate i-N+1, the point adjacent to the target point on the left side and the point with horizontal coordinate i-N+1 are both first adjacent points; i+1 represents the horizontal coordinate of the point adjacent to the target point on the right side of the target point, i+N represents the point adjacent to the target point on the right side and the point with horizontal coordinate i+N-1, the point adjacent to the target point on the right side and the point with horizontal coordinate i+N-1 are both second adjacent points; f k represents the target parameter of the point with horizontal coordinate k; f k+1 represents the target parameter of the point with horizontal coordinate k+1.

[0104] Step 103, generating the spine line of the human body based on the asymmetry index of the target point.

[0105] Since the asymmetry index of the target point represents the asymmetry of a specific region, the asymmetry of a specific region in the central region of the back of the human body can reflect the distortion of the back, and the distortion of the back can reflect the lateral bending condition of the back, so the present application can accurately and efficiently generate the spine line of the human body based on the asymmetry index of the target point under the condition of reducing the operation of professional medical personnel, and is suitable for large-scale screening of spinal scoliosis.

[0106] In a possible implementation, before step 103 is performed, it can further include:

[0107] If there is an outlier value in the asymmetry index of the target point, the target asymmetry index of the target point corresponding to the outlier value is determined based on the asymmetry index of the adjacent point of the target point corresponding to the outlier value;

[0108] The asymmetry index of the target point corresponding to the outlier value is updated as the target asymmetry index.

[0109] After determining the asymmetry indicators of any point (i.e. target point) in the point cloud data in the back central region by step 102, all the asymmetry indicators can be cleaned to ensure the reliability and stability of the data for subsequent spine line generation. Cleaning all the asymmetry indicators includes identifying and processing outliers in all the asymmetry indicators to avoid interference with further analysis. Specifically, it includes: first removing the NaN values in all the asymmetry indicators; then calculating the first quartile, the third quartile and the interquartile range (IQR), based on which the upper and lower boundaries for identifying outliers are determined, and based on the upper and lower boundaries for identifying outliers, it is determined whether there are outliers in the non-NaN values of all the asymmetry indicators; if there are outliers, the neighboring points of the target point corresponding to the outliers (i.e. the points adjacent to the target point corresponding to the outliers in the back central region) are determined, based on the asymmetry indicators of these neighboring points, the target asymmetry indicator of the target point corresponding to the outliers is determined (for example, the average of the asymmetry indicators of these neighboring points is taken as the target asymmetry indicator), and the target asymmetry indicator is used to replace the outliers, so as to reduce the influence of the outliers while maintaining the smoothness and consistency of the data. Of course, the outliers can also be directly set to NaN values to further reduce the outliers that may have a negative impact on the analysis results.

[0110] In one possible implementation, step 103 described above can include:

[0111] For any target line, based on the asymmetry indicators of the target points on the target line, extreme points on the target line are determined;

[0112] Based on the neighboring points in the extreme points on each target line, at least one extreme point set is formed;

[0113] The longest extreme point set in the at least one extreme point set is fitted to obtain the spine line of the human body.

[0114] Before the extreme points on the target line, the asymmetry indicators of the target points on each target line can also be smoothed, and on this basis, the extreme points (including local maximum or local minimum) are detected. These extreme points represent some significant features on the surface. According to the detected extreme points, a connected two-dimensional grid can be created, with 0 representing non-extreme points and 1 representing extreme points, to represent the connection relationship between the extreme points. By analyzing the set of connection points, especially the longest set, the extreme points of the set can be fitted to obtain the most significant center line of the surface. The center line represents the main shape feature of the surface and can be used as the spine line of the back to analyze whether there is a lateral curvature of the spine on the left and right sides. Figure 6An example graph of a set of extreme points is shown, Figure 6 The red dots in FIG. 6 are extreme points, and adjacent extreme points form a set of extreme points. Figure 6 It can be seen that there are multiple sets of extreme points, and the set of extreme points near the center of the central region of the back has the longest length. The spinal column line can be generated by fitting the set of extreme points with the longest length.

[0115] In this embodiment, to evaluate the reliability of the fitted midline, the point cloud data in the central region of the back of 161 adolescent idiopathic scoliosis (AIS) patients with a Cobb angle ranging from 2.7 degrees to 39.4 degrees is fitted for the spinal column line. The average execution time of the electronic device from inputting the original point cloud to outputting the final reconstructed spinal column line is less than three seconds. Subsequently, these fitted spinal column lines are systematically compared with the spinal column lines extracted from the corresponding X-ray images. Specifically, two key methods are used to measure the similarity between the fitted spinal column lines and the spinal column lines extracted from the corresponding X-ray images: Fourier descriptor and curvature feature analysis. The similarity between curves can be captured from different angles and the errors caused by various factors can be reduced. The experimental results are shown in Table 1 below, which shows that the spinal column lines fitted in this embodiment can better match the spinal column lines extracted from the X-ray images under the two key methods, and exhibit strong correlation.

[0116] Table 1

[0117]

[0118] The Fourier descriptor is based on frequency domain analysis and can effectively capture the overall morphological features of the spinal column and has strong noise resistance. Its high correlation (0.946) indicates that this method can stably match the spinal column midline and is suitable for cases with regular morphological features. At the same time, its low variance (0.00516) and low standard deviation (0.009) mean that this method has small volatility and stronger stability between different individuals.

[0119] The curvature feature method directly depends on the geometric information of the spinal column surface and can well capture local morphological changes. Its correlation reaches 0.903, indicating that this method can accurately fit the spinal column line to some extent, especially in cases with complex local features. It is shown that the fitted spinal column line and the spinal column line in the X-ray image have high correlation.

[0120] In the embodiment of the present application, the point cloud data of the central region of the back of the human body is obtained, and for any point (i.e., a target point) in the point cloud data, the asymmetry index of the target point is determined based on each point in a specific region corresponding to the target point. The asymmetry index of the target point represents the asymmetry of the specific region. The asymmetry of the specific region in the central region of the back of the human body can reflect the distortion of the back. The distortion of the back can reflect the scoliosis of the back. Therefore, the present application can accurately and efficiently generate the spine line of the human body based on the asymmetry index of the target point under the condition of reducing the operation of professional medical personnel, and is suitable for large-scale screening of scoliosis.

[0121] It should be understood that the size of the serial number of each step in the above embodiment does not mean the order of execution. The execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

[0122] According to the spine line generation method described in the above embodiment, Figure 7 The structure diagram of the spine line generation device provided by the embodiment of the present application is shown. For the convenience of description, only the part related to the embodiment of the present application is shown.

[0123] Referring to Figure 7 The spine line generation device comprises:

[0124] The point cloud acquisition module 701 is configured to acquire point cloud data of a central region of a back of a human body.

[0125] The index determination module 702 is configured to determine, for any target point, an asymmetry index of the target point based on each point in a specific region corresponding to the target point. The target point is any point in the point cloud data of the central region of the back of the human body. The specific region includes the target point, N first adjacent points located on the left side of the target point, and N second adjacent points located on the right side of the target point. N is an integer greater than zero. The N first adjacent points, the N second adjacent points, and the target point are on the same target line. The target line is a line in the central region of the back of the human body parallel to the shoulder of the human body. The asymmetry index of the target point represents the asymmetry of the specific region.

[0126] The spine line generation module 703 is configured to generate a spine line of the human body based on the asymmetry index of the target point.

[0127] Optionally, the index determination module 702 comprises:

[0128] The curvature determination unit is configured to determine the principal curvature of each point.

[0129] determine a target parameter of each point based on the principal curvatures of the point, the target parameter being at least one of mean curvature, Gaussian curvature, principal curvature difference, second-order smoothness and third-order smoothness;

[0130] an index determination unit configured to determine an asymmetry index of the target point based on the target parameter of each point.

[0131] Optionally, the index determination unit comprises:

[0132] a first determination sub-unit configured to determine an asymmetry index corresponding to each target parameter based on the target parameter of each point.

[0133] a second determination sub-unit configured to determine the asymmetry index of the target point based on the asymmetry index corresponding to each target parameter.

[0134] Optionally, the first determination sub-unit is specifically configured to:

[0135] for any target parameter, calculate absolute values of differences of the target parameter between all adjacent points in the specific region;

[0136] determine the asymmetry index corresponding to the target parameter based on the absolute values of the differences of the target parameter between all adjacent points.

[0137] Optionally, the spine line generation module 703 is specifically configured to:

[0138] for any target line, determine an extreme point on the target line based on the asymmetry index of the target point on the target line;

[0139] based on adjacent points in the extreme points on each target line, form at least one extreme point set;

[0140] fit the extreme point set with the longest length in the at least one extreme point set to obtain the spine line of the human body.

[0141] Optionally, the spine line generation device further comprises:

[0142] a number determination module configured to determine a number N of the first adjacent point and the second adjacent point based on the position information of the target point in the back central region.

[0143] Optionally, the spine line generation device further comprises:

[0144] The target determination module is used to determine the target asymmetry index of the target point corresponding to the outlier based on the asymmetry index of the neighboring points of the target point corresponding to the outlier if there is an outlier in the asymmetry index of the target point.

[0145] The indicator update module is used to update the asymmetric indicator of the target point corresponding to the outlier to the target asymmetric indicator.

[0146] Optionally, the point cloud acquisition module 701 described above is specifically used for:

[0147] Acquire depth images and point cloud data of the back of the human body;

[0148] Based on the depth image of the back, the key points of the left shoulder, right shoulder, neck, and waist of the human body are determined.

[0149] Based on the key points of the left shoulder, the right shoulder, the neck, and the waist, the point cloud data of the central region of the back is determined from the point cloud data of the back.

[0150] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0151] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 8 As shown, the electronic device 8 of this embodiment includes: at least one processor 80 ( Figure 8 (Only one is shown in the diagram), memory 81, and computer program 82 stored in said memory 81 and executable on said at least one processor 80, which, when executed, implements the steps in any of the above method embodiments.

[0152] The electronic device may include, but is not limited to, a processor 80 and a memory 81. Those skilled in the art will understand that... Figure 8 This is merely an example of electronic device 8 and does not constitute a limitation on electronic device 8. It may include more or fewer components than shown, or combine certain components, or different components. For example, it may also include input / output devices, network access devices, etc.

[0153] The processor 80 can be a central processing unit (CPU), and can also be other general-purpose processors, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0154] The memory 81 can be an internal storage unit of the electronic device 8 in some embodiments, such as a hard disk or a memory of the electronic device 8. The memory 81 can also be an external storage device of the electronic device 8 in other embodiments, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 8. Further, the memory 81 can include both the internal storage unit and the external storage device of the electronic device 8. The memory 81 is used to store an operating system, an application program, a boot loader, data, and other programs, such as program codes of the computer program, etc. The memory 81 can also be used to temporarily store data that has been output or will be output.

[0155] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above functional units and modules is exemplified, and in actual application, the above functions can be completed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to complete all or part of the above-described functions. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit, and the integrated unit can be realized in the form of hardware or in the form of software functional unit. In addition, the specific names of each functional unit and module are only for easy distinction, and do not limit the protection scope of the present application. The specific working process of the units and modules in the system can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0156] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the present application can implement all or part of the processes in the above-mentioned embodiment methods through a computer program to instruct relevant hardware to complete, and the computer program can be stored in a computer readable storage medium. When the computer program is executed by a processor, the steps of each method embodiment described above can be implemented. The computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate forms. The computer readable medium at least includes any entity or device capable of carrying the computer program code to the device / equipment, recording medium, computer memory, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), electrical carrier signal, telecommunication signal and software distribution medium. For example, U disk, mobile hard disk, magnetic disk or optical disk, etc. In some jurisdictions, according to legislation and patent practice, the computer readable medium can not be an electrical carrier signal and a telecommunication signal.

[0157] In the above embodiments, the description of each embodiment has its own focus, and the parts not described or recorded in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0158] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in connection with the embodiments disclosed herein can be realized by electronic hardware, or a combination of computer software and electronic hardware. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0159] In the embodiments provided by the present application, it should be understood that the disclosed apparatus / equipment and method can be implemented by other ways. For example, the above-mentioned apparatus / equipment embodiments are only schematic, and the division of the modules or units is only a logical function division, and there can be another division way in actual implementation, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection between each displayed or discussed unit can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or other forms.

[0160] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may also be distributed to multiple network units. Part or all of the units can be selected to achieve the purpose of the embodiment scheme according to actual needs.

[0161] The above embodiments are only used to illustrate the technical solutions of the present application, but not limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can still be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A method for generating a spinal line, characterized in that, The spine line generation method comprises: Obtaining point cloud data of a central region of a back of a human body; For any target point, determining an asymmetry index of the target point based on each point in a specific region corresponding to the target point, the target point being any point in the point cloud data of the central region of the back, the specific region including the target point, N first neighboring points located on a left side of the target point, and N second neighboring points located on a right side of the target point, N being an integer greater than zero, the N first neighboring points, the N second neighboring points, and the target point being on a same target line, the target line being a line in the central region of the back parallel to a shoulder of the human body, the asymmetry index of the target point representing asymmetry of the specific region; Generating a spine line of the human body based on the asymmetry index of the target point; The method comprises: Determining principal curvatures of the each point; Determining a target parameter of the each point based on the principal curvatures of the each point, the target parameter being at least one of mean curvature, Gaussian curvature, principal curvature difference, second-order smoothness, and third-order smoothness; Determining the asymmetry index of the target point based on the target parameter of the each point; The method comprises: For any target line, determining extreme points on the target line based on the asymmetry index of the target point on the target line; Forming at least one extreme point set based on adjacent points in the extreme points on the target lines; Fitting the extreme point set with the longest length in the at least one extreme point set to obtain the spine line of the human body.

2. The spine line generation method of claim 1, wherein, The method comprises: Determining an asymmetry index corresponding to each target parameter based on the target parameter of the each point; Determining the asymmetry index of the target point based on the asymmetry index corresponding to each target parameter.

3. The spine line generation method of claim 2, wherein, The method comprises: For any target parameter, calculating absolute values of differences of the target parameter between all adjacent points in the specific region; Determining the asymmetry index corresponding to the target parameter based on the absolute values of the differences of the target parameter between the all adjacent points.

4. The spine line generation method according to any one of claims 1 to 3, characterized in that, Before determining the asymmetry index of the target point based on each point in a specific region corresponding to the target point, the method further comprises: Determining the number N of the first neighboring points and the second neighboring points based on position information of the target point in the central region of the back.

5. The spine line generation method according to any one of claims 1 to 3, characterized in that, Before generating the spine line of the human body based on the asymmetry index of the target point, the method further comprises: If there is an outlier in the asymmetry index of the target point, determining a target asymmetry index of the target point corresponding to the outlier based on asymmetry indices of adjacent points of the target point corresponding to the outlier. update the asymmetry index of the target point corresponding to the outlier to the target asymmetry index.

6. The spine line generation method according to any one of claims 1 to 3, characterized in that, The point cloud data of the central region of the back of the human body is acquired, including: acquiring a depth image and point cloud data of the back of the human body; determining a left shoulder key point, a right shoulder key point, a neck key point, and a waist key point of the human body based on the depth image of the back; determining point cloud data of the central region of the back from the point cloud data of the back based on the left shoulder key point, the right shoulder key point, the neck key point, and the waist key point.

7. A spinal cord line generating device characterized by, The spine line generation device comprises: a point cloud acquisition module configured to acquire point cloud data of a central region of a back of a human body; an index determination module configured to, for any target point, determine an asymmetry index of the target point based on each point in a specific region corresponding to the target point, the target point being any point in the point cloud data of the central region of the back, the specific region including the target point, N first neighboring points located on the left side of the target point, and N second neighboring points located on the right side of the target point, N being an integer greater than zero, the N first neighboring points, the N second neighboring points, and the target point being on a same target line, the target line being a line in the central region of the back parallel to the shoulder of the human body, the asymmetry index of the target point representing the asymmetry of the specific region; a spine line generation module configured to generate a spine line of the human body based on the asymmetry index of the target point; The index determination module comprises: a curvature determination unit configured to determine a principal curvature of the each point; a parameter determination unit configured to determine a target parameter of the each point based on the principal curvature of the each point, the target parameter being at least one of an average curvature, a Gaussian curvature, a principal curvature difference, a second-order smoothness, and a third-order smoothness; an index determination unit configured to determine the asymmetry index of the target point based on the target parameter of the each point; The spine line generation module is specifically configured to: for any target line, determine extreme points on the target line based on the asymmetry indices of the target points on the target line; based on adjacent points in the extreme points on each target line, form at least one extreme point set; fit the extreme point set with the longest length in the at least one extreme point set to obtain the spine line of the human body.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program, so that the electronic device implements the method in any one of claims 1 to 6.

Images