Spine key point detection method and system

Abstract

The invention relates to the technical field of body visual detection, and particularly discloses a spine key point detection method and system, and the method comprises the steps: shooting an image in a preset direction of a user, and carrying out the contour recognition of the image, and obtaining an actual human body model; querying feature points in the actual human body model, fitting a feature curve based on the feature points, and taking the feature curve as a spine feature of the user in each direction; comparing the spine features in each direction in the actual human body model with standard features, marking key points according to comparison results, and synchronously determining spine scores; and training a neural network model from spine features to spine scores. The spine features are extracted based on an image recognition algorithm, a comparison scheme based on a human body model is provided, the spine score is judged, meanwhile, a neural network model for training the spine features to the spine score is provided, when the spine is simply evaluated, the spine score can be quickly responded, and the efficiency is extremely high.

Description

Technical Field

[0001] The present invention relates to the technical field of body visual detection, and in particular to a method and system for detecting key points of a spine. Background Art

[0002] Commonly used scoliosis screening tools include traditional scoliosis measuring rulers and portable electronic scoliosis screening devices. The principle of both is to assess the severity of scoliosis by measuring the rotation angle of the spinal vertebrae.

[0003] The electronic scoliosis screening device uses three-dimensional electronic acceleration angle sensing and gravity inclination technology to measure the angle values ​​of spinal deformity, abnormal posture, and abnormal mobility in real time and accurately, thereby evaluating whether the spine is normal. However, the electronic scoliosis measuring device is expensive, and requires a handheld mobile terminal to follow the subject's movements for measurement, which is relatively cumbersome to operate. Therefore, more convenient spinal detection equipment came into being.

[0004] Most of the existing convenient spinal detection equipment are based on visual detectors. However, visual detectors need to go through a large number of calculation processes before they can output results, which is inefficient. How to improve the response speed of the visual detector is a technical problem that the technical solution of the present invention wants to solve. Summary of the invention

[0005] The purpose of the present invention is to provide a method and system for detecting key points of the spine to solve the problems raised in the above-mentioned background technology.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] A spine key point detection method and system, including.

[0008] Collecting the user's gender, height, weight and age at least once, and searching for a standard human body model in a preset human body model library according to the collected gender, height, weight and age; the standard human body model is a three-dimensional contour model;

[0009] Take an image in the direction preset by the user, perform contour recognition on the image, and obtain an actual human body model;

[0010] Querying feature points in the actual human body model, and fitting feature curves based on the feature points as the spine features of the user in each direction;

[0011] Query the spine features of the standard human body model in each direction as standard features, compare the spine features in each direction of the actual human body model with its standard features, mark key points according to the comparison results, and simultaneously determine the spine score;

[0012] A neural network model is trained from spinal features to spinal scores, and the trained neural network model is used as a trigger for the comparison process in the actual application stage.

[0013] As a further solution of the present invention: the step of collecting the gender, height, weight and age of the user at least once, and searching for a standard human body model in a preset human body model library according to the collected gender, height, weight and age comprises:

[0014] Receive the detection request input by the user and open the acquisition module;

[0015] The collecting module collects the gender, height, weight and age of the user at least once, and the collected gender, height, weight and age are stored in the form of an array; the elements in the array correspond to time;

[0016] Count the mode values ​​in each array in a collection period, and use the mode value as the final gender, height, weight and age;

[0017] Use gender, height, weight and age as indexes to query standard human models in the preset human model library.

[0018] As a further solution of the present invention, the step of capturing an image in a direction preset by a user, performing contour recognition on the image, and obtaining an actual human body model comprises:

[0019] Create a baseline model based on final gender, height, weight, and age;

[0020] The user image is photographed in a preset direction, and the contour of the user image is recognized; the directions include at least a front view direction, a rear view direction, a top view direction, and a side view direction, and also include other directions that are at preset angles to the front view reference plane, the top view reference plane, and the front view reference plane; the directions are represented by vectors;

[0021] When the direction belongs to the forward viewing direction, the rearward viewing direction, the downward viewing direction and the side viewing direction, the reference model is adjusted according to the contour recognition result;

[0022] When the direction belongs to other directions at a preset angle to the front view reference plane, the top view reference plane, and the front view reference plane, the adjusted reference model is verified according to the contour recognition result.

[0023] As a further solution of the present invention: the step of querying feature points in the actual human body model and fitting feature curves based on the feature points as the spine features of the user in each direction includes:

[0024] Querying the characteristic area according to the preset position relationship; the characteristic points include the left acromion, the right acromion, the left inferior angle of the scapula, the right inferior angle of the scapula, the left lumbar fossa, the right lumbar fossa, the left iliac crest, the right iliac crest and the line where the spinous process is located;

[0025] Selecting feature points in the feature area, connecting the feature points, and fitting a feature curve;

[0026] Obtain the projection of the characteristic curve in each direction to obtain the spine feature in each direction;

[0027] The process of fitting the characteristic curve is:

[0028] Define the fitting function z(x,y)=ax 2 +by 2 +cxy+dx+ey+f;

[0029] Input the feature point coordinates into the fitting function z(x,y), and solve the fitting parameters a, b, c, d, e and f based on the least squares method to minimize the objective function

[0030] The mean square error is calculated to evaluate the fitting effect; the mean square error is as follows:

[0031]

[0032] As a further solution of the present invention: the steps of comparing the spine features in each direction of the actual human body model with the standard features, marking key points according to the comparison results, and simultaneously determining the spine score include:

[0033] Query the spine features of the standard human body model in each direction as standard features;

[0034] Read the spinal features in each direction of the actual human body model, compare them with the standard features, mark the abnormal segments, and simultaneously calculate the abnormality degree;

[0035] When the abnormality reaches a preset abnormality threshold, the nearest feature point is marked as a key point;

[0036] Count the key points and their abnormalities to determine the spinal score;

[0037] The process of reading the spinal features in each direction of the actual human body model, comparing them with the standard features, marking the abnormal segments, and simultaneously calculating the abnormality degree includes:

[0038] Based on the DTW algorithm, the abnormal spinal features and their standard features are matched, and the distance between each pair of matching points is calculated in turn. When the distance is greater than the preset distance threshold, it is marked as an abnormal point. The abnormal points are connected to obtain the abnormal segment, and the mean distance of each abnormal point in the abnormal segment is calculated as the abnormality degree.

[0039] As a further solution of the present invention: the step of training the neural network model from spinal features to spinal scores and using the trained neural network model as a trigger for the comparison process in the actual application stage comprises:

[0040] The spinal features and spinal scores were counted to construct a sample set;

[0041] The neural network model is trained based on the sample set, and when the error rate of the neural network model is less than a preset error threshold, the trained neural network model is output;

[0042] When the user's spinal features in each direction are obtained, the spinal features are input into the trained neural network model to obtain a spinal score;

[0043] After every preset number of tests, the comparison process is applied to the spinal features with spinal scores greater than the preset score threshold to verify the accuracy of the neural network model.

[0044] The technical solution of the present invention also provides a spine key point detection system, the system comprising:

[0045] A human body model query module is used to collect the gender, height, weight and age of the user at least once, and query a standard human body model in a preset human body model library according to the collected gender, height, weight and age; the standard human body model is a three-dimensional contour model;

[0046] The contour recognition module is used to capture images in the direction preset by the user, perform contour recognition on the images, and obtain an actual human body model;

[0047] A feature extraction and fitting module, used for querying feature points in the actual human body model, and fitting feature curves based on the feature points as the spinal features of the user in each direction;

[0048] The spine comparison and determination module is used to query the spine features of the standard human body model in each direction, and compare the spine features in each direction of the actual human body model with its standard features as standard features, mark key points according to the comparison results, and simultaneously determine the spine score;

[0049] The pre-trigger module is used to train the neural network model from spinal features to spinal scores, and the trained neural network model is used as a trigger for the comparison process in the actual application stage.

[0050] As a further solution of the present invention: the human body model query module includes:

[0051] The acquisition module opening unit is used to receive the detection request input by the user and open the acquisition module;

[0052] A data collection and storage unit, used for collecting the gender, height, weight and age of the user at least once based on the collection module, and storing the collected gender, height, weight and age in the form of an array; the elements in the array correspond to time;

[0053] A mode value counting unit, used to count the mode values ​​in each array in a collection period, and use the mode value as the final gender, height, weight and age;

[0054] The majority value matching unit is used to use gender, height, weight and age as indexes to query standard human body models in a preset human body model library.

[0055] As a further solution of the present invention: the contour recognition module includes:

[0056] A benchmark model creation unit, used to create a benchmark model according to the final gender, height, weight and age;

[0057] The recognition execution unit is used to capture the user image in a preset direction and perform contour recognition on the user image; the directions include at least a front view direction, a rear view direction, a top view direction and a side view direction, and also include other directions that are at preset angles to the front view reference plane, the top view reference plane and the front view reference plane; the directions are represented by vectors;

[0058] A model adjustment unit, used for adjusting the reference model according to the contour recognition result when the direction belongs to the front view direction, the rear view direction, the top view direction and the side view direction;

[0059] The model verification unit is used to verify the adjusted reference model according to the contour recognition result when the direction belongs to other directions with a preset angle to the front view reference plane, the top view reference plane and the front view reference plane.

[0060] As a further solution of the present invention: the feature extraction and fitting module includes:

[0061] A region query unit, used to query a feature region according to a preset position relationship; the feature points include the left acromion, the right acromion, the left inferior angle of the scapula, the right inferior angle of the scapula, the left lumbar fossa, the right lumbar fossa, the left iliac crest, the right iliac crest and the line where the spinous process is located;

[0062] A curve fitting unit, used for selecting feature points in the feature area, connecting the feature points, and fitting a feature curve;

[0063] A projection calculation unit, used to obtain the projection of the characteristic curve in each direction to obtain the spine feature in each direction;

[0064] The process of fitting the characteristic curve is:

[0065] Define the fitting function z(x,y)=ax 2+by 2 +cxy+dx+ey+f;

[0066] Input the feature point coordinates into the fitting function z(x,y), and solve the fitting parameters a, b, c, d, e and f based on the least squares method to minimize the objective function

[0067] The mean square error is calculated to evaluate the fitting effect; the mean square error is as follows:

[0068]

[0069] Compared with the prior art, the beneficial effects of the present invention are: the present invention extracts spinal features based on an image recognition algorithm, provides a comparison scheme based on a human body model, and determines the spinal score. At the same time, a neural network model that trains spinal features to spinal scores can quickly respond to spinal scores when performing a simple assessment of the spine, with extremely high efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention.

[0071] Figure 1 This is a flowchart of the spine key point detection method.

[0072] Figure 2 This is the first sub-process flowchart of the spine key point detection method.

[0073] Figure 3 This is the second sub-process flowchart of the spine key point detection method.

[0074] Figure 4 This is the third sub-process flowchart of the spine key point detection method.

[0075] Figure 5 This is the fourth sub-process flowchart of the spine key point detection method.

[0076] Figure 6 This is the fifth sub-process flowchart of the spine key point detection method.

[0077] Figure 7 This is a structural block diagram of the spine key point detection system.

[0078] Figure 8 Schematic diagram of the key points of the spine.

[0079] Fig. 9 Schematic diagram of key points from the right perspective DETAILED DESCRIPTION

[0080] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0081] Figure 1 : is a flowchart of a spine key point detection method. In an embodiment of the present invention, a spine key point detection method is provided, the method comprising:

[0082] Step S100: collecting the gender, height, weight and age of the user at least once, and searching for a standard human body model in a preset human body model library according to the collected gender, height, weight and age; the standard human body model is a three-dimensional contour model;

[0083] The technical solution of the present invention will pre-create some standard human body models. These standard human body models use gender, height, weight and age as indexes. When the user wants to detect the spine, the execution body of this method collects the user's gender, height, weight and age, and the corresponding standard human body model can be queried based on the collected gender, height, weight and age.

[0084] It should be noted that this application has made specific limitations on the collection process. The collection is a multi-frequency collection process, not just one time. This is essentially an anti-shake solution, which makes the collection results of height and weight more accurate. The two parameters of age and gender are very easy to obtain. Just open an information receiving end to receive user input.

[0085] Step S200: photographing an image in a direction preset by a user, performing contour recognition on the image, and obtaining an actual human body model;

[0086] The user's image is captured by a pre-placed camera. The shooting direction of the camera is the preset direction in the above content. After the image is captured, the image contour is recognized and a three-dimensional model is constructed using the same data structure as the standard human body model, which is called the actual human body model.

[0087] Step S300: querying feature points in the actual human body model, and fitting feature curves based on the feature points as the spine features of the user in each direction;

[0088] like Figure 8As shown, after the actual human body model is determined, the feature points are queried in the actual human body model. There are mainly ten positions of feature points involved in this application, among which 1 to 9 represent the left acromion, the right acromion, the lower angle of the left scapula, the lower angle of the right scapula, the left lumbar fossa, the right lumbar fossa, the left iliac crest, the right iliac crest and the line where the spinous process is located; the left acromion, the right acromion, the lower angle of the left scapula, the lower angle of the right scapula and the line where the spinous process is located are in the upper part of the body, and the left lumbar fossa, the right lumbar fossa, the left iliac crest and the right iliac crest are in the lower part of the body. By fitting the two parts separately, two curves can be obtained, and the curves can be used as spinal features.

[0089] Two curves can be fitted in each direction, that is, there will be independent spine features in each direction.

[0090] Step S400: querying the spine features of the standard human body model in each direction as standard features, comparing the spine features of the actual human body model in each direction with the standard features, marking key points according to the comparison results, and simultaneously determining the spine score;

[0091] The spine features in each direction of the standard human body model are queried as standard features. The spine features in each direction of the actual human body model are compared with its standard features to determine whether the spine is normal. During the comparison process, some key points need to be marked and the spine score is determined synchronously, wherein the key points are used to characterize the area where the abnormality exists, and the spine score is used to characterize the degree of abnormality.

[0092] Step S500: training a neural network model from spinal features to spinal scoring, and using the trained neural network model as a trigger for the comparison process in the actual application stage;

[0093] During the comparison process, each time a spinal feature and its spinal score is obtained, the present application regards it as a sample, counts all samples, obtains a sample set, divides the sample set into a test set and a validation set, and trains the neural network model. The response speed of the neural network model is very high. When the error rate of the neural network model is small, it can be put into use. During the use stage, the user can directly obtain the spinal score with the help of the trained neural network model. When more detailed spinal identification content is needed, the comparison process of step S400 is applied.

[0094] Human skeleton key point detection is one of the basic algorithms of computer vision, and plays a fundamental role in the research of other related fields of computer vision, such as behavior recognition, character tracking, gait recognition and other related fields. Specific applications are mainly concentrated in intelligent video surveillance, patient monitoring systems, human-computer interaction, virtual reality, human animation, smart home, smart security, athlete auxiliary training, etc. Traditional human skeleton key point detection algorithms are basically based on the idea of ​​template matching based on geometric priors. The core lies in how to use templates to represent the entire human structure, including the representation of key points, the representation of limb structures, and the representation of the relationship between different limb structures. A good template matching idea can simulate more posture ranges, so that it can better match and detect the corresponding human posture. Based on the feasibility of human skeleton key point detection application, a human spine key point detection algorithm based on deep learning is designed. By using the powerful learning ability of neural networks and training a large amount of spine key point data, the neural network can detect spine key points by itself.

[0095] Figure 2 This is a first sub-flow diagram of the spine key point detection method, wherein the step of collecting the user's gender, height, weight and age at least once, and searching for a standard human body model in a preset human body model library according to the collected gender, height, weight and age includes:

[0096] Step S101: receiving a detection request input by a user and opening a collection module;

[0097] Step S102: Gender, height, weight and age of the user are collected at least once based on the collection module, and the collected gender, height, weight and age are stored in the form of an array; the elements in the array correspond to time;

[0098] Step S103: Counting the mode values ​​in each array in a collection period, and taking the mode values ​​as the final gender, height, weight and age;

[0099] Step S104: using gender, height, weight and age as indexes, searching for standard human body models in a preset human body model library.

[0100] In one example of the technical solution of the present invention, a detection request input by a user is received, a collection module is opened, and the gender, height, weight and age of the user are collected at least once based on the collection module: the mode values ​​in each array in a collection period are counted, and the mode values ​​are used as the final gender, height, weight and age; this is essentially an anti-shake process, which is mainly applied to height and weight, and gender and age do not need anti-shake and can be input once.

[0101] Use gender, height, weight and age as indexes to search for standard human body models in the preset human body model library. This is a database data reading solution based on four parameters.

[0102] Figure 3 This is a second sub-flow diagram of the spine key point detection method, wherein the steps of shooting an image in a direction preset by a user, performing contour recognition on the image, and obtaining an actual human body model include:

[0103] Step S201: creating a baseline model according to the final gender, height, weight and age;

[0104] Step S202: photographing a user image in a preset direction and performing contour recognition on the user image; the directions at least include a front view direction, a rear view direction, a top view direction and a side view direction, and also include other directions at preset angles to the front view reference plane, the top view reference plane and the front view reference plane; the directions are represented by vectors;

[0105] Step S203: When the direction is a front view direction, a rear view direction, a top view direction, or a side view direction, adjusting the reference model according to the contour recognition result;

[0106] Step S204: when the direction belongs to other directions that have a preset angle with the front view reference plane, the top view reference plane and the front view reference plane, the adjusted reference model is verified according to the contour recognition result.

[0107] In an example of the technical solution of the present invention, a specific actual human body model generation scheme is provided. First, a benchmark model is created according to the final gender, height, weight and age. The simplest benchmark model is to directly read the corresponding standard human body model. Of course, the management party of the execution subject of this method can also pre-build a benchmark model library, and read the benchmark model in the benchmark model library according to gender, height, weight and age.

[0108] A user image is photographed in a preset direction, and contour recognition is performed on the user image. The directions mainly include two categories, one is the front view direction, the rear view direction, the top view direction and the side view direction, and the other is other directions. The other directions include other directions that are at preset angles to the front view reference plane, the top view reference plane and the front view reference plane, and the directions are represented by the normal vector of the plane.

[0109] Figure 4 The third sub-flow chart of the spine key point detection method is as follows. The steps of querying feature points in the actual human body model and fitting feature curves based on the feature points as the spine features of the user in each direction include:

[0110] Step S301: querying a feature area according to a preset position relationship; the feature points include the left acromion, the right acromion, the left inferior angle of the scapula, the right inferior angle of the scapula, the left lumbar fossa, the right lumbar fossa, the left iliac crest, the right iliac crest and the line where the spinous process is located;

[0111] Step S302: selecting feature points in the feature area, connecting the feature points, and fitting a feature curve;

[0112] Step S303: Obtain the projection of the characteristic curve in each direction to obtain the spine feature in each direction.

[0113] The positions of the feature points are almost fixed. For example, the left acromion is at the junction of the left limb and the trunk. These positions are called positional relationships. The feature area is queried according to the preset positional relationship, and the feature points (generally the center point) are selected in the feature area. The feature points are connected to fit the feature curve. It has been explained in the above content that the left acromion, the right acromion, the lower angle of the left scapula and the upper part of the spinous process are in the upper part of the body. Therefore, the feature points corresponding to the acromion, the right acromion, the lower angle of the left scapula and the upper part of the spinous process are fitted to obtain a feature curve; the left iliac crest, the right iliac crest, the lower part of the spinous process, the left lumbar fossa and the right lumbar fossa are in the lower part of the body. The feature points corresponding to the left iliac crest, the right iliac crest, the lower part of the spinous process, the left lumbar fossa and the right lumbar fossa are fitted to obtain another feature curve.

[0114] By obtaining the projection of the characteristic curve in the plane in each direction, the spine feature in each direction can be obtained.

[0115] The process of fitting the characteristic curve is:

[0116] Define the fitting function z(x,y)=ax 2 +by 2 +cxy+dx+ey+f;

[0117] Input the feature point coordinates into the fitting function z(x,y), and solve the fitting parameters a, b, c, d, e and f based on the least squares method to minimize the objective function

[0118] The mean square error is calculated to evaluate the fitting effect; the mean square error is as follows:

[0119]

[0120] The above content provides a fitting solution based on a quadratic polynomial, through which the feature points are fitted; wherein the z-axis generally represents the height direction, the y-axis generally represents the front-back direction of the body, and the x-axis represents the direction in the horizontal plane that is perpendicular to the y-axis.

[0121] As a preferred embodiment of the technical solution of the present invention, after querying the feature points in the actual human body model, a more direct and simple method can be used to extract the spinal features, which can be used as a simplified solution. If the spinal features extracted in this way are determined to have problems, no subsequent judgment is required, that is, the solution provided in step S300. A more direct and simple solution for extracting spinal features is as follows:

[0122] Joint angle measurement is an important means of evaluating the severity of spinal curvature. By measuring the angle of the spine, the degree and direction of the spinal curvature and whether there is twisting can be determined. Different types and degrees of curvature may require different countermeasures, which is of great reference value. In addition, spinal curvature is not a fixed state, it may change over time, and regular measurement of joint angles can help users better understand their physical condition.

[0123] Specifically, Figure 8 As shown in FIG. 1 , after obtaining the coordinates of each spinal joint point, we measure the angles between the line 1-2, the line 3-4, the line 5-6, and the line 7-8 and the horizontal plane according to the angle measurement formula. The angle measurement formula is: α = argcos(b 2 +c 2 -a 2 ) / 2bc; where α represents the angle between a certain line and the horizontal plane, a represents the side opposite to the angle α in any triangle formed by the line and the horizontal plane, b and c represent the sides adjacent to the angle α in any triangle formed by the line and the horizontal plane; argcos(x) represents the inverse function of x.

[0124] After calculating the angle, the angle is compared with the preset standard angle to determine whether spinal curvature exists and the type and degree of curvature.

[0125] Figure 5 The fourth sub-flow chart of the spine key point detection method, wherein the steps of comparing the spine features in each direction of the actual human body model with the standard features, marking the key points according to the comparison results, and synchronously determining the spine score include:

[0126] Step S401: querying the spine features of the standard human body model in each direction as standard features;

[0127] Step S402: reading the spinal features in each direction of the actual human body model, comparing them with the standard features, marking the abnormal segments, and synchronously calculating the abnormality degree;

[0128] Step S403: when the abnormality reaches a preset abnormality threshold, mark the nearest feature point as a key point;

[0129] Step S404: Count the key points and their abnormality to determine the spine score.

[0130] In an example of the technical solution of the present invention, a specific comparison process is provided. First, spinal features in each direction are obtained based on a standard human body model, which are called standard features. Then, spinal features in each direction in the actual human body model are read and compared with the standard features. The parts with differences are marked as abnormal segments, and the degree of abnormality is determined according to the size of the difference. Then, the degree of abnormality is compared with a preset abnormal threshold. When the degree of abnormality is large enough, that is, when the degree of abnormality reaches the preset abnormal threshold, the feature point closest to the abnormal segment is marked as a key point. Finally, the key points and their degrees of abnormality are counted to obtain a spinal score.

[0131] The process of reading the spinal features in each direction of the actual human body model, comparing them with the standard features, marking the abnormal segments, and synchronously calculating the abnormality degree includes:

[0132] Based on the DTW algorithm, the abnormal spinal features and their standard features are matched, and the distance between each pair of matching points is calculated in turn. When the distance is greater than the preset distance threshold, it is marked as an abnormal point. The abnormal points are connected to obtain the abnormal segment, and the average distance of each abnormal point in the abnormal segment is calculated as the abnormality degree.

[0133] The process of comparing spine features with standard features is essentially the process of comparing three-dimensional curves. With the help of the DTW algorithm, the points on the two features can be matched one by one, and then the distance can be calculated. The abnormal points are judged by the distance, and the abnormal points are connected to get the abnormal segments. Then the average distance of the abnormal points on the abnormal segments is calculated as the abnormality. The larger the distance, the greater the abnormality.

[0134] As a preferred embodiment of the technical solution of the present invention, images at other angles can also be identified to determine whether there is a problem with the spine, such as images in the left-view direction and images in the right-view direction. Fig. 9 As shown, at this time, fewer points can be selected for fitting the curve, only three are needed, and the curve comparison process can adopt the solution provided in the above content.

[0135] Figure 6 The fifth sub-flow chart of the spine key point detection method, wherein the steps of training the neural network model from spine features to spine scores and using the trained neural network model as a trigger for the comparison process in the actual application stage include:

[0136] Step S501: Counting spine features and spine scores to construct a sample set;

[0137] Step S502: training a neural network model based on the sample set, and outputting the trained neural network model when the error rate of the neural network model is less than a preset error threshold;

[0138] Step S503: When the spine features of the user in each direction are obtained, the spine features are input into the trained neural network model to obtain a spine score;

[0139] Step S504: every preset number of detections, a comparison process is applied to the spinal features whose spinal scores are greater than a preset score threshold to verify the accuracy of the neural network model.

[0140] Steps S501 to S504 are specific model training and application stages. On the basis that the spine characteristics and spine scores have been limited and determined, constructing a sample set and training a neural network model are conventional technical means, and the existing model training process can be used.

[0141] After the model training is completed, when the user's spinal features in each direction are obtained, the spinal features are input into the trained neural network model to obtain a spinal score as the result fed back to the user. At this time, the user can only receive one parameter, the spinal score. If the user needs, the execution body of this method receives the query request input by the user based on the preset request receiving port, further applies the comparison process, obtains the abnormal segment and its abnormality, and feeds back to the user.

[0142] In addition, when the spine score is high, the comparison process is automatically applied every preset number of tests. The purpose of this automatic application process is to test the accuracy of the neural network model.

[0143] Figure 7 1 is a structural block diagram of a spine key point detection system. In an embodiment of the present invention, a spine key point detection system is provided. The system 10 includes:

[0144] The human body model query module 11 is used to collect the gender, height, weight and age of the user at least once, and query the standard human body model in the preset human body model library according to the collected gender, height, weight and age; the standard human body model is a three-dimensional contour model;

[0145] The contour recognition module 12 is used to capture an image in a direction preset by a user, perform contour recognition on the image, and obtain an actual human body model;

[0146] A feature extraction and fitting module 13 is used to query feature points in the actual human body model and fit feature curves based on the feature points as the spinal features of the user in each direction;

[0147] The spine comparison and determination module 14 is used to query the spine features of the standard human body model in each direction, and compare the spine features in each direction of the actual human body model with its standard features as standard features, mark key points according to the comparison results, and simultaneously determine the spine score;

[0148] The pre-trigger module 15 is used to train the neural network model from spinal features to spinal scores, and use the trained neural network model as a trigger for the comparison process in the actual application stage.

[0149] Furthermore, the human body model query module 11 includes:

[0150] The acquisition module opening unit is used to receive the detection request input by the user and open the acquisition module;

[0151] A data collection and storage unit, used for collecting the gender, height, weight and age of the user at least once based on the collection module, and storing the collected gender, height, weight and age in the form of an array; the elements in the array correspond to time;

[0152] A mode value counting unit, used to count the mode values ​​in each array in a collection period, and use the mode value as the final gender, height, weight and age;

[0153] The majority value matching unit is used to use gender, height, weight and age as indexes to query standard human body models in a preset human body model library.

[0154] Specifically, the contour recognition module 12 includes:

[0155] A benchmark model creation unit, used to create a benchmark model according to the final gender, height, weight and age;

[0156] The recognition execution unit is used to capture the user image in a preset direction and perform contour recognition on the user image; the directions include at least a front view direction, a rear view direction, a top view direction and a side view direction, and also include other directions that are at preset angles to the front view reference plane, the top view reference plane and the front view reference plane; the directions are represented by vectors;

[0157] A model adjustment unit, used for adjusting the reference model according to the contour recognition result when the direction belongs to the front view direction, the rear view direction, the top view direction and the side view direction;

[0158] The model verification unit is used to verify the adjusted reference model according to the contour recognition result when the direction belongs to other directions with a preset angle to the front view reference plane, the top view reference plane and the front view reference plane.

[0159] Furthermore, the feature extraction and fitting module 13 includes:

[0160] A region query unit, used to query a feature region according to a preset position relationship; the feature points include the left acromion, the right acromion, the left inferior angle of the scapula, the right inferior angle of the scapula, the left lumbar fossa, the right lumbar fossa, the left iliac crest, the right iliac crest and the line where the spinous process is located;

[0161] A curve fitting unit, used for selecting feature points in the feature area, connecting the feature points, and fitting a feature curve;

[0162] A projection calculation unit, used to obtain the projection of the characteristic curve in each direction to obtain the spine feature in each direction;

[0163] The process of fitting the characteristic curve is:

[0164] Define the fitting function z(x,y)=ax 2 +by 2 +cxy+dx+ey+f;

[0165] Input the feature point coordinates into the fitting function z(x,y), and solve the fitting parameters a, b, c, d, e and f based on the least squares method to minimize the objective function

[0166] The mean square error is calculated to evaluate the fitting effect; the mean square error is as follows:

[0167]

[0168] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for detecting key points of the spine, characterized in that: The method comprises: Collecting the user's gender, height, weight and age at least once, and searching for a standard human body model in a preset human body model library according to the collected gender, height, weight and age; the standard human body model is a three-dimensional contour model; Take an image in the direction preset by the user, perform contour recognition on the image, and obtain an actual human body model; Querying feature points in the actual human body model, and fitting feature curves based on the feature points as the spine features of the user in each direction; Query the spine features of the standard human body model in each direction as standard features, compare the spine features in each direction of the actual human body model with its standard features, mark key points according to the comparison results, and simultaneously determine the spine score; A neural network model is trained from spinal features to spinal scores, and the trained neural network model is used as a trigger for the comparison process in the actual application stage.

2. The spine key point detection method according to claim 1, characterized in that: The step of collecting the gender, height, weight and age of the user at least once and searching for a standard human body model in a preset human body model library according to the collected gender, height, weight and age comprises: Receive the detection request input by the user and open the acquisition module; The collecting module collects the gender, height, weight and age of the user at least once, and the collected gender, height, weight and age are stored in the form of an array; the elements in the array correspond to the time; Count the mode values ​​in each array in a collection period, and use the mode value as the final gender, height, weight and age; Use gender, height, weight and age as indexes to query standard human models in the preset human model library.

3. The spine key point detection method according to claim 1, characterized in that: The step of shooting an image in a direction preset by a user and performing contour recognition on the image to obtain an actual human body model comprises: Create a baseline model based on final gender, height, weight, and age; The user image is photographed in a preset direction, and the contour of the user image is recognized; the directions include at least a front view direction, a rear view direction, a top view direction, and a side view direction, and also include other directions that are at preset angles to the front view reference plane, the top view reference plane, and the front view reference plane; the directions are represented by vectors; When the direction belongs to the forward viewing direction, the rearward viewing direction, the downward viewing direction and the side viewing direction, the reference model is adjusted according to the contour recognition result; When the direction belongs to other directions at a preset angle to the front view reference plane, the top view reference plane, and the front view reference plane, the adjusted reference model is verified according to the contour recognition result.

4. The spine key point detection method according to claim 1, characterized in that: The step of querying feature points in the actual human body model and fitting feature curves based on the feature points as the spine features of the user in each direction comprises: Querying the characteristic area according to the preset position relationship; the characteristic points include the left acromion, the right acromion, the left inferior angle of the scapula, the right inferior angle of the scapula, the left lumbar fossa, the right lumbar fossa, the left iliac crest, the right iliac crest and the line where the spinous process is located; Selecting feature points in the feature area, connecting the feature points, and fitting a feature curve; Obtain the projection of the characteristic curve in each direction to obtain the spine feature in each direction; The process of fitting the characteristic curve is: Define the fitting function z(x,y)=ax 2 +by 2 +cxy+dx+ey+f; Input the feature point coordinates into the fitting function z(x,y), and solve the fitting parameters a, b, c, d, e and f based on the least squares method to minimize the objective function The mean square error is calculated to evaluate the fitting effect; the mean square error is as follows:

5. The spine key point detection method according to claim 1, characterized in that: The steps of comparing the spine features in each direction of the actual human body model with the standard features, marking key points according to the comparison results, and simultaneously determining the spine score include: Query the spine features of the standard human body model in each direction as standard features; Read the spinal features in each direction of the actual human body model, compare them with the standard features, mark the abnormal segments, and simultaneously calculate the abnormality degree; When the abnormality reaches a preset abnormality threshold, the nearest feature point is marked as a key point; Count the key points and their abnormalities to determine the spinal score; The process of reading the spinal features in each direction of the actual human body model, comparing them with the standard features, marking the abnormal segments, and simultaneously calculating the abnormality degree includes: Based on the DTW algorithm, the abnormal spinal features and their standard features are matched, and the distance between each pair of matching points is calculated in turn. When the distance is greater than the preset distance threshold, it is marked as an abnormal point. The abnormal points are connected to obtain the abnormal segment, and the average distance of each abnormal point in the abnormal segment is calculated as the abnormality degree.

6. The spine key point detection method according to claim 1, characterized in that: The steps of training the neural network model from spine features to spine scores and using the trained neural network model as a trigger for the comparison process in the actual application stage include: The spinal features and spinal scores were counted to construct a sample set; The neural network model is trained based on the sample set, and when the error rate of the neural network model is less than a preset error threshold, the trained neural network model is output; When the user's spinal features in each direction are obtained, the spinal features are input into the trained neural network model to obtain a spinal score; After every preset number of tests, the comparison process is applied to the spinal features with spinal scores greater than the preset score threshold to verify the accuracy of the neural network model.

7. A spine key point detection system, characterized in that: The system comprises: A human body model query module is used to collect the gender, height, weight and age of the user at least once, and query a standard human body model in a preset human body model library according to the collected gender, height, weight and age; the standard human body model is a three-dimensional contour model; The contour recognition module is used to capture images in the direction preset by the user, perform contour recognition on the images, and obtain an actual human body model; A feature extraction and fitting module, used for querying feature points in the actual human body model, and fitting feature curves based on the feature points as the spinal features of the user in each direction; The spine comparison and determination module is used to query the spine features of the standard human body model in each direction, and compare the spine features in each direction of the actual human body model with its standard features as standard features, mark key points according to the comparison results, and simultaneously determine the spine score; The pre-trigger module is used to train the neural network model from spinal features to spinal scores, and the trained neural network model is used as a trigger for the comparison process in the actual application stage.

8. The spine key point detection system according to claim 7, characterized in that: The human body model query module comprises: The acquisition module opening unit is used to receive the detection request input by the user and open the acquisition module; A data collection and storage unit, used for collecting the gender, height, weight and age of the user at least once based on the collection module, and storing the collected gender, height, weight and age in the form of an array; the elements in the array correspond to time; A mode value counting unit, used to count the mode values ​​in each array in a collection period, and use the mode value as the final gender, height, weight and age; The majority value matching unit is used to use gender, height, weight and age as indexes to query standard human body models in a preset human body model library.

9. The spine key point detection system according to claim 7, characterized in that: The contour recognition module comprises: A benchmark model creation unit, used to create a benchmark model according to the final gender, height, weight and age; The recognition execution unit is used to capture the user image in a preset direction and perform contour recognition on the user image; the directions include at least a front view direction, a rear view direction, a top view direction and a side view direction, and also include other directions that are at preset angles to the front view reference plane, the top view reference plane and the front view reference plane; the directions are represented by vectors; A model adjustment unit, used for adjusting the reference model according to the contour recognition result when the direction belongs to the front view direction, the rear view direction, the top view direction and the side view direction; The model verification unit is used to verify the adjusted reference model according to the contour recognition result when the direction belongs to other directions with a preset angle to the front view reference plane, the top view reference plane and the front view reference plane.

10. The spine key point detection system according to claim 7, characterized in that: The feature extraction and fitting module comprises: A region query unit, used to query a feature region according to a preset position relationship; the feature points include the left acromion, the right acromion, the left inferior angle of the scapula, the right inferior angle of the scapula, the left lumbar fossa, the right lumbar fossa, the left iliac crest, the right iliac crest and the line where the spinous process is located; A curve fitting unit, used for selecting feature points in the feature area, connecting the feature points, and fitting a feature curve; A projection calculation unit, used to obtain the projection of the characteristic curve in each direction to obtain the spine feature in each direction; The process of fitting the characteristic curve is: Define the fitting function z(x,y)=ax 2 +by 2 +cxy+dx+ey+f; Input the feature point coordinates into the fitting function z(x,y), and solve the fitting parameters a, b, c, d, e and f based on the least squares method to minimize the objective function The mean square error is calculated to evaluate the fitting effect; the mean square error is as follows:

Images