Construction and application method of disease prediction model, model, equipment and medium

Abstract

The invention relates to the field of medicine, and discloses a construction and application method of a disease prediction model, a model, equipment and a medium. According to the scheme, a gait displacement judgment module is introduced into the disease prediction model and is used for presetting a first threshold value based on coordinate features of each same articulation point of a patient in two adjacent frames of gait images of a gait video; and judging whether gait displacement occurs to the patient between two adjacent frames of gait images of the gait video or not, deleting one frame of gait image to obtain a processed gait video when the patient does not displace between the two adjacent frames of gait images of the gait video, and performing disease prediction on the patient by a subsequent module based on the processed gait video. According to the disease prediction model, the same or similar gait images in the gait video are deleted, and the processed gait video is utilized to perform disease prediction on the patient, so that the accuracy of disease prediction is improved, and the performance of the disease prediction model is improved.

Description

Technical Field

[0001] The present application relates to the medical field, and in particular to a method, model, device, and medium for constructing and applying a disease prediction model. Background Art

[0002] With the development of medical imaging technology and biometric recognition technology, gait analysis, as a non-contact biometric recognition method, has been widely used in the field of clinical medicine. By analyzing the patient's gait, doctors can diagnose certain neurological diseases, musculoskeletal diseases, and other health problems that affect walking patterns.

[0003] At present, relevant researchers have trained many disease prediction models using gait videos, aiming to assist doctors in disease diagnosis by performing gait analysis through trained disease prediction models, that is, using disease prediction models to obtain disease prediction results of patients based on their gait videos, and using the disease prediction results to assist doctors in making more accurate medical diagnoses. The disease prediction model can automatically extract the gait features of patients by processing gait videos. When training the disease prediction model, these features will be learned and used by the model to predict the target disease. However, since the gait video includes multiple frames of gait images, when the disease prediction model is trained using gait videos in the prior art, there are many frames of gait images in the gait video that are the same or similar (that is, if the patient does not move in adjacent frames, the images presented in the two frames are the same or similar), then these frames are not of much information value for the model to learn gait features. Therefore, using gait videos with many frames of the same or similar gait images to train the model increases unnecessary computational burden, reduces the efficiency of model training, and may even introduce noise interference to the model, affecting the model performance, thereby reducing the accuracy of disease prediction.

[0004] In summary, the existing disease prediction models have poor performance, resulting in inaccurate disease prediction results. Therefore, there is an urgent need for a disease prediction model with better performance to improve the accuracy of disease prediction results. Summary of the invention

[0005] The present application provides a method, model, device, and medium for constructing and applying a disease prediction model to solve the problem that the existing disease prediction model has poor performance, resulting in inaccurate disease prediction results.

[0006] The first aspect of the present application provides a method for constructing a disease prediction model, wherein the disease prediction model is used to predict a disease based on a patient's gait video, the method comprising:

[0007] Get a gait video set, where the gait video set includes multiple gait videos, each gait video includes multiple frames of gait images, each gait video corresponds to a patient, and the patient's body structure includes multiple joint points:

[0008] Construct an initial disease prediction model, wherein the initial disease prediction model of the present application includes:

[0009] A feature extraction module, used to take each gait video as a target gait video, and perform a feature extraction operation on each joint point of the patient in each frame of the gait image of the target gait video, so as to obtain a coordinate feature of each joint point of the patient in each frame of the gait image of the target gait video;

[0010] a gait displacement judgment module, for judging whether the patient has gait displacement between two adjacent frames of gait images of the target gait video based on the coordinate features of each identical joint point of the patient in two adjacent frames of gait images of the target gait video and a preset first threshold, and deleting one of the gait images when the patient has not displaced between two adjacent frames of gait images of the target gait video, so as to obtain a processed target gait video;

[0011] A disease prediction module, used for obtaining a disease prediction result corresponding to the patient in the target gait video based on the processed target gait video;

[0012] Among them, the gait displacement judgment module is communicated with the feature extraction module and the disease prediction module respectively;

[0013] The initial disease prediction model was trained multiple times using the gait video set until convergence.

[0014] In some embodiments of the present application, the feature extraction module of the present application is configured to obtain the coordinate features of each joint point of the patient in each frame of the gait image of the target gait video in the following manner:

[0015] Normalize the resolution of the target gait video;

[0016] A feature extraction operation is performed on each joint point of the patient in each frame of the target gait video after resolution normalization processing to obtain the coordinate features of each joint point of the patient in each frame of the target gait video.

[0017] In some embodiments of the present application, the feature extraction module of the present application is configured to obtain the coordinate features of each joint point of the patient in each frame of the gait image of the target gait video in the following manner:

[0018] The YoLoV8 method is used to perform feature extraction operations on each joint point of the patient in each frame of the gait image of the target gait video to obtain the coordinate features of each joint point of the patient in each frame of the gait image of the target gait video.

[0019] In some embodiments of the present application, the gait displacement judgment module of the present application is configured to obtain the processed target gait video in the following manner:

[0020] Performing target detection on the patient in each frame of the gait image of the target gait video to obtain the circumscribed rectangular bounding box of the patient in each frame of the gait image of the target gait video, and extracting the length and width coordinates of the circumscribed rectangular bounding box of the patient in each frame of the gait image of the target gait video to obtain the length and width coordinate features of the circumscribed rectangular bounding box of the patient in each frame of the gait image of the target gait video;

[0021] Based on the coordinate features of each identical joint point of the patient in two adjacent frames of gait images of the target gait video, the length and width coordinates of the circumscribed rectangular bounding box of the patient in two adjacent frames of gait images, a preset first threshold and a preset second threshold are used to determine whether a gait displacement occurs in the patient between two adjacent frames of gait images of the target gait video, and when no displacement occurs in the patient between two adjacent frames of gait images of the target gait video, one of the gait images is deleted to obtain a processed target gait video.

[0022] In some embodiments of the present application, the gait displacement determination module of the present application is further configured to obtain the processed target gait video in the following manner:

[0023] The width of the patient's circumscribed rectangular bounding box in two adjacent frames of gait images of the target gait video constitutes a first bounding box pair, and the length of the patient's circumscribed rectangular bounding box in two adjacent frames of gait images of the target gait video constitutes a second bounding box pair, and the Euclidean distance between the wide coordinate features of the two rectangular bounding boxes of the first bounding box pair and the Euclidean distance between the long coordinate features of the two rectangular bounding boxes of the second bounding box pair in the two adjacent frames of gait images of the target gait video are calculated respectively;

[0024] Each identical joint point of the patient in two adjacent frames of gait images of the target gait video constitutes a joint point pair, and the Euclidean distance between the coordinate features of the two joint points of each joint point pair in two adjacent frames of gait images of the target gait video is calculated;

[0025] Based on the Euclidean distance between the coordinate features of two joint points of all joint pairs in two adjacent frames of gait images of the target gait video, the Euclidean distance between the wide coordinate features of the two rectangular bounding boxes of the first bounding box pair, the Euclidean distance between the long coordinate features of the two rectangular bounding boxes of the second bounding box pair, a preset first threshold, and a preset second threshold, it is determined whether a gait displacement occurs to the patient between two adjacent frames of gait images of the target gait video, and when no displacement occurs to the patient between two adjacent frames of gait images of the target gait video, one of the gait images is deleted to obtain a processed target gait video.

[0026] In some embodiments of the present application, the coordinate feature of each joint point includes a horizontal coordinate feature of each joint point and a vertical coordinate feature of each joint point, and the gait displacement judgment module of the present application is further configured to judge whether a gait displacement occurs in the patient between two adjacent frames of gait images of the target gait video in the following manner:

[0027] The patient's

[0028]

[0029] in, represents the wide coordinate feature of the circumscribed rectangular bounding box of the patient in the t-th frame gait image of the target gait video, Represents the wide coordinate feature of the patient's circumscribed rectangular bounding box in the t+1th frame of the target gait video, Δx rect represents the Euclidean distance between the wide coordinate features of the two rectangular bounding boxes of the first bounding box pair in the t-th and t+1-th adjacent frames of the target gait video, Represents the long coordinate feature of the patient's circumscribed rectangular bounding box in the t-th frame gait image of the target gait video, Represents the long coordinate feature of the patient's circumscribed rectangular bounding box in the t+1th frame of the target gait video, Δy rect represents the Euclidean distance between the long coordinate features of the two rectangular bounding boxes of the second bounding box pair in the t-th and t+1-th adjacent frames of the target gait video, represents the ordinate feature of the i-th joint point of the patient in the t-th frame gait image of the target gait video, represents the ordinate feature of the i-th joint point of the patient in the t+1-th frame gait image of the target gait video, Δy i represents the Euclidean distance between the ordinate features of the two joint points of the i-th joint pair in the t-th and t+1-th adjacent frames of the target gait video, a 1 is the preset first threshold, a 2 A second threshold is preset.

[0030] The second aspect of the present application provides a disease prediction model, which is used to predict a disease based on a patient's gait video, wherein the gait video includes multiple frames of gait images, the gait video corresponds to a patient, and the patient's body structure includes multiple joints. The disease prediction model of the present application includes:

[0031] A feature extraction module is used to perform a feature extraction operation on each joint point of the patient in each frame of the gait image of the gait video to obtain a coordinate feature of each joint point of the patient in each frame of the gait image of the gait video;

[0032] a gait displacement judgment module, for judging whether the patient has gait displacement between two adjacent frames of gait images of the gait video based on the coordinate features of each identical joint point of the patient in two adjacent frames of gait images of the gait video and a preset first threshold, and deleting one of the gait images when the patient has not displaced between two adjacent frames of gait images of the gait video to obtain a processed target gait video;

[0033] The disease prediction module is used to obtain the disease prediction result corresponding to the patient in the gait video based on the processed gait video.

[0034] The third aspect of the present application provides an application method of a disease prediction model, the method comprising:

[0035] Obtain a video of the patient's gait;

[0036] The disease prediction model in the above embodiment is used to obtain the disease prediction result corresponding to the patient based on the patient's gait video.

[0037] The fourth aspect of the present application provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the method described in any one of the first and third aspects of the above embodiments are implemented.

[0038] The fifth aspect of the present application provides a computer-readable storage medium having program instructions stored thereon, and when the program instructions are executed by a processor, the method described in any one of the first aspect and the third aspect of the above embodiments is implemented.

[0039] This application has the following beneficial effects:

[0040] The construction scheme of the disease prediction model of the present application introduces a gait displacement judgment module into the disease prediction model. The module is used to judge whether the patient has gait displacement between two adjacent frames of gait images of the gait video based on the coordinate features of each identical joint point of the patient in two adjacent frames of gait images of the gait video and a preset first threshold value, and when the patient has not displaced between two adjacent frames of gait images of the gait video, delete one of the gait images to obtain a processed gait video, and use the disease prediction module to obtain the disease prediction result corresponding to the patient in the gait video based on the processed gait video. The construction scheme of the disease prediction model of the present application reduces the unnecessary computational burden of the model when predicting diseases, improves the efficiency of model training, and at the same time, the disease prediction model constructed by the present application improves the accuracy of the disease prediction results and the performance of the disease prediction model by deleting the same or similar gait images in the gait video and using the processed gait video to predict diseases for patients. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application.

[0042] Figure 1 This is a flow chart of an embodiment of a method for constructing a disease prediction model provided by the present application;

[0043] Figure 2 This is a schematic diagram of the framework of an embodiment of a disease prediction model provided by the present application;

[0044] Figure 3 is an example schematic diagram of a circumscribed rectangular bounding box and joint points of a patient in a gait image provided by the present application;

[0045] Figure 4 It is a flow chart of an embodiment of a method for determining whether a patient has displacement in two adjacent frames of gait images provided by the present application;

[0046] Figure 5 This is a flow chart of an embodiment of an application method of a disease prediction model provided by the present application;

[0047] Figure 6 It is a schematic diagram of a framework of an embodiment of an electronic device provided by the present application;

[0048] Figure 7 It is a schematic diagram of a framework of an embodiment of a computer-readable storage medium provided by the present application. DETAILED DESCRIPTION

[0049] The scheme of the embodiment of the present application is described in detail below in conjunction with the drawings of the specification.

[0050] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.

[0051] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the objects associated before and after are in an "or" relationship. In addition, "many" in this article means two or more than two. In addition, the term "at least one" in this article means any combination of at least two of any one or more of a plurality of, for example, including at least one of A, B, and C, can mean including any one or more elements selected from the set consisting of A, B, and C.

[0052] As described in the background technology, the prior art uses gait videos with many frames of identical or similar gait images to train the model, which increases unnecessary computational burden, reduces the efficiency of model training, and may even introduce noise interference to the model, affecting model performance, thereby reducing the accuracy of disease prediction.

[0053] In order to solve the above problems, the present application proposes a new disease prediction model construction scheme. In the construction scheme of the present application, a gait displacement judgment module is introduced into the disease prediction model. The module is used to judge whether the patient has gait displacement between two adjacent frames of gait images of the gait video based on the coordinate features of each identical joint point of the patient in two adjacent frames of gait images of the gait video and a preset first threshold value, and when the patient has not displaced between two adjacent frames of gait images of the gait video, one of the gait images is deleted to obtain a processed gait video, and the disease prediction module is used to obtain the disease prediction result corresponding to the patient in the gait video based on the processed gait video. The disease prediction model construction scheme of the present application reduces the unnecessary computational burden of the model when predicting diseases, improves the efficiency of model training, and at the same time, the disease prediction model constructed by the present application improves the accuracy of the disease prediction results and the performance of the disease prediction model by deleting the same or similar gait images in the gait video and using the processed gait video to predict diseases for patients.

[0054] The present application is described in detail below with reference to the accompanying drawings and specific embodiments.

[0055] According to one embodiment of the present application, Figure 1As shown, the present application provides a method for constructing a disease prediction model, the method comprising: S1, obtaining a gait video set, wherein the gait video set includes multiple gait videos, each gait video includes multiple frames of gait images, each gait video corresponds to a patient, and the patient's body structure includes multiple joints; S2, constructing an initial disease prediction model, wherein, as Figure 2 As shown, the initial disease prediction model of the present application includes: a feature extraction module, which is used to take each gait video as a target gait video, and perform a feature extraction operation on each joint point of the patient in each frame of the gait image of the target gait video to obtain the coordinate features of each joint point of the patient in each frame of the gait image of the target gait video; a gait displacement judgment module, which is used to judge whether the patient has gait displacement between two adjacent frames of gait images of the target gait video based on the coordinate features of each same joint point of the patient in two adjacent frames of gait images of the target gait video and a preset first threshold, and when the patient has not displaced between two adjacent frames of gait images of the target gait video, delete one of the gait images to obtain a processed target gait video; a disease prediction module, which is used to obtain the disease prediction result corresponding to the patient in the target gait video based on the processed target gait video, wherein the gait displacement judgment module is communicated with the feature extraction module and the disease prediction module respectively; S3, using the gait video set to perform multiple iterative training on the initial disease prediction model until convergence.

[0056] According to one embodiment of the present application, in each iterative training process, the parameters of the disease prediction model are updated according to the following loss function:

[0057]

[0058] Among them, y is the true value of the disease corresponding to the patient in the CT image, The disease prediction result corresponding to the patient in the target gait video.

[0059] It can be seen from the above embodiments that the true result of whether the patient has the corresponding disease is obtained based on the patient's CT image, and the parameters of the disease prediction model are updated based on the loss value between the true disease result and the disease prediction result obtained by the model, thereby improving the performance of the disease prediction model.

[0060] In order to more intuitively understand the functions performed by the feature extraction module and the gait displacement judgment module in the disease prediction model constructed by the construction scheme of the present application, the feature extraction module and the gait displacement judgment module of the disease prediction model are described in detail below.

[0061] 1. Feature Extraction Module

[0062] The feature extraction module is used to take each gait video as a target gait video, and perform a feature extraction operation on each joint point of the patient in each frame of the gait image of the target gait video, so as to obtain the coordinate features of each joint point of the patient in each frame of the gait image of the target gait video.

[0063] Among them, according to one embodiment of the present application, the feature extraction module of the present application is configured to: perform resolution normalization processing on the target gait video; perform feature extraction operation on each joint point of the patient in each frame gait image of the target gait video after resolution normalization processing, so as to obtain the coordinate features of each joint point of the patient in each frame gait image of the target gait video.

[0064] For example, resolution normalization processing refers to the feature extraction module resizing the gait video and fixing it to a 1080p resolution (1920×1080 pixels). This can be achieved through image processing methods. The commonly used image processing method is to use the INTER_AREA interpolation method to interpolate the gait video of the original pixels to obtain a gait image that adapts to the target resolution. In order to ensure the consistency and reliability of subsequent processing, the feature extraction module will perform unified resolution processing on the captured gait video. This means that all gait images in all gait videos will be adjusted to the same resolution to eliminate the scale differences between different images. It should be noted that the 1080p resolution is only one of the embodiments of the present application. The resolution of each frame of the gait image in the specific gait video can be determined according to factors such as task requirements and hardware conditions, and is not specifically limited here.

[0065] From the above examples, we can see that resolution normalization can eliminate scale differences between different videos, ensure that all gait images have consistent resolution, and enable the model to better focus on learning gait features instead of being disturbed by irrelevant factors such as image size or resolution. This helps the model perform better on unseen data, that is, improves the model's generalization ability.

[0066] After research, it was found that the YoloV8 method can obtain more stable and accurate coordinate points. Therefore, according to one embodiment of the present application, the feature extraction module of the present application is also configured to: use the YoLoV8 method to perform feature extraction operations on each joint point of the patient in each frame of the gait image of the target gait video, so as to obtain the coordinate features of each joint point of the patient in each frame of the gait image of the target gait video. It should be noted that the YoLoV8 method is only one of the bottom-up human body posture estimation methods used in this application to extract the joint features of patients in the image. With the continuous development of human body posture estimation methods, this application can also use other human body posture estimation methods to extract the joint point features of patients in the image, which is not specifically limited here.

[0067] It can be seen from the above embodiments that more accurate joint features of the patient in the image can be extracted using the YoLoV8 method.

[0068] 2. Gait displacement judgment module

[0069] The gait displacement judgment module is used to judge whether the patient has gait displacement between two adjacent frames of gait images of the target gait video based on the coordinate features of each identical joint point of the patient in two adjacent frames of gait images of the target gait video and a preset first threshold, and when the patient has not displaced between two adjacent frames of gait images of the target gait video, delete one of the gait images to obtain a processed target gait video.

[0070] Among them, according to one embodiment of the present application, the gait displacement judgment module of the present application is configured to obtain the processed target gait video in the following manner: perform target detection on the patient in each frame of the gait image of the target gait video to obtain the patient's circumscribed rectangular bounding box in each frame of the gait image of the target gait video, and extract the length and width coordinates of the patient's circumscribed rectangular bounding box in each frame of the gait image of the target gait video to obtain the length and width coordinate features of the patient's circumscribed rectangular bounding box in each frame of the gait image of the target gait video; based on the coordinate features of each identical joint point of the patient in two adjacent frames of the gait image of the target gait video, the length and width coordinates of the patient's circumscribed rectangular bounding box in two adjacent frames of the gait image, a preset first threshold and a preset second threshold, judge whether the patient has gait displacement between two adjacent frames of the gait image of the target gait video, and when the patient has not displaced between two adjacent frames of the gait image of the target gait video, delete one of the gait images to obtain the processed target gait video. Figure 3 As shown in FIG, it shows the circumscribed rectangular bounding box and joint point diagram of the patient in the gait image. Figure 3 It can be seen that the coordinate point (x rect ,y rect ) are the coordinates of the patient’s circumscribed rectangular bounding box in the gait image, (x i ,y i ) are the coordinates of the patient’s joint points in the gait image, i = 0, 1, …, n.

[0071] Among them, according to an embodiment of the present application, the gait displacement judgment module of the present application is also configured to obtain the processed target gait video in the following manner: the width of the patient's circumscribed rectangular bounding box in two adjacent frames of gait images of the target gait video constitutes a first bounding box pair, and the length of the patient's circumscribed rectangular bounding box in two adjacent frames of gait images of the target gait video constitutes a second bounding box pair, and the Euclidean distance between the wide coordinate features of the two rectangular bounding boxes of the first bounding box pair and the Euclidean distance between the long coordinate features of the two rectangular bounding boxes of the second bounding box pair in two adjacent frames of gait images of the target gait video are calculated respectively; each identical joint point of the patient in two adjacent frames of gait images of the target gait video constitutes a joint point pair, and calculates Calculate the Euclidean distance between the coordinate features of the two joint points of each joint point pair in two adjacent frames of gait images of the target gait video; based on the Euclidean distance between the coordinate features of the two joint points of all joint pairs in two adjacent frames of gait images of the target gait video, the Euclidean distance between the wide coordinate features of the two rectangular bounding boxes of the first bounding box pair, the Euclidean distance between the long coordinate features of the two rectangular bounding boxes of the second bounding box pair, the preset first threshold, and the preset second threshold, determine whether the patient has gait displacement between two adjacent frames of gait images of the target gait video, and when the patient has not displaced between two adjacent frames of gait images of the target gait video, delete one of the gait images to obtain the processed target gait video. According to one embodiment of the present application, the coordinate features of each joint point include the horizontal coordinate features of each joint point and the vertical coordinate features of each joint point, and the gait displacement judgment module of the present application is also configured to determine whether the patient has gait displacement between two adjacent frames of gait images of the target gait video in the following manner:

[0072]

[0073] in, represents the wide coordinate feature of the circumscribed rectangular bounding box of the patient in the t-th frame gait image of the target gait video, Represents the wide coordinate feature of the patient's circumscribed rectangular bounding box in the t+1th frame of the target gait video, Δx rect represents the Euclidean distance between the wide coordinate features of the two rectangular bounding boxes of the first bounding box pair in the t-th and t+1-th adjacent frames of the target gait video, Represents the long coordinate feature of the patient's circumscribed rectangular bounding box in the t-th frame gait image of the target gait video, Represents the long coordinate feature of the patient's circumscribed rectangular bounding box in the t+1th frame of the target gait video, Δy rectrepresents the Euclidean distance between the long coordinate features of the two rectangular bounding boxes of the second bounding box pair in the t-th and t+1-th adjacent frames of the target gait video, represents the ordinate feature of the i-th joint point of the patient in the t-th frame gait image of the target gait video, represents the ordinate feature of the i-th joint point of the patient in the t+1-th frame gait image of the target gait video, Δy i represents the Euclidean distance between the ordinate features of the two joint points of the i-th joint pair in the t-th and t+1-th adjacent frames of the target gait video, a 1 is the preset first threshold, a 2 A second threshold is preset.

[0074] For example, Figure 4 As shown in FIG. 1 , it shows a method for determining whether displacement occurs between two frames of gait images. Figure 4 It can be seen that the present application selects the coordinate feature difference of the left and right joint points of the patient's shoulder, elbow, hip and knee in the two adjacent frames of gait images of the tth and t+1th as the basis for judging the gait direction, and calculates the coordinate feature difference between the patient's left shoulder and right shoulder, left elbow and right elbow, left hip and right hip, left knee and right knee in each frame of gait image, and judges the gait direction of the patient in the frame of gait image based on the calculated four differences. When all four difference values ​​are positive, and all positive values ​​are greater than the preset threshold, it is determined that the patient is walking backward; when all four difference values ​​are negative, and the absolute values ​​of all negative values ​​are greater than the preset threshold, it is determined that the patient is walking forward; other than the two, it is determined to be turning. It should be noted that selecting the coordinate feature difference of the left and right joint points of the shoulder, elbow, hip and knee as the basis for judging the gait direction is only one of the embodiments of the present application, and the specific basis for judgment is not specifically limited here.

[0075] It can be seen from the above embodiments that by combining the patient's circumscribed rectangular bounding box and the coordinates of each joint point, it is possible to more accurately determine whether the patient has displaced in two adjacent frames of gait images, determine gait displacement and delete redundant frames, thereby improving the efficiency of the model in processing data.

[0076] In addition, using the method for constructing a disease prediction model of the above-mentioned embodiment of the present application, according to one embodiment of the present application, the present application provides a disease prediction model, which is used to predict a disease based on a gait video of a patient, wherein the gait video includes multiple frames of gait images, the gait video corresponds to a patient, and the patient's body structure includes multiple joints. The disease prediction model of the present application includes: a feature extraction module, which is used to perform a feature extraction operation on each joint point of the patient in each frame of the gait image of the gait video, so as to obtain the coordinate features of each joint point of the patient in each frame of the gait image of the gait video; A gait displacement judgment module is used to judge whether the patient has gait displacement between two adjacent frames of gait images of the gait video based on the coordinate features of each identical joint point of the patient in two adjacent frames of gait images of the gait video and a preset first threshold, and when the patient has not displaced between two adjacent frames of gait images of the gait video, delete one of the gait images to obtain a processed target gait video; a disease prediction module is used to obtain a disease prediction result corresponding to the patient in the gait video based on the processed gait video; wherein the gait displacement judgment module is communicatively connected to the feature extraction module and the disease prediction module respectively.

[0077] In summary, compared with the prior art of using gait videos with identical or similar gait images in many frames to train the model, the construction scheme of the disease prediction model of the present application introduces a gait displacement judgment module into the disease prediction model, which is used to judge whether the patient has gait displacement between two adjacent frames of gait images of the gait video based on the coordinate features of each identical joint point of the patient in two adjacent frames of gait images of the gait video and a preset first threshold, and when the patient has not displaced between two adjacent frames of gait images of the gait video, delete one of the gait images to obtain a processed gait video, and use the disease prediction module to obtain the disease prediction result corresponding to the patient in the gait video based on the processed gait video. The construction scheme of the disease prediction model of the present application reduces the unnecessary computational burden when the model is used for disease prediction and training, and improves the efficiency of model training. At the same time, the disease prediction model constructed by the present application improves the accuracy of the disease prediction result and the performance of the disease prediction model by deleting the identical or similar gait images in the gait video and using the processed gait video to predict the patient's disease.

[0078] In addition, according to the disease prediction model of the above embodiment of the present application, the present application provides an application method of the disease prediction model, such as Figure 5 As shown, the method includes: T1, obtaining a gait video of the patient; T2, using the disease prediction model in the above embodiment to obtain a disease prediction result corresponding to the patient based on the gait video of the patient.

[0079] For example, scoliosis is the most common spinal deformity among children and adolescents, and is the third largest "killer" of children's health after obesity and myopia. Its incidence and severity rate are significantly higher than the international average. Since the skeletal system of adolescents is in the growth and development stage, there is a risk of further aggravation of scoliosis along with the overall development of the spine. If it is not discovered and intervened early, as scoliosis continues to worsen, it will compress important organs such as the heart and lungs, affecting their normal development, causing abnormal functions of important organs such as the heart and lungs of adolescents, and making it impossible to complete daily activities normally. In mild cases, it will cause back pain, which will significantly reduce the patient's quality of life; in severe cases, it will shorten their lifespan. At the same time, patients often bear a profound psychological burden due to the obvious abnormalities in their appearance. Therefore, implementing "early detection" is the most critical link in the prevention and control of scoliosis.

[0080] Currently, the commonly used screening methods at home and abroad include visual inspection, Adams flexion test (physical examination), corrugated image examination, scoliosis measuring ruler, portable electronic scoliosis screening tool and X-ray measurement of Cobb angle. Their advantages and disadvantages can be seen in Table 1.

[0081] Table 1

[0082]

[0083] From the analysis of Table 1, we can see that the current screening methods require students to take off their shirts and rely on professional doctor examinations and manual measurements. In addition to being inefficient, there is also a certain degree of subjective error, which can easily lead to a high false positive rate.

[0084] The current mainstream scoliosis screening and assessment methods have the following difficulties:

[0085] 1) The early symptoms of scoliosis are not obvious and are easily ignored. Many patients do not have obvious pain and functional impairment in the early stage, and the scoliosis angle is small and difficult to detect.

[0086] 2) Limitations of screening methods. Currently, physical examinations such as the Adam's flexion test are mainly used, which have limited sensitivity and specificity. Although X-rays are the gold standard, they are not suitable for routine screening due to radiation exposure.

[0087] 3) The selection of screening subjects is not uniform. Some are mainly school-age children, while others include adolescents. The incidence and characteristics of scoliosis in different age groups are different, which makes screening difficult.

[0088] 4) Low screening coverage and poor follow-up compliance. Due to lack of awareness, some students and parents do not pay enough attention to it, and there are difficulties in school organization, resulting in low screening coverage and high loss of follow-up rate for positive cases.

[0089] 5) Lack of unified diagnosis and treatment standards. The medical level varies from place to place, and there is a lack of standards for the diagnosis, treatment and follow-up of scoliosis, which will affect the triage and patient management after screening.

[0090] 6) The cost-effectiveness of screening needs to be weighed. Large-scale screening requires manpower and material resources, and requires a balance between input and output.

[0091] The disease prediction model of the present application uses human posture estimation technology (Pose estimation) to obtain the coordinates of the joint points during the human gait process to predict whether the patient suffers from scoliosis, without the need for the patient to undress or stick sensors on the human body surface. This technology is used to capture the dynamic biological information of the human body during movement within a time period, rather than the static information at a certain moment. Combining optical systems and computer vision technology, the patient's motion characteristics are identified and tracked based on RGB 2D images (i.e., videos taken by ordinary mobile phone cameras), and the characteristic data of the target dynamic movement are extracted. The disease prediction model of the present application can be used to more conveniently and accurately predict whether the patient has scoliosis, and the doctor can make a more accurate diagnosis based on the prediction results. It should be noted that the use of the disease prediction model constructed in the present application to predict whether the patient's spine is scoliotic is only an embodiment of the present application. The disease prediction model of the present application can also be used for other results that can predict whether the patient suffers from the corresponding disease based on the patient's joint point information. The application scenario of the disease prediction model here is not specifically limited.

[0092] Based on the inventive concept of the above embodiments, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described in the above embodiments when executing the computer program. Figure 5 Provide detailed explanation.

[0093] like Figure 6 As shown, it shows the electronic device of the present application, which may specifically include a processor 110 and a memory 120. The memory 120 is coupled to the processor 110.

[0094] The processor 110 is used to control the operation of the electronic device. The processor 110 may also be referred to as a CPU (Central Processing Unit). The processor 110 may be an integrated circuit chip having the ability to process signals. The processor 110 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. A general-purpose processor may be a microprocessor or the processor 110 may also be any conventional processor, etc.

[0095] The memory 120 is used to store computer programs, and may be a RAM, a ROM, or other types of storage terminals. Specifically, the memory 120 may include one or more computer-readable storage media, which may be non-transitory or transient. The memory 120 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage terminals, flash memory storage terminals. In some embodiments, the non-transitory computer-readable storage medium in the memory 120 is used to store at least one program code.

[0096] The processor 110 is used to execute the computer program stored in the memory 120 to implement the methods described in the various method embodiments of the present application.

[0097] In some embodiments, the electronic device may further include: a peripheral terminal interface 130 and at least one peripheral terminal. The processor 110, the memory 120 and the peripheral terminal interface 130 may be connected via a bus or a signal line. Each peripheral terminal may be connected to the peripheral terminal interface 130 via a bus, a signal line or a circuit board. Specifically, the peripheral terminal includes: at least one of a radio frequency circuit 140, a display screen 150, an audio circuit 160 and a power supply 170.

[0098] The peripheral terminal interface 130 may be used to connect at least one peripheral terminal related to I / O (Input / output) to the processor 110 and the memory 120. In some embodiments, the processor 110, the memory 120, and the peripheral terminal interface 130 are integrated on the same chip or circuit board; in some other implementations, any one or two of the processor 110, the memory 120, and the peripheral terminal interface 130 may be implemented on a separate chip or circuit board, which is not limited in this embodiment.

[0099] The radio frequency circuit 140 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 140 communicates with the communication network and other IoT devices through electromagnetic signals, and the radio frequency circuit 140 is the communication circuit of the electronic device. The radio frequency circuit 140 converts the electrical signal into an electromagnetic signal for transmission, or converts the received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 140 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, an operator identity module card, and the like. The radio frequency circuit 140 can communicate with other terminals through at least one wireless communication protocol. The wireless communication protocol includes but is not limited to: the World Wide Web, a metropolitan area network, an intranet, various generations of mobile communication networks (2G, 3G, 4G and 5G), a wireless local area network and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 140 may also include circuits related to NFC (Near Field Communication), which is not limited in this application.

[0100] The display screen 150 is used to display the UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen 150 is a touch display screen, the display screen 150 also has the ability to collect touch signals on the surface or above the surface of the display screen 150. The touch signal can be input to the processor 110 as a control signal for processing. At this time, the display screen 150 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen 150 can be one, set on the front panel of the electronic device; in other embodiments, the display screen 150 can be at least two, respectively set on different surfaces of the electronic device or in a folding design; in other embodiments, the display screen 150 can be a flexible display screen, set on a curved surface or a folding surface of the electronic device. Even, the display screen 150 can also be set to a non-rectangular irregular shape, that is, a special-shaped screen. The display screen 150 can be made of materials such as LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode, organic light-emitting diode).

[0101] The audio circuit 160 may include a microphone and a speaker. The microphone is used to collect sound waves from the operator and the environment, and convert the sound waves into electrical signals and input them into the processor 110 for processing, or input them into the radio frequency circuit 140 to achieve voice communication. For the purpose of stereo acquisition or noise reduction, there can be multiple microphones, which are respectively arranged in different parts of the electronic device. The microphone can also be an array microphone or an omnidirectional acquisition microphone. The speaker is used to convert the electrical signal from the processor 110 or the radio frequency circuit 140 into sound waves. The speaker can be a traditional film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for purposes such as ranging. In some embodiments, the audio circuit 160 may also include a headphone jack.

[0102] The power supply 170 is used to power various components in the electronic device. The power supply 170 can be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 170 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged through a wired line, and a wireless rechargeable battery is a battery that is charged through a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0103] For a detailed description of the functions and execution processes of each functional module or component in the electronic device embodiment of the present application, reference can be made to the description in the above-mentioned method embodiments of the present application, which will not be repeated here.

[0104] In the several embodiments provided in the present application, it should be understood that the disclosed electronic devices and methods can be implemented in other ways. For example, the various embodiments of the electronic devices described above are only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0105] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0106] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0107] Based on the inventive concept of the above embodiment, the present application also provides a computer-readable storage medium storing a computer program, which can be executed by a processor to implement the above projection area calculation method. Figure 7 The execution process of the above-mentioned embodiment in a computer-readable storage medium is described.

[0108] like Figure 7 As shown, it shows the computer-readable storage medium of the present application. If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium 200. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions / computer programs to enable an IoT device (which can be a personal computer, server, or network terminal, etc.) or a processor (processor) to perform all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: various media such as a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk, and electronic terminals such as computers, mobile phones, laptops, tablet computers, cameras, etc. having the above-mentioned storage medium.

[0109] The description of the execution process of the program data in the computer-readable storage medium can refer to the description in the above-mentioned method embodiments of the present application, which will not be repeated here.

[0110] The above descriptions are merely embodiments of the present application and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

[0111] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

Claims

1. A method for constructing a disease prediction model, wherein the disease prediction model is used to predict a disease based on a patient's gait video, characterized in that: The method comprises: Obtain a gait video set, wherein the gait video set includes multiple gait videos, each of the gait videos includes multiple frames of gait images, each of the gait videos corresponds to a patient, and the body structure of the patient includes multiple joint points: Constructing an initial disease prediction model, wherein the initial disease prediction model includes: A feature extraction module, used to take each gait video as a target gait video, and perform a feature extraction operation on each joint point of the patient in each frame of the gait image of the target gait video, so as to obtain a coordinate feature of each joint point of the patient in each frame of the gait image of the target gait video; a gait displacement judgment module, configured to judge whether the patient has gait displacement between two adjacent frames of gait images of the target gait video based on the coordinate features of each identical joint point of the patient in two adjacent frames of gait images of the target gait video and a preset first threshold, and to delete one of the gait images when the patient has not displaced between two adjacent frames of gait images of the target gait video, so as to obtain the processed target gait video; A disease prediction module, used to obtain a disease prediction result corresponding to the patient in the target gait video based on the processed target gait video; Wherein, the gait displacement judgment module is respectively connected to the feature extraction module and the disease prediction module in communication; The initial disease prediction model is trained multiple times iteratively using the gait video set until convergence.

2. The method for constructing a disease prediction model according to claim 1, characterized in that: The feature extraction module is configured to obtain the coordinate features of each joint point of the patient in each frame of the gait image of the target gait video in the following manner: Performing resolution normalization processing on the target gait video; A feature extraction operation is performed on each joint point of the patient in each frame of the target gait video after the resolution normalization process to obtain the coordinate features of each joint point of the patient in each frame of the target gait video.

3. The method for constructing a disease prediction model according to claim 1, characterized in that: The feature extraction module is configured to obtain the coordinate features of each joint point of the patient in each frame of the gait image of the target gait video in the following manner: The YoLoV8 method is used to perform a feature extraction operation on each joint point of the patient in each frame of the gait image of the target gait video to obtain the coordinate features of each joint point of the patient in each frame of the gait image of the target gait video.

4. The method for constructing a disease prediction model according to claim 1, characterized in that: The gait displacement judgment module is configured to obtain the processed target gait video in the following manner: Performing target detection on the patient in each frame of the target gait video to obtain a circumscribed rectangular bounding box of the patient in each frame of the target gait video, and extracting the length and width coordinates of the circumscribed rectangular bounding box of the patient in each frame of the target gait video to obtain the length and width coordinate features of the circumscribed rectangular bounding box of the patient in each frame of the target gait video; Based on the coordinate features of each identical joint point of the patient in two adjacent frames of gait images of the target gait video, the length and width coordinates of the circumscribed rectangular boundary box of the patient in two adjacent frames of gait images, a preset first threshold and a preset second threshold, it is determined whether the patient has a gait displacement between two adjacent frames of gait images of the target gait video, and when the patient has not displaced between two adjacent frames of gait images of the target gait video, one of the gait images is deleted to obtain the processed target gait video.

5. The method for constructing a disease prediction model according to claim 4, characterized in that: The gait displacement judgment module is further configured to obtain the processed target gait video in the following manner: The width of the circumscribed rectangular bounding box of the patient in two adjacent frames of gait images of the target gait video constitutes a first bounding box pair, and the length of the circumscribed rectangular bounding box of the patient in two adjacent frames of gait images of the target gait video constitutes a second bounding box pair, and the Euclidean distance between the wide coordinate features of the two rectangular bounding boxes of the first bounding box pair and the Euclidean distance between the long coordinate features of the two rectangular bounding boxes of the second bounding box pair in the two adjacent frames of gait images of the target gait video are calculated respectively; Each identical joint point of the patient in two adjacent frames of gait images of the target gait video constitutes a joint point pair, and the Euclidean distance between the coordinate features of the two joint points of each joint point pair in two adjacent frames of gait images of the target gait video is calculated; Based on the Euclidean distance between the coordinate features of two joint points of all joint pairs in two adjacent frames of gait images of the target gait video, the Euclidean distance between the wide coordinate features of the two rectangular bounding boxes of the first bounding box pair, the Euclidean distance between the long coordinate features of the two rectangular bounding boxes of the second bounding box pair, a preset first threshold, and a preset second threshold, it is determined whether the patient has a gait displacement between two adjacent frames of gait images of the target gait video, and when the patient has not displaced between two adjacent frames of gait images of the target gait video, one of the gait images is deleted to obtain the processed target gait video.

6. The method for constructing a disease prediction model according to claim 5, characterized in that: The coordinate feature of each joint point includes a horizontal coordinate feature of each joint point and a vertical coordinate feature of each joint point, and the gait displacement judgment module is further configured to judge whether the patient has gait displacement between two adjacent frames of gait images of the target gait video in the following manner: in, represents the wide coordinate feature of the circumscribed rectangular bounding box of the patient in the t-th frame gait image of the target gait video, represents the wide coordinate feature of the circumscribed rectangular bounding box of the patient in the t+1th frame gait image of the target gait video, Δx rect represents the Euclidean distance between the wide coordinate features of the two rectangular bounding boxes of the first bounding box pair in the t-th and t+1-th adjacent frames of the target gait video, represents the long coordinate feature of the circumscribed rectangular bounding box of the patient in the t-th frame gait image of the target gait video, represents the long coordinate feature of the patient's circumscribed rectangular bounding box in the t+1th frame gait image of the target gait video, Δy rect represents the Euclidean distance between the long coordinate features of the two rectangular bounding boxes of the second bounding box pair in the t-th and t+1-th adjacent frames of the target gait video, represents the ordinate feature of the i-th joint point of the patient in the t-th frame gait image of the target gait video, represents the ordinate feature of the u-th joint point of the patient in the t+1-th frame gait image of the target gait video, Δy i It represents the Euclidean distance between the ordinate features of the two joint points of the i-th joint pair in the t-th and t+1-th adjacent frames of the target gait video, where a1 is a preset first threshold and a2 is a preset second threshold.

7. A disease prediction model constructed using the method for constructing a disease prediction model according to any one of claims 1 to 6, wherein the disease prediction model is used to predict a disease based on a patient's gait video, wherein: The gait video includes multiple frames of gait images, the gait video corresponds to a patient, the body structure of the patient includes multiple joints, and the model includes: A feature extraction module, used for performing a feature extraction operation on each joint point of the patient in each frame of the gait image of the gait video, so as to obtain a coordinate feature of each joint point of the patient in each frame of the gait image of the gait video; a gait displacement judgment module, configured to judge whether the patient has gait displacement between two adjacent frames of gait images of the gait video based on the coordinate features of each identical joint point of the patient in two adjacent frames of gait images of the gait video and a preset first threshold, and to delete one of the gait images when the patient has not displaced between two adjacent frames of gait images of the gait video, so as to obtain the processed target gait video; A disease prediction module, used for obtaining a disease prediction result corresponding to the patient in the gait video based on the processed gait video; Wherein, the gait displacement judgment module is communicatively connected with the feature extraction module and the disease prediction module respectively.

8. A method for applying a disease prediction model, characterized in that: The method comprises: Obtain a video of the patient's gait; The disease prediction model as claimed in claim 7 is used to obtain a disease prediction result corresponding to the patient based on the gait video of the patient.

9. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method according to any one of claims 1 to 6 and 8 when executing the computer program.

10. A computer-readable storage medium having program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the method described in any one of claims 1-6 and 8 is implemented.

Images