Construction and application method of disease prediction model, model, equipment and medium

Abstract

The invention relates to the field of medicine, and discloses a construction and application method of a disease prediction model, the model, equipment and a medium. According to the scheme, a standardization module in a disease prediction model is configured to perform standardization processing on each frame of gait image of gait videos collected by camera devices at different positions based on a calculated distance value between each joint point of a patient in each frame of gait image of a target gait video and an original point of a new two-dimensional coordinate system and a preset distance value; according to the method, the distance value between each joint point of the patient and the original point of the new two-dimensional coordinate system is calculated to dynamically carry out standardization processing on the gait video, so that motion information of different angles can be mapped to communicated semantic latitudes, and the motion information of the patient in each frame of gait image can be well reserved; therefore, a more accurate disease prediction result can be obtained based on the processed gait video, and the model performance is improved.

Description

Technical Field

[0001] The present application relates to the medical field, and in particular to a method, model, device, and medium for constructing and applying a disease prediction model. Background Art

[0002] With the development of medical imaging technology and biometric recognition technology, gait analysis, as a non-contact biometric recognition method, has been widely used in the field of clinical medicine. By analyzing gait, doctors can diagnose certain neurological diseases, musculoskeletal diseases, and other health problems that affect walking patterns.

[0003] At present, relevant researchers have trained many disease prediction models using gait videos, aiming to assist doctors in disease diagnosis by performing gait analysis through trained disease prediction models, that is, using disease prediction models to obtain disease prediction results of patients based on their gait videos, and using the disease prediction results to assist doctors in making more accurate medical diagnoses. Among them, when collecting gait videos, the camera is usually placed in a position parallel to the patient's activity plane, for example, the camera is fixed in front of the patient, and the patient walks on a specified path to obtain his gait video. However, due to the fixed position of the camera, the size of the patient in the gait video will change with the distance between the patient and the camera. When the patient is far away from the camera, its proportion in the gait image decreases, resulting in unclear motion information, which results in the model being unable to learn the patient's motion information in each frame of the gait image when the disease prediction model is trained using the patient's gait video, resulting in poor performance of the trained model.

[0004] Related researchers introduced fixed standardization parameters (also known as scaling parameters) to standardize each frame of the gait video to ensure that the motion information between frames in the gait video is aligned, thereby retaining the patient's motion characteristics in each frame. However, this method has limitations, especially under different camera position settings, fixed standardization parameters (such as pre-set standardization parameters) are difficult to adapt to all situations. However, when training disease prediction models and applying trained disease prediction models, many gait videos are generally processed, and the camera position settings of each gait video are different. The fixed standardization parameters used in the existing standardization method for each frame of gait images in gait videos are obtained when the camera is set at a specified position. This standardization scheme cannot meet the settings of other camera postures (for example, shooting a patient's gait video at a higher angle, such shooting causes more patient motion information to be covered, such as shortened leg length, some lower limb information being blocked by the trunk during gait, and the upper body trunk standing screen ratio becoming higher).

[0005] In summary, the existing method for standardizing each frame of gait images in gait videos uses fixed standardization parameters to standardize each frame of gait images of gait videos collected by cameras at different positions. However, it is unable to well retain the patient's motion information in each frame of gait images, resulting in inaccurate disease prediction results when using the processed gait videos to perform disease prediction based on the disease prediction model, that is, the performance of the model is poor. Summary of the invention

[0006] The present application provides a method, model, device, and medium for constructing and applying a disease prediction model to solve the problem that the existing method uses fixed standardized parameters to standardize each frame of gait images of gait videos collected by cameras at different positions, resulting in inaccurate disease prediction results when the processed gait video is used to predict the disease based on the disease prediction model, that is, the performance of the model is poor.

[0007] The first aspect of the present application provides a method for constructing a disease prediction model, wherein the disease prediction model is used to predict a disease based on a patient's gait video, the method comprising:

[0008] Acquire a gait video set, wherein the gait video set includes a plurality of gait videos, each gait video includes a plurality of frames of gait images, each gait video corresponds to a patient, and the patient's body structure includes a plurality of joint points, and the plurality of joint points include at least a left hip, a right hip, a left shoulder, and a right shoulder joint point;

[0009] Construct an initial disease prediction model, wherein the initial disease prediction model of the present application includes:

[0010] A global coordinate extraction module is used to take each gait video as a target gait video and obtain the initial horizontal and vertical coordinate features of each joint point of the patient in each frame of the gait image of the target gait video based on a preset initial two-dimensional coordinate system;

[0011] A human body structure coordinate construction module is used to construct a new two-dimensional coordinate system based on the origin coordinates of the preset initial two-dimensional coordinate system and the initial horizontal and vertical coordinate features of the left and right hip joints of the patient in each frame of the gait image of the target gait video, and obtain new horizontal and vertical coordinate features of each joint of the patient in each frame of the gait image of the target gait video based on the new two-dimensional coordinate system;

[0012] The standardization module is used to obtain the target distance between each joint point of the patient in each frame of the gait image of the target gait video based on the new horizontal and vertical coordinate features of the patient's left hip, right hip, left shoulder and right shoulder joint points in each frame of the gait image of the target gait video, the calculated distance value between each joint point of the patient in each frame of the gait image of the target gait video and the origin of the new two-dimensional coordinate system, and the preset distance value.

[0013] Horizontal and vertical coordinate characteristics;

[0014] A disease prediction module is used to obtain a disease prediction result corresponding to the patient in the target gait video based on the target horizontal and vertical coordinates of each joint point of the patient in each frame of the gait image of the target gait video;

[0015] Among them, the human body structure coordinate construction module is communicated with the global coordinate extraction module and the standardization module respectively, and the standardization module is also communicated with the disease prediction module;

[0016] The initial disease prediction model was trained multiple times using the gait video set until convergence.

[0017] In some embodiments of the present application, the standardization module of the present application is configured to obtain the target horizontal and vertical coordinate features of each joint point of the patient in each frame of the gait image of the target gait video in the following manner:

[0018] Based on the new horizontal and vertical coordinate features of the left and right hip joints of the patient in each frame of the gait image of the target gait video, new horizontal and vertical coordinate features of the center point between the left and right hip joints of the patient in each frame of the gait image of the target gait video are obtained, and based on the new horizontal and vertical coordinate features of the left and right shoulder joints of the patient in each frame of the gait image of the target gait video, new horizontal and vertical coordinate features of the center point between the left and right shoulder joints of the patient in each frame of the gait image of the target gait video are obtained;

[0019] Based on the new horizontal and vertical coordinate features of the center point between the left hip and right hip joints of the patient in each frame of the gait image of the target gait video and the new horizontal and vertical coordinate features of the center point between the left shoulder and right shoulder joints of the patient in each frame of the gait image of the target gait video, calculate the distance value between the center point between the left hip and right hip joints of the patient and the center point between the left shoulder and right shoulder joints in each frame of the gait image of the target gait video;

[0020] Based on the new horizontal and vertical coordinate features of each joint point of the patient in each frame of the gait image of the target gait video, the distance value between each joint point of the patient in each frame of the gait image of the target gait video and the origin of the new two-dimensional coordinate system is calculated;

[0021] The distance values ​​between each joint point of the patient and the origin of the new two-dimensional coordinate system in each frame of the gait image of the target gait video and the preset distance values ​​are sorted to screen out the largest distance value, and the target distance value is obtained based on the maximum distance value and the distance value between the center point between the left hip and right hip joint points of the patient and the center point between the left shoulder and right shoulder joint points in each frame of the gait image of the target gait video;

[0022] Based on the target distance value and the new horizontal and vertical coordinate features of each joint point of the patient in each frame of the gait image of the target gait video, the target horizontal and vertical coordinate features of each joint point of the patient in each frame of the gait image of the target gait video are obtained.

[0023] In some embodiments of the present application, the standardization module of the present application is configured to obtain the target distance value in the following manner:

[0024] pose_size=tosor_size×torso_size_multiplier

[0025] Among them, pose_size represents the target distance value, tosor_size represents the distance value between the center point between the left hip and right hip joints and the center point between the left shoulder and right shoulder joints of the patient in each frame of the gait image of the target gait video, and torso_size_multiplier represents the maximum distance value.

[0026] In some embodiments of the present application, the standardization module of the present application is further configured to obtain the target horizontal and vertical coordinate features of each joint point of the patient in each frame of the gait image of the target gait video in the following manner:

[0027] The new horizontal and vertical coordinate features of each joint point of the patient in each frame of the target gait video are normalized, and based on the new horizontal and vertical coordinate features of the left hip, right hip, left shoulder and right shoulder joint points of the patient in each frame of the target gait video after the normalization, the calculated distance value between each joint point of the patient in each frame of the target gait video and the origin of the new two-dimensional coordinate system, and the preset distance value, the target horizontal and vertical coordinate features of each joint point of the patient in each frame of the target gait video are obtained.

[0028] In some embodiments of the present application, the standardization module of the present application is further configured to normalize the new horizontal and vertical coordinate features of each joint point of the patient in each frame of the gait image of the target gait video in the following manner:

[0029] Based on the new horizontal and vertical coordinate features of the left and right hip joints of the patient in each frame of the gait image of the target gait video, new horizontal and vertical coordinate features of the center point between the left and right hip joints of the patient in each frame of the gait image of the target gait video are obtained, and based on the new horizontal and vertical coordinate features of the left and right shoulder joints of the patient in each frame of the gait image of the target gait video, new horizontal and vertical coordinate features of the center point between the left and right shoulder joints of the patient in each frame of the gait image of the target gait video are obtained;

[0030] Based on the new horizontal and vertical coordinate features of the center point between the left hip and right hip joints of the patient in each frame of the gait image of the target gait video and the new horizontal and vertical coordinate features of the center point between the left shoulder and right shoulder joints of the patient in each frame of the gait image of the target gait video, calculate the vector from the center point between the left shoulder and right shoulder joints of the patient in each frame of the gait image of the target gait video to the center point between the left hip and right hip joints;

[0031] A vertical vector is arbitrarily selected, and the angle formed by the vertical vector and the vector from the center point between the left shoulder and right shoulder joints of the patient to the center point between the left hip and right hip joints in each frame of the gait image of the target gait video is calculated, and based on the angle, the new horizontal and vertical coordinate features of each joint point of the patient in each frame of the gait image of the target gait video are normalized.

[0032] In some embodiments of the present application, the standardization module of the present application is further configured to normalize the new horizontal and vertical coordinate features of each joint point of the patient in each frame of the gait image of the target gait video in the following manner:

[0033]

[0034]

[0035] Wherein, θ is the angle formed by the vertical vector and the vector from the center point between the left shoulder and right shoulder joints of the patient to the center point between the left hip and right hip joints in each frame of the gait image of the target gait video, Represents the new horizontal coordinate feature of the i-th joint point of the patient in each frame of the gait image of the target gait video, Represents the new ordinate feature of the i-th joint point of the patient in each frame of the gait image of the target gait video, represents the new horizontal coordinate feature of the i-th joint point of the patient in each frame of the gait image of the normalized target gait video, Represents the new vertical coordinate feature of the i-th joint point of the patient in each frame of the gait image of the normalized target gait video.

[0036] A second aspect of the present application provides a disease prediction model, which is used to predict a disease based on a patient's gait video, wherein the gait video includes multiple frames of gait images, the gait video corresponds to a patient, the patient's body structure includes multiple joints, and the multiple joints include at least left hip, right hip, left shoulder, and right shoulder joints. The disease prediction model of the present application includes:

[0037] A global coordinate extraction module, used to obtain the initial horizontal and vertical coordinate features of each joint point of the patient in each frame of the gait image of the gait video based on a preset initial two-dimensional coordinate system;

[0038] A human body structure coordinate construction module is used to construct a new two-dimensional coordinate system based on the origin coordinates of the preset initial two-dimensional coordinate system and the initial horizontal and vertical coordinate features of the left and right hip joints of the patient in each frame of the gait image of the gait video, and obtain new horizontal and vertical coordinate features of each joint of the patient in each frame of the gait video based on the new two-dimensional coordinate system;

[0039] A standardization module, for obtaining target horizontal and vertical coordinate features of each joint point of the patient in each frame of the gait image of the gait video based on new horizontal and vertical coordinate features of the left hip, right hip, left shoulder and right shoulder joint points of the patient in each frame of the gait image of the gait video, a calculated distance value between each joint point of the patient in each frame of the gait image of the gait video and the origin of the new two-dimensional coordinate system, and a preset distance value;

[0040] A disease prediction module is used to obtain a disease prediction result corresponding to the patient in the gait video based on the target horizontal and vertical coordinates of each joint point of the patient in each frame of the gait video;

[0041] Among them, the human body structure coordinate construction module is communicated with the global coordinate extraction module and the standardization module respectively, and the standardization module is also communicated with the disease prediction module.

[0042] The third aspect of the present application provides an application method of a disease prediction model, the method comprising:

[0043] Obtain a video of the patient's gait;

[0044] The disease prediction result of the patient is obtained based on the patient's gait video using the disease prediction model in the above embodiment.

[0045] The fourth aspect of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the method described in any one of the first and third aspects of the above embodiments are implemented.

[0046] The fifth aspect of the present application provides a computer-readable storage medium having program instructions stored thereon, and when the program instructions are executed by a processor, the method described in any one of the first aspect and the third aspect of the above embodiments is implemented.

[0047] The present application has the following beneficial effects: in the construction scheme of the disease prediction model of the present application, the standardization module in the disease prediction model is configured to perform standardization processing on each frame of gait image of the gait video collected by camera devices at different positions based on the calculated distance value and preset distance value between each joint point of the patient and the origin of the new two-dimensional coordinate system in each frame of gait image of the target gait video. This method of dynamically standardizing the gait video by calculating the distance value between each joint point of the patient and the origin of the new two-dimensional coordinate system can map motion information at different angles to a common semantic latitude, so as to achieve good retention of the motion information of the patient in each frame of the gait image, so that more accurate disease prediction results can be obtained based on the processed gait video, thereby improving the model performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application.

[0049] Figure 1 This is a flow chart of an embodiment of a method for constructing a disease prediction model provided by the present application;

[0050] Figure 2 This is a schematic diagram of a framework of an embodiment of a disease prediction model provided by the present application;

[0051] Figure 3 It is a flow chart of an embodiment of a method for calculating target horizontal and vertical coordinates of each joint point of a patient in each frame of a gait image of a target gait video provided by the present application;

[0052] Figure 4 It is a flow chart of an embodiment of a method for normalizing new horizontal and vertical coordinate features of each joint point of a patient in each frame of a gait image of a target gait video provided by the present application;

[0053] FIG5( a ) is a schematic diagram of experimental results of coordinate system center transfer using the y-axis coordinate value of the patient's left shoulder as an example provided by the present application;

[0054] FIG5( b ) is a schematic diagram of the experimental results provided by the present application of the standardization processing using the y-axis coordinate value of the patient's left shoulder as an example;

[0055] FIG5( c ) is a schematic diagram of experimental results of coordinate system center transfer using the x-axis coordinate value of the patient's left shoulder as an example provided by the present application;

[0056] FIG5( d ) is a schematic diagram of an experimental result provided by the present application of performing a normalization process using the y-axis coordinate value of the patient's left shoulder as an example;

[0057] Figure 6This is a flow chart of an embodiment of an application method of a disease prediction model provided by the present application;

[0058] Figure 7 It is a schematic diagram of a framework of an embodiment of an electronic device provided by the present application;

[0059] Figure 8 It is a schematic diagram of a framework of an embodiment of a computer-readable storage medium provided by the present application. DETAILED DESCRIPTION

[0060] The scheme of the embodiment of the present application is described in detail below in conjunction with the drawings of the specification.

[0061] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.

[0062] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the objects associated before and after are in an "or" relationship. In addition, "many" in this article means two or more than two. In addition, the term "at least one" in this article means any combination of at least two of any one or more of a plurality of, for example, including at least one of A, B, and C, can mean including any one or more elements selected from the set consisting of A, B, and C.

[0063] As described in the background technology, the existing method for standardizing each frame of gait images in gait videos uses fixed standardization parameters to standardize each frame of gait images in gait videos collected by cameras at different positions. However, it is unable to well retain the patient's motion information in each frame of gait images, resulting in inaccurate disease prediction results when using the processed gait video to perform disease prediction based on the disease prediction model, i.e., the performance of the model is poor.

[0064] In order to solve the above problems, the present application provides a new disease prediction model construction scheme. In the construction scheme of the present application, the standardization module in the disease prediction model is configured to standardize each frame of the gait video collected by camera devices at different positions based on the calculated distance value and preset distance value between each joint point of the patient and the origin of the new two-dimensional coordinate system in each frame of the gait image of the target gait video. This method of dynamically standardizing the gait video by calculating the distance value between each joint point of the patient and the origin of the new two-dimensional coordinate system can map motion information at different angles to a common semantic latitude, so as to achieve good retention of the patient's motion information in each frame of the gait image, so that more accurate disease prediction results can be obtained based on the processed gait video, thereby improving model performance.

[0065] The present application is described in detail below with reference to the accompanying drawings and specific embodiments.

[0066] According to one embodiment of the present application, Figure 1 As shown, the present application provides a method for constructing a disease prediction model, which is used to predict diseases based on gait videos of patients. The method includes: S1, obtaining a gait video set, wherein the gait video set includes multiple gait videos, each gait video includes multiple frames of gait images, each gait video corresponds to a patient, and the patient's body structure includes multiple joints, and the multiple joints include at least left hip, right hip, left shoulder and right shoulder joints; S2, constructing an initial disease prediction model, wherein, as Figure 2As shown, the initial disease prediction model of the present application includes: a global coordinate extraction module, which is used to take each gait video as a target gait video, and obtain the initial horizontal and vertical coordinate features of each joint point of the patient in each frame of the gait image of the target gait video based on a preset initial two-dimensional coordinate system; a human body structure coordinate construction module, which is used to construct a new two-dimensional coordinate system based on the origin coordinates of the preset initial two-dimensional coordinate system and the initial horizontal and vertical coordinate features of the left hip and right hip joint points of the patient in each frame of the gait image of the target gait video, and obtain new horizontal and vertical coordinate features of each joint point of the patient in each frame of the gait image of the target gait video based on the new two-dimensional coordinate system; a normalization module, which is used to obtain the initial horizontal and vertical coordinate features of the left hip, right hip, left shoulder and right hip joint points of the patient in each frame of the gait image of the target gait video based on the origin coordinates of the preset initial two-dimensional coordinate system and the initial horizontal and vertical coordinate features of the left hip and right hip joint points of the patient in each frame of the gait image of the target gait video. and the new horizontal and vertical coordinate features of the right shoulder joint, the calculated distance value between each joint point of the patient and the origin of the new two-dimensional coordinate system in each frame of the gait image of the target gait video, and the preset distance value, to obtain the target horizontal and vertical coordinate features of each joint point of the patient in each frame of the gait image of the target gait video; a disease prediction module, which is used to obtain the disease prediction result corresponding to the patient in the target gait video based on the target horizontal and vertical coordinates of each joint point of the patient in each frame of the gait image of the target gait video; wherein the human body structure coordinate construction module is communicated with the global coordinate extraction module and the standardization module respectively, and the standardization module is also communicated with the disease prediction module; S3, using the gait video set to perform multiple iterative training on the initial disease prediction model until convergence.

[0067] According to one embodiment of the present application, in each iterative training process, the parameters of the disease prediction model are updated according to the following loss function:

[0068]

[0069] Among them, y is the true value of the disease corresponding to the patient in the CT image, The disease prediction result corresponding to the patient in the target gait video.

[0070] It can be seen from the above embodiments that the true result of whether the patient has the corresponding disease is obtained based on the patient's CT image, and the parameters of the disease prediction model are updated based on the loss value between the true disease result and the disease prediction result obtained by the model, thereby improving the performance of the disease prediction model.

[0071] In order to more intuitively understand the functions performed by the human body structure coordinate construction module and the standardization module in the disease prediction model constructed by the construction scheme of the present application, the human body structure coordinate construction module and the standardization module of the disease prediction model are explained in detail below.

[0072] 1. Human body structure coordinate construction module

[0073] The human body structure coordinate construction module is used to construct a new two-dimensional coordinate system based on the origin coordinates of the preset initial two-dimensional coordinate system and the initial horizontal and vertical coordinate features of the patient's left hip and right hip joint points in each frame of the gait image of the target gait video, and obtain the new horizontal and vertical coordinate features of each joint point of the patient in each frame of the gait image of the target gait video based on the new two-dimensional coordinate system.

[0074] According to one embodiment of the present application, the human body structure coordinate construction module is configured to construct a new two-dimensional coordinate system in the following manner:

[0075]

[0076]

[0077] in, Represents the new horizontal coordinate of the center point between the left and right hip joints of the patient in each frame of the gait image of the target gait video, Represents the new ordinate of the center point between the left and right hip joints of the patient in each frame of the gait image of the target gait video, represents the initial horizontal coordinate of the patient's left hip joint point in each frame of the target gait video, represents the initial horizontal coordinate of the patient's right hip joint point in each frame of the target gait video, represents the initial ordinate of the left hip joint point of the patient in each frame of the gait image of the target gait video, represents the initial ordinate of the patient's right hip joint point in each frame of the target gait video, x0 represents the abscissa of the origin of the preset initial two-dimensional coordinate system, y0 represents the ordinate of the origin of the preset initial two-dimensional coordinate system, and x new Represents the horizontal coordinate of the origin of the new two-dimensional coordinate system, y new The ordinate that represents the origin of the new two-dimensional coordinate system.

[0078] It can be seen from the above embodiments that the present application uses the center of the human body as the new origin of the coordinate system. Changes in the patient's spatial position relationship do not affect the relationship between the coordinate data and the patient's position, so that no matter how the camera posture height and the overhead angle are set, the fidelity of the signal in the image is protected.

[0079] 2. Standardization Module

[0080] The standardization module is used to obtain the target horizontal and vertical coordinate features of each joint point of the patient in each frame of the gait image of the target gait video based on the new horizontal and vertical coordinate features of the left hip, right hip, left shoulder and right shoulder joint points of the patient in each frame of the gait image of the target gait video, the calculated distance value between each joint point of the patient in each frame of the gait image of the target gait video and the origin of the new two-dimensional coordinate system, and the preset distance value.

[0081] According to one embodiment of the present application, Figure 3 As shown, the standardization module of the present application is configured to execute steps P1-P5 to obtain the target horizontal and vertical coordinate features of each joint point of the patient in each frame of the gait image of the target gait video. The technical content of steps P1-P5 is described in detail below.

[0082] (1) Step P1

[0083] In step P1, based on the new horizontal and vertical coordinate features of the left and right hip joints of the patient in each frame of the target gait video, the new horizontal and vertical coordinate features of the center point between the left and right hip joints of the patient in each frame of the gait image of the target gait video are obtained, and based on the new horizontal and vertical coordinate features of the left and right shoulder joints of the patient in each frame of the gait image of the target gait video, the new horizontal and vertical coordinate features of the center point between the left and right shoulder joints of the patient in each frame of the gait video are obtained.

[0084] (2) Step P2

[0085] In step P2, based on the new horizontal and vertical coordinate features of the center point between the left hip and right hip joints of the patient in each frame of the gait image of the target gait video and the new horizontal and vertical coordinate features of the center point between the left shoulder and right shoulder joints of the patient in each frame of the gait image of the target gait video, the distance value between the center point between the left hip and right hip joints of the patient and the center point between the left shoulder and right shoulder joints in each frame of the gait image of the target gait video is calculated.

[0086] According to one embodiment of the present application, the standardization module of the present application is configured to obtain the distance value between the center point between the left hip and right hip joint points of the patient and the center point between the left shoulder and right shoulder joint points in each frame of the gait image of the target gait video in the following manner:

[0087]

[0088] Among them, tosor_size represents the distance between the center point between the left hip and right hip joints of the patient and the center point between the left shoulder and right shoulder joints in each frame of the gait image of the target gait video. Represents the new horizontal coordinate of the center point between the left shoulder and right shoulder joint points of the patient in each frame of the gait image of the target gait video, Represents the new ordinate of the center point between the left shoulder and right shoulder joint points of the patient in each frame of the gait image of the target gait video, Represents the new horizontal coordinate of the center point between the left and right hip joints of the patient in each frame of the gait image of the target gait video, Represents the new vertical coordinate of the center point between the left hip and right hip joints of the patient in each frame of the gait image of the target gait video.

[0089] (3) Step P3

[0090] In step P3, based on the new horizontal and vertical coordinate features of each joint point of the patient in each frame of the gait image of the target gait video, the distance value between each joint point of the patient in each frame of the gait image of the target gait video and the origin of the new two-dimensional coordinate system is calculated.

[0091] According to one embodiment of the present application, the standardization module of the present application is configured to obtain the distance value between each joint point of the patient and the origin of the new two-dimensional coordinate system in each frame of the gait image of the target gait video in the following manner:

[0092]

[0093] Among them, distance i Represents the distance value between the i-th joint point of the patient in each frame of the gait image of the target gait video and the origin of the new two-dimensional coordinate system, represents the new horizontal coordinate of the i-th joint point of the patient in each frame of the gait image of the target gait video, represents the new ordinate of the i-th joint point of the patient in each frame of the gait image of the target gait video, x new Represents the horizontal coordinate of the origin of the new two-dimensional coordinate system, y new The ordinate that represents the origin of the new two-dimensional coordinate system.

[0094] (4) Step P4

[0095] In step P4, the distance values ​​between each joint point of the patient and the origin of the new two-dimensional coordinate system in each frame of the gait image of the target gait video and the preset distance values ​​are sorted to screen out the largest distance value, and the target distance value is obtained based on the maximum distance value and the distance value between the center point between the left hip and right hip joints of the patient and the center point between the left shoulder and right shoulder joints in each frame of the gait image of the target gait video.

[0096] According to one embodiment of the present application, the standardization module of the present application is configured to obtain the target distance value in the following manner:

[0097] pose_size=tosor_size×torso_size_multiplier

[0098] Among them, pose_size represents the target distance value, tosor_size represents the distance value between the center point between the left hip and right hip joints and the center point between the left shoulder and right shoulder joints of the patient in each frame of the gait image of the target gait video, and torso_size_multiplier represents the maximum distance value. Among them,

[0099] torsor_size_multiplier=max(distance i , distance_middle_trunk)

[0100] i=0,1,...,n

[0101] Among them, distance i It represents the distance value between the i-th joint point of the patient in each frame of the gait image of the target gait video and the origin of the new two-dimensional coordinate system, and distance_middle_trunk represents the preset distance value.

[0102] As can be seen from the above embodiments, the present application traverses the distances of all the patient's related nodes to the human body structure coordinate system (i.e., the new two-dimensional coordinate system), and compares all distance values ​​with the preset empirical threshold to select the largest distance value as torso_size_multiplier. This achieves high-precision dynamic feature capture capabilities under different camera postures and when the body posture has information loss.

[0103] (5) Step P5

[0104] In step P5, based on the target distance value and the new horizontal and vertical coordinate features of each joint point of the patient in each frame of the gait image of the target gait video, the target horizontal and vertical coordinate features of each joint point of the patient in each frame of the gait image of the target gait video are obtained.

[0105] According to one embodiment of the present application, the standardization module of the present application is configured to obtain the target horizontal and vertical coordinate features of each joint point of the patient in each frame of the gait image of the target gait video in the following manner:

[0106]

[0107] in, Represents the target horizontal coordinate feature of the patient's i-th joint point in each frame of the target gait video, represents the target ordinate feature of the patient's i-th joint point in each frame of the target gait video, pose_size represents the target distance value, Represents the new horizontal coordinate feature of the i-th joint point of the patient in each frame of the gait image of the target gait video, Represents the new ordinate feature of the i-th joint point of the patient in each frame of the gait image of the target gait video.

[0108] It can be seen from the above embodiments that standardizing the joint point coordinate features based on the target distance value improves the consistency and comparability of the data and enhances the generalization ability and robustness of the model.

[0109] Among them, according to one embodiment of the present application, the standardization module of the present application is also configured to obtain the target horizontal and vertical coordinate features of each joint point of the patient in each frame of the gait image of the target gait video in the following manner: normalize the new horizontal and vertical coordinate features of each joint point of the patient in each frame of the gait image of the target gait video, and based on the new horizontal and vertical coordinate features of the left hip, right hip, left shoulder and right shoulder joint points of the patient in each frame of the gait image of the target gait video after the normalization, the calculated distance value between each joint point of the patient in each frame of the gait image of the target gait video and the origin of the new two-dimensional coordinate system, and the preset distance value, obtain the target horizontal and vertical coordinate features of each joint point of the patient in each frame of the gait image of the target gait video.

[0110] It can be seen from the above embodiments that by normalizing the new horizontal and vertical coordinate features of each joint point of the patient in each frame of the gait image of the target gait video, the coordinate features of all joint points can be scaled to the same range, eliminating the scale differences between different individuals and different videos, ensuring the consistency and comparability of the data, thereby enhancing the generalization ability and robustness of the model.

[0111] According to one embodiment of the present application, Figure 4As shown, the standardization module of the present application is also configured to perform the following steps P51-P53 to normalize the new horizontal and vertical coordinate features of each joint point of the patient in each frame of the gait image of the target gait video: P51, based on the new horizontal and vertical coordinate features of the left hip and right hip joint points of the patient in each frame of the gait image of the target gait video, obtain the new horizontal and vertical coordinate features of the center point between the left hip and right hip joint points of the patient in each frame of the gait image of the target gait video, and based on the new horizontal and vertical coordinate features of the left shoulder and right shoulder joint points of the patient in each frame of the gait image of the target gait video, obtain the new horizontal and vertical coordinate features of the center point between the left shoulder and right shoulder joint points of the patient in each frame of the gait image of the target gait video; P52, based on the new horizontal and vertical coordinate features of the target gait video The new horizontal and vertical coordinate features of the center point between the left hip and right hip joints of the patient in each frame of the gait image and the new horizontal and vertical coordinate features of the center point between the left shoulder and right shoulder joints of the patient in each frame of the gait image of the target gait video are calculated, and the vector from the center point between the left shoulder and right shoulder joints of the patient in each frame of the gait image of the target gait video to the center point between the left hip and right hip joints is calculated; P53, arbitrarily select a vertical vector, and calculate the angle formed by the vertical vector and the vector from the center point between the left shoulder and right shoulder joints of the patient in each frame of the gait image of the target gait video to the center point between the left hip and right hip joints, and based on the angle, the new horizontal and vertical coordinate features of each joint point of the patient in each frame of the gait image of the target gait video are normalized. According to one embodiment of the present application, in step P53, the new horizontal and vertical coordinate features of each joint point of the patient in each frame of the gait image of the target gait video are normalized in the following manner:

[0112]

[0113] Wherein, θ is the angle formed by the vertical vector and the vector from the center point between the left shoulder and right shoulder joints of the patient to the center point between the left hip and right hip joints in each frame of the gait image of the target gait video, Represents the new horizontal coordinate feature of the i-th joint point of the patient in each frame of the gait image of the target gait video, Represents the new ordinate feature of the i-th joint point of the patient in each frame of the gait image of the target gait video, represents the new horizontal coordinate feature of the i-th joint point of the patient in each frame of the gait image of the normalized target gait video, Represents the new vertical coordinate feature of the i-th joint point of the patient in each frame of the gait image of the normalized target gait video.

[0114] The solution for standardizing the gait video in the above embodiment can well preserve the patient's motion information in each frame of the gait image when standardizing each frame of the gait video collected by the camera devices at different positions.

[0115] In addition, the disease prediction model construction method of the above-mentioned embodiment of the present application is adopted. According to one embodiment of the present application, the present application provides a disease prediction model, which is used to predict diseases based on the patient's gait video, wherein the gait video includes multiple frames of gait images, the gait video corresponds to one patient, and the patient's body structure includes multiple joints, and the multiple joints include at least left hip, right hip, left shoulder and right shoulder joints. The disease prediction model of the present application includes: a global coordinate extraction module, which is used to obtain the initial horizontal and vertical coordinate features of each joint point of the patient in each frame of the gait image of the gait video based on a preset initial two-dimensional coordinate system; a human body structure coordinate construction module, which is used to construct a new two-dimensional coordinate system based on the origin coordinates of the preset initial two-dimensional coordinate system and the initial horizontal and vertical coordinate features of the left hip and right hip joints of the patient in each frame of the gait image of the gait video. system, and obtain new horizontal and vertical coordinate features of each joint point of the patient in each frame of gait image of the gait video based on the new two-dimensional coordinate system; a standardization module is used to obtain target horizontal and vertical coordinate features of each joint point of the patient in each frame of gait image of the gait video based on the new horizontal and vertical coordinate features of the left hip, right hip, left shoulder and right shoulder joint points of the patient in each frame of gait image of the gait video, the calculated distance value between each joint point of the patient in each frame of gait image of the gait video and the origin of the new two-dimensional coordinate system, and the preset distance value; a disease prediction module is used to obtain the corresponding disease prediction result of the patient in the gait video based on the target horizontal and vertical coordinates of each joint point of the patient in each frame of gait image of the gait video; wherein the human body structure coordinate construction module is communicated with the global coordinate extraction module and the standardization module respectively, and the standardization module is also communicated with the disease prediction module.

[0116] In order to verify the effectiveness of the disease prediction model construction scheme of the above embodiment of the present application, that is, the construction scheme of the present application can improve the performance of the disease prediction model, the inventors conducted a series of experiments, and the specific experimental results are as follows:

[0117] As shown in FIG5(a), it shows a schematic diagram of the experimental results of coordinate system center transfer using the y-axis coordinate value of the patient's left shoulder as an example, wherein the blue curve in FIG5(a) represents the y-axis coordinate point of the patient's left shoulder during walking captured by the high-position camera, and the yellow curve represents the y-axis coordinate point of the patient's left shoulder during walking captured by the low-position camera. The coordinate values ​​of the two curves of the first picture of FIG5(a) (i.e., the left picture of FIG5(a)) can be obtained by calculation, and the signal correlation of the two curves (i.e., Signal correlation) = 0.8865914415073374; after the coordinate system center transfer (i.e., Normalizetranslation) processing, the second picture of FIG5(a) (i.e., the right picture of FIG5(a)) is obtained, and the signal correlation of the two curves = 0.9747832994849105.

[0118] As shown in FIG5(b), it shows a schematic diagram of the experimental results of normalizing the y-axis coordinate value of the patient's left shoulder as an example, wherein the blue curve in FIG5(b) represents the y-axis coordinate point of the patient's left shoulder during walking captured by the high-position camera, and the yellow curve represents the y-axis coordinate point of the patient's left shoulder during walking captured by the low-position camera. The coordinate values ​​of the two curves in the first left picture of FIG5(b) can be obtained by calculation, and the signal correlation (i.e., Signal correlation) of the two curves is 0.985382055 (i.e., the second picture on the right of FIG5(b) (i.e., the upper right corner of FIG5(b)) is obtained by the normalization processing method of the present application), and the coordinate values ​​of the two curves in the third picture of FIG5(b) (i.e., the lower right corner of FIG5(b)) can be obtained by calculation, and the signal correlation (i.e., Signal correlation) of the two curves is 0.9989567336023486 (i.e., the lower right corner of FIG5(b) is obtained by the normalization processing method of the prior art).

[0119] As shown in FIG5(c), it shows a schematic diagram of the experimental results of the coordinate system center transfer using the x-axis coordinate value of the patient's left shoulder as an example, wherein the blue curve in FIG5(c) represents the x-axis coordinate point of the patient's left shoulder during walking captured by the high-position camera, and the yellow curve represents the x-axis coordinate point of the patient's left shoulder during walking captured by the low-position camera. The coordinate values ​​of the two curves of the first picture of FIG5(c) (i.e., the left picture of FIG5(c)) can be obtained by calculation, and the signal correlation of the two curves (i.e., Signal correlation) = 0.996660016434495; after the coordinate system center transfer (i.e., Normalizetranslation) processing, the second picture of FIG5(c) (i.e., the right picture of FIG5(c)) is obtained, and the signal correlation of the two curves = 0.9146136763494871.

[0120] As shown in Figure 5(d), it shows a schematic diagram of the experimental results of standardization processing using the x-axis coordinate value of the patient's left shoulder as an example, wherein the blue curve in Figure 5(d) represents the x-axis coordinate point of the patient's left shoulder during walking captured by the high-position camera, and the yellow curve represents the x-axis coordinate point of the patient's left shoulder during walking captured by the low-position camera. The coordinate values ​​of the two curves in the second picture of Figure 5(d) (i.e., the left picture of Figure 5(d)) can be obtained by calculation, and the signal correlation (i.e., Signal correlation) of the two curves = 0.94485445 (i.e., the second picture on the right of Figure 5(d) (i.e., the upper right picture of Figure 5(d)) is obtained by the standardization processing method of the present application), and the coordinate values ​​of the two curves in the third picture on the right of Figure 5(d) (i.e., the lower right picture of Figure 5(d)) can be obtained by calculation, and the signal correlation (i.e., Signal correlation) of the two curves = 0.938326988631854 (i.e., the lower right picture of Figure 5(d) is obtained by the standardization processing method of the prior art).

[0121] Therefore, taking the x and y axis coordinate values ​​of the patient's left shoulder as an example, the average value of the signal correlation of the two curves before the coordinate system center is transferred is: 0.9416255; taking the x and y axis coordinate values ​​of the patient's left shoulder as an example, the average value of the signal correlation of the two curves after the coordinate system center is transferred is: 0.94469845; taking the x and y axis coordinate values ​​of the patient's left shoulder as an example, the average value of the signal correlation of the two curves after the coordinate system center is transferred and then standardized is: 0.9651 (standardization processing method of the present application) and 0.9686418 (standardization processing method of the prior art).

[0122] The above experimental results show that the joint point coordinate standardization method proposed in this application continuously preserves the data pattern and achieves the purpose of data standardization. The gait video processed by the standardization method of this application can well retain the patient's motion information in each frame of the gait image, so that more accurate disease prediction results can be obtained based on the processed gait video, thereby improving the model performance.

[0123] To summarize, compared with the existing method for standardizing each frame of gait images in gait videos, which uses fixed standardization parameters to standardize each frame of gait images in gait videos collected by cameras at different positions, the present application configures the standardization module in the disease prediction model to standardize each frame of gait images in gait videos collected by cameras at different positions based on the calculated distance value and preset distance value between each joint point of the patient and the origin of the new two-dimensional coordinate system in each frame of gait images of the target gait video. This method of dynamically standardizing the gait video by calculating the distance value between each joint point of the patient and the origin of the new two-dimensional coordinate system can well retain the patient's motion information in each frame of the gait image, so that more accurate disease prediction results can be obtained based on the processed gait video, thereby improving the model performance.

[0124] In addition, according to the disease prediction model of the above embodiment, the present application provides an application method of the disease prediction model, such as Figure 6 As shown, the method includes: T1, obtaining a gait video of a patient; T2, obtaining a disease prediction result of the patient based on the gait video of the patient using a disease prediction model as in the above embodiment.

[0125] For example, scoliosis is the most common spinal deformity among children and adolescents, and is the third largest "killer" of children's health after obesity and myopia. Its incidence and severity rate are significantly higher than the international average. Since the skeletal system of adolescents is in the growth and development stage, there is a risk of further aggravation of scoliosis along with the overall development of the spine. If it is not discovered and intervened early, as scoliosis continues to worsen, it will compress important organs such as the heart and lungs, affecting their normal development, causing abnormal functions of important organs such as the heart and lungs of adolescents, and making it impossible to complete daily activities normally. In mild cases, it will cause back pain, which will significantly reduce the patient's quality of life; in severe cases, it will shorten their lifespan. At the same time, patients often bear a profound psychological burden due to the obvious abnormalities in their appearance. Therefore, implementing "early detection" is the most critical link in the prevention and control of scoliosis.

[0126] Currently, the commonly used screening methods at home and abroad include visual inspection, Adams flexion test (physical examination), corrugated image examination, scoliosis measuring ruler, portable electronic scoliosis screening tool and X-ray measurement of Cobb angle. Their advantages and disadvantages can be seen in Table 1.

[0127] Table 1

[0128]

[0129] From the analysis of Table 1, we can see that the current screening methods require students to take off their shirts and rely on professional doctor examinations and manual measurements. In addition to being inefficient, there is also a certain degree of subjective error, which can easily lead to a high false positive rate.

[0130] The current mainstream scoliosis screening and assessment methods have the following difficulties:

[0131] 1) The early symptoms of scoliosis are not obvious and are easily ignored. Many patients do not have obvious pain and functional impairment in the early stage, and the scoliosis angle is small and difficult to detect.

[0132] 2) Limitations of screening methods. Currently, physical examinations such as the Adam's flexion test are mainly used, which have limited sensitivity and specificity. Although X-rays are the gold standard, they are not suitable for routine screening due to radiation exposure.

[0133] 3) The selection of screening subjects is not uniform. Some are mainly school-age children, while others include adolescents. The incidence and characteristics of scoliosis in different age groups are different, which makes screening difficult.

[0134] 4) Low screening coverage and poor follow-up compliance. Due to lack of awareness, some students and parents do not pay enough attention to it, and there are difficulties in school organization, resulting in low screening coverage and high loss of follow-up rate for positive cases.

[0135] 5) Lack of unified diagnosis and treatment standards. The medical level varies from place to place, and there is a lack of standards for the diagnosis, treatment and follow-up of scoliosis, which will affect the triage and patient management after screening.

[0136] 6) The cost-effectiveness of screening needs to be weighed. Large-scale screening requires manpower and material resources, and requires a balance between input and output.

[0137] The disease prediction model of the present application uses human posture estimation technology (Pose estimation) to obtain the coordinates of the joint points during the human gait process to predict whether the patient suffers from scoliosis, without the need for the patient to undress or stick sensors on the human body surface. This technology is used to capture the dynamic biological information of the human body during movement within a time period, rather than the static information at a certain moment. Combining optical systems and computer vision technology, the patient's motion characteristics are identified and tracked based on RGB 2D images (i.e., videos taken by ordinary mobile phone cameras), and the characteristic data of the target dynamic movement are extracted. The disease prediction model of the present application can be used to more conveniently and accurately predict whether the patient has scoliosis, and the doctor can make a more accurate diagnosis based on the prediction results. It should be noted that the use of the disease prediction model constructed in the present application to predict whether the patient's spine is scoliotic is only an embodiment of the present application. The disease prediction model of the present application can also be used for other results that can predict whether the patient suffers from the corresponding disease based on the patient's joint point information. The application scenario of the disease prediction model here is not specifically limited.

[0138] Based on the inventive concept of the above embodiments, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described in the above embodiments when executing the computer program. Figure 7 Provide detailed explanation.

[0139] like Figure 7 As shown, it shows the electronic device of the present application, which may specifically include a processor 110 and a memory 120. The memory 120 is coupled to the processor 110.

[0140] The processor 110 is used to control the operation of the electronic device. The processor 110 may also be referred to as a CPU (Central Processing Unit). The processor 110 may be an integrated circuit chip having the ability to process signals. The processor 110 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. A general-purpose processor may be a microprocessor or the processor 110 may also be any conventional processor, etc.

[0141] The memory 120 is used to store computer programs, and may be a RAM, a ROM, or other types of storage terminals. Specifically, the memory 120 may include one or more computer-readable storage media, which may be non-transitory or transient. The memory 120 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage terminals, flash memory storage terminals. In some embodiments, the non-transitory computer-readable storage medium in the memory 120 is used to store at least one program code.

[0142] The processor 110 is used to execute the computer program stored in the memory 120 to implement the methods described in the various method embodiments of the present application.

[0143] In some embodiments, the electronic device may further include: a peripheral terminal interface 130 and at least one peripheral terminal. The processor 110, the memory 120 and the peripheral terminal interface 130 may be connected via a bus or a signal line. Each peripheral terminal may be connected to the peripheral terminal interface 130 via a bus, a signal line or a circuit board. Specifically, the peripheral terminal includes: at least one of a radio frequency circuit 140, a display screen 150, an audio circuit 160 and a power supply 170.

[0144] The peripheral terminal interface 130 may be used to connect at least one peripheral terminal related to I / O (Input / output) to the processor 110 and the memory 120. In some embodiments, the processor 110, the memory 120, and the peripheral terminal interface 130 are integrated on the same chip or circuit board; in some other implementations, any one or two of the processor 110, the memory 120, and the peripheral terminal interface 130 may be implemented on a separate chip or circuit board, which is not limited in this embodiment.

[0145] The radio frequency circuit 140 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 140 communicates with the communication network and other IoT devices through electromagnetic signals, and the radio frequency circuit 140 is the communication circuit of the electronic device. The radio frequency circuit 140 converts the electrical signal into an electromagnetic signal for transmission, or converts the received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 140 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, an operator identity module card, and the like. The radio frequency circuit 140 can communicate with other terminals through at least one wireless communication protocol. The wireless communication protocol includes but is not limited to: the World Wide Web, a metropolitan area network, an intranet, various generations of mobile communication networks (2G, 3G, 4G and 5G), a wireless local area network and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 140 may also include circuits related to NFC (Near Field Communication), which is not limited in this application.

[0146] The display screen 150 is used to display the UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen 150 is a touch display screen, the display screen 150 also has the ability to collect touch signals on the surface or above the surface of the display screen 150. The touch signal can be input to the processor 110 as a control signal for processing. At this time, the display screen 150 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen 150 can be one, set on the front panel of the electronic device; in other embodiments, the display screen 150 can be at least two, respectively set on different surfaces of the electronic device or in a folding design; in other embodiments, the display screen 150 can be a flexible display screen, set on a curved surface or a folding surface of the electronic device. Even, the display screen 150 can also be set to a non-rectangular irregular shape, that is, a special-shaped screen. The display screen 150 can be made of materials such as LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode, organic light-emitting diode).

[0147] The audio circuit 160 may include a microphone and a speaker. The microphone is used to collect sound waves from the operator and the environment, and convert the sound waves into electrical signals and input them into the processor 110 for processing, or input them into the radio frequency circuit 140 to achieve voice communication. For the purpose of stereo acquisition or noise reduction, there can be multiple microphones, which are respectively arranged in different parts of the electronic device. The microphone can also be an array microphone or an omnidirectional acquisition microphone. The speaker is used to convert the electrical signal from the processor 110 or the radio frequency circuit 140 into sound waves. The speaker can be a traditional film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for purposes such as ranging. In some embodiments, the audio circuit 160 may also include a headphone jack.

[0148] The power supply 170 is used to power various components in the electronic device. The power supply 170 can be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 170 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged through a wired line, and a wireless rechargeable battery is a battery that is charged through a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0149] For a detailed description of the functions and execution processes of each functional module or component in the electronic device embodiment of the present application, reference can be made to the description in the above-mentioned method embodiments of the present application, which will not be repeated here.

[0150] In the several embodiments provided in the present application, it should be understood that the disclosed electronic devices and methods can be implemented in other ways. For example, the various embodiments of the electronic devices described above are only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0151] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0152] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0153] Based on the inventive concept of the above embodiment, the present application also provides a computer-readable storage medium storing a computer program, which can be executed by a processor to implement the above projection area calculation method. Figure 8 The execution process of the above-mentioned embodiment in a computer-readable storage medium is described.

[0154] like Figure 8As shown, it shows the computer-readable storage medium of the present application. If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium 200. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions / computer programs to enable an IoT device (which can be a personal computer, server, or network terminal, etc.) or a processor (processor) to perform all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: various media such as a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk, and electronic terminals such as computers, mobile phones, laptops, tablet computers, cameras, etc. having the above-mentioned storage medium.

[0155] The description of the execution process of the program data in the computer-readable storage medium can refer to the description in the above-mentioned method embodiments of the present application, which will not be repeated here.

[0156] The above descriptions are merely embodiments of the present application and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

[0157] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

Claims

1. A method for constructing a disease prediction model, wherein the disease prediction model is used to predict a disease based on a patient's gait video, characterized in that: The method comprises: Acquire a gait video set, wherein the gait video set includes a plurality of gait videos, each of the gait videos includes a plurality of frames of gait images, each of the gait videos corresponds to a patient, and the body structure of the patient includes a plurality of joint points, and the plurality of joint points include at least a left hip, a right hip, a left shoulder, and a right shoulder joint point; Constructing an initial disease prediction model, wherein the initial disease prediction model includes: A global coordinate extraction module, used to take each of the gait videos as a target gait video, and obtain the initial horizontal and vertical coordinate features of each joint point of the patient in each frame of the gait image of the target gait video based on a preset initial two-dimensional coordinate system; A human body structure coordinate construction module, for constructing a new two-dimensional coordinate system based on the origin coordinates of the preset initial two-dimensional coordinate system and the initial horizontal and vertical coordinate features of the left and right hip joints of the patient in each frame of the gait image of the target gait video, and obtaining new horizontal and vertical coordinate features of each joint of the patient in each frame of the gait image of the target gait video based on the new two-dimensional coordinate system; A standardization module, for obtaining target horizontal and vertical coordinate features of each joint point of the patient in each frame of the gait image of the target gait video based on new horizontal and vertical coordinate features of the left hip, right hip, left shoulder and right shoulder joint points of the patient in each frame of the gait image of the target gait video, a calculated distance value between each joint point of the patient in each frame of the gait image of the target gait video and the origin of the new two-dimensional coordinate system, and a preset distance value; A disease prediction module, used to obtain a disease prediction result corresponding to the patient in the target gait video based on the target horizontal and vertical coordinates of each joint point of the patient in each frame of the gait image of the target gait video; The human body structure coordinate construction module is respectively connected to the global coordinate extraction module and the standardization module, and the standardization module is also connected to the disease prediction module; The initial disease prediction model is trained multiple times iteratively using the gait video set until convergence.

2. The method for constructing a disease prediction model according to claim 1, characterized in that: The standardization module is configured to obtain the target horizontal and vertical coordinate features of each joint point of the patient in each frame of the gait image of the target gait video in the following manner: Based on the new horizontal and vertical coordinate features of the left and right hip joints of the patient in each frame of the gait image of the target gait video, new horizontal and vertical coordinate features of the center point between the left and right hip joints of the patient in each frame of the gait image of the target gait video are obtained, and based on the new horizontal and vertical coordinate features of the left and right shoulder joints of the patient in each frame of the gait image of the target gait video, new horizontal and vertical coordinate features of the center point between the left and right shoulder joints of the patient in each frame of the gait image of the target gait video are obtained; Based on the new horizontal and vertical coordinate features of the center point between the left hip and right hip joints of the patient in each frame of the gait image of the target gait video and the new horizontal and vertical coordinate features of the center point between the left shoulder and right shoulder joints of the patient in each frame of the gait image of the target gait video, calculate the distance value between the center point between the left hip and right hip joints of the patient and the center point between the left shoulder and right shoulder joints in each frame of the gait image of the target gait video; Based on the new horizontal and vertical coordinate features of each joint point of the patient in each frame of the gait image of the target gait video, calculate the distance value between each joint point of the patient in each frame of the gait image of the target gait video and the origin of the new two-dimensional coordinate system; Sorting the distance value between each joint point of the patient and the origin of the new two-dimensional coordinate system and the preset distance value in each frame of the gait image of the target gait video to screen out the largest distance value, and obtaining the target distance value based on the maximum distance value and the distance value between the center point between the left hip and right hip joint points of the patient and the center point between the left shoulder and right shoulder joint points in each frame of the gait image of the target gait video; Based on the target distance value and the new horizontal and vertical coordinate features of each joint point of the patient in each frame of the gait image of the target gait video, the target horizontal and vertical coordinate features of each joint point of the patient in each frame of the gait image of the target gait video are obtained.

3. The method for constructing a disease prediction model according to claim 2, characterized in that: The standardization module is configured to obtain the target distance value in the following manner: pose_size=tosor_size×torso_size_multiplier Among them, pose_size represents the target distance value, tosor_size represents the distance value between the center point between the left hip and right hip joints of the patient and the center point between the left shoulder and right shoulder joints in each frame of the gait image of the target gait video, and torso_size_multiplier represents the maximum distance value.

4. The method for constructing a disease prediction model according to claim 1, characterized in that: The standardization module is further configured to obtain target horizontal and vertical coordinate features of each joint point of the patient in each frame of the gait image of the target gait video in the following manner: The new horizontal and vertical coordinate features of each joint point of the patient in each frame of the gait image of the target gait video are normalized, and based on the normalized new horizontal and vertical coordinate features of the left hip, right hip, left shoulder and right shoulder joint points of the patient in each frame of the gait image of the target gait video, the calculated distance value between each joint point of the patient in each frame of the gait image of the target gait video and the origin of the new two-dimensional coordinate system, and the preset distance value, the target horizontal and vertical coordinate features of each joint point of the patient in each frame of the gait image of the target gait video are obtained.

5. The method for constructing a disease prediction model according to claim 4, characterized in that: The standardization module is further configured to normalize the new horizontal and vertical coordinate features of each joint point of the patient in each frame of the gait image of the target gait video in the following manner: Based on the new horizontal and vertical coordinate features of the left and right hip joints of the patient in each frame of the gait image of the target gait video, new horizontal and vertical coordinate features of the center point between the left and right hip joints of the patient in each frame of the gait image of the target gait video are obtained, and based on the new horizontal and vertical coordinate features of the left and right shoulder joints of the patient in each frame of the gait image of the target gait video, new horizontal and vertical coordinate features of the center point between the left and right shoulder joints of the patient in each frame of the gait image of the target gait video are obtained; Based on the new horizontal and vertical coordinate features of the center point between the left hip and right hip joints of the patient in each frame of the gait image of the target gait video and the new horizontal and vertical coordinate features of the center point between the left shoulder and right shoulder joints of the patient in each frame of the gait image of the target gait video, calculate the vector from the center point between the left shoulder and right shoulder joints of the patient in each frame of the gait image of the target gait video to the center point between the left hip and right hip joints; A vertical vector is arbitrarily selected, and the angle formed by the vertical vector and the vector from the center point between the left shoulder and right shoulder joints of the patient to the center point between the left hip and right hip joints in each frame of the gait image of the target gait video is calculated, and based on the angle, the new horizontal and vertical coordinate features of each joint point of the patient in each frame of the gait image of the target gait video are normalized.

6. The method for constructing a disease prediction model according to claim 5, characterized in that: The standardization module is further configured to normalize the new horizontal and vertical coordinate features of each joint point of the patient in each frame of the gait image of the target gait video in the following manner: Wherein, θ is the angle formed by the vertical vector and the vector from the center point between the left shoulder and right shoulder joint points of the patient to the center point between the left hip and right hip joint points in each frame of the gait image of the target gait video, represents the new horizontal coordinate feature of the i-th joint point of the patient in each frame of the gait image of the target gait video, represents the new ordinate feature of the i-th joint point of the patient in each frame of the gait image of the target gait video, represents the new horizontal coordinate feature of the i-th joint point of the patient in each frame of the gait image of the target gait video after normalization, Represents the new ordinate feature of the i-th joint point of the patient in each frame of the gait image of the target gait video after normalization.

7. A disease prediction model based on the method for constructing a disease prediction model according to any one of claims 1 to 6, wherein the disease prediction model is used to predict a disease based on a patient's gait video, wherein: The gait video includes multiple frames of gait images, the gait video corresponds to a patient, the body structure of the patient includes multiple joints, the multiple joints include at least left hip, right hip, left shoulder and right shoulder joints, and the model includes: A global coordinate extraction module, used for obtaining the initial horizontal and vertical coordinate features of each joint point of the patient in each frame of the gait image of the gait video based on a preset initial two-dimensional coordinate system; A human body structure coordinate construction module, for constructing a new two-dimensional coordinate system based on the origin coordinates of the preset initial two-dimensional coordinate system and the initial horizontal and vertical coordinate features of the left and right hip joints of the patient in each frame of the gait image of the gait video, and obtaining new horizontal and vertical coordinate features of each joint of the patient in each frame of the gait video based on the new two-dimensional coordinate system; A standardization module, for obtaining target horizontal and vertical coordinate features of each joint point of the patient in each frame of the gait image of the gait video based on new horizontal and vertical coordinate features of the left hip, right hip, left shoulder and right shoulder joint points of the patient in each frame of the gait image of the gait video, a calculated distance value between each joint point of the patient in each frame of the gait image of the gait video and the origin of the new two-dimensional coordinate system, and a preset distance value; A disease prediction module, used for obtaining a disease prediction result corresponding to the patient in the gait video based on the target horizontal and vertical coordinates of each joint point of the patient in each frame of the gait image of the gait video; The human body structure coordinate construction module is respectively connected in communication with the global coordinate extraction module and the standardization module, and the standardization module is also connected in communication with the disease prediction module.

8. A method for applying a disease prediction model, characterized in that: The method comprises: Obtain a video of the patient's gait; The disease prediction model as claimed in claim 7 is used to obtain the disease prediction result of the patient based on the gait video of the patient.

9. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method according to any one of claims 1 to 6 and 8 when executing the computer program.

10. A computer-readable storage medium having program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the method described in any one of claims 1-6 and 8 is implemented.

Images