Data processing method and device

Abstract

The invention provides a data processing method and device. The method comprises the following steps: acquiring video data and physiological data of a user; and determining the health condition of the user based on the acquired video data of the user and the physiological data of the user.

Description

Technical Field

[0001] The present application relates to the technical field of storage data processing, and more specifically, to a data access method and device. Background Art

[0002] Nowadays, more and more people are living in a fast-paced life, which leads to a trend of more and more diseases becoming younger. There are many smart devices on the market that can detect body information through a variety of sensors, indicating that people are paying more and more attention to their own health issues. However, the models used by the devices on the market are mostly general models, which often makes the standards of human health monitoring results absolute.

[0003] In addition, wearable devices mainly focus on monitoring internal body data such as heart rate, blood oxygen level and body temperature, and pay limited attention to external information such as voice, facial expressions and body movements. Relying solely on internal physiological data may introduce bias in assessing personal health status and hinder the effective analysis of their overall health information.

[0004] Therefore, there is an urgent need for a data processing method and device that can perform personalized analysis on a user's health status to more accurately determine the user's physical health status. Summary of the invention

[0005] The object of the present invention is to provide a data processing method and device to at least solve the above-mentioned problems in the related art, or not to solve any of the above-mentioned problems.

[0006] According to one aspect of an embodiment of the present disclosure, there is provided a data processing method, including: acquiring video data of a user and physiological data of the user; and determining a health status of the user based on the acquired video data of the user and the physiological data of the user.

[0007] According to the embodiments of the present disclosure, the user's physical health condition can be determined more accurately based on the user's video data and the user's physiological data.

[0008] Optionally, the step of determining the health status of the user includes: acquiring limb data of the user based on the video data; and determining the health status of the user based on the acquired limb data of the user and the physiological data of the user.

[0009] Optionally, the step of obtaining the user's limb data based on the video data includes: determining the backbone node coordinates of the user in the video; and processing the node coordinates in the backbone based on preset processing rules, and obtaining the user's limb data based on the processed coordinates.

[0010] Optionally, the preset rule is wavelet filtering.

[0011] Optionally, the user's limb data includes at least one of the following items: an angle between body parts, a swing frequency, amplitude, speed or acceleration of a backbone node.

[0012] Optionally, the step of determining the user's health status based on the acquired user's limb data and the user's physiological data includes: predicting the user's limb data and the user's physiological data based on a first model; determining whether to correct the user's actual limb data and the user's physiological data based on a comparison between the predicted limb data and the user's physiological data and the acquired user's limb data and the user's physiological data; when the difference between the predicted limb data and the user's physiological data and the acquired user's limb data and the user's physiological data is greater than a threshold, correcting the acquired user's limb data and the user's physiological data; and determining the user's physical health status based on the corrected acquired user's limb data and / or user's physiological data.

[0013] Optionally, the step of determining the user's health condition based on the acquired limb data and physiological data of the user includes: determining the user's health condition by inputting the acquired limb data and physiological data of the user into a second model corresponding to the user.

[0014] Optionally, the method further comprises: updating the prediction model based on the corrected limb data and the user's physiological data.

[0015] Optionally, the step of correcting the acquired limb data of the user and the physiological data of the user includes: correcting the actual limb data of the user and the physiological data of the user based on cubic spline interpolation.

[0016] Optionally, the physiological data includes the user's heart rate and blood oxygen.

[0017] According to another aspect of an embodiment of the present disclosure, a data processing device is provided, including: an acquisition unit configured to acquire video data of a user and physiological data of the user; and a determination unit configured to determine the health status of the user based on the acquired video data of the user and the physiological data of the user.

[0018] Optionally, the determination unit is configured to: acquire limb data of the user based on the video data; and determine the health status of the user based on the acquired limb data of the user and the physiological data of the user.

[0019] Optionally, the determination unit is configured to: determine the backbone node coordinates of the user in the video; and process the node coordinates in the backbone based on preset processing rules, and obtain the limb data of the user based on the processed coordinates.

[0020] Optionally, the preset rule is wavelet filtering.

[0021] Optionally, the user's limb data includes at least one of the following items: an angle between body parts, a swing frequency, amplitude, speed or acceleration of a backbone node.

[0022] Optionally, the determination unit is configured to: predict the user's limb data and the user's physiological data based on a first model; determine whether to correct the user's actual limb data and the user's physiological data based on a comparison between the predicted limb data and the user's physiological data and the acquired limb data and the user's physiological data; when the difference between the predicted limb data and the user's physiological data and the acquired limb data and the user's physiological data is greater than a threshold, correct the acquired limb data and the user's physiological data; and determine the user's physical health status based on the corrected acquired limb data and / or the user's physiological data.

[0023] Optionally, the determination unit is configured to determine the health status of the user by inputting the acquired limb data of the user and the physiological data of the user into a second model corresponding to the user.

[0024] Optionally, the data processing device further includes: an updating unit configured to update the prediction model based on the corrected limb data and the user's physiological data.

[0025] Optionally, the determination unit is configured to correct the user's actual limb data and the user's physiological data based on cubic spline interpolation.

[0026] Optionally, the physiological data includes the user's heart rate and blood oxygen.

[0027] According to another aspect of an embodiment of the present disclosure, a computer-readable storage medium storing a computer program is provided, wherein when the computer program is executed by a processor, the processor is enabled to implement the data processing method as described above. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The above and other objects and features of the present invention will become more apparent from the following description in conjunction with the accompanying drawings which exemplarily illustrate embodiments of the present invention, in which:

[0029] Figure 1 Examples of various sports scenarios for users are shown;

[0030] Figure 2 A flow chart of a data processing method according to an embodiment of the present disclosure is shown;

[0031] Figure 3 A schematic diagram showing a process of acquiring video data and physiological data of a user according to an embodiment of the present disclosure;

[0032] Figure 4A diagram illustrating an example of a denoising process according to an example of the present disclosure;

[0033] Figure 5 A schematic diagram showing a process of determining limb data according to an embodiment of the present disclosure;

[0034] Figure 6 A schematic diagram showing a process of correcting limb data and physiological data according to an embodiment of the present disclosure;

[0035] Figure 7 A diagram showing an example based on determining a user's physical health condition according to an embodiment of the present disclosure;

[0036] Figure 8 A diagram showing an example of determining a user's health status according to an embodiment of the present disclosure; and

[0037] Fig. 9 A block diagram showing the structure of a data processing device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0038] Hereinafter, various embodiments of the present disclosure are described with reference to the accompanying drawings, wherein the same reference numerals are used to represent the same or similar elements, features and structures. However, it is not intended that the present disclosure be limited to specific embodiments by the various embodiments described herein, and it is intended that: the present disclosure covers all modifications, equivalents and / or substitutes of the present disclosure, as long as they are within the scope of the attached claims and their equivalents. The terms and words used in the following specification and claims are not limited to their dictionary meanings, but are only used to enable a clear and consistent understanding of the present disclosure. Therefore, it should be apparent to those skilled in the art that the following description of various embodiments of the present disclosure is provided for illustrative purposes only, and not for the purpose of limiting the present disclosure defined by the attached claims and their equivalents.

[0039] It should be understood that the singular includes the plural unless the context clearly indicates otherwise.The terms "include", "comprising" and "having" used herein indicate the presence of disclosed functions, operations or elements, but do not exclude other functions, operations or elements.

[0040] For example, the expression "A or B" or "at least one of A and / or B" may indicate A and B, A or B. For example, the expression "A or B" or "at least one of A and / or B" may indicate (1) A, (2) B, or (3) both A and B.

[0041] In various embodiments of the present disclosure, it is intended that when a component (e.g., a first component) is referred to as being "coupled" or "connected" to another component (e.g., a second component) or being "coupled" or "connected" to another component (e.g., the second component), the component may be directly connected to the other component or may be connected through another component (e.g., a third component). In contrast, when a component (e.g., a first component) is referred to as being "directly coupled" or "directly connected" to another component (e.g., the second component) or being directly coupled to or directly connected to another component (e.g., the second component), there is no other component (e.g., the third component) between the component and the other component.

[0042] The expression "configured to" used in describing various embodiments of the present disclosure may be used interchangeably with expressions such as "suitable for", "having the ability to", "designed to", "suitable for", "manufactured to", and "capable of", for example, depending on the circumstances. The term "configured to" may not necessarily indicate that it is "specially designed to" in terms of hardware. On the contrary, the expression "a device configured to..." in some cases may indicate that the device and another device or part "can..." For example, the expression "a processor configured to perform A, B, and C" may indicate a dedicated processor (e.g., an embedded processor) for performing the corresponding operations or a general-purpose processor (e.g., a central processing unit CPU or an application processor (AP)) for performing the corresponding operations by executing at least one software program stored in a memory device.

[0043] The terms used herein are to describe certain embodiments of the present disclosure, but are not intended to limit the scope of other embodiments. Unless otherwise noted herein, all terms used herein (including technical or scientific terms) may have the same meaning as those generally understood by those skilled in the art. Typically, the terms defined in the dictionary should be considered to have the same meaning as the contextual meaning in the relevant field, and, unless clearly defined herein, should not be understood differently or understood to have an overly formal meaning. In any case, the terms defined in the present disclosure are not intended to be interpreted as excluding embodiments of the present disclosure.

[0044] Figure 1 Examples of various sports scenarios of users are shown.

[0045] For example, when a user is doing exercises such as skipping rope, the user's physiological indicators (such as heart rate, blood oxygen, etc.) can be detected through a wearable device (such as a smart bracelet). If it is determined that, for example, the heart rate or blood oxygen is abnormal, the user can be reminded through the smart bracelet or other devices wirelessly connected to the smart bracelet.

[0046] Obviously, the judgment of user health abnormality is made only based on the user's physiological data, and does not take into account the user's limb information (for example, the degree of bending of the user's legs or arms during skipping, the user's sitting posture when sitting for a long time, etc.). Therefore, although the user's limb information sometimes reflects the potential health risks that the user may face, it is impossible to warn the user based on this information.

[0047] The data processing method according to the embodiments of the present disclosure can at least solve the above problems.

[0048] Figure 2 A flow chart of a data processing method according to an embodiment of the present disclosure is shown.

[0049] Reference Figure 2 , in step S201, the video data and physiological data of the user are obtained.

[0050] In step S202, the health status of the user is determined based on the acquired video data of the user and the physiological data of the user.

[0051] As understood by those skilled in the art, the video data of a user may indicate a video or picture taken of the user.

[0052] As an example, the acquired user video data may be aligned or matched with the acquired user physiological data based on time, and the user's physical health condition may be determined based on the aligned or matched user video data and user physiological data.

[0053] As an example, video data of a user may be obtained by capturing a video of the user with an optical sensor (eg, a camera) located near the user.

[0054] As an example, an optical sensor may be used to identify, track, and capture a video or image of a user, and the captured video or image may be stored.

[0055] As an example, the physiological data of a user may be obtained through a wearable device worn on the user.

[0056] Figure 3 A schematic diagram showing a process of acquiring video data and physiological data of a user according to an embodiment of the present disclosure is shown.

[0057] Reference Figure 3 , the user's video data can be obtained by a camera located near the user's location, and the camera can send the obtained video data to the data processing device through the intelligent gateway. The user's physiological data can be obtained by a wearable device worn by the user, and the obtained physiological data can be sent to the data processing device through the intelligent gateway to determine the user's health status based on the video data and the physiological data.

[0058] As an example, when a person enters a specific space (e.g., indoors), the wearable device worn by the user can automatically access the local area network and use heart rate broadcasting / Bluetooth / wireless network, etc. to send the recently detected human body data stored on the wearable device to the smart gateway.

[0059] As an example, the data processing device may be a mobile phone, a desktop computer, a notebook computer, a tablet computer, or a server.

[0060] As an example, the step of determining the health status of the user may include: acquiring limb data of the user based on the video data; and determining the health status of the user based on the limb data of the user and the physiological data of the user.

[0061] As an example, the step of obtaining the user's limb data based on the video data may include: determining the coordinates of the user's backbone nodes (or human body joints) in the video; and processing the backbone node coordinates based on preset processing rules, and obtaining the user's limb data based on the processed coordinates.

[0062] For ease of description, terms related to nodes such as trunk nodes, backbone nodes, body nodes or human joint points described in this article can be used interchangeably.

[0063] As an example, the coordinates of the user's backbone nodes can be obtained through the YOLO technology. The YOLO technology can detect and track backbone nodes or human joints. The number and position of backbone nodes or human joints are not fixed, and the user can adjust them as needed. For example, some backbone nodes or human joints can be selected and the selected joints can be detected and tracked.

[0064] As an example, the video of the user can be decomposed into pictures, and then the YOLO technology is used to process the pictures, mark the backbone nodes of the characters in the pictures, and determine the coordinates of the marked backbone nodes.

[0065] As an example, the coordinate system used to determine the coordinates of the backbone nodes may be a world coordinate system, a camera coordinate system, or an image coordinate system.

[0066] As an example, the preset rule may be wavelet filtering.

[0067] When the user is moving (for example, the user is moving periodically), the backbone nodes will fluctuate, which may cause errors in the coordinates of the acquired backbone nodes. For example, due to the low camera frame rate, the acquired coordinates of the user's backbone nodes are inconsistent with the actual coordinates of the backbone nodes.

[0068] Therefore, the noise of the coordinates of the backbone nodes can be eliminated by performing preset processing on the coordinates of the backbone nodes.

[0069] Figure 4A diagram illustrating an example of a denoising process according to examples of the present disclosure.

[0070] Reference Figure 4 , which shows a schematic diagram of the wavelet denoising process for the coordinates of the left hand, with reference to the wavelet filtering formula in the figure, where a: scale factor, indicating the level or resolution of the wavelet. b: translation parameter, indicating the position of the wavelet function on the time axis. X(t) is the signal, t is the time, ψ(t) is the wavelet function, *: complex conjugate sign, λ is the preset threshold (threshold), for example, the threshold processing method can be a hard threshold (hard_threshold) processing. Through denoising, the coordinates of the left hand (including x-coordinates and y-coordinates) at each moment acquired based on the picture can be made more consistent with the actual coordinates of the left hand.

[0071] As an example, the user's limb data includes: the angle between body parts, the swing (or movement) frequency, amplitude, speed or acceleration of the trunk nodes, etc.

[0072] As examples, the user's body parts may include upper arms, lower arms, thighs, calves, head, neck, abdomen, chest, etc.

[0073] As an example, body parts of a user may be defined using limb nodes.

[0074] For example, an arm may be defined by an elbow node and a hand node, and an upper arm may be defined by a shoulder node (shoulder joint point) and an elbow node.

[0075] For example, the angles between body parts may include the angle between the upper arm and the lower arm, the angle between the thigh and the calf, the angle between the abdomen and the leg, the angle between the head (or neck) and the chest, etc.

[0076] Those skilled in the art should understand that the above-mentioned angle, swing frequency, amplitude, speed or acceleration etc. can be calculated based on the acquired coordinates of the trunk nodes.

[0077] As an example, the user's motion type may also be identified based on the YOLO technology.

[0078] As an example, the YOLO recognition technology can be used to identify the nodes of the human body in the picture. After the human body nodes are identified, the left and bottom frames of the picture are used as direct coordinate systems to describe the coordinates of the nodes of the limbs, and this is used as basic data to obtain the angles between body parts, the swing frequency, amplitude, speed and / or acceleration of the limb nodes. Then, the user's movement category can be determined based on the above calculation results.

[0079] For example, if the angle between the upper arm and the lower arm is small and the swing frequency of the hand node and the elbow node is high, it can be determined that the user is running.

[0080] Those skilled in the art should understand that the method for determining the movement type may be preset or determined based on the coordinate data of the trunk nodes or the above-mentioned calculation results using a trained model.

[0081] Figure 5 A schematic diagram showing a process of determining limb data according to an embodiment of the present disclosure is shown.

[0082] Reference Figure 5 , (X, Y), (x, y), (x1, y1) and (x2, y2) can be determined by YOLO technology, which respectively represent the coordinates of the left hand at the first moment, the coordinates of the left hand at the second moment, the coordinates of the left elbow node at the second moment, and the coordinates of the left shoulder node at the second moment. Based on the above coordinates, the swing amplitude (distance), speed, acceleration of the left hand node from the first moment to the second moment, and the angle θ between the left upper arm and the left forearm at the second moment can be calculated. Then, the user's movement type can be determined based on the determined limb data.

[0083] As an example, the user's motion type may be determined using a trained model based on acquired coordinate data of body nodes.

[0084] As an example, the user's motion type can be determined based on the acquired limb data. For example, the user's motion type can be determined based on the acquired limb data using a trained model.

[0085] Those skilled in the art should understand that subsequent analysis of the data may be combined with the type of exercise to select a corresponding data model for the user, thereby achieving accuracy in data analysis.

[0086] For example, when a user is running on a treadmill, the user's limb data may include at least one of the following items: the angle between the user's upper arm and lower arm, the swing frequency, speed, acceleration of the hand nodes, the angle between the thigh and calf, and the swing frequency, amplitude, speed and acceleration of the foot nodes.

[0087] As an example, when a user sits for a long time, the user's limb data may include the angle between the user's abdomen and legs, the angle of lowering the head (for example, the angle between the head and chest), etc.

[0088] As an example, the determination step may include: predicting the user's limb data and the user's physiological data based on a first model; determining whether to correct the obtained user's limb data and the user's physiological data based on a comparison between the predicted limb data and the user's physiological data and the acquired user's limb data and the user's physiological data; when the difference between the predicted limb data and the user's physiological data and the acquired user's limb data and the user's physiological data is greater than a threshold, correcting the acquired user's limb data and the user's physiological data; and determining the user's physical health condition based on the corrected acquired user's limb data and the user's physiological data.

[0089] Since the wearable device used by the user to measure the user's physiological indicators may fall off and / or the camera used to shoot the user's video may be blocked, the acquired limb data and / or physiological data of the user may not truly reflect the user's actual limb posture and physiological data. The present disclosure corrects the acquired limb data and physiological data of the user based on the predicted data, and the corrected limb data and physiological data can better reflect the user's actual limb posture and physiological data.

[0090] As an example, the first model may be obtained from a server.

[0091] As an example, a user profile corresponding to the user may be determined from a server according to the user's identity information, and a user personalized model, ie, a first model, in the user profile may be called.

[0092] As an example, the first model may be trained by the user's historical limb data and / or the user's physiological data and uploaded to the server, and may be downloaded for use.

[0093] As an example, the first model may be further updated based on the revised acquired limb data of the user and / or the physiological data of the user.

[0094] As an example, the step of correcting the acquired limb data of the user and / or the physiological data of the user includes: correcting the acquired limb data of the user and / or the physiological data of the user based on cubic spline interpolation.

[0095] Figure 6 A schematic diagram showing a process of correcting limb data and physiological data according to an embodiment of the present disclosure is shown.

[0096] Reference Figure 6First, the server downloads a first model (which may be called a data anomaly analysis model) based on the user's identity information (for example, the user's device information). The first model predicts the user's current limb data and physiological data based on historical limb data and physiological data. The predicted user's current physiological data and limb data are compared with the user's current physiological data and limb data obtained through optical sensors and wearable devices. The data points with large differences are corrected using a cubic spline difference algorithm to obtain corrected physiological data and limb data.

[0097] As an example, the first model may be further trained and updated based on the corrected physiological data and limb data.

[0098] According to an embodiment of the present disclosure, based on the existing first model, user personalized data is used to fine-tune parameters of the user model to strengthen the correlation between the first model and the user behavior.

[0099] As an example, the step of determining the user's physical health condition may include: determining the user's physical health condition using a second model corresponding to the user based on acquired video data of the user and physiological data of the user.

[0100] As an example, the health status of the user may be determined by inputting the acquired limb data of the user and the physiological data of the user into a second model corresponding to the user.

[0101] As an example, the second model may be obtained from a server.

[0102] As an example, the second model is obtained through training based on the user's historical limb data and the user's physiological data and physical health condition.

[0103] As an example, the second model may be uploaded to the server in advance, and when the second model is used, the second model may be downloaded from the server.

[0104] As an example, multimodal data is formed by integrating limb data and physiological data of the data and the second model is trained according to a health condition corresponding to the multimodal data.

[0105] As an example, the user's physical health status can be determined by comparing existing health data. For example, through comparison, it can be determined that the user's limbs have bad trajectories, or that the heart rate, blood oxygen and other information are abnormal in certain sports states.

[0106] As an example, by inputting the limb data and physiological data acquired at the current time or time period into the trained second model, possible physical problems can be analyzed. For example, although the heart rate and blood samples are within the normal value of ordinary people, compared with the user's previous heart rate and blood pressure, the user's current heart rate is slow and the blood pressure is low. The second model can prompt the user that there are cardiovascular problems.

[0107] As an example, the first model and the second model described in this article can be machine learning models trained based on an attention mechanism.

[0108] Figure 7 A diagram showing an example based on determining a user's physical health condition according to an embodiment of the present disclosure.

[0109] Reference Figure 7 , a second model (which may be called a health abnormality analysis model) can be downloaded from the server based on the identity of the user, and the user's physical health status can be determined by inputting the user's limb data and physiological data into the second model, such as outputting the user's health analysis results.

[0110] As an example, a user profile for a user may be created and uploaded to a server. The user profile may include the user's personal information, the user's personal device information, the first model, and the second model. The user's personal information may include information such as height and weight.

[0111] The initial first model and / or second model can be trained based on video data and physiological data of a general population to obtain the model.

[0112] As an example, in sports events, the physical health of athletes can be analyzed based on video information and physiological data information obtained by cameras on drones and wearable devices worn by athletes. Each athlete has his or her own personalized model, which can more effectively detect the physical condition of athletes.

[0113] For example, if there is an abnormality in the limb data or physiological data of athlete A, the second model may be used to determine that there is a problem with the physical health of athlete A. For example, when it is determined that there is a problem with the physical health of athlete A, an alarm may be issued. The medical team immediately begins to rescue the athlete upon receiving the alarm. In this way, the first aid time will be shortened and the efficiency of the medical team in the rescue mission will be improved.

[0114] As an example, the physical health of a swimmer can be analyzed based on video information and physiological data obtained by a camera installed by the pool and a wearable device worn by the swimmer. Different swimmers can have their own personalized models to effectively detect the swimmer's physical condition. For example, if the limb data or physiological data of swimmer A is abnormal, it can be determined that there is a problem with the health of swimmer A based on the model corresponding to swimmer A. When it is determined that there is a problem with the physical health of swimmer A, an alarm can be issued. The lifeguard rescues him immediately after receiving the alarm. In this way, the lifeguard can rescue the swimmer before he drowns. This can avoid the situation where rescue is not timely.

[0115] Figure 8A diagram showing an example of determining a user's health status according to an embodiment of the present disclosure.

[0116] Referring to the figure, when a user runs on an indoor treadmill, the user's video data can be obtained through the indoor camera and the user's physiological data can be obtained through the wearable device worn by the user. Based on the user's identity information, a first model corresponding to the user (which can predict the user's physiological data) can be downloaded from the server. The user's physiological data is corrected based on the first model, and then the corrected physiological data and the user's limb data are input into the second model downloaded from the cloud server to determine the user's health status, for example, output a health analysis report.

[0117] As an example, indoor cameras can be used to obtain employee videos and / or wearable devices worn by employees can be used to obtain the user's physiological data. Different employees can have their own user-specific models, which can effectively detect the employee's physical condition by combining the employee's limb data and. For example, employee A, employee B, and employee C are all in a sedentary state, and the heart rates and postures of employees A, B, and C are roughly the same, but B previously had lumbar muscle strain. Since employee B sits for a long time, the model corresponding to employee B can determine that employee B has health problems based on employee B's limb data and physiological data, thereby reminding user B not to sit for a long time.

[0118] Reference above Figures 1 to 8 The data processing method according to the embodiment of the present disclosure is described. Fig. 9 A data processing apparatus according to an embodiment of the present disclosure is described.

[0119] Fig. 9 A block diagram showing the structure of a data processing device 900 according to an embodiment of the present disclosure.

[0120] Reference Fig. 9 , the data processing device 900 may include an acquiring unit 901 and a determining unit 902.

[0121] Those skilled in the art should understand that the data processing device 900 may further include other components, and at least one of the components included in the data processing device 900 may be combined or separated.

[0122] As an example, the acquisition unit 901 may be configured to acquire video data of a user and physiological data of the user.

[0123] As an example, the determination unit 902 may be configured to determine the health status of the user based on the acquired video data of the user and the physiological data of the user.

[0124] As an example, the determining unit 902 may be configured to: acquire limb data of the user based on the video data; and determine the health status of the user based on the acquired limb data of the user and the physiological data of the user.

[0125] As an example, the determination unit 902 may be configured to: determine the backbone node coordinates of the user in the video; and process the node coordinates in the backbone based on preset processing rules, and obtain the limb data of the user based on the processed coordinates.

[0126] As an example, the preset rule is wavelet filtering.

[0127] As an example, the user's limb data includes at least one of the following items: the angle between body parts, the swing frequency, amplitude, speed or acceleration of the backbone nodes.

[0128] As an example, the determination unit 902 can be configured to: predict the user's limb data and the user's physiological data based on the first model; determine whether to correct the user's actual limb data and the user's physiological data based on a comparison between the predicted limb data and the user's physiological data and the acquired limb data and the user's physiological data; when the difference between the predicted limb data and the user's physiological data and the acquired limb data and the user's physiological data is greater than a threshold, correct the acquired limb data and the user's physiological data; and determine the user's physical health status based on the corrected acquired limb data and / or the user's physiological data.

[0129] As an example, the determining unit 902 may be configured to determine the health status of the user by inputting the acquired limb data of the user and the physiological data of the user into a second model corresponding to the user.

[0130] As an example, the data processing device 900 may further include: an updating unit (not shown) configured to update the first model based on the corrected limb data and the user's physiological data.

[0131] As an example, the determining unit 902 may be configured to correct the actual limb data of the user and the physiological data of the user based on cubic spline interpolation.

[0132] As an example, the physiological data includes the user's heart rate and blood oxygen.

[0133] According to an embodiment of the present disclosure, a computer-readable storage medium storing instructions may also be provided, wherein when the instructions are executed by at least one processor, the at least one processor performs the data processing method according to the embodiment of the present disclosure. Examples of computer-readable storage media here include: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disk storage, hard disk drive (HDD), solid state drive (SSD), card storage (such as, multimedia card, secure digital (SD) card or extreme digital (XD) card), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid state disk and any other device, any other device is configured to store computer programs and any associated data, data files and data structures in a non-transitory manner and provide the computer programs and any associated data, data files and data structures to a processor or computer so that the processor or computer can execute the computer program. The computer program in the above-mentioned computer-readable storage medium can be run in an environment deployed in a computer device such as a client, a host, an agent device, a server, etc. In addition, in one example, the computer program and any associated data, data files and data structures are distributed on a networked computer system, so that the computer program and any associated data, data files and data structures are stored, accessed and executed in a distributed manner by one or more processors or computers.

[0134] According to an embodiment of the present disclosure, a computer program product may also be provided, and instructions in the computer program product may be executed by a processor of a computer device to complete the data processing method described herein.

[0135] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

Claims

1. A data processing method, comprising: Obtaining video data of the user and physiological data of the user; The health status of the user is determined based on the acquired video data of the user and the physiological data of the user.

2. The method of claim 1, wherein the step of determining the health condition of the user comprises: Acquire the user's body data based on the video data; as well as The health status of the user is determined based on the acquired limb data of the user and the physiological data of the user.

3. The method according to claim 2, wherein the step of acquiring the user's body data based on the video data comprises: Determine the backbone node coordinates of the user in the video; as well as The node coordinates in the backbone are processed based on preset processing rules, and the limb data of the user is obtained based on the processed coordinates. The method according to claim 3 , wherein the preset rule is wavelet filtering.

5. The method according to claim 3, wherein the user's limb data comprises at least one of the following items: an angle between body parts, a swing frequency, amplitude, speed or acceleration of a backbone.

6. The method according to claim 2, wherein the step of determining the user's health condition based on the acquired user's limb data and the user's physiological data comprises: predicting the user's limb data and the user's physiological data based on the first model; determining whether to correct the actual limb data of the user and the physiological data of the user based on a comparison of the predicted limb data and the physiological data of the user with the acquired limb data of the user and the physiological data of the user; When the difference between the predicted limb data and the user's physiological data and the acquired limb data and the user's physiological data is greater than a threshold, correcting the acquired limb data and the user's physiological data; as well as The user's physical health condition is determined based on the modified acquired user's limb data and / or the user's physiological data.

7. The method according to claim 2, wherein the step of determining the user's health condition based on the acquired limb data and physiological data of the user comprises: The health status of the user is determined by inputting the acquired limb data of the user and the physiological data of the user into a second model corresponding to the user.

8. The method of claim 7, further comprising: The prediction model is updated based on the revised limb data and the user's physiological data.

9. The method according to claim 6, wherein the step of correcting the acquired user's limb data and the user's physiological data comprises: The user's actual limb data and the user's physiological data are corrected based on cubic spline interpolation.

10. The method according to claim 1, wherein the physiological data comprises a user's heart rate and blood oxygen.

11. A data processing device, comprising: an acquisition unit, configured to acquire video data of a user and physiological data of the user; as well as The determination unit is configured to determine the health status of the user based on the acquired video data of the user and the physiological data of the user.

12. The data processing device according to claim 11, wherein the determining unit is configured to: Acquiring the user's body data based on the video data; and The health status of the user is determined based on the acquired limb data of the user and the physiological data of the user.

13. The data processing device according to claim 12, wherein the determining unit is configured to: Determining backbone node coordinates of users in the video; and The node coordinates in the backbone are processed based on preset processing rules, and the limb data of the user is obtained based on the processed coordinates. The data processing device according to claim 13 , wherein the preset rule is wavelet filtering. 15 . The data processing device according to claim 13 , wherein the user's limb data comprises at least one of the following items: an angle between body parts, a swing frequency, amplitude, speed or acceleration of a backbone node.

16. The data processing device according to claim 12, wherein the determining unit is configured to: predicting the user's limb data and the user's physiological data based on the first model; determining whether to correct the actual limb data of the user and the physiological data of the user based on a comparison of the predicted limb data and the physiological data of the user with the acquired limb data of the user and the physiological data of the user; When the difference between the predicted limb data and the user's physiological data and the acquired limb data and the user's physiological data is greater than a threshold, correcting the acquired limb data and the user's physiological data; as well as The user's physical health condition is determined based on the modified acquired user's limb data and / or the user's physiological data.

17. The data processing device according to claim 12, wherein the determining unit is configured to: The health status of the user is determined by inputting the acquired limb data of the user and the physiological data of the user into a second model corresponding to the user.

18. The data processing apparatus according to claim 17, further comprising: An updating unit is configured to update the prediction model based on the corrected limb data and the user's physiological data. 19 . The data processing device according to claim 16 , wherein the determining unit is configured to correct the actual limb data of the user and the physiological data of the user based on cubic spline interpolation.

20. The data processing device according to claim 11, wherein the physiological data includes a user's heart rate and blood oxygen.

21. A computer-readable storage medium storing a computer program, wherein: When the computer program is executed by a processor, the processor is caused to implement the data processing method according to any one of claims 1 to 10.

Images