Health monitoring and correcting method and equipment based on posture recognition, medium and product

Abstract

The invention discloses a health monitoring and correction method and device based on posture recognition, a medium and a product, and relates to the field of computer vision and health management cross technologies, and the method comprises the steps: obtaining RGB image data, depth map data and radar observation data in real time; respectively carrying out normalization processing on the RGB image data and the depth map data; the RGB image data after normalization processing and the depth map data after normalization processing are fused into RGB-D image data; an HRNet-Spine model is constructed, and the HRNet-Spine The RGB-D image data and the radar observation data are input into an HRNet-Spine model, and a spine key point detection result is obtained; obtaining a health score of the current posture based on a spine key point detection result; a gesture correction suggestion is generated based on the health score for the current gesture. According to the invention, efficient and accurate personnel health monitoring and real-time posture correction can be realized.

Description

Technical Field

[0001] The present application relates to the interdisciplinary field of computer vision and health management, and in particular to a method, device, medium, and product for health monitoring and correction based on posture recognition. Background Art

[0002] Modern office workers often spend extended periods of time sitting, averaging over eight hours per day. Incorrect sitting posture and poor sitting habits, particularly those affecting the cervical spine and spinal column, are major contributors to health risks such as cervical spondylosis. However, existing health management technologies face numerous challenges in office environments, failing to effectively provide real-time, comprehensive health monitoring and correction.

[0003] Existing solutions, such as wearable devices, usually require wearing pressure or acceleration sensors, which can easily affect the normal work of office workers and cannot accurately monitor posture. In addition, existing solutions have the following defects: (1) Insufficient capture of multi-angle posture features. The existing system can only perform posture recognition from a single perspective and cannot fully capture the posture features in dynamic changes, especially for the spine and back posture of office workers, the recognition accuracy is low. (2) Lack of real-time dynamic evaluation algorithm. The existing system cannot evaluate whether the user's posture is healthy in real time and provide timely feedback based on the work environment and body changes, resulting in poor health management results. (3) Low matching degree of personalized correction strategy. The existing technology lacks posture correction solutions customized according to individual differences and cannot provide personalized health guidance based on the work environment and physiological characteristics of different users. Summary of the Invention

[0004] The purpose of this application is to provide a health monitoring and correction method, equipment, medium and product based on posture recognition, which can achieve efficient and accurate personnel health monitoring and real-time posture correction.

[0005] To achieve the above objectives, this application provides the following solutions:

[0006] In a first aspect, the present application provides a health monitoring and correction method based on posture recognition, comprising:

[0007] Acquire RGB image data, depth map data, and radar observation data in real time; the RGB image data and the depth map data both contain user gestures;

[0008] Normalizing the RGB image data and the depth map data respectively;

[0009] The normalized RGB image data and the normalized depth map data are fused into RGB-D image data;

[0010] Build the HRNet-Spine model;

[0011] Inputting the RGB-D image data and the radar observation data into the HRNet-Spine model to obtain a spine key point detection result;

[0012] Obtaining a health score for the current posture based on the spinal key point detection result;

[0013] Posture correction suggestions are generated based on the health score of the current posture.

[0014] Optionally, the construction process of the HRNet-Spine model includes:

[0015] Obtain a sample data set; the sample data set includes a plurality of sample pairs having sample RGB-D image data and sample radar observation data as input and sample spine key points as output;

[0016] Build an initial model;

[0017] The sample data set is used to train the initial model until the output of the initial model meets the set requirements, thereby obtaining a trained initial model, and the trained initial model is used as the HRNet-Spine model; the HRNet-Spine model includes: a feature extraction module, a multi-resolution subnet module, a feature fusion module and an adaptive channel calibration module.

[0018] Optionally, the initial model is trained using the sample data set until the output of the initial model meets set requirements, thereby obtaining a trained initial model, including:

[0019] Inputting the sample RGB-D image data and the sample radar observation data into the initial model to obtain a sample spine key point detection result;

[0020] Obtaining a loss function value based on the sample spine key point detection result and the sample spine key point, and determining whether the loss function value reaches a set value;

[0021] When the loss function value does not reach the set value, adjusting the parameters of the initial model, and returning to the step of inputting the sample RGB-D image data and the sample radar observation data into the initial model to obtain the sample spine key point detection result;

[0022] When the loss function value reaches a set value, a trained initial model is obtained.

[0023] Optionally, inputting the RGB-D image data and the radar observation data into the HRNet-Spine model to obtain spine key point detection results includes:

[0024] Using the feature extraction module, based on the RGB-D image data and the radar observation data, respectively, to obtain an RGB-D data feature map and a radar data feature map;

[0025] Using the multi-resolution subnet module, a plurality of RGB-D feature maps and a plurality of radar feature maps are obtained based on the RGB-D data feature map and the radar data feature map respectively;

[0026] Using the feature fusion module, the RGB-D feature map and the radar feature map are processed to obtain a fused feature map;

[0027] Using the adaptive channel calibration module, obtaining an output feature map based on the fused feature map;

[0028] A spine key point detection result is obtained based on the output feature map.

[0029] Optionally, the multi-resolution subnet module includes a first branch, a second branch, a third branch and a fourth branch; the first branch is used to generate a first RGB-D feature map and a first radar feature map based on the RGB-D data feature map and the radar data feature map; the second branch is used to generate a second RGB-D feature map and a second radar feature map based on the RGB-D data feature map and the radar data feature map; the third branch is used to generate a third RGB-D feature map and a third radar feature map based on the RGB-D data feature map and the radar data feature map; the fourth branch is used to generate a fourth RGB-D feature map and a fourth radar feature map based on the RGB-D data feature map and the radar data feature map.

[0030] Optionally, the process of acquiring radar observation data in real time includes:

[0031] Initial radar observation data is acquired in real time, and filtering is performed on the initial radar observation data to obtain the radar observation data.

[0032] Optionally, augmented reality technology is used to generate posture correction suggestions based on the health score of the current posture.

[0033] In a second aspect, the present application provides a computer device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any one of the above-mentioned health monitoring and correction methods based on posture recognition.

[0034] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any one of the above-mentioned health monitoring and correction methods based on posture recognition.

[0035] In a fourth aspect, the present application provides a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of any one of the above-mentioned health monitoring and correction methods based on posture recognition.

[0036] According to the specific embodiments provided in this application, this application has the following technical effects:

[0037] This application provides a health monitoring and correction method, device, medium, and product based on posture recognition. The method performs normalization processing on real-time acquired RGB image data and depth map data. The normalized RGB image data and normalized depth map data, along with real-time radar observation data, are input into a constructed HRNet-Spine model to obtain spinal key point detection results. A health score for the current posture is obtained based on the spinal key point detection results. Posture correction suggestions are generated based on the current posture health score. The method evaluates the user's posture in real time and provides posture correction suggestions. This method enables efficient and accurate personnel health monitoring and real-time posture correction. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0039] Figure 1 A flowchart of a health monitoring and correction method based on posture recognition provided in one embodiment of the present application;

[0040] Figure 2 This is a schematic diagram of the architecture of a health monitoring and correction method based on posture recognition in one embodiment of the present application;

[0041] Figure 3 A schematic diagram of the HRNet-Spine model framework flow chart provided in one embodiment of the present application;

[0042] Figure 4 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0043] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0044] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0045] In an exemplary embodiment, Figure 1 As shown, a health monitoring and correction method based on posture recognition is provided. The method is executed by a computer device. Specifically, it can be executed by a computer device such as a terminal or a server alone, or by a terminal and a server together. In the embodiment of the present application, the method is described by taking the method of obtaining a health score of the current posture based on the user's posture and generating posture correction suggestions as an example, including:

[0046] Step 100: Acquire RGB image data, depth map data, and radar observation data in real time. Both the RGB image data and the depth map data contain user gestures.

[0047] Step 200: Normalize the RGB image data and the depth map data, respectively, and fuse the normalized RGB image data and the normalized depth map data into RGB-D image data.

[0048] Step 300: Build the HRNet-Spine model.

[0049] Step 400: Input the RGB-D image data and radar observation data into the HRNet-Spine model to obtain the spine key point detection results.

[0050] Step 500: Obtain a health score of the current posture based on the spinal key point detection results.

[0051] Step 600 : Generate posture correction suggestions based on the health score of the current posture.

[0052] As an optional implementation, in order to improve the accuracy of spinal key point detection, the construction process of the HRNet-Spine model in step 300 includes:

[0053] Step 310: Obtain a sample dataset. The sample dataset includes a plurality of sample pairs that take sample RGB-D image data and sample radar observation data as input and take sample spine key points as output.

[0054] Step 320: construct an initial model.

[0055] In step 330, the initial model is trained using the sample dataset until the output of the initial model meets the set requirements, resulting in a trained initial model. This trained initial model is used as the HRNet-Spine model. The HRNet-Spine model includes a feature extraction module, a multi-resolution subnet module, a feature fusion module, and an adaptive channel calibration module.

[0056] Wherein, step 330 includes:

[0057] In step 331, the sample RGB-D image data and the sample radar observation data are input into the initial model to obtain the sample spine key point detection results.

[0058] Step 332: Obtain a loss function value based on the sample spine key point detection result and the sample spine key point, and determine whether the loss function value reaches a set value.

[0059] Step 333: When the loss function value does not reach the set value, the parameters of the initial model are adjusted, and the process returns to the step of inputting the sample RGB-D image data and the sample radar observation data into the initial model to obtain the sample spine key point detection result.

[0060] In step 334, when the loss function value reaches the set value, the trained initial model is obtained.

[0061] Furthermore, in order to improve the accuracy of spine key point detection, the weighted mean square error (MSE) loss can be used to determine the loss function value. The loss function is expressed as:

[0062]

[0063] Among them, L MSE represents the loss function value, λ p The weight of each key point, y p is the true value of the p-th key point, is the predicted value of the pth keypoint. M represents the number of spine keypoints.

[0064] This loss function directly applies to the regression of spine keypoints and is combined with parameter updates for each module of the initial model to train the initial model. When the loss function reaches a set value, the trained initial model is obtained. This trained initial model is used as the HRNet-Spine model, achieving optimal performance in feature extraction, keypoint detection, and subsequent posture scoring.

[0065] Based on the steps in the above embodiment, step 400 includes:

[0066] In step 410 , a feature extraction module is used to obtain an RGB-D data feature map and a radar data feature map based on the RGB-D image data and the radar observation data, respectively.

[0067] In step 420 , a multi-resolution subnet module is used to obtain a plurality of RGB-D feature maps and a plurality of radar feature maps based on the RGB-D data feature map and the radar data feature map, respectively.

[0068] Step 430: Using a feature fusion module, the multiple RGB-D feature maps and the multiple radar feature maps are processed to obtain a fused feature map.

[0069] In step 440 , an adaptive channel calibration module is used to obtain an output feature map based on the fused feature map.

[0070] Step 450: Obtain spine key point detection results based on the output feature map.

[0071] The multi-resolution subnet module includes a first branch, a second branch, a third branch, and a fourth branch. The first branch is used to generate a first RGB-D feature map and a first radar feature map based on the RGB-D data feature map and the radar data feature map. The second branch is used to generate a second RGB-D feature map and a second radar feature map based on the RGB-D data feature map and the radar data feature map. The third branch is used to generate a third RGB-D feature map and a third radar feature map based on the RGB-D data feature map and the radar data feature map. The fourth branch is used to generate a fourth RGB-D feature map and a fourth radar feature map based on the RGB-D data feature map and the radar data feature map.

[0072] In practical applications, the user's posture data (i.e., RGB image data and depth map data) can be obtained through an RGB-D camera, and the millimeter-wave radar can be used to obtain additional user posture data (i.e., initial radar observation data).

[0073] As an optional embodiment, the process of acquiring radar observation data in real time in step 100 includes: acquiring initial radar observation data in real time and filtering the initial radar observation data to obtain radar observation data. For example, posture data can be supplemented and smoothed using millimeter-wave radar-assisted skeletal tracking. To improve data accuracy, a Kalman filter is used to process the initial radar observation data. The Kalman filter process is as follows:

[0074] Prediction steps:

[0075] Prediction equation: in, is the predicted value of the current state, A is the state transfer matrix, B is the control matrix, μ k is the control input, is the state value at time k-1 (i.e. the previous moment).

[0076] Covariance prediction: in, is the predicted covariance matrix, P k-1 is the covariance matrix of the previous moment, and Q is the process noise covariance matrix.

[0077] Observation update steps:

[0078] Kalman gain calculation: Among them, K k is the Kalman gain, H is the observation matrix, and R is the observation noise covariance matrix.

[0079] Covariance update: Among them, P k Represents the covariance matrix of the current state, and I is the identity matrix.

[0080] Status Update: Among them, x Radar is the filtered data (i.e. radar observation data), z k is the observed value, It represents the state prediction value at time k (i.e. the current moment), which is obtained by the state value at the previous moment through the prediction equation.

[0081] In step 600 , augmented reality technology is used to generate posture correction suggestions based on the health score of the current posture.

[0082] In an exemplary embodiment, in combination with the steps in the above embodiments, an example is given of using a health monitoring and correction method based on posture recognition to perform a health score on the user's posture based on the constructed HRNet-Spine model and generate posture correction suggestions.

[0083] In step S1, an RGB-D camera is used to capture the user's full-body posture (including RGB image data and depth map data). Millimeter-wave radar is used to obtain additional user posture data (i.e., initial radar observation data) to fill in visual blind spots and enhance posture recognition accuracy in complex environments.

[0084] The initial radar observation data is filtered to obtain radar observation data.

[0085] To keep the data in the same range, normalize the RGB image data and depth map data separately. For example, use Min-Max normalization to scale the pixel values ​​of the RGB image to [0, 1] to ensure that all input values ​​are in the same range for easy subsequent processing.

[0086]

[0087] Among them, X is the pixel value in the RGB image, X min and X max are the minimum and maximum values ​​of the channel respectively, X norm Represents the normalized RGB pixel data.

[0088] The depth map is normalized using Z-Score. Since the range of depth values ​​changes dynamically, Z-Score normalization is used to process the depth data.

[0089]

[0090] Where D is the depth value in the depth map, μ D is the mean of the depth map, σ D is the standard deviation, x D Represents the normalized depth map data.

[0091] In step S2, the normalized RGB image data and the normalized depth map data are fused into RGB-D image data. For example, the normalized RGB image data and the normalized depth map data are spatially aligned and channel-joined.

[0092] x RGB-D =w1x RGB +w2x D .

[0093] Among them, x RGB Represents the normalized RGB image data, x D Represents the normalized depth map data, x RGB-D is the RGB-D image data. The weights w1, w2 are determined by their respective covariance matrices P i Sure:

[0094]

[0095] The process of determining the covariance matrix of an RGB-D image involves calculating the statistical correlation between each channel, i.e., x RGB-D The covariance matrix of . Each element of the covariance matrix is ​​defined as:

[0096]

[0097] Among them, P RGB-D is the covariance matrix; N is the total number of samples, that is, the number of pixels in the RGB image participating in the statistics. k The observed value of variable x in the kth sample, y k is the observed value of variable y in the kth sample, represents the mean of the variable x, represents the mean of the variable y.

[0098] Step S3: Input the RGB-D image data and radar observation data into the HRNet-Spine model to obtain the spine key point detection results, such as Figure 3 shown.

[0099] S31 uses a feature extraction module to generate RGB-D data feature maps and radar data feature maps based on the RGB-D image data and radar observation data, respectively. For example, to ensure that the input data is fully represented in the convolutional neural network, the HRNet-Spine model includes a dedicated feature extraction module at the input stage to process the RGB-D image data and radar observation data separately. This module uses standard convolution operations combined with a high-resolution preservation mechanism to extract effective semantic features while preserving the spatial information of the input image data as much as possible, laying the foundation for subsequent multi-scale feature fusion.

[0100] Specifically, the input image data is first scanned through a sliding window of a 3×3 convolution kernel. Each time the convolution kernel moves one pixel in the spatial dimension, local feature information at that location is extracted. To prevent boundary information from being lost during the convolution process and to maintain the resolution of the output image consistent with the input, 1-pixel zero padding is performed on the edges of the input image. This "zero padding + stride of 1" strategy effectively increases feature representation density and avoids loss of spatial information.

[0101] After convolution, the image is downsampled twice with a stride of 2 to reduce its size. Each downsampling operation halves the image's resolution while preserving key structural information. By gradually constructing a multi-scale feature pyramid, the network can capture semantic information at different levels while maintaining high-resolution features, achieving feature modeling from detailed to global scale.

[0102] The RGB-D image data is processed by the feature extraction module, which extracts the joint features of the RGB image and depth map to obtain the RGB-D data feature map. The radar observation data is processed by the feature extraction module, which extracts the features of the radar observation data to obtain the radar data feature map.

[0103] S32 uses a multi-resolution subnetwork module to generate multiple RGB-D feature maps and multiple radar feature maps based on the RGB-D data feature map and radar data feature map, respectively. For example, the multi-resolution subnetwork module consists of four stages (Stage 1 to Stage 4, i.e., the first branch, the second branch, the third branch, and the fourth branch). Each stage gradually introduces a new low-resolution branch and continuously extracts features through convolution operations within each stage.

[0104] Stage 1 (high-resolution branch, also known as the first branch): The input is the RGB-D data feature map and the radar data feature map, which are fed into their respective convolutional layers to generate the first-stage high-resolution feature map (including the first RGB-D feature map and the first radar feature map). Its resolution is the same as the original image size (1× resolution). This branch focuses on extracting image details such as edges and textures.

[0105] Stage 2 (adding a low-resolution branch to obtain the second branch): In this branch, the first RGB-D feature map and the first radar feature map are downsampled using a convolution operation with a stride of 2 and a kernel size of 3×3, respectively, to obtain a feature map with a resolution of 1 / 2 (including the second RGB-D feature map and the second radar feature map). This branch extracts mid-scale structural information while preserving some details. While the second branch is processing, the first branch is retained and updated, allowing it to continue to obtain features at the original resolution.

[0106] Stage 3 (adding a 1 / 4 resolution branch to obtain the third branch): The second RGB-D feature map and the second radar feature map are each subjected to a convolution operation with a stride of 2 to obtain a lower resolution (1 / 4) feature map (including the third RGB-D feature map and the third radar feature map), which is used to extract broader semantic information. Similarly, the first and second branches are still retained and updated in parallel.

[0107] Stage 4 (adding a 1 / 8 resolution branch to obtain the fourth branch): The third RGB-D feature map and the third radar feature map are further downsampled to obtain the fourth RGB-D feature map and the fourth radar feature map with a resolution of 1 / 8. This branch is used to model global context information and enhance the whole-body perception capability in pose estimation.

[0108] S33, using the feature fusion module, processes the RGB-D feature map and the radar feature map to obtain the fused feature map (corresponding to Figure 3 For example, a weighted fusion mechanism is used to process the RGB-D feature maps and radar feature maps obtained by the multi-resolution subnet module in the above steps. That is, the RGB-D feature maps and radar feature maps obtained from the four branches are weightedly fused through the attention mechanism. Weights α and β are assigned to the features of each modality, where α + β = 1, and then the two feature maps are weightedly fused:

[0109] F j =αF RGB-D +βF Radar .

[0110] Among them, F jrepresents the feature map after weighted fusion of the RGB-D feature map and the radar feature map in the j-th branch. α and β are dynamic weights calculated by the attention mechanism. F RGB-D Represents the RGB-D data feature map, F Radar Represents a radar data feature map.

[0111] Through weighted fusion, four weighted fused feature maps (i.e., feature maps with 1×, 1 / 2×, 1 / 4×, and 1 / 8× resolutions) are obtained, which are represented as X1, X2, X3, and X4 respectively. The four fused feature maps are denoted as follows:

[0112]

[0113] in, Represents the fused feature map, + represents element-by-element addition, ensuring that feature information of different scales can be effectively fused. UPSample is bilinear interpolation upsampling, used to match the size of higher resolution features. down is a 3×3 convolution with a stride of 2 to match the size of the lower resolution features.

[0114] S34 uses an adaptive channel calibration module (SE module) to obtain an output feature map based on the fused feature maps. For example, to ensure channel consistency of features at different resolutions, a 1×1 convolution is applied to each resolution feature, the number of channels is adjusted, and an adaptive attention mechanism is used for fusion to enhance the adaptability of feature fusion. The Squeeze-and-Excitation (SE) mechanism is introduced to dynamically adjust the feature channel weights:

[0115]

[0116] Among them, F SE is the output feature map after SE module adjustment, represents the nth fused feature map, W1 and W2 are two fully connected layers, GAP represents global average pooling, δ represents the ReLU activation function, and σ represents the Sigmoid activation function.

[0117] Specifically, the fused feature map The input is fed into the SE module to further enhance the expressive power in the channel dimension. The SE module first uses 1×1 convolution to unify the number of channels in each branch. It then introduces a compression-excitation mechanism. Through global average pooling and a two-layer fully connected network, it calculates channel attention weights, weighting the importance of different channel features and enhancing the network's perception of key semantic areas.

[0118] S35: Determine the spine keypoint detection results based on the output feature map. For example, the output feature map from the SE module is fed into two prediction branches: a heatmap prediction module and a keypoint coordinate regression module. In this embodiment, the keypoint direct regression module is used as the primary recognition path, while heatmap supervision is used as an auxiliary method to improve the responsiveness and positioning accuracy of key areas.

[0119] In the direct regression path, the network maps the output feature map to a set of keypoint coordinate vectors through a series of convolutional layers and fully connected layers. Each output feature map is mapped to a set of three-dimensional coordinates (x l ,y l ,z l ), where l represents the spinal keypoint number. Multiple keypoint locations are preset based on bony landmarks (e.g., typical locations of the cervical, thoracic, and lumbar vertebrae). Training labels are created through manual annotation to train the keypoint coordinate regression module. Ultimately, the module outputs a coordinate sequence consisting of multiple keypoints, representing the complete spatial structure of the spine.

[0120] Furthermore, to enhance the accuracy and stability of posture recognition, keypoint visibility labels and spatial order constraints are introduced during the training phase of the keypoint coordinate regression module to ensure anatomical consistency of the predicted points. When continuous multi-frame data is input, keypoint trajectories can also be filtered and smoothed in the temporal dimension to improve the robustness and practicality of posture monitoring in practical applications.

[0121] In order to more accurately assess the health of the spine, not only the angles of two adjacent points are calculated, but at least three points are used to estimate the curvature of the spine.

[0122] The curvature calculation formula is expressed as:

[0123]

[0124] in, and are the vectors between three adjacent key points, and θ is the bending angle.

[0125] Based on this, the spine key point detection results are obtained.

[0126] Step S4: Calculate the health score of the current posture based on the spinal key point detection results. For example, the health score of the overall posture is calculated based on the difference between the angle of each spinal key point and the target health angle. The scoring formula is expressed as:

[0127]

[0128] Among them, θ i is the angle of the current i-th spine key point, is the target health angle of the i-th keypoint, A is the number of spine keypoints, and Spine Score represents the raw health score of the current posture. The closer the Spine Score is to 0, the healthier the current posture.

[0129] To make the score more intuitive, the posture score is mapped to the interval [0,100]:

[0130]

[0131] Among them, θ max is the preset maximum allowable deviation, Score norm represents the pose score after mapping.

[0132] Step S5: Generate posture correction suggestions based on the health score of the current posture (corresponding to Figure 2 For example, based on the above scoring results, key points with a score below 60 are filtered. Using augmented reality (AR) technology, the direction of the AR projection is calculated based on the 3D coordinate deviation of the spine key points, and posture correction suggestions are generated and provided in real time. The projection formula is expressed as:

[0133]

[0134] Among them, Δ p is the deviation direction, θ current represents the spine angle of the current posture, θ target The ideal angle of the spine in the target posture (i.e. the ideal healthy posture) is expressed. is the normal direction vector of the spine.

[0135] In order to implement the above-mentioned health monitoring and correction method based on posture recognition, the present application can adopt the following method: Figure 2 The office health monitoring and correction method architecture is shown. The multimodal perception layer is used to execute step 100 (equivalent to step S1), the core algorithm layer is used to execute steps 200-400 (equivalent to steps S2-S3), and the feedback execution layer is used to execute steps 500 and 600 (equivalent to steps S4-S5).

[0136] Based on the above embodiments provided in this application, this application has the following advantages:

[0137] 1. Integrating RGB-D camera and millimeter-wave radar data, using a hybrid fusion model combining early fusion (spatial alignment and channel concatenation before processing in the multi-resolution subnetwork module) and late fusion (dynamic weighting through an attention mechanism), this approach addresses the issue of insufficient pose capture from a single perspective. This effectively addresses the limitations of traditional systems in capturing blind spots and dynamically changing poses in complex office environments.

[0138] 2. Based on traditional HRNet, we have specifically optimized spinal keypoint detection by incorporating SE modules, zero-padded convolution kernels, and multi-resolution subnet module fusion to ensure accurate spinal posture extraction while maintaining high-resolution details. This not only extracts the locations of spinal keypoints, but also accurately calculates the angles between adjacent keypoints and the overall spinal curvature, providing a reliable basis for subsequent posture scoring.

[0139] 3. A Dynamic Posture Scoring (DPS) system was developed to quantify postural health by comparing actual spinal angles with healthy target angles and mapping them to an intuitive [0,100] scoring scale. Real-time monitoring and scoring provide timely feedback when posture deviates from a healthy state, and personalized interventions can be implemented through AR projection correction guidance.

[0140] 4. Utilizing augmented reality technology, the system calculates the corrective direction based on 3D key point deviations and provides real-time visual cues to guide users in adjusting poor sitting posture. This addresses the issue of traditional health management devices (such as wearables) significantly interfering with user activities (e.g., office work) by enabling seamless monitoring and intelligent correction.

[0141] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 4 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O) and a communication interface. The processor, memory and input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store relevant data involved in the health monitoring and correction method based on posture recognition. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a health monitoring and correction method based on posture recognition is implemented.

[0142] Those skilled in the art will understand that Figure 4 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present application and does not constitute a limitation on the computer device to which the solution of the present application is applied. A specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement. In an exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps of the above-mentioned method embodiments when executing the computer program.

[0143] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0144] In an exemplary embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0145] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0146] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0147] The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may include, but are not limited to, general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic units, data processing logic units based on quantum computing, and the like.

[0148] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0149] This document uses specific examples to illustrate the principles and implementation methods of this application. The description of the above examples is only intended to help understand the method and core concept of this application. At the same time, for those skilled in the art, based on the concept of this application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A health monitoring and correction method based on posture recognition, characterized in that: include: Acquire RGB image data, depth map data and radar observation data in real time; The RGB image data and the depth map data both contain user gestures; Normalizing the RGB image data and the depth map data respectively; The normalized RGB image data and the normalized depth map data are fused into RGB-D image data; Build the HRNet-Spine model; Inputting the RGB-D image data and the radar observation data into the HRNet-Spine model to obtain a spine key point detection result; Obtaining a health score for the current posture based on the spinal key point detection result; Posture correction suggestions are generated based on the health score of the current posture.

2. The health monitoring and correction method based on posture recognition according to claim 1, characterized in that: The construction process of the HRNet-Spine model includes: Obtain a sample data set; the sample data set includes a plurality of sample pairs having sample RGB-D image data and sample radar observation data as input and sample spine key points as output; Build an initial model; The sample data set is used to train the initial model until the output of the initial model meets the set requirements, thereby obtaining a trained initial model, and the trained initial model is used as the HRNet-Spine model; the HRNet-Spine model includes: a feature extraction module, a multi-resolution subnet module, a feature fusion module and an adaptive channel calibration module.

3. The health monitoring and correction method based on posture recognition according to claim 2, characterized in that: The initial model is trained using the sample data set until the output of the initial model meets the set requirements, thereby obtaining a trained initial model, including: Inputting the sample RGB-D image data and the sample radar observation data into the initial model to obtain a sample spine key point detection result; Obtaining a loss function value based on the sample spine key point detection result and the sample spine key point, and determining whether the loss function value reaches a set value; When the loss function value does not reach the set value, adjusting the parameters of the initial model, and returning to the step of inputting the sample RGB-D image data and the sample radar observation data into the initial model to obtain the sample spine key point detection result; When the loss function value reaches a set value, a trained initial model is obtained.

4. The health monitoring and correction method based on posture recognition according to claim 2, characterized in that: Inputting the RGB-D image data and the radar observation data into the HRNet-Spine model to obtain spine key point detection results, including: Using the feature extraction module, based on the RGB-D image data and the radar observation data, respectively, to obtain an RGB-D data feature map and a radar data feature map; Using the multi-resolution subnet module, a plurality of RGB-D feature maps and a plurality of radar feature maps are obtained based on the RGB-D data feature map and the radar data feature map respectively; Using the feature fusion module, processing the plurality of RGB-D feature maps and the plurality of radar feature maps to obtain a fused feature map; Using the adaptive channel calibration module, obtaining an output feature map based on the fused feature map; A spine key point detection result is obtained based on the output feature map.

5. The health monitoring and correction method based on posture recognition according to claim 4, characterized in that: The multi-resolution subnet module includes a first branch, a second branch, a third branch, and a fourth branch; the first branch is used to generate a first RGB-D feature map and a first radar feature map based on the RGB-D data feature map and the radar data feature map; The second branch is used to generate a second RGB-D feature map and a second radar feature map based on the RGB-D data feature map and the radar data feature map; the third branch is used to generate a third RGB-D feature map and a third radar feature map based on the RGB-D data feature map and the radar data feature map; the fourth branch is used to generate a fourth RGB-D feature map and a fourth radar feature map based on the RGB-D data feature map and the radar data feature map.

6. The health monitoring and correction method based on posture recognition according to claim 1, characterized in that: The process of acquiring radar observation data in real time includes: Initial radar observation data is acquired in real time, and filtering is performed on the initial radar observation data to obtain the radar observation data.

7. The health monitoring and correction method based on posture recognition according to claim 1, characterized in that: Augmented reality technology is used to generate posture correction suggestions based on the health score of the current posture.

8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the health monitoring and correction method based on posture recognition according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the health monitoring and correction method based on posture recognition according to any one of claims 1 to 7 is implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the health monitoring and correction method based on posture recognition according to any one of claims 1 to 7 is implemented.

Images