An image recognition method for human body posture assessment

Abstract

The present application relates to the technical field of image data processing, and discloses an image recognition method for human body posture evaluation, comprising: synchronously acquiring color image data and depth image data, calculating skeleton joint space coordinates and confidence measure values; screening a stable frame set by using the confidence measure values and inter-frame displacement amounts, registering and converting the skeleton joint space coordinates and a three-dimensional human body general mesh model into a body surface feature point space coordinate sequence by using a spatial affine transformation matrix; calculating a collinear angle cosine value to determine a space-time consistency weight coefficient, and when the space-time consistency weight coefficient is lower than a threshold value, reducing the weight and correcting mesh model non-rigid deformation parameters by using a residual error, and outputting an evaluation index value; the present application eliminates coordinate mutation steps at the junction of modes, and guarantees the continuity of anatomical feature space geometry.

Description

Technical Field

[0001] This invention belongs to the field of image data processing technology, and in particular relates to an image recognition method for human body posture assessment. Background Technology

[0002] Currently, within the scope of human motion feature analysis technology, using a depth camera to simultaneously acquire color image streams and depth image streams to identify the spatial coordinates of skeletal joints is a fundamental method for providing three-dimensional topological features. Within this technological framework, the color lens provides two-dimensional edge details, and the depth component reconstructs a three-dimensional point cloud of the body surface through ranging. The two types of image streams establish a spatial geometric relationship through a projection matrix, and the chip calculates the coordinates of the rotation center of the internal joints of the human body. To achieve spatial alignment of anatomical features, the skeletal motion center is transformed into a specific physical landmark point.

[0003] However, when the above methods are applied to refined measurement scenarios, the volumetric centroid calculation mechanism they rely on leads to a fundamental mismatch. Since the keypoints acquired by the depth camera are essentially implicit centers of motion rather than the anatomical features required clinically, a non-rigid spatial bias exists between them due to tissue thickness. Furthermore, in actual acquisition environments, clothing wrinkles easily generate false edge gradients, and multipath reflections introduce continuous noise into the point cloud, resulting in spatial breaks between 2D keypoints and the 3D manifold. Static block approximation paths typically ignore skeletal topological geometric manifold constraints, causing downstream markers to experience step-like coordinate space jumps at the multimodal advantage boundary. Inherent deformation and bias limitations exist at the multimodal sensing boundary, and related data processing methods also suffer from insufficient computation at the software level. For example, Chinese invention patent application CN116309712A discloses a functional motion analysis system and method based on a depth camera. It reduces the dimensionality of the acquired three-dimensional skeletal point coordinates by two-dimensional projection along a specific anatomical plane, simplifies the coordinate information in the depth space and calculates the angle. This simplified path, which relies on the overall plane projection, depends on the ideal environment of linear motion and no occlusion. However, when faced with the complex non-rigid deformation of the human body, false edges of clothing folds and multi-path multi-dimensional reflection noise at the edges, this application causes abrupt changes in coordinate space due to the discarding of the continuous manifold constraint in the depth direction. Furthermore, the fixed depth threshold division mechanism cannot break the linear continuity assumption of spatial registration from the bottom layer, making it difficult to meet the requirements of fine-grained anti-drift long-term body posture tracking.

[0004] Therefore, the technical problem to be solved by this invention is how to break the linear continuity assumption of spatial registration, eliminate coordinate jumps at the fusion boundary through feature adaptive collaboration, and establish a drift-resistant long-period image recognition benchmark within a deterministic algorithm framework. Summary of the Invention

[0005] This invention aims to solve the problems of global anatomical feature spatial coordinate jumps and long-period tracking technology drift caused by non-rigid mismatch between internal joint rotation centers and anatomical landmarks on the body surface, interference from false edges of clothing folds, and differences in floating-point numbers of computing platforms when using multimodal depth camera feature fusion.

[0006] In this technical solution, an image recognition method for human posture assessment includes the following steps: Step S1: Acquire multi-view color image data and depth image data of the subject through hardware synchronous output of the infrared depth acquisition component; calculate the spatial coordinates of the subject's skeletal joints and the corresponding confidence metric from the depth image data, and detect the coordinates of the subject's two-dimensional key points on the body surface from the color image data. Step S2: Select stable frames according to confidence metric and inter-frame displacement to obtain a set of stable frames; use spatial affine transformation matrix to register the spatial coordinates of skeletal joints with the three-dimensional human general mesh model, and convert the spatial coordinates of skeletal joints into a sequence of spatial coordinates of body surface feature points by curve fitting of the spatial trajectory of the subject's back spine. Step S3: Calculate the cosine of the collinearity angle between the gradient direction of the image edge in the color image data and the direction of the surface normal vector in the depth image data to obtain the dimensionless spatiotemporal consistency weight coefficient; when the spatiotemporal consistency weight coefficient is lower than the set decoupling threshold, reduce the regression weight of the corresponding two-dimensional key point coordinates on the body surface in the coordinate correction regression model, and use the calculated coordinate residuals to correct the non-rigid deformation parameters of the three-dimensional human body general mesh model, and output the spatially continuous human body posture evaluation index value.

[0007] Preferably, the image recognition method includes a temporal smoothing step of the spatial coordinate sequence of body surface feature points: smoothing the three-dimensional position components and velocity components contained in the spatial coordinate sequence of body surface feature points, wherein the measurement noise covariance matrix in the smoothing filter is scaled and adjusted in real time according to the confidence metric; and combining the median of the position coordinates of each body surface feature point in the stable frame set to output the smoothed coordinate data, so as to eliminate measurement noise caused by feature point jumps and respiratory movements.

[0008] Preferably, in step S1, while acquiring multi-view color image data and depth image data, background acquisition feature data is recorded. The background acquisition feature data includes the software development kit version, camera firmware version, processor model, and image acquisition timestamp. The background acquisition feature data is associated with the spatial coordinate sequence of body surface feature points and stored in the database for subsequent long-term longitudinal data comparison and verification.

[0009] Preferably, the stable frame selection in step S2 includes the following steps: Step S21, calculating the continuous displacement of the skeletal joints of the subject's torso and the continuous displacement of the skeletal joints of the limbs between adjacent image frames; Step S22, when the continuous displacement of the skeletal joints of the torso is less than 5 mm and the continuous displacement of the skeletal joints of the limbs is less than 8 mm, the corresponding image frame is determined as a stable frame and included in the stable frame set, wherein the stable frame set contains more than or equal to 60 frames.

[0010] Preferably, the image recognition method further includes introducing a quality verification control step in the stable frame set: when the number of frames in the stable frame set is less than 30, the current evaluation process is terminated and a prompt message indicating insufficient data acquisition quality is output; when the proportion of frames in the stable frame set whose confidence metric value is greater than or equal to the set confidence threshold value is less than 60%, the weight coefficient of the corresponding feature point in subsequent calculations is reduced.

[0011] Preferably, step S2 involves registering the spatial coordinates of skeletal joints with a three-dimensional human general mesh model using a spatial affine transformation matrix, and performing curve fitting on the spatial trajectory of the subject's back spine. This includes: calculating the three-dimensional point cloud data of the human body surface from the depth image data, registering the three-dimensional point cloud data of the human body surface with a three-dimensional human general mesh model, and calculating the spatial curvature change characteristics of the longitudinal bending trajectory corresponding to the central region of the subject's back, thereby converting the spatial coordinates of skeletal joints into a sequence of spatial coordinates of surface feature points.

[0012] Preferably, in step S3, when the spatiotemporal consistency weight coefficient is lower than the set decoupling threshold, the regression weight of the corresponding two-dimensional key point coordinates on the body surface in the coordinate correction regression model is reduced, and the non-rigid deformation parameters of the three-dimensional human body general mesh model are corrected using the calculated coordinate residuals. This includes: calculating the absolute value of the cosine of the collinearity angle between the gradient direction of the image edge and the direction of the surface normal vector in the dual-modal feature fusion space to obtain the dimensionless spatiotemporal consistency weight coefficient; when the spatiotemporal consistency weight coefficient is lower than the decoupling threshold, it is determined that the corresponding feature point has a matching deviation caused by the deformation of the subject's outer clothing, thereby reducing the regression coefficient of the corresponding feature point in the coordinate correction regression model to limit the transmission of abnormal displacement errors.

[0013] Preferably, the image recognition method further includes a long-cycle tracking calculation step: obtaining multiple sets of historical human body posture assessment index values ​​of the subject at different historical acquisition times, and constructing a feature evolution time series according to the chronological order of acquisition times; calculating the position deviation slope of the feature evolution time series over time, and outputting an abnormal deviation prompt signal representing the change in body posture when the position deviation slope continuously exceeds the set fluctuation range.

[0014] Preferably, the image recognition method further includes a body posture assessment result display step: integrating the spatially continuous human body posture assessment index values, the analysis data of the feature evolution time series, and the feature data collected in the background to generate a digital assessment report text containing the spatial distribution coordinate parameters of the subject's feature points; transmitting the digital assessment report text to the interface of the display terminal for graphical layout display to provide a spatial shape feature parameter retrieval map.

[0015] Compared with existing technologies, the image recognition method for human body posture assessment of the present invention has the following advantages: 1. In image recognition for human body posture assessment, a high-purity effective frame set is constructed by synchronously outputting color and depth image streams through hardware, combined with the elimination of convergent frames at the initial acquisition stage and the screening of inter-frame displacement thresholds. While extracting the internal joint rotation center from the depth stream, two-dimensional dense key points on the body surface are extracted from the color stream and a three-dimensional point cloud on the surface is generated. The two-dimensional dense key points are back-projected to three-dimensional space using calibration parameters, and a regression error model is established in the overlapping area. This achieves modal complementary correction with three-dimensional data as the benchmark in high-precision areas and two-dimensional data as the supplement in areas with superior detail. The deep interweaving of multi-source spatial feature streams under spatiotemporal constraints enables the system to resist interference from local pseudo-semantic edges and parasitic noise from edge multipaths.

[0016] 2. For the spatial transformation from the internal joint rotation center to anatomical landmarks, a fixed bias mapping, a statistical shape model image involving rigid body affine transformation, and cubic B-spline curve fitting along the spinal groove trajectory are invoked to accurately transform the coordinates of the three-dimensional joint tracking points to the clinical surface anatomical landmarks. Then, a Kalman filter containing three-dimensional position and velocity components is applied to the spatial coordinate sequence of the landmarks. The measurement noise covariance matrix is ​​adaptively scaled according to the confidence level output by the tracking engine. Finally, the median output of the effective frame set is combined. This multi-track spatial mapping collaborative adaptive temporal smoothing method eliminates the measurement noise caused by feature point jumps and non-rigid respiratory motion in single-frame image recognition.

[0017] 3. A nonlinear adaptive correction of a cross-modal gradient consensus field is constructed in the dual-modal feature fusion space. The dimensionless spatiotemporal consensus weight is obtained by calculating the direction cosine alignment between the gradient vector of the color flow image edge and the surface normal vector of the depth flow. When the weight is lower than the consensus threshold, it is determined that there is a pseudo-semantic conflict caused by clothing wrinkles at the current point. The system automatically reduces the regression weight of the corresponding key point in the error model and injects the residual back into the upstream statistical shape model as a topological relaxation constraint parameter for the deformation of the skeletal mesh region. This closed-loop correction chain of cross-level and cross-modal feature bidirectional flow eliminates the coordinate jump steps at the boundary between the two-dimensional detail advantage area and the three-dimensional accuracy advantage area, ensuring the geometric manifold continuity of the global anatomical feature space. Attached Figure Description

[0018] Figure 1 This is a flowchart of the feature processing of the human body posture assessment image recognition method of the present invention; Figure 2 This is a component interaction architecture diagram of the image recognition method implemented in this invention. Detailed Implementation

[0019] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0020] An image recognition method for human posture assessment includes the following steps: Step S1: Acquire multi-view color image data and depth image data of the subject through hardware synchronous output of the infrared depth acquisition component; calculate the spatial coordinates of the subject's skeletal joints and the corresponding confidence metric from the depth image data, and detect the coordinates of the subject's two-dimensional key points on the body surface from the color image data. Step S2: Select stable frames according to confidence metric and inter-frame displacement to obtain a set of stable frames; use spatial affine transformation matrix to register the spatial coordinates of skeletal joints with the three-dimensional human general mesh model, and convert the spatial coordinates of skeletal joints into a sequence of spatial coordinates of body surface feature points by curve fitting of the spatial trajectory of the subject's back spine. Step S3: Calculate the cosine of the collinearity angle between the gradient direction of the image edge in the color image data and the direction of the surface normal vector in the depth image data to obtain the dimensionless spatiotemporal consistency weight coefficient; when the spatiotemporal consistency weight coefficient is lower than the set decoupling threshold, reduce the regression weight of the corresponding two-dimensional key point coordinates on the body surface in the coordinate correction regression model, and use the calculated coordinate residuals to correct the non-rigid deformation parameters of the three-dimensional human body general mesh model, and output the spatially continuous human body posture evaluation index value.

[0021] Preferably, the image recognition method includes a temporal smoothing step of the spatial coordinate sequence of body surface feature points: smoothing the three-dimensional position components and velocity components contained in the spatial coordinate sequence of body surface feature points, wherein the measurement noise covariance matrix in the smoothing filter is scaled and adjusted in real time according to the confidence metric; and combining the median of the position coordinates of each body surface feature point in the stable frame set to output the smoothed coordinate data, so as to eliminate measurement noise caused by feature point jumps and respiratory movements.

[0022] Preferably, in step S1, while acquiring multi-view color image data and depth image data, background acquisition feature data is recorded. The background acquisition feature data includes the software development kit version, camera firmware version, processor model, and image acquisition timestamp. The background acquisition feature data is associated with the spatial coordinate sequence of body surface feature points and stored in the database for subsequent long-term longitudinal data comparison and verification.

[0023] Preferably, the stable frame selection in step S2 includes the following steps: Step S21, calculating the continuous displacement of the skeletal joints of the subject's torso and the continuous displacement of the skeletal joints of the limbs between adjacent image frames; Step S22, when the continuous displacement of the skeletal joints of the torso is less than 5 mm and the continuous displacement of the skeletal joints of the limbs is less than 8 mm, the corresponding image frame is determined as a stable frame and included in the stable frame set, wherein the stable frame set contains more than or equal to 60 frames.

[0024] Preferably, the image recognition method further includes introducing a quality verification control step in the stable frame set: when the number of frames in the stable frame set is less than 30, the current evaluation process is terminated and a prompt message indicating insufficient data acquisition quality is output; when the proportion of frames in the stable frame set whose confidence metric value is greater than or equal to the set confidence threshold value is less than 60%, the weight coefficient of the corresponding feature point in subsequent calculations is reduced.

[0025] Preferably, step S2 involves registering the spatial coordinates of skeletal joints with a three-dimensional human general mesh model using a spatial affine transformation matrix, and performing curve fitting on the spatial trajectory of the subject's back spine. This includes: calculating the three-dimensional point cloud data of the human body surface from the depth image data, registering the three-dimensional point cloud data of the human body surface with a three-dimensional human general mesh model, and calculating the spatial curvature change characteristics of the longitudinal bending trajectory corresponding to the central region of the subject's back, thereby converting the spatial coordinates of skeletal joints into a sequence of spatial coordinates of surface feature points.

[0026] Preferably, in step S3, when the spatiotemporal consistency weight coefficient is lower than the set decoupling threshold, the regression weight of the corresponding two-dimensional key point coordinates on the body surface in the coordinate correction regression model is reduced, and the non-rigid deformation parameters of the three-dimensional human body general mesh model are corrected using the calculated coordinate residuals. This includes: calculating the absolute value of the cosine of the collinearity angle between the gradient direction of the image edge and the direction of the surface normal vector in the dual-modal feature fusion space to obtain the dimensionless spatiotemporal consistency weight coefficient; when the spatiotemporal consistency weight coefficient is lower than the decoupling threshold, it is determined that the corresponding feature point has a matching deviation caused by the deformation of the subject's outer clothing, thereby reducing the regression coefficient of the corresponding feature point in the coordinate correction regression model to limit the transmission of abnormal displacement errors.

[0027] Preferably, the image recognition method further includes a long-cycle tracking calculation step: obtaining multiple sets of historical human body posture assessment index values ​​of the subject at different historical acquisition times, and constructing a feature evolution time series according to the chronological order of acquisition times; calculating the position deviation slope of the feature evolution time series over time, and outputting an abnormal deviation prompt signal representing the change in body posture when the position deviation slope continuously exceeds the set fluctuation range.

[0028] Preferably, the image recognition method further includes a body posture assessment result display step: integrating the spatially continuous human body posture assessment index values, the analysis data of the feature evolution time series, and the feature data collected in the background to generate a digital assessment report text containing the spatial distribution coordinate parameters of the subject's feature points; transmitting the digital assessment report text to the interface of the display terminal for graphical layout display to provide a spatial shape feature parameter retrieval map.

[0029] Example 1: When the system faces motion tracking drift caused by false edge interference from clothing folds and multipath parasitic noise, the infrared depth acquisition component hardware synchronously outputs multi-view color image data and depth image data of the subject. The processor calculates the spatial coordinates of the subject's skeletal joints and the corresponding confidence metric from the depth image data. Simultaneously, it detects the coordinates of two-dimensional key points on the subject's body surface from the color image data and generates three-dimensional point cloud data of the human body surface from the depth image data. To eliminate fluctuations in the initial acquisition stage and obtain stable image frames, the processor discards the convergence frames of the infrared depth acquisition component during the initial acquisition stage. The number of convergence frames is not less than the number of frames obtained by multiplying the sampling frame rate by 2 seconds. Based on the confidence metric and the inter-frame displacement, the processor calculates the continuous displacement of the subject's skeletal joints in the trunk and the continuous displacement of the skeletal joints in the limbs. When the skeletal joints in the trunk... When the continuous displacement of the iliac joint is less than 5 mm and the continuous displacement of the skeletal joints of the limbs is less than 8 mm, the corresponding image frame is determined as a stable frame and included in the stable frame set. The number of effective frames in the stable frame set is controlled to be greater than or equal to 60 frames. The determination thresholds of 5 mm and 8 mm are based on the measurement uncertainty of the infrared depth acquisition component at a frame sampling rate of 30 Hz and a spatial depth resolution of 1 mm, and are derived in conjunction with the physiological static boundary value of the human body. In actual engineering calibration, when the subject maintains a normal static standing posture, due to unavoidable breathing movements and spontaneous slight swaying of the core muscle groups, the maximum physiological displacement of the skeletal joints of the trunk between adjacent frames is within 3.5 mm. As for the distal joints of the limbs, due to the leverage amplification effect of the long bones, the amplitude of the occasional physiological swaying at the ends is within 6 mm.Combining the discrete measurement uncertainty variance introduced by the inherent 1mm spatial depth resolution of the acquisition components, and by synthesizing the physiological displacement amplitude and measurement uncertainty with standard uncertainty, the upper limits for continuous inter-frame displacement judgment of the torso and limbs are scientifically limited to 5mm and 8mm, respectively. This serves as an absolute engineering criterion for filtering out active limb shaking of the subject and occasional timing jitter of the equipment. Below this threshold, the current image frame is confirmed to be in a high-purity, stable acquisition state; above this threshold, it means the image frame is subject to non-steady-state motion interference, and the system will not include it in the stable frame set. To limit the influence of low-quality image sequences, when the processor detects that the number of valid frames in the stable frame set is small... At frame 30, the current posture assessment process is stopped and an alarm message indicating insufficient data acquisition quality is output. When the percentage of frames in which the confidence metric of a joint point in the stable frame set is greater than or equal to a preset confidence threshold is less than 60%, the processor automatically reduces the weight coefficient of that joint point in subsequent feature fusion and coordinate correction, and marks the downstream indicator corresponding to that joint point as a low-confidence acquisition state in the posture assessment report. The processor selectively converts the spatial coordinates of skeletal joint points into three-dimensional spatial coordinates of surface anatomical landmarks based on the anatomical type of the target surface anatomical landmarks. For surface anatomical landmarks such as the spinous process of the seventh cervical vertebra, the acromion, and the tragus, which have a linear approximation of offset, the processor uses a method based on... The linear mapping relationship with fixed bias or height ratio bias is used to calculate the spatial coordinates of the body surface. For pelvic surface landmarks such as the anterior superior iliac spine or posterior superior iliac spine, which involve three-dimensional rotation and individual morphological differences, the processor uses a spatial affine transformation matrix to register the spatial coordinates of skeletal joints with a three-dimensional human general mesh model. Then, it converts them into the corresponding spatial coordinate sequence of body surface feature points through a rigid body affine transformation mapping relationship based on statistical shape model and singular value decomposition. The process of solving the spatial affine transformation matrix using singular value decomposition includes: constructing a current skeletal coordinate matrix composed of the three-dimensional spatial coordinates of multiple skeletal joints of the currently measured subject, and the corresponding three-dimensional coordinates of the skeletal joints preset in the three-dimensional human general mesh model. The processor constructs a standard template coordinate matrix. It then calculates the geometric centroids of the corresponding point sets in these two coordinate matrices and subtracts the coordinates of their respective centroids to obtain two decentralized sets of bone point coordinates. A 3x3 cross-covariance matrix is ​​constructed by calculating the sum of the spatial vector cross products of the decentralized standard template coordinate vector and the current bone coordinate vector. Next, the processor performs singular value decomposition on this 3x3 cross-covariance matrix, decomposing it into a left singular vector matrix, a singular value diagonal matrix, and the transpose of the right singular vector matrix. Finally, the optimal rigid body rotation matrix is ​​directly calculated by performing matrix multiplication between the right singular vector matrix and the transpose of the left singular vector matrix.Finally, by combining the displacement difference vector between the geometric centroids of the two point sets, a spatial affine transformation matrix containing rotation and translation dimensions is obtained, achieving accurate rigid body registration between the spatial coordinates of the skeletal joints and the mesh model.

[0030] For anatomical landmarks related to spinal curvature indices such as thoracic kyphosis or lumbar lordosis, the processor registers 3D point cloud data of the human body surface with a 3D general human body mesh model. It calculates the spatial curvature variation characteristics of the longitudinal bending trajectory corresponding to the central region of the subject's back, thereby extracting representative points at the level of three vertebrae along the spinal groove trajectory. A cubic B-spline curve is then used to fit the global spatial curve of these representative points. Subsequently, the equivalent angle index is calculated by obtaining the inverse cosine value of the tangent vector at the endpoints of the global spatial curve. This process eliminates measurement errors caused by respiratory movements and occasional limb microswaying. To address noise, the processor performs joint-by-joint Kalman filtering on the calculated spatial coordinate sequence of body surface feature points for time-series smoothing. During the smoothing process, the state vector is set to a six-dimensional vector containing three-dimensional position and velocity components. The size of the measurement noise covariance matrix in the filtering program is adjusted based on the confidence metric. Combined with the median of the position coordinates of each body surface feature point in the stable frame set, the coordinates of the smoothed and stabilized spatial marker points are calculated and output. In practice, to eliminate the dimensionality discrepancy between the dimensionless confidence metric and the dimensionless measurement noise covariance matrix... For matching, the processor performs the following cross-dimensional bridging and dimensionless adjustment: Based on the 1mm spatial resolution performance of the depth camera, a baseline measurement noise covariance matrix with the dimension of length square is pre-defined, and the initial noise elements representing the three-dimensional position components on its diagonal are all set to the square of 1.0mm; During real-time smoothing filtering, the processor reads the dimensionless confidence metric value between 0.0 and 1.0 output of the current frame, calculates the reciprocal of this confidence metric value as a scalar adjustment coefficient; Finally, the processor combines this dimensionless scalar adjustment coefficient with the aforementioned baseline measurement noise covariance matrix with the dimension of length square. The diagonal elements of the quasi-measurement noise covariance matrix are directly multiplied to dynamically scale and adjust the measurement noise covariance matrix of the current frame online. When the confidence metric is at its maximum value of 1.0, the scalar adjustment coefficient is 1.0, and the matrix maintains its baseline size. When the confidence metric is reduced to 0.2 due to clothing pseudo-edge noise interference, its reciprocal scalar adjustment coefficient is automatically increased to 5.0, thereby amplifying the measurement noise covariance matrix by a factor of 5. This causes the Kalman filter gain to automatically decrease, making the system more inclined to trust the state prediction sequence of the previous moment under the premise of uniformity of dimensions, thus eliminating jump noise.

[0031] After initial smoothing, the processor projects the coordinates of the two-dimensional key points on the body surface into the same three-dimensional coordinate system as the skeletal joint coordinates, using the internal and external parameters calibrated at the factory by the depth camera. It then calculates the cosine of the collinearity angle between the gradient direction of the image edges in the color image data and the direction of the surface normal vector in the depth image data, thus obtaining the spatiotemporal consistency weight coefficient. When the spatiotemporal consistency weight coefficient is lower than the set decoupling threshold, it indicates the presence of pseudo-semantic edge interference caused by clothing wrinkles. The processor then reduces the regression weight of the corresponding two-dimensional key point coordinates on the body surface in the coordinate correction regression model and uses the calculated coordinate residuals to correct the non-rigid deformation parameters of the three-dimensional human body general mesh model within the overlapping area. This allows the three-dimensional point cloud data of the human body surface to serve as a spatial positioning coordinate reference in dense point cloud regions. The coordinates of two-dimensional key points on the body surface serve as supplementary localization vectors in the image edge region, completing dual-modal complementary correction. When calculating the cosine of the collinearity angle, in order to eliminate the dimensional discontinuity between two-dimensional and three-dimensional geometric quantities, the processor obtains the two-dimensional image edge gradient direction vector in the neighborhood of the two-dimensional key points on the body surface in the color image. This vector is composed of the first-order spatial directional derivative components of the pixel gray value in the horizontal and vertical directions. The processor uses the camera's intrinsic and extrinsic parameter matrices to back-project and reconstruct the pixel coordinates of the two-dimensional key point and the corresponding depth image depth value into a three-dimensional spatial point, and calculates the tangent plane basis vector of the point in the three-dimensional point cloud manifold. By performing a three-dimensional spatial projection transformation on the two-dimensional image edge gradient direction vector along the tangent plane basis vector, it is mapped into a three-dimensional edge gradient direction vector in the three-dimensional geometric space.Finally, the processor calculates the dot product between the mapped edge gradient direction 3D vector and the surface normal direction 3D vector obtained from the depth point cloud through local least squares fitting, and then divides it by the product of the magnitudes of the two 3D vectors to solve for the absolute value of the cosine of the collinear angle that is completely in the same 3D spatial dimension, thus obtaining the dimensionless spatiotemporal consistency weight coefficient. Based on the spatial coordinate sequence of body surface feature points output after dual-modal complementary correction, the processor calculates the subject's multi-dimensional postural angle indices, including at least one of the following calculated using linear mapping relationships: head and neck forward tilt angle, shoulder height difference angle, pelvic tilt angle, or knee-tibia-femoral angle. The pelvic tilt angle, thoracic kyphosis angle, or lumbar lordosis angle calculated using rigid body affine transformation mapping or cubic B-spline curve fitting are determined by the following: Since the spatial coordinates of these marker points contain background data such as software development kit version, camera firmware version, processor model, and image acquisition timestamps, the processor can not only perform long-term longitudinal tracking comparison and hardware fingerprint consistency verification through database storage, but also combine the calculated confidence interval and the 95% minimum detectable change value to classify and label each body angle indicator according to the credibility of the adopted proxy mapping strategy. The final output includes indicator values, confidence intervals, and the most... The small-scale structured body posture assessment report detects changes in values ​​and reliability levels, eliminates coordinate jumps at modal boundaries, maintains the continuity of the global anatomical feature spatial geometric manifold, and ensures the numerical stability of body posture assessment indicators. During long-term tracking and comparison, the processor extracts historical body posture assessment indicator values ​​collected multiple times within the past six months from the database in chronological order, constructing a feature evolution time series. A univariate linear regression equation is then used to calculate the slope of the feature evolution time series' positional deviation over time. The absolute value threshold for the set fluctuation range is specifically set at a 0.05° angular positional deviation every 30 days. This threshold... When the deviation slope is less than 0.05° every 30 days at the boundary of the value range, the system determines that it is within the range of normal daily physiological swaying of the human body and the inherent noise measurement error of the equipment. However, when the position deviation slope calculated by the processor continuously exceeds this fluctuation range, and the single body angle index collected for three consecutive times exceeds the theoretical extreme value boundary of the aforementioned minimum detectable change value of 95%, the system determines that the subject's body structure has undergone organic or habitual skeletal deviation variation. The processor then outputs a deviation abnormality prompt signal representing the change in body posture in the digital assessment report text, thereby establishing a noise-resistant digital benchmark for long-term clinical comparison.

[0032] Example 2: Under conditions of interference from the false edges of the outer covering of the subject and multipath parasitic spatial noise, the experimental platform uses an infrared depth camera array with parameters of 30Hz frame sampling rate, 1mm spatial depth resolution, and a maximum measurement range of 5m. An external laser interferometer tracker with a measurement accuracy of 0.05mm is simultaneously introduced as a spatial coordinate reference calibration source. The test data sources include publicly available human body posture datasets and on-site multi-view entity acquisition image sequences. Under these conditions, the fluctuation duration of the initial segment of the joint point spatial coordinate acquisition is... The selection thresholds for the stable frame set are mutually restrictive. If the sampling range is too narrow, there will be insufficient effective sampling frames. If the sampling range is too wide, hardware timing jitter and occasional slight body shaking will be introduced. To address this, the system adopts a spatiotemporal joint decoupling method. By limiting the continuous displacement of the bone joints of the torso to less than 5 mm and the continuous displacement of the bone joints of the limbs to less than 8 mm, and continuously discarding the number of frames obtained by multiplying the sampling frame rate by 2 seconds at the beginning of the acquisition phase of the software development kit, a stable sampling interval is established between image stream utilization and parasitic interference suppression.

[0033] The experiment included a test group using the method of this invention, a first control group with the spatiotemporal consistency weight coefficient calculation program removed, and a second control group with the continuous inter-frame displacement threshold of the skeletal joints of the torso expanded to 20 mm and without removing convergence frames. Based on the number of layers of the subject's outer covering and the difference in surface wrinkle protrusion density, a first interference gradient, a second interference gradient, and a third interference gradient were set as boundary conditions for the test input. During the data acquisition phase, the processor simultaneously received depth image data including multipath noise and color image data output by the infrared depth acquisition component. The depth image data here exhibited a state of continuous distortion of local geometric manifold boundaries due to interference from the edges of the subject's clothing wrinkles. Under the third interference gradient, the input data streams of the first control group, the second control group, and the test group using the method of this invention were all at the functional level of modal feature boundary aliasing.

[0034] Under the first interference gradient, the first control group, the second control group, and the test group using the method of this invention all output the subject's body angle index. As the interference from the package increases from the first interference gradient to the third interference gradient, the second control group, lacking convergence frame filtering and continuous inter-frame displacement limiting, experiences high-frequency disturbances accumulating in the coordinate calculation loop, resulting in continuous jumps in the spatial coordinates of skeletal joints. This increases the calculated deviation of the thoracic kyphosis angle to 5.34°, and the number of effective frames in the stable frame set drops below 30. The first control group, lacking constraints on the collinearity of edge gradient directions in color image data and surface normal vector directions in depth image data, uses its coordinate correction regression model to directly use the coordinates of two-dimensional key points on the body surface at the fold edge pixels as the spatial iteration reference for anatomical landmarks, leading to an increase in the calculation error of the head and neck forward tilt angle to 4.62°. In contrast, the test group using the method of this invention shows a higher error in the third interference gradient. Under the perturbation gradient condition, the cosine of the collinearity angle between the edge gradient direction of the color image data calculated by the processor and the surface normal direction of the depth image data decreases. The output spatiotemporal consistency weight coefficient decreases from 0.92 under the first perturbation gradient to 0.23. The coordinate correction regression model adjusts the regression weight of the corresponding two-dimensional key point coordinates on the body surface according to the spatiotemporal consistency weight coefficient, and uses the coordinate residuals in the spatially overlapping area to correct the non-rigid deformation parameters of the three-dimensional human body general mesh model. This ensures that the spatial coordinate sequence of the body surface feature points remains converged to the spatial coordinate baseline of the external laser interferometer tracker in the dense point cloud region. Finally, the calculation error of the head and neck forward tilt angle is maintained at 0.42°, and the measurement variation coefficient of the shoulder height difference angle is reduced to 1.15%. This establishes that the complementary constraints of the spatial geometric data of the depth flow and the two-dimensional detail features of the color flow can maintain the continuity of the spatial geometric manifold of the anatomical features under the edge aliasing state.

[0035] Example 3: When the system faces the technical problem of signal drift in skeletal vertex tracking and increased uncertainty in edge positioning caused by the subject's body shape being at the edge of its limit or by abnormally loose coverings, the infrared depth acquisition component outputs multi-view color image data and depth image data of the subject. The processor calculates the spatial coordinates of skeletal joints and the corresponding confidence metric from the depth image data, and simultaneously detects the coordinates of two-dimensional key points on the body surface from the color image data. To expose the spatial manifold transformation path between the spatial coordinates of skeletal joints and the anatomical landmarks on the body surface in a white-box manner, the system calls a parameter-driven overall topology template framework. The construction and update cycle of the overall topology template framework includes three stages: In the offline setup phase, a large-scale, high-resolution 3D human surface topology database is processed. Principal component decomposition is used to extract principal component control vectors and orthogonal deformation bases representing shape changes, thereby constructing a 3D universal human mesh model under a standard body shape. In the online matching phase, after the processor obtains the spatial coordinates of skeletal joints from the subject's current depth image data, a spatial affine transformation matrix is ​​calculated using a rigid body affine transformation mapping relationship based on singular value decomposition. This matrix aligns the skeletal joints in the 3D universal human mesh model with the currently measured spatial coordinates of the skeletal joints, aligning the entire 3D universal human mesh model to the subject's actual spatial orientation. In the dynamic correction phase, to offset... The local morphological non-rigid distortion caused by the covering object is addressed by the processor running an optimization loop based on the iterative nearest-point algorithm. This loop calculates the spatial shortest distance residual between the 3D point cloud data of the human body surface generated from depth image data and the vertices of the 3D general human body mesh model. A statistical regularized smoothing constraint operator is introduced to solve and correct the non-rigid deformation parameters of the 3D general human body mesh model. Specifically, the statistical regularized smoothing constraint operator is implemented by introducing a Dirichlet energy penalty term into the overall objective function of the optimization loop. In each cascaded optimization iteration based on the iterative nearest-point algorithm, this operator extracts the topological connectivity between adjacent vertices on the surface of the general mesh model and calculates the non-rigid deformation parameters of adjacent vertices under the current deformation. The first-order spatial difference of the rigid deformation displacement vector, i.e. the first-order spatial variation of the grid Laplacian coordinates, has a constraint penalty mechanism: when the non-rigid deformation of adjacent vertices causes a spatial abrupt change or step due to the protrusion of the loose outer wrapping of the subject, the absolute value of the corresponding first-order spatial difference increases, thereby causing a surge in the value of the Dirichlet energy penalty term. At this time, the optimization loop solves the global minimum of the overall objective function containing the penalty term and applies stiffness smoothing constraints to the spatial first-order partial derivatives of the non-rigid deformation parameters, so that the non-rigid deformation trend between adjacent vertices maintains the continuity and smoothness of the local spatial manifold, thereby automatically filtering out the non-rigid high-frequency pseudo-distortion noise caused by the wrinkles of the wrapping in the algorithm flow.

[0036] To establish the determination process for spatiotemporal consistency weight coefficients and eliminate the implicit parameter determination structure in computational loops, the system deploys an explicit data flow operator path: Based on inherent hardware parameters, the processor back-projects the pixel coordinates in the color image data to the three-dimensional spatial coordinate system containing the spatial coordinates of the skeletal joints; the processor establishes a pixel-level neighborhood search window in the color image data, calculates the spatial directional derivative of local grayscale values ​​to obtain the gradient direction components of the image edges, and simultaneously calculates the corresponding surface by applying least-squares fitting to the local tangent planes of the three-dimensional point cloud in the depth image data. Normal vector direction component; the processor calculates the cosine of the collinearity angle between the image edge gradient direction component after spatial rotation transformation and the surface normal vector direction component, thus generating a spatiotemporal consistency weight coefficient; the decoupling threshold is based on the background stray measurement noise variance of the infrared depth acquisition component under standard plane calibration environment. In the test, when the cosine of the collinearity angle is lower than the decoupling threshold set at 0.30, it indicates that the current observation point is interfered with by the false edge of the outer wrapping material. The processor runs an adaptive weight decay operator to reduce the regression weight of the corresponding two-dimensional key point to 20% of the original weight. The non-rigid deformation parameters of the 3D human general mesh model are corrected by utilizing the 3D point cloud coordinate residuals in the spatially overlapping region. This allows for the unfolding of closed-loop constraint logic at the intersection of bimodal features. Specifically, the coordinate correction regression model employs a weighted least squares regression algorithm. The input data of this model includes the target 3D coordinate vector formed by back-projecting the 2D key points on the body surface into 3D space using calibration parameters, the current 3D coordinate vector of the corresponding control vertex of the 3D human general mesh model, and the diagonal weight matrix formed by arranging the reduced regression weights diagonally. The loss function of this regression model is defined as the sum of the square of the spatial Euclidean distance between each control vertex and the target 3D coordinates and the product of the corresponding regression weight. In the dynamic optimization loop for correcting non-rigid deformation parameters, the processor calculates the spatial coordinate residual matrix between each key point pair and performs matrix multiplication with the diagonal weight matrix. Using the optimization principle of finding the minimum value of the least squares objective function, the processor solves inversely for the incremental vector of non-rigid deformation parameters of the global control vertex of the mesh model, thereby outputting the corrected global anatomical feature spatial mesh model and ensuring the continuity of the global geometric manifold.

[0037] With the coordinated operation of the aforementioned white-box data flow path and adaptive adjustment factors, when the test group progresses to a complex scenario with multipath parasitic spatial noise, the processor stably calculates the corresponding spatial coordinate sequence of anatomical landmarks on the body surface, controlling the coordinate jump phenomenon at the modal boundary. The calculated output data of head and neck forward tilt angle, shoulder height difference angle, pelvic tilt angle, and thoracic kyphosis angle fully conform to the monotonicity of the subject's actual body geometry. The calculated confidence interval and the 95% minimum detectable change value are both within the preset quality and safety boundaries. Finally, a structured body assessment report containing the continuity of the global anatomical feature spatial geometric manifold is generated, maintaining the feasibility and data self-consistency of the technical solution under extreme conditions.

[0038] Example 4: When the system faces variations in infrared speckle reflectance and multipath parasitic spatial noise in a newly deployed acquisition environment, in order to eliminate the inherent measurement bias of the infrared depth acquisition component under different physical fields, the system performs a pre-deployment calibration procedure before receiving multi-view color image data and depth image data: The infrared depth acquisition component hardware synchronously outputs a continuous and stable frame sequence for a preset standard plane calibration object. The processor extracts the depth measurement point cloud of the central region from the original depth image corresponding to the stable frame sequence and performs statistical calculations on the spatial variance of the depth measurement point cloud. When the calculated spatial variance is less than the stability judgment threshold of 0.02mm, the processor establishes the average depth in the current continuous acquisition frame as the spatial reference calibration parameter of the infrared depth acquisition component in this deployment environment, and then deducts the spatial reference calibration parameter in full from the data stream to complete the in-situ physical correction of the hardware.

[0039] To further block the propagation of multipath parasitic noise at the edges caused by different deployment environments, the processor performs spatial smoothing enhancement processing on the depth image data after in-situ hardware physical correction. This involves analyzing the peak distribution pattern of the depth grayscale histogram obtained by the infrared depth acquisition component and dynamically associating the step gradient of the edge contrast enhancement operator with the histogram. Figure 1 The zero-crossing time constant of the first derivative is used, and the spatial gain operator, which has been fixed by pre-fitting offline, is automatically loaded in the pixel-level edge regression model. This adaptively maintains the feature sensitivity at key points of the image edge within the preset process index threshold window, and ultimately ensures that the output spatial coordinate sequence of body surface feature points maintains the continuity of the geometric manifold in both dense point cloud regions and image detail regions, thus maintaining the numerical stability of the body posture assessment system for the analysis of the underlying acquired feature parameters.

[0040] The embodiments of this application have been described above with reference to the accompanying drawings. Unless otherwise specified, the embodiments and features in the embodiments of this application can be combined with each other. This application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit of this application and the scope of protection of this invention, and all of these forms are within the protection scope of this application.

Claims

1. An image recognition method for human body posture assessment, characterized in that, Includes the following steps: Step S1: Acquire multi-view color image data and depth image data of the subject through hardware synchronous output of the infrared depth acquisition component; calculate the spatial coordinates of the subject's skeletal joints and the corresponding confidence metric from the depth image data, and detect the coordinates of the subject's two-dimensional key points on the body surface from the color image data. Step S2: Select stable frames according to confidence metric and inter-frame displacement to obtain a set of stable frames; use spatial affine transformation matrix to register the spatial coordinates of skeletal joints with the three-dimensional human general mesh model, and convert the spatial coordinates of skeletal joints into a sequence of spatial coordinates of body surface feature points by curve fitting of the spatial trajectory of the subject's back spine. Step S3: Calculate the cosine of the collinearity angle between the gradient direction of the image edge in the color image data and the direction of the surface normal vector in the depth image data to obtain the dimensionless spatiotemporal consistency weight coefficient; when the spatiotemporal consistency weight coefficient is lower than the set decoupling threshold, reduce the regression weight of the corresponding two-dimensional key point coordinates on the body surface in the coordinate correction regression model, and use the calculated coordinate residuals to correct the non-rigid deformation parameters of the three-dimensional human body general mesh model, and output the spatially continuous human body posture evaluation index value.

2. The image recognition method for human body posture assessment according to claim 1, characterized in that, The image recognition method includes a temporal smoothing step for the spatial coordinate sequence of body surface feature points: smoothing filters the three-dimensional position components and velocity components contained in the spatial coordinate sequence of body surface feature points, wherein the measurement noise covariance matrix in the smoothing filter is scaled and adjusted in real time according to the confidence metric; combined with the median of the position coordinates of each body surface feature point in the stable frame set, the smoothed coordinate data is output to eliminate measurement noise caused by feature point jumps and respiratory movements.

3. The image recognition method for human body posture assessment according to claim 1, characterized in that, In step S1, while acquiring multi-view color image data and depth image data, background acquisition feature data is recorded. The background acquisition feature data includes software development kit version, camera firmware version, processor model, and image acquisition timestamp. The background acquisition feature data is associated with the spatial coordinate sequence of body surface feature points and stored in the database for subsequent long-term longitudinal data comparison and verification.

4. The image recognition method for human body posture assessment according to claim 1, characterized in that, Step S2, which involves selecting stable frames, includes the following steps: Step S21, calculating the continuous displacement of the skeletal joints of the subject's torso and the continuous displacement of the skeletal joints of the limbs between adjacent image frames; Step S22, when the continuous displacement of the skeletal joints of the torso is less than 5 mm and the continuous displacement of the skeletal joints of the limbs is less than 8 mm, the corresponding image frame is determined to be a stable frame and included in the stable frame set, wherein the stable frame set contains more than or equal to 60 frames.

5. The image recognition method for human body posture assessment according to claim 4, characterized in that, The image recognition method also includes introducing a quality verification control step in the stable frame set: when the number of frames in the stable frame set is less than 30, the current evaluation process is terminated and a prompt message indicating insufficient data acquisition quality is output; when the proportion of frames in the stable frame set whose confidence metric value is greater than or equal to the set confidence threshold value is less than 60%, the weight coefficient of the corresponding feature point in subsequent calculations is reduced.

6. The image recognition method for human body posture assessment according to claim 1, characterized in that, In step S2, the spatial coordinates of the skeletal joints are registered with the three-dimensional human general mesh model using a spatial affine transformation matrix, and curve fitting is performed on the spatial trajectory of the subject's back spine. This includes: solving the three-dimensional point cloud data of the human body surface from the depth image data, registering the three-dimensional point cloud data of the human body surface with the three-dimensional human general mesh model, and calculating the spatial curvature change characteristics of the longitudinal bending trajectory corresponding to the central region of the subject's back, thereby converting the spatial coordinates of the skeletal joints into a sequence of spatial coordinates of the surface feature points.

7. The image recognition method for human body posture assessment according to claim 1, characterized in that, In step S3, when the spatiotemporal consistency weight coefficient is lower than the set decoupling threshold, the regression weight of the corresponding two-dimensional key point coordinates on the body surface in the coordinate correction regression model is reduced, and the non-rigid deformation parameters of the three-dimensional human body general mesh model are corrected using the calculated coordinate residuals. This includes: calculating the absolute value of the cosine of the collinear angle between the gradient direction of the image edge and the direction of the surface normal vector in the dual-modal feature fusion space to obtain the dimensionless spatiotemporal consistency weight coefficient; when the spatiotemporal consistency weight coefficient is lower than the decoupling threshold, it is determined that the corresponding feature point has a matching deviation caused by the deformation of the subject's outer clothing, thereby reducing the regression coefficient of the corresponding feature point in the coordinate correction regression model to limit the transmission of abnormal displacement errors.

8. The image recognition method for human body posture assessment according to claim 1, characterized in that, Image recognition methods also include long-term tracking and calculation steps: obtaining multiple sets of historical human body posture assessment index values ​​of subjects at different historical acquisition times, and constructing a feature evolution time series according to the chronological order of acquisition times; The slope of the positional deviation of the feature evolution time series over time is calculated. When the slope of the positional deviation continues to exceed the set fluctuation range, an abnormal deviation warning signal representing the change in body posture is output.

9. The image recognition method for human body posture assessment according to claim 3, characterized in that, The image recognition method also includes a body posture assessment result display step: by integrating the spatially continuous human body posture assessment index values, the analysis data of the feature evolution time series, and the feature data collected in the background, a digital assessment report text containing the spatial distribution coordinate parameters of the subject's feature points is generated; the digital assessment report text is transmitted to the interface of the display terminal for graphical layout and display, so as to provide a spatial shape feature parameter retrieval map.

Images