An AI vision-based human motion posture correction method, device and medium

Abstract

This invention discloses a method, device, and medium for correcting human motion posture based on AI vision, relating to the field of AI vision technology. The method includes: acquiring multi-channel image sequences of human posture using multiple cameras; performing temporal alignment and normalization on the multi-channel image sequences; obtaining a human probability map using a pre-trained human semantic segmentation network; performing morphological cleanup to obtain a human silhouette sequence; performing back-projection according to camera intrinsic and extrinsic parameters to obtain a 3D joint sequence from a 3D AI visual volume; extracting the human motion manifold to perform contact-sensing neural motion priors and outputting prior signals; and using posture feasibility scores and joint importance weights to weightedly fuse the geometric difference vector and the center of gravity correction component to obtain a comprehensive joint correction vector. This invention generates a 3D AI visual volume through back-projection and extracts the central skeleton, using constant bone segment lengths for constraint and correction, thus enhancing the reliability of posture stability analysis and alignment with standard action templates.

Description

Technical Field

[0001] This invention relates to the field of AI vision technology, and in particular to a method, device and medium for correcting human motion posture based on AI vision. Background Technology

[0002] In recent years, human pose estimation and 3D motion analysis have gradually become important research directions in AI vision and intelligent interaction. With the development of multi-view imaging technology and deep learning semantic segmentation methods, researchers can use multiple cameras to acquire image sequences and extract human body regions through semantic segmentation networks to achieve foreground modeling and motion analysis of the human body. Existing methods often combine optical flow estimation, edge detection, and mid-axis skeleton extraction, providing fundamental support for subsequent action recognition, virtual interaction, and sports rehabilitation assessment.

[0003] However, existing technologies generally suffer from two shortcomings: First, in the fusion of multi-camera sequences, conventional methods fail to adequately address brightness differences and noise interference, resulting in insufficient cross-view data consistency and thus affecting the accuracy of human foreground segmentation. Second, existing segmentation and skeleton extraction methods lack refined corrections in edge processing and motion consistency screening, often resulting in false foregrounds, edge burrs, or holes, thereby reducing the accuracy of silhouette sequences and 3D skeletons. These shortcomings directly limit the reliability of human stability assessment and complex motion template alignment. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides a human motion posture correction method based on AI vision to solve the problems of insufficient segmentation consistency and 3D skeleton extraction accuracy in multi-camera human sequences.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: In a first aspect, the present invention provides a method for correcting human motion posture based on AI vision, comprising, Multiple image sequences of human poses are acquired by multiple cameras. The multiple image sequences are time-aligned and normalized. A pre-trained human semantic segmentation network is used to obtain a human probability map and generate a foreground human mask. The foreground human mask is morphologically cleaned by acquiring basic motion cues to obtain a human silhouette sequence. The human silhouette sequence is back-projected according to the camera's intrinsic and extrinsic parameters to generate a 3D AI visual volume. The 3D central skeleton is extracted from the 3D AI visual volume, and the joints are located to obtain a 3D joint sequence. The foot contact points in the 3D joint sequence are projected onto the ground plane to obtain the supporting polygon. By extracting the human motion manifold of the three-dimensional joint coordinate sequence, contact perception neural motion priors are obtained and prior signals are output. The geometric difference vector and physical consistency correction component between the preset standard motion template and the real-time 3D joint coordinate sequence are calculated. The geometric difference vector and the center of gravity correction component are then weighted and fused using the posture feasibility score and joint importance weight to obtain the joint comprehensive correction vector.

[0007] As a preferred embodiment of the AI ​​vision-based human motion posture correction method of the present invention, the specific steps for generating the foreground human body mask are as follows: The probability values ​​of the human body probability map are quantized into several gray levels. The probability value of each pixel is linearly mapped to the gray level, and the probability representative value of each gray level is calculated. The number of pixels falling into each gray level is counted on each frame of the image, and a frequency histogram is drawn. Several points are selected from the frequency histogram to divide the pixels of each frame of the image into foreground human body points and background points, thus obtaining candidate segmentation points; The candidate segmentation point is determined as a valid segmentation point by calculating the cumulative frequency and cumulative first moment on both sides of the candidate segmentation point. Calculate the inter-class variance for each effective split point, and take the effective split point with the largest inter-class variance as the best split point; The probability representation value of the optimal segmentation point is mapped back to the probability value, and the human body probability map is binarized to generate a foreground human body mask.

[0008] As a preferred embodiment of the AI ​​vision-based human motion posture correction method of the present invention, the specific steps for extracting the three-dimensional central skeleton from the three-dimensional AI visual volume are as follows: Calculate the minimum Euclidean distance from each voxel in the 3D AI vision volume to the boundary of the 3D AI vision volume, and calculate the gradient vector of each voxel based on the minimum Euclidean distance to extract the ridge points. Using the ridge point as the seed point, the gradient flow direction is followed point by point and connected to form a continuous three-dimensional central skeleton.

[0009] As a preferred embodiment of the AI ​​vision-based human motion posture correction method of the present invention, the specific steps for obtaining a three-dimensional joint sequence through joint localization are as follows: Within the 3D central skeleton, skeleton endpoints, skeleton bifurcation points, and potential joints are statistically analyzed as joint candidate points. Based on the human standing motion, obtain the three-dimensional joint coordinates of the standing position and calculate the bone segment length constant; By using the bone segment length constant to correct the joint candidate points, a three-dimensional joint coordinate sequence is obtained.

[0010] As a preferred embodiment of the AI ​​vision-based human motion posture correction method of the present invention, the specific steps of extracting the human motion manifold of the three-dimensional joint coordinate sequence to perform contact perception neural motion prior and outputting the prior signal are as follows. A motion posture evaluation model is constructed based on neural networks; Generate human motion manifold vectors in the encoder of the motion posture evaluation model; In the decoder of the motion posture evaluation model, the dimension of the human motion manifold vector is gradually expanded, and the three-dimensional joint coordinate sequence is reconstructed under the constraint of constant bone segment length. In the posture feasibility branch of the motion posture evaluation model, the Gaussian kernel similarity between the three-dimensional joint sequence and the reconstructed three-dimensional joint sequence is calculated as the posture feasibility score output based on the Euclidean distance between the three-dimensional joint sequence and the reconstructed three-dimensional joint sequence. In the joint attention branch of the motion posture evaluation model, the weights of joint importance are assigned using a normalized exponential distribution based on the latent vectors.

[0011] As a preferred embodiment of the AI ​​vision-based human motion posture correction method of the present invention, the specific steps for obtaining basic motion cues are as follows: The foreground human body image sequence is cropped pixel by pixel based on the foreground human body mask. The second moment within each pixel window of the foreground human body image sequence is calculated to obtain the corner points. Based on the spatial and temporal gradients of the corner points, the least squares solution is used to solve the sparse optical flow vector of each corner point, and the sparse optical flow vector field is reconstructed in the entire foreground human body region. Based on the gradient magnitude of the spatial gradient at the corner points, calculate the mean and standard deviation, and then filter the set of contour points. The set of contour points is projected onto a multi-channel image sequence to generate a human body contour edge map.

[0012] As a preferred embodiment of the AI ​​vision-based human motion posture correction method of the present invention, the specific steps for morphological cleaning of the foreground human body mask are as follows: An intermediate mask is obtained by filling the gaps and holes in the foreground human body mask; Based on the sparse optical flow vector field, the intermediate mask is screened for optical flow consistency to obtain a motion-consistent mask; By using the human body contour edge map to correct the edge consistency of the motion-consistent mask, a sequence of human silhouettes is obtained.

[0013] As a preferred embodiment of the AI ​​vision-based human motion posture correction method of the present invention, the physical consistency correction component refers to the distance between the projection of the human centroid and the supporting polygon. The basic motion cues include a sparse optical flow vector field and a human body contour edge map; The prior signals include posture feasibility scores and joint importance weights.

[0014] In a second aspect, the present invention provides a computer device including a memory and a processor, wherein the memory stores a computer program, wherein when the computer program is executed by the processor, it implements any step of the AI ​​vision-based human motion posture correction method described in the first aspect of the present invention.

[0015] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the AI ​​vision-based human motion posture correction method described in the first aspect of the present invention.

[0016] The beneficial effects of this invention are as follows: By combining optical flow consistency and edge correction mechanisms, this invention can effectively eliminate static false foregrounds and compensate for mask defects, ensuring the continuity and boundary integrity of the human silhouette sequence. On this basis, a three-dimensional AI visual volume is generated by back projection and the central skeleton is extracted. Constraints and corrections are made in conjunction with bone segment length constants, which can significantly improve the accuracy and stability of joint positioning. This not only improves the accuracy of the three-dimensional joint sequence but also enhances the reliability of posture stability analysis and alignment with standard action templates, providing more robust support for intelligent interaction and motion safety assessment. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart of a human motion posture correction method based on AI vision.

[0019] Figure 2 This is a flowchart of the joint integrated correction vector.

[0020] Figure 3 A flowchart for generating a foreground human body mask and human silhouette sequence.

[0021] Figure 4 A flowchart for constructing a 3D AI visual object and extracting its skeleton joints. Detailed Implementation

[0022] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0023] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0024] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0025] Reference Figures 1-4 This is one embodiment of the present invention, which provides a human motion posture correction method based on AI vision, including the following steps: S1. Acquire multiple image sequences of human poses through multiple cameras, perform time alignment and normalization on the multiple image sequences, obtain human probability maps using a pre-trained human semantic segmentation network, generate foreground human masks, obtain basic motion cues, perform morphological cleanup on the foreground human masks, and obtain human silhouette sequences.

[0026] Among them, normalizing the multi-image sequence means that for each image sequence, the mean and standard deviation of the pixel intensity of the image sequence within the time window are used as statistics to normalize the pixel intensity of the multi-image sequence so that the brightness distribution of different cameras is within the same scale range. Using pre-trained human semantic segmentation networks, such as DeepLab, HRNet, and BodyPix, the probability value of each pixel belonging to the human body in a multi-channel image sequence is obtained. Taking DeepLab as an example, the multi-channel image sequence is scaled according to the input size of DeepLab, and color conversion and channel normalization are performed. The encoding and decoding structure of DeepLab is used to reason forward frame by frame and pixel by pixel to obtain the category tensor, including the human body tensor and the background tensor. The category tensor is probabilistically converted by the activation function Softmax to obtain the pixel category probability value, including the probability value of the pixel belonging to the background and the probability value of the pixel belonging to the background. The probability value of each pixel belonging to the human body is arranged according to the spatial position to obtain the human body probability map. The probability values ​​in the human body probability map are converted into binary labels to generate a foreground human body mask. The specific steps are as follows: The continuous interval of probability values ​​in the human probability map is uniformly quantized into several gray levels, that is, each gray level represents an interval. The probability value of each pixel is linearly mapped to the gray level. The mean of the probability values ​​at the two boundaries of the gray level is used as the probability representative value of the gray level. The number of pixels falling into each gray level is counted on each frame of the image, and a frequency histogram is drawn. Based on the total number of pixels in each frame of the image, the frequency histogram is normalized to obtain the frequency histogram distribution, and the first moment of the frequency histogram is calculated. In the frequency histogram, several gray levels are selected as candidate segmentation points. The pixels of each frame image are temporarily divided into foreground human figures and background figures. That is, pixels with a probability representation value not less than the gray level of the candidate segmentation point are temporarily classified as foreground human figures, and pixels with a probability representation value less than the gray level of the candidate segmentation point are temporarily classified as background figures. The probability representation values ​​corresponding to the gray levels at both ends cannot be used as candidate segmentation points. The cumulative frequency and cumulative first moment on both sides of the candidate segmentation point, i.e., the foreground human figure side and the background figure side, are calculated as follows: ; ; ; ; In the formula, and These represent the cumulative frequencies on one side of the foreground human figure and on the other side of the background figure, respectively. and Let represent the cumulative first moments on one side of the foreground human figure point and on the other side of the background point, respectively. The frequency histogram of the number of times The normalized frequency of the i-th gray level, i.e. the i-th The ratio of the number of pixels in each gray level to the total number of pixels in the image. For the first The probability representative value of each gray level. The first moment of the frequency histogram, For grayscale indexing, Number of gray levels; Candidate segmentation points whose cumulative frequencies on both sides are greater than 0 are determined as valid segmentation points. Calculate the ratio of the cumulative first moment to the cumulative frequency on both sides of the effective split point to obtain the conditional mean. Based on the conditional mean, cumulative frequency, and cumulative first moment, calculate the inter-class variance for each effective split point, and select the effective split point with the largest inter-class variance as the optimal split point. The calculation formula is as follows: ; In the formula, For inter-class variance, and These represent the conditional mean values ​​on one side of the foreground human figure point and on one side of the background point, respectively. The probability values ​​of the optimal segmentation points are mapped back to the probability values ​​in the human body probability map to obtain the segmentation probability values. These segmentation probability values ​​are then used to binarize the human body probability map, generating a foreground human body mask. The calculation formula is as follows: ; In the formula, The first in the foreground human body mask A mask of 1 pixel, The first in the human body probability diagram The probability value of each pixel. For the segmentation probability value, Indicates the first One pixel was determined to belong to the foreground human body area. Indicates the first Each pixel belongs to the background area; Basic motion cues are acquired within the foreground human body mask and morphological cleanup is performed to obtain a sequence of human silhouettes. The specific steps are as follows: Among them, the basic motion cues include the sparse optical flow vector field in the foreground human body region and the human body contour edge map; The multi-path image sequence is cropped pixel by pixel based on the foreground human body mask to obtain the foreground human body image sequence. The spatial gradients of the foreground human body image sequence in the horizontal and vertical directions are calculated, and a second moment is formed in the window centered on each pixel. The second moment is decomposed into features to obtain the feature values ​​in the horizontal and vertical directions. Pixels with a ratio of two feature values ​​greater than or equal to 1 are taken as corner points. Based on the brightness uniformity assumption, the temporal gradient of corner points is calculated using the optical flow constraint equation. Based on the spatial and temporal gradients of the corner points, matrix equations are established regarding the horizontal and vertical displacements of the corner points within the temporal gradient. Least squares are used to solve these matrix equations, yielding the sparse optical flow vector for each corner point. Using the corner point's optical flow vector as a fixed anchor point and the foreground human body region as the solution domain, initial values ​​(e.g., 0) are randomly defined for the sparse optical flow vectors of non-corner pixels. By applying Laplacian smoothing to the sparse optical flow vectors of non-corner pixels within the solution domain, a smoothly varying optical flow vector field is obtained within the region, where the sparse optical flow vectors of the corner points remain unchanged. To ensure the smooth continuity of the sparse optical flow vector field within the solution domain, the sparse optical flow vector of each non-corner pixel is required to be equal to the average of the sparse optical flow vectors in its neighborhood, such as a 4-neighborhood. Calculate the gradient magnitude based on the spatial gradients in the horizontal and vertical directions; Calculate the mean and standard deviation of the gradient magnitudes of all pixels in the foreground human body region, adaptively filter the contour points, and obtain the contour point set. The calculation formula is as follows: ; In the formula, For the first Contour points in an image sequence from a camera. These are the pixel coordinates. For the first Pixels in an image sequence from a camera gradient magnitude, The mean of the gradient magnitude. The standard deviation of the gradient magnitude. For camera indexing; The contour point set is projected onto a multi-channel image sequence to generate a human contour edge map. The human contour edge map is in the form of a binary mask. The mask value of the pixels belonging to the contour point set in the multi-channel image sequence is 1, and the mask value of the pixels that do not belong to the contour point set is 0. Morphological cleanup of the foreground human body mask is performed using a sparse optical flow vector field and a human body contour edge map. Specifically, by performing a closing operation on the foreground human body mask, the human body boundary is smoothed, gaps in the foreground region are filled, and sharp burrs generated by semantic segmentation are eliminated, resulting in an intermediate mask. The calculation formula is as follows: ; In the formula, For the first A middle mask of 1 pixel, For structural elements, i.e., circular neighborhoods. For the dilation operation, the foreground region expands along the shape of the structuring element. For the erosion operation, the region expanded by the dilation operation is shrunk back; The flood fill method is used to fill the holes inside the foreground region in the intermediate mask. Specifically, on the boundary pixels of the entire image, a starting point is randomly specified, and the image is continuously expanded according to the connectivity rules, with 4-neighborhood or 8-neighborhood until the boundary is encountered. All background pixels connected to the boundary are marked, and the unmarked background area is the hole. The pixels in these holes are reassigned to the intermediate mask. It should be noted that while the functions of closing and hole filling may seem to overlap, the closing function's filling ability is limited by the radius of the structuring element, and can only fill gaps or small holes. Hole filling, on the other hand, targets completely closed background areas within the foreground area, and can be understood as large holes. The two functions are complementary and do not overlap. Optical flow consistency screening of intermediate masks is performed based on sparse optical flow vector field. Specifically, the sparse optical flow vectors of all pixels in the foreground human body region are extracted from the sparse optical flow vector field, and the average optical flow vector amplitude of the pixels in the foreground human body region is calculated, which is the average value of the amplitude of the sparse optical flow vectors of all pixels. If the average optical flow vector amplitude of a certain foreground human body region is close to zero in adjacent time windows, it is determined to be a static pseudo foreground and is removed. The retained foreground human body regions constitute the temporally continuous foreground regions, and a motion-consistent mask is obtained. The motion-consistent mask is corrected for edge consistency using the human body contour edge map to obtain a human silhouette sequence. Specifically, the set of edge pixels of the foreground human body region in the motion-consistent mask is extracted, and the average intensity of the edge pixels is calculated in the local neighborhood, such as the 8-neighborhood. If the intensity of the contour points in the human body contour edge map is greater than the average intensity, it is determined that the edge pixels match the actual human body edge, and a human silhouette sequence is obtained. Otherwise, a closing operation is performed to fine-tune the boundary position.

[0027] S2. The human silhouette sequence is back-projected according to the camera's intrinsic and extrinsic parameters to generate a 3D AI visual volume. The 3D central skeleton is extracted from the 3D AI visual volume, and joint positioning is performed to obtain a 3D joint sequence. The foot contact points in the 3D joint sequence are projected onto the ground plane to obtain the supporting polygon.

[0028] The projection functions between the 3D world coordinates and 2D image coordinates of multiple synchronized cameras are established using the camera's intrinsic parameter matrix, rotation matrix, and translation vector, respectively. The calculation formula is as follows: ; In the formula, For the first Two-dimensional image pixel coordinates of the synchronized camera. For the first Depth in the coordinate system of a synchronous camera, i.e., from point to the first... The distance between the two synchronized cameras For the first The intrinsic parameter matrix of the synchronized camera, For the first Rotation matrix of the synchronous camera, In three-dimensional world coordinates, For the first The translation vector of the platform synchronization, For camera indexing; Voxels are divided in the three-dimensional world coordinate space. The center of each voxel is back-projected according to the projection function to obtain the pixel coordinates of the voxel on each camera. It is determined whether the voxel falls into the human silhouette sequence of the corresponding frame. If a voxel is in the human silhouette sequence of all cameras, it is retained; otherwise, it is removed. All retained voxels are aggregated to obtain the three-dimensional AI visual volume. Calculate the minimum Euclidean distance from each voxel in the 3D AI vision volume to the boundary of the 3D AI vision volume. By approximating the minimum Euclidean distance of adjacent voxels, calculate the gradient vector from each voxel to the boundary of the 3D AI vision volume and extract ridge points. The ridge points satisfy the following conditions: in the direction of the gradient vector, the ridge point is a local maximum, that is, the distance value no longer increases in the positive and negative directions of the gradient. In the normal plane of the gradient vector direction, both eigenvalues ​​of the Hessian matrix of the ridge point are negative. Using the ridge points as seed points, track and connect them point by point along the gradient flow direction to form a continuous 3D central axis skeleton. Based on the three-dimensional central skeleton and the human anatomical structure, the joints are initially located to obtain candidate joint points. By calculating the bone segment lengths, the candidate joint points are corrected to obtain the three-dimensional joint coordinates. The specific steps are as follows: Within the three-dimensional central skeleton, the number of adjacent points of each voxel is counted. Voxels with only one adjacent point are identified as skeleton endpoints, corresponding to the head, hands, and feet of the human body. Voxels with more than one adjacent point are identified as skeleton bifurcation points, corresponding to the shoulder joint, hip joint, and elbow joint of the human body. Based on the path relationship between the skeleton endpoints and skeleton bifurcation points, the main trunk and limbs of the human body are divided. The skeleton curves from the endpoints to the bifurcation points are obtained by connecting the three-dimensional central axis skeleton. For each point on the skeleton curve, a local window is constructed using the adjacent point set. The tangent vector of each point on the skeleton curve is calculated, and the rate of change of the tangent vector with the curve position is approximated to obtain the local curvature of each point on the skeleton curve. The curvature abrupt change points on the skeleton curve are marked. Based on human anatomy, topological matching rules are set. Specifically, the head endpoint is connected to the neck bifurcation point, the upper limb extends from the shoulder bifurcation point to the hand endpoint, the lower limb extends from the hip bifurcation point to the foot endpoint, and the trunk skeleton is represented by a curve from the neck to the pelvis. Skeleton endpoints, skeleton bifurcation points, and curvature abrupt change points are marked as joint candidate points; While maintaining the user's standing posture, acquire a sequence of standing 2D images and extract the 2D pixel coordinate sequence of the joints. Using standard linear least squares, combine the 2D pixel coordinates of the joints with the projection matrix, where the projection matrix is ​​a homogeneous linear form of the projection function, to calculate the 3D joint coordinates. The calculation formula is as follows: ; In the formula, For the standing two-dimensional image sequence, the first The first frame Three-dimensional joint coordinates of a standing unit. For the standing two-dimensional image sequence, the first The first standing two-dimensional image sequence of the synchronized camera The first frame The homogeneous standing 2D pixel coordinates of each joint For the first The projection matrix of the synchronous camera, Represents the cross product of vectors. For joint indexing, For time frame indexing; It should be noted that the right side of the above equation does not represent an identity, but rather a geometric constraint: the standing 3D joint coordinates, after being mapped by the projection matrix, should be collinear with the standing 2D pixel coordinates in homogeneous space. Based on the sequence of standing 2D pixel coordinates acquired by multiple cameras, the standing 3D joint coordinates are obtained by solving the system of equations using the standard linear least squares method. In camera imaging, the 3D points are projected onto the 2D image plane through the camera's intrinsic and extrinsic parameters. If ordinary Cartesian coordinates are used, this projection involves nonlinear division. By using homogeneous coordinates, the calculation process is transformed into matrix multiplication, simplifying it to linear algebraic operations. The bone segment length constant is obtained by calculating the Euclidean distance between the proximal and distal joint coordinates of the bone segment in three-dimensional standing position frame by frame and calculating the average value. Based on the constant length of bone segments, fixed-length projection is performed on the candidate joint points to correct them, thus obtaining a three-dimensional joint coordinate sequence. In a three-dimensional joint coordinate sequence, the joints of both feet near the ground are detected to obtain the plantar contact points. After projecting all plantar contact points onto the ground plane, the convex hull region is calculated to obtain the support polygon, which represents the plantar convex hull region. If one foot is on the ground, the support polygon is the outline of the sole of that foot; if both feet are on the ground, the support polygon is the outline of both feet. The overall center of mass of the human body is projected onto the ground plane. If the body falls within the support polygon, it indicates that the human posture is stable; otherwise, it indicates that there is a risk of tipping over. This method is used to determine whether the center of mass of the human body is within a stable range.

[0029] S3. By extracting the human motion manifold of the three-dimensional joint coordinate sequence, contact perception neural motion prior is performed, and the prior signal is output.

[0030] The prior signals include posture feasibility scores and joint importance weights; Based on the 3D joint coordinate sequence and supporting polygons, a motion posture evaluation model is constructed using a neural network to perform contact-sensing neural motion priors. The specific steps are as follows: Motion posture evaluation models include encoders, decoders, and posture evaluation networks; Specifically, the encoder maps the 3D joint coordinate sequence to a low-dimensional latent manifold space. In particular, a joint-by-joint linear layer maps the 3D joint sequence to a unified feature space. Local and global motion features are extracted in the fully connected layer. After nonlinear activation, the human motion manifold vector is output. In the decoder, the dimension of the human motion manifold vector is gradually expanded by stacking fully connected layers. Under the constraint of constant bone segment length, the 3D joint coordinate sequence is reconstructed. The pose evaluation network includes a pose feasibility branch and an attention branch. In the pose feasibility branch, the Gaussian kernel similarity between the 3D joint coordinate sequence and the reconstructed 3D joint coordinate sequence is calculated based on the Euclidean distance between the 3D joint coordinate sequence and the reconstructed 3D joint coordinate sequence as the pose feasibility score output. In the joint attention branch, the normalized exponential distribution is used to allocate the joint importance weights based on the latent vector. The importance weights of joints in different actions in the AMASS dataset are used to obtain the training dataset for the motion posture evaluation model. The training dataset is input into the motion posture evaluation model, which outputs a prior signal. The loss function of the prior signal includes the consistency loss function of the posture feasibility score and the attention regularization loss of the joint importance weights. By minimizing the loss function of the prior signal during the training process, the parameters of the motion posture evaluation model are updated using gradient descent methods, such as the Adam optimizer, until the loss function of the prior signal converges, thus completing the training.

[0031] S4. Calculate the geometric difference vector between the preset standard motion template and the real-time three-dimensional joint coordinate sequence, as well as the physical consistency correction component. Then, use the posture feasibility score and joint importance weight to perform weighted fusion of the geometric difference vector and the center of gravity correction component to obtain the joint comprehensive correction vector.

[0032] Pre-set standard motion templates, such as running motion templates, based on the AMASS dataset, include all three-dimensional joint coordinate sequences of the entire posture change process during running. By calculating the geometric difference vector of each joint in each frame between the standard motion template and the real-time three-dimensional joint coordinate sequence, the offset direction and magnitude between the current posture and the target posture can be reflected. Based on the position of the human body's center of mass and the supporting polygon, the distance between the projection of the human body's center of mass and the supporting polygon is calculated as a physical consistency correction component. When the physical consistency correction component is greater than 0, a centroid correction component is applied along the direction towards the inside of the supporting polygon; otherwise, no operation is required. Using the attitude feasibility score and joint importance weight as weighting factors, the geometric difference vector and the center of gravity correction component are weighted and fused to obtain the joint comprehensive correction vector. Based on the maximum change amplitude of joints in different movements in the preset standard movement template, as well as the maximum and minimum flexion angles of human joints, the upper limit of the correction amplitude of the joint comprehensive correction vector is set. The amplitude is clipped according to the joint comprehensive correction vector of each joint to obtain the joint comprehensive correction amplitude. If the joint comprehensive correction amplitude exceeds the upper limit of the correction amplitude, the upper limit of the correction amplitude shall be used as the standard; otherwise, the joint comprehensive correction amplitude shall remain unchanged.

[0033] This embodiment also provides a computer device applicable to the human motion posture correction method based on AI vision, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the human motion posture correction method based on AI vision as proposed in the above embodiment.

[0034] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0035] This embodiment also provides a storage medium storing a computer program, which, when executed by a processor, implements the AI ​​vision-based human motion posture correction method proposed in the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0036] In summary, this invention, by combining optical flow consistency and edge correction mechanisms, can effectively eliminate static false foregrounds and compensate for mask defects, maintaining the continuity and boundary integrity of human silhouette sequences. On this basis, it generates a 3D AI visual volume through back projection and extracts the central skeleton, using bone segment length constants for constraint and correction. This not only improves the accuracy of 3D joint sequences but also enhances the reliability of posture stability analysis and alignment with standard action templates, providing more robust support for AI vision and motion posture correction.

[0037] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for correcting human motion posture based on AI vision, characterized in that: include, Multiple image sequences of human poses are acquired by multiple cameras. The multiple image sequences are time-aligned and normalized. A pre-trained human semantic segmentation network is used to obtain a human probability map and generate a foreground human mask. The foreground human mask is morphologically cleaned by acquiring basic motion cues to obtain a human silhouette sequence. The human silhouette sequence is back-projected according to the camera's intrinsic and extrinsic parameters to generate a 3D AI visual volume. The 3D central skeleton is extracted from the 3D AI visual volume, and the joints are located to obtain a 3D joint sequence. The foot contact points in the 3D joint sequence are projected onto the ground plane to obtain the supporting polygon. By extracting the human motion manifold of the three-dimensional joint coordinate sequence, contact perception neural motion priors are obtained and prior signals are output. The geometric difference vector and physical consistency correction component between the preset standard motion template and the real-time 3D joint coordinate sequence are calculated. The geometric difference vector and the center of gravity correction component are then weighted and fused using the posture feasibility score and joint importance weight to obtain the joint comprehensive correction vector.

2. The human motion posture correction method based on AI vision as described in claim 1, characterized in that: The specific steps for generating the foreground human body mask are as follows: The probability values ​​of the human body probability map are quantized into several gray levels. The probability value of each pixel is linearly mapped to the gray level, and the probability representative value of each gray level is calculated. The number of pixels falling into each gray level is counted on each frame of the image, and a frequency histogram is drawn. Several points are selected from the frequency histogram to divide the pixels of each frame of the image into foreground human body points and background points, thus obtaining candidate segmentation points; The candidate segmentation point is determined as a valid segmentation point by calculating the cumulative frequency and cumulative first moment on both sides of the candidate segmentation point. Calculate the inter-class variance for each effective split point, and take the effective split point with the largest inter-class variance as the best split point; The probability representation value of the optimal segmentation point is mapped back to the probability value, and the human body probability map is binarized to generate a foreground human body mask.

3. The human motion posture correction method based on AI vision as described in claim 1, characterized in that: The specific steps for extracting the 3D central skeleton from a 3D AI visual object are as follows. Calculate the minimum Euclidean distance from each voxel in the 3D AI vision volume to the boundary of the 3D AI vision volume, and calculate the gradient vector of each voxel based on the minimum Euclidean distance to extract the ridge points. Using the ridge point as the seed point, the gradient flow direction is followed point by point and connected to form a continuous three-dimensional central skeleton.

4. The human motion posture correction method based on AI vision as described in claim 1, characterized in that: The specific steps for obtaining a three-dimensional joint sequence through joint localization are as follows. Within the 3D central skeleton, skeleton endpoints, skeleton bifurcation points, and potential joints are statistically analyzed as joint candidate points. Based on the human standing motion, obtain the three-dimensional joint coordinates of the standing position and calculate the bone segment length constant; By using the bone segment length constant to correct the joint candidate points, a three-dimensional joint coordinate sequence is obtained.

5. The human motion posture correction method based on AI vision as described in claim 1, characterized in that: The process of extracting the human motion manifold from the three-dimensional joint coordinate sequence to perform contact perception neural motion priors and outputting prior signals involves the following specific steps: A motion posture evaluation model is constructed based on neural networks; Generate human motion manifold vectors in the encoder of the motion posture evaluation model; In the decoder of the motion posture evaluation model, the dimension of the human motion manifold vector is gradually expanded, and the three-dimensional joint coordinate sequence is reconstructed under the constraint of constant bone segment length. In the posture feasibility branch of the motion posture evaluation model, the Gaussian kernel similarity between the three-dimensional joint sequence and the reconstructed three-dimensional joint sequence is calculated as the posture feasibility score output based on the Euclidean distance between the three-dimensional joint sequence and the reconstructed three-dimensional joint sequence. In the joint attention branch of the motion posture evaluation model, the weights of joint importance are assigned using a normalized exponential distribution based on the latent vectors.

6. The human motion posture correction method based on AI vision as described in claim 1, characterized in that: The specific steps for obtaining basic motion cues are as follows. The foreground human body image sequence is cropped pixel by pixel based on the foreground human body mask. The second moment within each pixel window of the foreground human body image sequence is calculated to obtain the corner points. Based on the spatial and temporal gradients of the corner points, the least squares solution is used to solve the sparse optical flow vector of each corner point, and the sparse optical flow vector field is reconstructed in the entire foreground human body region. Based on the gradient magnitude of the spatial gradient at the corner points, calculate the mean and standard deviation, and then filter the set of contour points. The set of contour points is projected onto a multi-channel image sequence to generate a human body contour edge map.

7. The human motion posture correction method based on AI vision as described in claim 1, characterized in that: The specific steps for morphological cleaning of the foreground human body mask are as follows. An intermediate mask is obtained by filling the gaps and holes in the foreground human body mask; Based on the sparse optical flow vector field, the intermediate mask is screened for optical flow consistency to obtain a motion-consistent mask; By using the human body contour edge map to correct the edge consistency of the motion-consistent mask, a sequence of human silhouettes is obtained.

8. The human motion posture correction method based on AI vision as described in claim 1, characterized in that: The physical consistency correction component refers to the distance between the projection of the human body's centroid and the supporting polygon; The basic motion cues include a sparse optical flow vector field and a human body contour edge map; The prior signals include posture feasibility scores and joint importance weights.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the AI ​​vision-based human motion posture correction method according to any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the AI ​​vision-based human motion posture correction method according to any one of claims 1 to 8.

Images