Method for recognizing and analyzing running behavior based on machine vision

Abstract

The application relates to the technical field of video recognition, and discloses a running behavior recognition and analysis method based on machine vision. The method comprises the following steps: acquiring a panoramic RGB video stream and extracting a contour deformation feature of a moving target in a spherical projection coordinate system; calculating the confidence weight of two-dimensional joint point positioning based on a pixel radial distance, combining the consistency of bone lengths, and reconstructing a three-dimensional bone point sequence under time and space constraints; performing variational mode decomposition on the angular velocity of knee and ankle joints and extracting a high-frequency mode to quantize the muscle compensation degree by a high-frequency energy proportion; calculating a differential entropy increment based on an individual dynamic reference window, combining a phase lag of the angular velocity to form a compensation phase deviation, and making a posture comparison reference conform to individual motion characteristics; and jointly determining the differential entropy increment and a weighted comprehensive index by a double-threshold value, positioning an abnormal joint according to the direction of the compensation phase deviation, and generating corresponding visual guidance information to provide intuitive and efficient visual evidence for running posture correction.

Description

Technical Field

[0001] This application relates to the field of video recognition technology, specifically to a method for recognizing and analyzing running behavior based on machine vision. Background Technology

[0002] Machine vision-based human motion analysis technology has been gradually applied to scenarios such as running posture recognition and motion standardization assessment. Most existing visual running behavior analysis solutions acquire video streams through ordinary planar cameras, while some solutions combine deep learning models to complete motion classification and anomaly detection. Existing technologies mostly adopt target detection and tracking strategies in a planar coordinate system, relying on color, texture, or rectangular box features to distinguish moving targets, which is difficult to adapt to scenarios with a large field of view and multiple targets moving simultaneously.

[0003] Existing technologies typically use raw time-series signals such as joint angles and angular velocities directly for anomaly detection. They lack effective decomposition and extraction methods for the high-frequency characteristics of muscle compensation implied in the signals, making it difficult to establish a quantitative correlation between high-frequency energy changes and the degree of muscle compensation. Moreover, they generally use a universal fixed benchmark for posture comparison without establishing a dynamic benchmark window that incorporates individual movement habits, resulting in a lack of sensitive representation of changes in the complexity of joint movements. Furthermore, the judgment results are easily affected by individual differences and instantaneous posture fluctuations, leading to misjudgments or missed judgments.

[0004] In summary, the technical problem addressed by this application is how to achieve accuracy and stability in the identification and analysis of running behaviors of different individuals in multi-objective motion scenarios. Summary of the Invention

[0005] To address the shortcomings of existing technologies, the purpose of this application is to provide a machine vision-based method for recognizing and analyzing running behavior, thereby improving the accuracy and reliability of recognizing and analyzing the running behavior of different individuals, and making it more suitable for practical application scenarios such as campus sports and group training.

[0006] To achieve the above objectives, this application provides a machine vision-based running behavior recognition and analysis method, which acquires a panoramic RGB video stream, identifies and extracts the contour deformation features of the moving target in the spherical projection coordinate system, and locks the identity of the analysis target after assigning an identifier.

[0007] For the continuous video frames corresponding to the identified target, the confidence weight of the two-dimensional joint location is calculated based on the pixel radial distance. The three-dimensional skeleton point sequence under spatiotemporal constraints is reconstructed using the consistency between the confidence weight and the skeleton length as a joint constraint.

[0008] The angular temporal signal of the knee and ankle joint is extracted from the three-dimensional skeletal point sequence. The angular velocity is calculated and variational mode decomposition is performed to extract high-frequency modes. The energy proportion of high-frequency modes is used to quantify the degree of muscle compensation.

[0009] During the monitoring period, the differential entropy increment of the real-time angle time sequence signal relative to the individual dynamic reference window and the corresponding phase lag of the angular velocity are calculated, and the phase lag is used as a compensating phase deviation.

[0010] When the increment of differential entropy is greater than the first threshold, and the weighted value of the degree of muscle compensation and the compensation phase deviation is greater than the second threshold, the running is determined to be abnormal, and corresponding visual guidance information is generated according to the direction of the compensation phase deviation.

[0011] Furthermore, the identification of the target being identified includes:

[0012] The panoramic RGB video stream is mapped to a spherical projection coordinate system. The contour points of the moving target are divided into multiple contour rings according to the radial distance of the spherical surface. The contour stretching and offset of the contour rings relative to the spherical reference in continuous video frames are extracted to form the contour deformation features of the moving target.

[0013] Based on the contour deformation features between consecutive video frames, association matching is performed. Moving targets with a matching degree greater than the matching threshold are identified as the same moving target and assigned a label, so as to lock the identity of the target after associating consecutive video frames.

[0014] Furthermore, reconstructing the three-dimensional skeleton point sequence includes:

[0015] Map the continuous video frames corresponding to the identified analysis target to the image coordinate system and extract the two-dimensional key points;

[0016] Based on the temporal spacing of two-dimensional joints in consecutive video frames, the allowable deviation range of bone length is updated to determine the constraints on bone length consistency.

[0017] Using confidence weight and bone length consistency as joint constraints, the positioning deviation of two-dimensional joints relative to the spherical projection contour is weighted and corrected according to the confidence weight.

[0018] By combining spatiotemporal constraints, the corrected two-dimensional joints are reconstructed into three-dimensional skeletal points. After performing temporal coherence verification on the joint displacement and bone length fluctuation of the three-dimensional skeletal points, a three-dimensional skeletal point sequence is obtained.

[0019] Furthermore, the pixel radial distance from the two-dimensional joint point to the center point of the panoramic image is calculated. Combined with the radial distortion curve preset by the panoramic lens, the relationship between the pixel radial distance and the distortion error is established. Based on the reciprocal of the distortion error, the confidence weight of the two-dimensional joint point localization is obtained after normalization.

[0020] The spatiotemporal constraints indicate that the displacement of the three-dimensional skeleton points at the same joint between adjacent video frames is not greater than the displacement threshold, and the inter-frame fluctuation of the skeleton length does not exceed a reasonable deviation range.

[0021] Furthermore, the quantitative characterization of muscle compensation includes:

[0022] The angular timing signal of the knee and ankle joint is extracted from the three-dimensional skeleton point sequence and the corresponding angular velocity is calculated. Variational mode decomposition is performed on the angular velocity to obtain multiple modal components of different frequencies. High-frequency modes with frequencies in the preset frequency range are selected.

[0023] The energy values ​​of high-frequency modes are determined, and corresponding weights are assigned according to the correlation between high-frequency modes and knee-ankle linkage. The high-frequency energy is obtained by weighted summation, and the degree of muscle compensation is quantified by the energy ratio of the high-frequency energy sum to the total energy of all modal components.

[0024] Furthermore, the three-dimensional spatial coordinates of the hip, knee and ankle joints in the three-dimensional skeletal point sequence are selected, and the flexion and extension angles of the knee and ankle joints are calculated according to the spatial vector angle between adjacent skeletal points. The corresponding knee and ankle angle time sequence signals are obtained by arranging them in the time sequence of the video frames.

[0025] Calculate the phase difference of the angular velocities of the knee and ankle joints corresponding to the high-frequency mode, and determine the correlation between the high-frequency mode and the knee-ankle linkage based on the magnitude of the absolute value of the phase difference.

[0026] Furthermore, based on the historical angle time-series signal of the knee and ankle joint from the three-dimensional skeletal point sequence, an individual dynamic benchmark window is established and the benchmark differential entropy is determined; real-time angle time-series signal within the monitoring period is obtained, real-time differential entropy is calculated, and the difference between the real-time differential entropy and the benchmark differential entropy is used as the differential entropy increment;

[0027] The angular velocities of the knee and ankle joints are acquired synchronously within the monitoring period. The phase difference between the angular velocities of the knee and ankle joints is calculated by weighting the correlation between the high-frequency mode and the knee-ankle linkage, and the phase lag is used as a compensatory phase deviation.

[0028] Furthermore, generating the individual dynamic benchmark window includes: selecting the knee and ankle joint angle time-series signal from the first n frames of the monitoring period as the initial sample, calculating the angle fluctuation variance of the initial sample, and dynamically adjusting the window length according to the angle fluctuation variance to form the individual dynamic benchmark window.

[0029] The frame-by-frame differential entropy is calculated for the time-series angle signal of the knee and ankle joint within the individual dynamic reference window, and the average value of all frame-by-frame differential entropies is taken as the reference differential entropy.

[0030] Calculate the instantaneous phase difference of the angular velocities of the knee and ankle joints in each frame within the monitoring period. Multiply the instantaneous phase difference by the corresponding weight of the correlation degree and sum them up. Take the average of the summed results as the phase lag.

[0031] Furthermore, generating the visual guidance information includes:

[0032] When the increment of differential entropy is greater than the first threshold, and the weighted value of the degree of muscle compensation and the phase deviation of compensation is greater than the second threshold, the running is judged to be abnormal.

[0033] The abnormal joint is located based on the direction of the compensatory phase deviation, and the guidance level is determined by combining the magnitude of the absolute difference between the differential entropy increment and the first threshold, and the magnitude of the absolute difference between the weighted value and the second threshold.

[0034] The presentation parameters of the guidance signs are adjusted based on the correlation between high-frequency modalities and knee-ankle linkage to generate corresponding visual guidance information.

[0035] Furthermore, the positive direction of the compensatory phase deviation corresponds to knee joint abnormality, and the negative direction corresponds to ankle joint abnormality; the correlation between the high-frequency mode and the knee-ankle linkage is used as the adjustment coefficient for the flashing frequency of the guidance mark; the guidance mark is superimposed on the target joint position in the panoramic image corresponding to the panoramic RGB video stream, and the color of the guidance mark is set according to the guidance level.

[0036] Compared with the prior art, the beneficial effects achieved by this application are as follows:

[0037] By acquiring panoramic RGB video streams and extracting the contour deformation features of moving targets in a spherical projection coordinate system, stable identification and identity locking of analyzed targets are achieved in scenarios with a large field of view and multiple targets moving simultaneously, improving the reliability of target analysis in complex running scenarios. The confidence weight of 2D joint point positioning is calculated based on pixel radial distance, and a joint constraint is formed by combining the consistency of bone length. Under spatiotemporal constraints, the 3D bone point sequence is reconstructed, reducing the positioning deviation caused by the radial distortion of the panoramic lens, making the joint point correction conform to the actual physiological structure and movement law, and improving the reconstruction accuracy of 3D bone point.

[0038] By performing variational mode decomposition on the angular velocity of the knee and ankle joints and extracting high-frequency modes, the degree of muscle compensation is quantified by the proportion of high-frequency energy. High-frequency components highly correlated with muscle compensation are separated from the original motion signal to achieve an objective quantitative representation of the muscle compensation state. The differential entropy increment is calculated based on the individual dynamic benchmark window, and the phase lag of the angular velocity is combined to form a compensatory phase deviation, so that the posture comparison benchmark fits the individual motion characteristics, so as to sensitively capture abnormal changes in joint motion complexity and knee-ankle linkage coordination.

[0039] By using a dual threshold method combining differential entropy increment and weighted comprehensive index, misjudgments caused by fluctuations in a single index or individual differences are significantly reduced, improving the accuracy and robustness of running anomaly detection. Abnormal joints are located based on the direction of compensatory phase deviation, and corresponding visual guidance information is generated, enabling precise pointing and graded prompts for abnormal parts, providing intuitive and efficient visual basis for running posture correction. Attached Figure Description

[0040] Figure 1 This is a flowchart of a machine vision-based method for recognizing and analyzing running behavior.

[0041] Figure 2 A flowchart for reconstructing the 3D skeleton point sequence;

[0042] Figure 3 A logical flowchart for generating visual guidance information. Detailed Implementation

[0043] The technical solution of this application will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments and specific features in the embodiments are detailed descriptions of the technical solution of this application, rather than limitations thereof.

[0044] like Figure 1 As shown, this embodiment provides a machine vision-based method for recognizing and analyzing running behavior, including:

[0045] S1. Acquire panoramic RGB video stream, identify and extract the contour deformation features of moving targets in the spherical projection coordinate system, and lock the target identity after assigning labels.

[0046] Specifically, identifying the target identity for analysis includes:

[0047] The panoramic RGB video stream is mapped to a spherical projection coordinate system. The contour points of the moving target are divided into multiple contour rings according to the radial distance of the spherical surface. The contour stretching and offset of the contour rings relative to the spherical reference in continuous video frames are extracted to form the contour deformation features of the moving target.

[0048] Based on the contour deformation features between consecutive video frames, association matching is performed. Moving targets with a matching degree greater than the matching threshold are identified as the same moving target and assigned a label, so as to lock the identity of the target after associating consecutive video frames.

[0049] The original image acquired by the panoramic RGB video stream is in a circular panoramic format. When the contour of the moving target is processed directly in the planar coordinate system, the pixels in the edge area will be stretched and distorted, resulting in insufficient positioning accuracy of the moving target contour points. At the same time, the single-dimensional contour features cannot effectively distinguish different moving targets in a scene where multiple people are running at the same time. The spherical projection coordinate system is a three-dimensional spherical coordinate system established with the optical center of the panoramic lens as the center of the sphere. Each pixel corresponds to a unique spatial point on the sphere, and the radius of the sphere is set according to the panoramic lens parameters. The coordinate transformation of each frame of the two-dimensional image of the panoramic RGB video stream is performed by equidistant cylindrical projection, with the optical center of the panoramic lens set as the center of the sphere O and the radius of the sphere R = 1000 pixels.

[0050] The mapping relationship between two-dimensional pixel coordinates (u, v) and spherical coordinates (θ, φ, R) is θ = 2π × (u / W - 0.5) and φ = π × (0.5 - v / H), where W = 3840 pixels and H = 2160 pixels, which are the width and height of the panoramic image, respectively, θ is the azimuth angle, and φ is the polar angle. For example, the coordinates of a contour pixel of a moving target in a certain frame of panoramic image are (1920, 1080), which gives its spherical coordinates as (0, 0, 1000), corresponding to the front of the spherical coordinate system; the spherical coordinates of the edge contour pixel (3840, 1080) are (π, 0, 1000), corresponding to the right of the spherical coordinate system.

[0051] The radial distance of the spherical surface refers to the straight-line distance from the contour point to the center O of the sphere in the spherical coordinate system. Since the radius of the sphere is fixed, it is actually the spherical arc length distance corresponding to the polar angle φ of the contour point on the sphere. The multi-layer contour ring zone divides the contour points of the moving target into multiple concentric ring regions according to different radial distances of the spherical surface, realizing the layered extraction of contour features. The contour points of the moving target in each frame of the spherical projection image are extracted by the Canny edge detection algorithm. The edge detection threshold is set to 50~150 to adapt to the brightness changes in different scenes of panoramic video. Those skilled in the art can adjust the edge detection threshold accordingly in the actual process.

[0052] Calculate the spherical radial distance d = R × φ for each contour point. Divide the contour into 3 layers based on the range of d values. The first layer corresponds to d ∈ [0, 300] pixels, corresponding to the torso contour region of the moving target, with a polar angle φ ∈ [0, 0.3] radians. The second layer corresponds to d ∈ (300, 600] pixels, corresponding to the thigh contour region of the moving target, with a polar angle φ ∈ (0.3, 0.6] radians. The third layer corresponds to d ∈ (600, 1000] pixels, corresponding to the lower leg contour region of the moving target, with a polar angle φ ∈ (0.6, 1.0] radians. For example, if the spherical radial distance d = 250 pixels for the torso contour point, it is assigned to the first layer; if the thigh contour point d = 450 pixels, it is assigned to the second layer; and if the lower leg contour point d = 750 pixels, it is assigned to the third layer.

[0053] Using the first frame of the continuous video stream as the reference frame, the multi-layer contour ring bands of the moving target in that frame are extracted as a comparison reference to ensure the consistency of feature comparison between frames; the contour stretching amount is the difference between the average radial distance of a certain contour ring band in a subsequent frame and the average radial distance of the corresponding ring band in the reference frame, which represents the stretching change of the contour ring band; the spherical normal offset is the spherical normal displacement of the center of a certain contour ring band in a subsequent frame relative to the center of the corresponding ring band in the reference frame, which represents the spatial offset of the contour ring band.

[0054] The first frame of the continuous video stream is selected as the reference frame, and the average radial distance d of the three-layer contour ring of the moving target in the frame is extracted. 01 d02 d 03 and the spherical coordinates O of the center of each ring zone 01 O 02 O 03 As a spherical reference, for the subsequent t-th frame (t≥2), the average radial distance d of its 3-layer contour ring is extracted. t1 d t2 d t3 Calculate the contour stretching amount Δd for each layer. t =d t -d0, where d0 is the average radial distance of the reference frame ring band, d t Let Δd be the average radial distance of the ring band in frame t. t A positive sign indicates outward expansion of the contour ring, while a negative sign indicates inward expansion. Calculate the spherical normal distance δd between the center of each contour ring layer in frame t and the center of the corresponding layer in the reference frame. t As the spherical normal offset, the spherical normal is the direction perpendicular to the spherical tangent where the ring zone of this layer is located, ensuring that the offset accurately represents the change in the spatial position of the contour.

[0055] The contour stretching of the three contour rings is integrated with the spherical normal offset to form the contour deformation feature F=[Δd] of the moving target in this frame. t1 ,δd t1 Δd t2 ,δd t2 Δd t3 ,δd t3 This enables multi-dimensional representation of the contour deformation of moving targets; for example, the average radial distance d of the first layer of the reference frame's ring. 01 =250 pixels, average radial distance d of the first layer ring in frame t. t1 =255 pixels, the layer outline stretching Δd t1 =5 pixels; Center O of the first layer ring of the reference frame 01 The spherical coordinates are (0, 0.25, 1000), and the center O of the t-th frame is... t1 The spherical coordinates are (0.02, 0.25, 1000), and the spherical normal offset is δd. t1 =20 pixels, after integration, the outline deformation feature F=[5, 20, 3, 15, -2, 10].

[0056] This eliminates stretching distortion in the edge region of the panoramic RGB video stream, ensuring the accuracy of moving target contour point localization; enables hierarchical extraction of contour features, solving the problem that a single contour feature is insufficient to distinguish different moving targets; and comprehensively characterizes the contour deformation state of moving targets, providing feature basis for subsequent inter-frame association matching.

[0057] During running, the contour stretching and offset changes of a moving target follow fixed patterns and do not exhibit random abrupt changes. However, the contour deformation features of different moving targets are fundamentally different. Using the contour deformation features of a baseline frame as a reference, the similarity between the contour deformation features of each subsequent frame and the contour deformation features of the previous frame is calculated. The consistency of the target is determined by the matching degree. Specifically, cosine similarity is used to calculate the matching degree of contour deformation features between consecutive video frames. The closer the cosine similarity is to 1, the higher the similarity between the two frames, and the greater the probability that the targets are the same object. A matching time window of 5 frames is set, and a secondary matching is performed with the average vector of the contour deformation features of the previous 5 frames. The higher value of the two matchings is taken as the matching degree. The matching degree is cosβ=(F t• F t-1 ) / (||F t ||×||F t-1 ||), F t For the contour deformation feature of frame t, F t-1 Let F represent the contour deformation features of frame t-1, where • denotes the vector dot product, and ||F|| denotes the magnitude of the contour deformation feature F. During secondary matching, the feature F of frame t is calculated. t The average vector F of the contour deformation features of the first 5 frames avg The matching degree is cosβ'. If cosβ≥cosβ', then cosβ is used as the matching degree, otherwise cosβ' is used as the matching degree.

[0058] The matching threshold is a critical value used to distinguish whether targets in different frames belong to the same moving target. It is determined by the frame rate of the panoramic video and the speed of the moving target. Those skilled in the art can adjust the matching threshold accordingly in practice. Combining the frame rate of 30 frames per second for the panoramic video and the running speed of the moving target of 1~5m / s, the matching threshold is set to 0.85 to avoid matching deviations caused by changes in running posture, while eliminating feature interference from different moving targets. If the matching degree between frame t and frame t-1 is greater than the matching threshold of 0.85, the moving targets in the two frames are determined to be the same object; if the matching degree is less than or equal to 0.85, it is determined to be a new moving target. For example, the contour deformation feature F of frame t-1. t-1 =[5, 20, 3, 15, -2, 10], the contour deformation feature F of frame t. t =[4, 18, 2, 14, -3, 9], substituting these values ​​gives cosβ≈0.98, which is greater than 0.85, indicating that the two frames represent the same moving target; if the contour deformation feature F of a certain frame... new =[10, 30, 8, 25, 5, 20], and F t-1 The matching degree cosβ≈0.72, which is less than 0.85, is determined to be a new moving target.

[0059] Each frame sequence identified as the same moving target is assigned a unique code, and all consecutive video frames corresponding to the same identifier are associated to ensure that the identifier corresponds to the moving target within the monitoring period. Identifiers are assigned using an alphanumeric encoding method, such as A001 and A002, where letters distinguish target categories and numbers distinguish different targets. Taking the first frame of the continuous video stream as a baseline, the moving target in that frame is assigned the identifier A001. If the same target is identified in subsequent frames, this identifier is used; if a new target is identified, identifiers A002, A003, etc., are assigned sequentially. The corresponding identifier is also recorded. When a moving target reappears after a brief occlusion (occlusion time ≤ 3 frames), a matching process is performed to quickly restore its identity. If the occlusion time exceeds 3 frames, the target's identity is reassessed and a new identifier is assigned. For example, if the matching degree of the moving target in frames 1 to 20 of the monitoring period is greater than 0.85, the identifier A001 is assigned and associated with those 20 frames. If a new target appears in frame 21 with a matching degree less than or equal to 0.85, the identifier A002 is assigned. If target A001 reappears in frame 22, the feature vector sequence is matched, and the matching degree is greater than 0.85, so the identifier A001 is retained.

[0060] This fully leverages the temporal continuity of the contour deformation of the same moving target, improving the accuracy of inter-frame correlation matching; adapts to the frame rate of panoramic video and the characteristics of running motion, effectively distinguishing different moving targets and avoiding identity confusion; achieves continuous locking of the target's identity and automatic tracking of the moving target; ensures that the subsequently extracted 2D joints and reconstructed 3D skeletal points all belong to the same target, and that the continuous video frames after identity locking have good temporal continuity, providing reliable temporal data support for subsequent processing.

[0061] S2. For the continuous video frames corresponding to the identified target, calculate the confidence weight of the two-dimensional joint location based on the pixel radial distance. With the confidence weight and the consistency of the bone length as a joint constraint, reconstruct the three-dimensional bone point sequence under the spatiotemporal constraints.

[0062] Furthermore, such as Figure 2 As shown, the reconstructed 3D skeleton point sequence includes:

[0063] Map the continuous video frames corresponding to the identified analysis target to the image coordinate system and extract the two-dimensional key points;

[0064] Based on the temporal spacing of two-dimensional joints in consecutive video frames, the allowable deviation range of bone length is updated to determine the constraints on bone length consistency.

[0065] Using confidence weight and bone length consistency as joint constraints, the positioning deviation of two-dimensional joints relative to the spherical projection contour is weighted and corrected according to the confidence weight.

[0066] By combining spatiotemporal constraints, the corrected two-dimensional joints are reconstructed into three-dimensional skeletal points. After performing temporal coherence verification on the joint displacement and bone length fluctuation of the three-dimensional skeletal points, a three-dimensional skeletal point sequence is obtained.

[0067] Among them, the pixel radial distance from the two-dimensional joint point to the center point of the panoramic image is calculated. Combined with the radial distortion curve preset by the panoramic lens, the relationship between the pixel radial distance and the distortion error is established. Based on the reciprocal of the distortion error, the confidence weight of the two-dimensional joint point positioning is obtained after normalization.

[0068] Spatiotemporal constraints characterize that the displacement of the three-dimensional skeleton points at the same joint between adjacent video frames is not greater than the displacement threshold, and the inter-frame fluctuation of the skeleton length does not exceed the reasonable deviation range.

[0069] The image coordinate system refers to a two-dimensional planar coordinate system established with the upper left corner of the panoramic image as the origin, the horizontal x-axis to the right, and the vertical y-axis downward. Specifically, equidistant cylindrical projection is used to reverse-map the continuous video frames corresponding to the identified target to the image coordinate system. The mapping relationship is the inverse operation of the previous spherical projection mapping, that is, the two-dimensional pixel coordinates (u, v) are calculated from the spherical coordinates (θ, φ, R), i.e., u=W×(θ / (2π)+0.5) and v=H×(0.5-φ / π), where W=3840 pixels, H=2160 pixels, which is consistent with the resolution of the previous panoramic image, and R=1000 pixels is the radius of the sphere. For example, the coordinates of a certain joint point in the spherical coordinate system are (0, 0.25, 1000), which can be substituted into the image coordinate system to obtain its coordinates (1920, 810), realizing the accurate mapping from spherical coordinates to image coordinates.

[0070] For running scenarios, key joints related to running posture analysis are extracted, including the hip, knee, and ankle joints. Auxiliary joints corresponding to the torso, thigh, and calf are also extracted to ensure the integrity of skeletal morphology reconstruction. For each consecutive video frame mapped to the image coordinate system, grayscale conversion and Gaussian blur preprocessing are performed to eliminate interference from imaging noise on joint extraction. Then, the OpenPose algorithm is applied, setting a joint detection confidence threshold of 0.7. Joints with a detection confidence higher than this threshold are selected, eliminating those with noise interference. To identify false joints, those skilled in the art can adjust the detection confidence threshold and corresponding parameters of the OpenPose algorithm during the actual process. The extracted joints are arranged in frame time sequence to form two-dimensional joints. For example, in a certain frame image, the coordinates of the hip joint of the locked target are (1920, 810), the coordinates of the knee joint are (1920, 1080), and the coordinates of the ankle joint are (1920, 1350). The detection confidence is higher than 0.7, so they are used as two-dimensional joints. A false joint has a detection confidence of 0.65 and is removed.

[0071] This ensures the compatibility and accuracy of 2D joint extraction, effectively eliminates false joints caused by imaging noise, and improves the purity of 2D joint extraction. The 2D joint sequence provides a reliable basic data source for subsequent deviation correction and 3D reconstruction, laying the foundation for the reconstruction of the entire 3D skeleton point sequence.

[0072] During running, the length of the same bone does not change significantly. However, due to joint point positioning deviations and imaging noise, the temporal spacing of two-dimensional joints in consecutive video frames will fluctuate. If a fixed allowable deviation range for bone length is set, it cannot adapt to this dynamic fluctuation, which can easily lead to deviations in subsequent processing. The temporal spacing of two-dimensional joints refers to the pixel distance between the two ends of two-dimensional joints of the same bone in consecutive video frames, forming a spacing sequence according to the frame time sequence. For example, the pixel distance between the two-dimensional joints of the hip and knee joints corresponding to the thigh bone, arranged frame by frame, forms a temporal spacing sequence. The allowable deviation range of bone length refers to the allowable fluctuation range of the temporal spacing of the same bone relative to its average spacing, which is used to constrain the correction magnitude of joint point positioning deviations. The constraint of bone length consistency refers to limiting the fluctuation of the temporal spacing of the same bone by the allowable deviation range, ensuring that the bone length remains relatively fixed during movement.

[0073] First, the temporal spacing between the joints at both ends of the same bone is extracted from the two-dimensional joints, with a focus on extracting the temporal spacing between the hip and knee joints, and between the knee and ankle joints. The stability of the length of these two types of bones affects the accuracy of the subsequent knee and ankle joint angle extraction. The temporal spacing of 10 consecutive video frames is selected as the initial sample, and the average spacing μ of the initial sample is calculated as the baseline length of the bone. Then, the standard deviation σ of the initial sample is calculated, and the allowable deviation range of the bone length is dynamically updated based on the standard deviation. The allowable deviation range is set to [μ-2σ, μ+2σ] to adapt to the normal fluctuations of joint positioning in running and to eliminate abnormal deviations.

[0074] For example, the 10-frame temporal intervals for the hip and knee joints are 300, 302, 298, 301, 299, 303, 297, 300, 301, and 299 pixels, respectively. The calculated average interval μ = 300 pixels and the standard deviation σ = 1.8 pixels. The allowable deviation range for the updated bone length is [296.4, 303.6] pixels. If the temporal interval of a certain frame is 305 pixels, which is outside the range, it is judged as an abnormal interval, and the corresponding joint point will be adjusted in subsequent corrections.

[0075] To ensure the dynamic adaptability of the allowable deviation range, every 5 frames, the average spacing and standard deviation of the temporal interval of the last 10 frames are recalculated, and the allowable deviation range is updated. This ensures that the bone length consistency constraint is always adjusted based on the individual's current actual movement state, preventing the aforementioned allowable deviation range from losing its constraint effect due to the slow change in running posture over time, and ensuring the rationality and dynamism of the bone length consistency constraint. For example, at frame 15, the temporal spacing of the thigh bone in frames 6 to 15 is recalculated, with an average spacing μ = 301 pixels and a standard deviation σ = 1.7 pixels. The allowable deviation range of bone length is updated to [297.6, 304.4] pixels.

[0076] This allows the constraint on bone length consistency to fit the dynamic scenario of running, ensuring the reasonableness of the deviation range; it also enhances the dynamic adaptability of the constraint, ensuring that bone length remains relatively consistent throughout the monitoring period, and providing a clear constraint standard for subsequent correction of joint point positioning deviations.

[0077] Due to radial distortion and imaging angle in panoramic video, some joints may exhibit positioning deviations relative to the spherical projection contour of the moving target, affecting the accuracy of 3D reconstruction. The degree of positioning deviation varies among different joints, resulting in differences in their positioning reliability. Confidence weight is a quantitative indicator characterizing the positioning reliability of 2D joints, with a value range of [0, 1]. The closer the value is to 1, the more reliable the joint positioning and the smaller the positioning deviation. Positioning deviation refers to the pixel distance between the actual position of the 2D joint in the image coordinate system and its theoretical position on the spherical projection contour of the moving target. The positioning deviation is proportionally corrected according to the confidence weight; the higher the positioning reliability, the smaller the correction magnitude, and vice versa.

[0078] Calculate the radial distance r of each two-dimensional joint point to the center point of the panoramic image. The coordinates of the center point of the panoramic image are (W / 2, H / 2) = (1920, 1080). , where (x, y) are the actual coordinates of the original two-dimensional joint point; retrieve the correspondence between the preset pixel radial distance r and the distortion error e of the panoramic lens. The distortion error increases with the increase of the pixel radial distance, and the distortion of the edge region of the panoramic image becomes more obvious; based on the reciprocal of the distortion error, normalize it to the interval [0, 1] to obtain the confidence weight w of the two-dimensional joint point, i.e., w = 1 / (1+e), where e is the distortion error corresponding to the joint point.

[0079] For example, the image coordinates of a two-dimensional joint point of a knee joint are (1920, 1080), the radial distance of pixels r=0, the corresponding distortion error e=0.01, and the confidence weight w=1 / (1+0.01)≈0.99, indicating extremely high positioning reliability; the actual coordinates of a two-dimensional joint point of an ankle joint are (3072, 1080), the radial distance of pixels r=1152, the corresponding distortion error e=0.1, and the confidence weight w=1 / (1+0.1)≈0.91, indicating slightly lower positioning reliability.

[0080] The original 2D joint point is mapped to a spherical projection coordinate system to determine its theoretical position on the spherical projection contour of the moving target. This is achieved by extracting the contour boundary using a contour edge detection algorithm, combining this with the physiological position of the joint point to determine the theoretical position, and then mapping the theoretical position back to the image coordinate system to obtain the theoretical coordinates (x0, y0). (Positioning deviation) Where (x, y) are the actual coordinates of the original two-dimensional joint; for example, the actual coordinates of a hip joint are (1920, 810), and its theoretical coordinates are (1922, 812), indicating a positioning deviation. Pixel.

[0081] Using confidence weight and bone length consistency as joint constraints, a weighted correction is performed, i.e., x'=xw×(x-x0), y'=yw×(y-y0), where (x', y') are the corrected two-dimensional joint coordinates, w is the confidence weight, and (x0, y0) are the theoretical coordinates. During the correction process, it must be ensured that the temporal intervals of the bones corresponding to the corrected joints are within the allowable deviation range. For example, the above hip joint confidence weight w=0.99, actual coordinates (19... 20, 810), theoretical coordinates (1922, 812), substituting them into the corrected coordinates x'=1920-0.99×(1920-1922)=1921.98≈1922 pixels, y'=810-0.99×(810-812)=811.98≈812 pixels. The corrected positioning deviation is close to 0, and the corrected temporal interval between the hip joint and the knee joint is 300 pixels, which is within the allowable deviation range and meets the constraint requirements.

[0082] This allows for precise characterization of the positioning reliability of two-dimensional joints, ensuring that joints with large positioning deviations are effectively corrected; and ensuring that the corrected joints conform to the fixed length law of human bones, making the corrected two-dimensional joints both accurate and in line with human physiological structure, thus providing a high-quality data source for subsequent three-dimensional reconstruction.

[0083] The corrected 2D joint points can only reflect the planar posture of the moving target and cannot represent the spatial positional relationship of the joint points. In running, the spatial displacement of joint points and the fluctuation of bone length have temporal continuity. If the 3D bone points are directly reconstructed, problems such as abrupt changes in joint point displacement between frames and abnormal fluctuations in bone length are likely to occur, resulting in distortion of the 3D bone point sequence and failure to reflect the real running posture. Spatiotemporal constraints indicate that the displacement of the 3D bone points of the same joint point between adjacent video frames is not greater than the displacement threshold, and the inter-frame fluctuation of bone length does not exceed a reasonable deviation range, ensuring the continuity of joint point movement and the stability of bone length. The 3D spatial coordinates of the joint points are calculated by combining the image coordinates of the 2D joint points with the panoramic lens parameters. For the 3D bone points of consecutive frames, the spatiotemporal constraints are checked to see if the joint point displacement and bone length fluctuations meet the spatiotemporal constraints. Abnormal points are removed and corrected to ensure sequence continuity.

[0084] The allowable deviation range of the bone length corresponding to the two-dimensional joint points is converted to obtain the reasonable physiological range of the actual bone length in a single frame. That is, in the panoramic fixed shooting scene of this scheme, visual calibration is completed in advance through standard scale reference objects to determine that the conversion ratio between the image pixel spacing and the actual limb length of the human body is about 1.667mm / pixel. Based on this, the allowable deviation range of thigh bone length [297.6, 304.4] pixels obtained from the two-dimensional level is uniformly converted into the three-dimensional length range [496.1, 507.4] mm, which is used for pre-verification of whether the bone size reconstructed in a single frame conforms to the normal physiological state of human movement.

[0085] The inter-frame fluctuation of bone length refers to the absolute difference in the three-dimensional length of the same group of limb bones between adjacent frames. The standard deviation of the temporal interval of bone length is obtained, and after conversion to the actual scale, the reasonable deviation range of inter-frame bone length is determined to be 0~2σ×1.667mm / pixel, where σ is the standard deviation of the current temporal interval calculated in the aforementioned dynamic update process, and 1.667mm / pixel is the conversion ratio between the image pixel interval calibrated in advance using a standard scale reference and the actual length of a human limb. This reasonable deviation range is used to verify whether the fluctuation of bone length between adjacent frames is within the allowable range.

[0086] Specifically, a monocular vision-based 3D reconstruction method combines the intrinsic parameters (focal length f=12mm, pixel size s=1.4μm) and extrinsic parameters (lens mounting height, pitch angle) of a panoramic lens to convert the corrected 2D joint image coordinates (x', y') into 3D spatial coordinates (X, Y, Z), where X and Y are horizontal coordinates and Z is the vertical coordinate (height). During the reconstruction process, it is ensured that the displacement of the 3D skeleton points at the same joint in adjacent frames does not exceed a displacement threshold, which is set to 50mm to adapt to the joint movement during running. The normal displacement range of the node; the inter-frame fluctuation of the bone length does not exceed the reasonable deviation range; for example, the corrected coordinates of a knee joint's two-dimensional joint point are (1922, 812). Substituting the intrinsic and extrinsic parameters of the panoramic lens into the corrected coordinates of the two-dimensional joint point, we get the three-dimensional coordinates as (0, 0, 1500) mm; in the next frame, the corrected two-dimensional coordinates of the joint point are (1923, 813), and the reconstructed three-dimensional coordinates are (0.1, 0.1, 1500.2) mm. The displacement between adjacent frames is ≈0.14 mm, which is less than 50 mm, thus satisfying the spatiotemporal constraints.

[0087] Extract the 3D coordinate sequence of the same joint point in consecutive frames, calculate the 3D displacement of adjacent frames. If the displacement is greater than the displacement threshold, the joint point is determined to be an abnormal point and corrected by the average coordinate of the previous and next frames. Then extract the 3D length sequence of the same bone (hip and knee joints, knee and ankle joints), where the 3D length is the Euclidean distance between the 3D coordinates of the two ends of the bone. Calculate the bone length fluctuation of adjacent frames. If the fluctuation exceeds the reasonable deviation range, it is determined to be an abnormal fluctuation. Adjust the 3D coordinates of the corresponding joint point until the fluctuation meets the constraints.

[0088] For example, if the 3D length of the thigh bone in one frame is 500mm and in the next frame it is 515mm, the calculated inter-frame fluctuation of the bone length is 15mm. The previously obtained allowable deviation range for bone length is [297.6, 304.4] pixels. After conversion, the reasonable physiological range for the actual bone length in a single frame is [496.1, 507.4]mm. This is used to determine whether the length of the bone in a single frame conforms to physiological dimensions. Simultaneously, based on the standard deviation σ of the temporal interval of bone length, the reasonable deviation range of the inter-frame bone length is determined after actual scale conversion. That is, the reasonable deviation range is 0~2σ×1.667mm / pixel. Taking the current time σ=1.7 pixels as an example, the reasonable deviation range is 0~2×1.7×1.667≈0~5.67mm. This is used to verify whether the fluctuation of bone length in adjacent frames is reasonable. A fluctuation of 15mm exceeds the reasonable deviation range and is judged as abnormal. The 3D coordinates of the knee joint are adjusted, and the length is corrected to 504mm. The inter-frame fluctuation is 4mm, which is within the reasonable deviation range and meets the constraint requirements.

[0089] After verification, the three-dimensional skeletal points of all frames are arranged in frame time sequence to form a three-dimensional skeletal point sequence. The sequence contains the three-dimensional spatial coordinates of key joints such as the hip, knee and ankle joints of each frame, and the points of each frame meet the spatiotemporal constraints and have good inter-frame continuity. For example, the three-dimensional skeletal points of frames 1 to 30 in the monitoring period, after verification, the displacement of the joint points in adjacent frames is less than 50mm, and the fluctuation of bone length is within a reasonable deviation range. The three-dimensional skeletal point sequence is formed by arranging them in frame time sequence.

[0090] This enables precise conversion of corrected 2D joints to 3D skeleton points, solving the problem that 2D data cannot represent spatial posture; ensuring the continuity of joint movement and the stability of skeleton length during 3D reconstruction; effectively eliminating abnormal points and fluctuations, correcting reconstruction deviations, and ensuring the coherence and reliability of the 3D skeleton point sequence; and providing accurate 3D spatial data for subsequent processing.

[0091] S3. Extract the angular temporal signal of the knee and ankle joint from the three-dimensional skeletal point sequence, calculate the angular velocity and perform variational mode decomposition to extract high-frequency modes, and use the energy proportion of high-frequency modes to quantify the degree of muscle compensation.

[0092] Specifically, quantifying the degree of muscle compensation includes:

[0093] The angular timing signal of the knee and ankle joint is extracted from the three-dimensional skeleton point sequence and the corresponding angular velocity is calculated. Variational mode decomposition is performed on the angular velocity to obtain multiple modal components of different frequencies. High-frequency modes with frequencies in the preset frequency range are selected.

[0094] The energy values ​​of high-frequency modes are determined, and corresponding weights are assigned according to the correlation between high-frequency modes and knee-ankle linkage. The high-frequency energy is obtained by weighted summation, and the degree of muscle compensation is quantified by the energy ratio of the high-frequency energy sum to the total energy of all modal components.

[0095] Specifically, the three-dimensional spatial coordinates of the hip, knee, and ankle joints in the three-dimensional skeletal point sequence are selected. Based on the spatial vector angle between adjacent skeletal points, the flexion and extension angles of the knee and ankle joints are calculated respectively. The corresponding knee and ankle angle time sequence signals are obtained by arranging them according to the time sequence of video frames.

[0096] Calculate the phase difference of the angular velocities of the knee and ankle joints corresponding to the high-frequency mode, and determine the correlation between the high-frequency mode and the knee-ankle linkage based on the magnitude of the absolute value of the phase difference.

[0097] Muscle compensation directly alters the flexion and extension patterns of the knee and ankle joints, a change manifested through variations in their angles. The knee and ankle joint angle time-series signal refers to the flexion and extension angles of the knee and ankle joints, arranged sequentially from consecutive video frames. The knee angle represents the relative flexion and extension of the thigh and lower leg, while the ankle angle represents the relative flexion and extension of the lower leg and foot. In the three-dimensional skeletal point sequence, the angle between the spatial vectors of two adjacent skeletal points (hip and knee, knee and ankle) constituting the knee and ankle joints represents the flexion and extension angle of the corresponding joint. Angular velocity is the derivative of the knee and ankle joint angle time-series signal with respect to time, representing the rate of angle change.

[0098] Specifically, from the 3D skeletal point sequence, the 3D spatial coordinates of the hip, knee, and ankle joints in each frame are selected and denoted as hip joint P1 (X1,Y1,Z1), knee joint P2 (X2,Y2,Z2), and ankle joint P3 (X3,Y3,Z3), respectively. Spatial vectors of adjacent skeletal points are constructed. The spatial vectors corresponding to the knee joint are P1P2 (X2-X1,Y2-Y1,Z2-Z1) and P2P3 (X3-X2,Y3-Y2,Z3-Z2), and the angle between the two vectors is the flexion-extension angle of the knee joint. The spatial vectors corresponding to the ankle joint are P2P3 (X3-X2,Y3-Y2,Z3-Z2) and P3P4 (X4-X3,Y4-Y3,Z4-Z3), where P4 (X4,Y4,Z4) is the foot reference point, which is extracted synchronously from the 3D skeletal point sequence. The angle between the two vectors is the flexion-extension angle of the ankle joint.

[0099] The angle between spatial vectors is calculated using the dot product formula, λ = arccos[(a•b) / (|a|×|b|)], where a and b are the spatial vectors of adjacent bone points, • denotes the dot product, and |a| and |b| represent the magnitudes of the vectors. For example, in a frame, the hip joint P1 (0,0,1500) mm, the knee joint P2 (0,0,1000) mm, and the ankle joint P3 (0,0,500) mm have spatial vectors P1 and P2... (0,0,-500), P2P3 is (0,0,-500), substituting into the calculation, the knee joint angle λ=180°, that is, the straightened state; in another frame, P2 is (0,100,1000)mm, P3 is (0,200,500)mm, the spatial vector P1P2 is (0,100,-500), P2P3 is (0,100,-500), the knee joint angle is still 180°, indicating that the knee joint did not undergo flexion and extension changes in this frame.

[0100] The knee and ankle flexion and extension angles calculated for each frame are arranged sequentially according to the time sequence of the video frames to form knee angle time sequence signals and ankle angle time sequence signals respectively. For example, if there are 30 frames in the monitoring period, the knee angle (180°, 175°, 170°...) of each frame is arranged in frame order to form knee angle time sequence signals; similarly, ankle angle time sequence signals are formed.

[0101] The first-order difference algorithm is used to differentiate the angle time-series signal to obtain the angular velocity, i.e., ω. t =(λ t -λ t-1 ) / Δt, where ω t Let λ be the angular velocity of the t-th frame. t Let λ be the joint angle in frame t. t-1 Let λ be the joint angle in frame t-1, and Δt be the frame interval. Since the panoramic video frame rate is 30 frames / second, Δt = 1 / 30 second. For example, the knee joint angle λ in frame t-1. t-1 =180°, knee angle λ in frame t t =175°, substituting this into the equation gives the angular velocity ω. t =(175°-180°) / (1 / 30)=-150° / second, the negative sign indicates knee flexion; knee angle λ in frame t+1. t+1 =170°, angular velocity ω t+1 =(170°-175°) / (1 / 30)=-150° / second, indicating that the knee flexion rate is stable during this period.

[0102] This allows for a true reflection of the flexion and extension states of the knee and ankle joints, converting the spatial pose data of three-dimensional skeletal points into temporal signals to meet the needs of subsequent variational mode decomposition; accurately capturing the rate of change in knee and ankle joint angles, effectively reflecting rapid motion adjustments during muscle compensation; and providing a basis for subsequent analysis of muscle compensation in the two joints separately, ensuring the targeted quantification of the degree of muscle compensation.

[0103] Conventional signal decomposition methods struggle to accurately separate modal components of different frequencies, failing to effectively extract high-frequency components corresponding to muscle compensation. Through iterative optimization, the original time-series signal is decomposed into multiple independent modal components with different center frequencies, each corresponding to a signal component of a specific frequency. The signals of individual frequency components obtained after variational mode decomposition each have a unique center frequency. High-frequency modes refer to modal components whose center frequencies fall within a preset high-frequency range. This high-frequency range corresponds to the high-frequency fluctuation range of joint movement during muscle compensation and needs to be set in conjunction with the characteristics of running. Those skilled in the art can adjust the preset high-frequency range accordingly during actual operation.

[0104] Specifically, a variational mode decomposition algorithm is first employed, with the number of decomposition modes set to K=5 to adapt to the frequency distribution of the angular velocity signal in running motion. This avoids redundancy due to too many modes and insufficient decomposition due to too few modes. A penalty factor α=2000 is used to control the smoothness of the mode components and adapt to the noise characteristics of the angular velocity signal. The convergence accuracy ε=1×10⁻⁶. -7 To ensure the stability of the decomposition results, those skilled in the art can adjust the algorithm parameters accordingly in the actual process. The angular velocities of the knee and ankle joints are input into the variational mode decomposition algorithm, and after iterative optimization, five modal components of different frequencies are obtained. Each modal component corresponds to a center frequency, and they are named IMF1 (low frequency), IMF2, IMF3, IMF4, and IMF5 (high frequency) in ascending order of center frequency.

[0105] For example, after decomposing the knee joint angular velocity time-series signal, the center frequencies of the five modal components are 0.5Hz, 1.2Hz, 2.5Hz, 4.8Hz, and 8.3Hz, respectively. Among them, IMF1~IMF3 are low-frequency modes, and IMF4~IMF5 are high-frequency modes. After decomposing the ankle joint angular velocity time-series signal, the center frequencies are 0.6Hz, 1.3Hz, 2.7Hz, 5.1Hz, and 8.5Hz, respectively. IMF4~IMF5 are high-frequency modes.

[0106] Based on the physiological characteristics of muscle compensation during running, the frequency range of high-frequency modes is preset to 4Hz~10Hz. This range corresponds to the rapid contraction frequency during muscle compensation. Below 4Hz is considered normal exercise rhythm, and above 10Hz is considered noise interference. For all decomposed modal components, modal components with center frequencies in the 4Hz~10Hz range are selected as high-frequency modes. If the center frequency of a modal component exceeds this range, it is discarded. For example, after decomposing the knee joint, IMF4 (4.8Hz) and IMF5 (8.3Hz) both have center frequencies in the 4Hz~10Hz range and are identified as high-frequency modes; IMF3 (2.5Hz) has a center frequency below 4Hz and is discarded; after decomposing the ankle joint, IMF4 (5.1Hz) and IMF5 (8.5Hz) are identified as high-frequency modes, and the remaining low-frequency modes are discarded.

[0107] This enables precise decomposition of angular velocity timing signals, effectively separating modal components of different frequencies; ensuring the stability and accuracy of the decomposition results, accurately screening out the high-frequency modes corresponding to muscle compensation, and eliminating normal movement rhythms and noise interference.

[0108] Different high-frequency modes correspond to different degrees of muscle compensation. The higher the correlation with knee-ankle linkage, the greater the contribution of the high-frequency mode to muscle compensation. Simply summing the energies of all high-frequency modes cannot reflect the differences in contribution between different modes, resulting in insufficient quantification of the degree of muscle compensation. The energy value of a high-frequency mode refers to the sum of the squares of the amplitudes of each high-frequency mode signal, representing the intensity of the signal. The larger the energy value, the higher the intensity of the corresponding muscle compensation. The total high-frequency energy refers to the sum of the energy values ​​of all high-frequency modes after weighted summation based on correlation. The energy ratio refers to the ratio of the total high-frequency energy to the total energy of all modal components, used to standardize and quantify the degree of muscle compensation. The value range is [0,1]. The larger the ratio, the higher the degree of muscle compensation. The correlation between a high-frequency mode and knee-ankle linkage refers to the phase difference between the angular velocities of the knee and ankle joints corresponding to the high-frequency mode. The smaller the absolute value of the phase difference, the more coordinated the knee-ankle linkage, and the higher the correlation between the high-frequency mode and knee-ankle linkage, the greater its contribution to muscle compensation; conversely, the correlation is lower.

[0109] For each selected high-frequency mode, calculate its energy value E=Σ(x t 2 ), where x t Let E4 be the signal amplitude of the t-th frame of the high-frequency mode; for example, the amplitudes of each frame of the knee joint high-frequency mode IMF4 (4.8Hz) are 2.5° / sec, 2.7° / sec, 2.6° / sec, etc. Calculate the sum of squares of the amplitudes to obtain the energy value E4 = 125 (° / sec). 2 The energy value of IMF5 (8.3 Hz) is E5 = 86 (° / second). 2 The energy value of the ankle joint high-frequency mode IMF4 (5.1 Hz) is E4' = 118 (° / s). 2 The energy value of IMF5 (8.5 Hz) is E5' = 79 (° / s). 2 .

[0110] For each high-frequency mode, the angular velocities of the corresponding knee and ankle joints are extracted, and the instantaneous phase difference between them is calculated. The instantaneous phase of the angular velocity is extracted using Hilbert transform, and then the difference between the instantaneous phases of the knee and ankle joint angular velocities in the same frame is calculated to obtain the instantaneous phase difference Δφ. t The average of the absolute values ​​of the instantaneous phase differences across all frames is taken as the mean phase difference Q between the high-frequency mode and the knee-ankle linkage. The value of Q ranges from [0, π]. Based on the mean phase difference Q, the correlation degree B between the high-frequency mode and the knee-ankle linkage is calculated as B = (π - Q) / π. The value of the correlation degree B ranges from [0, 1]. The smaller Q is, the more coordinated the knee-ankle linkage is, and the higher the correlation degree B is; conversely, the larger Q is, the lower the correlation degree B is. For example, the instantaneous phase of the angular velocity corresponding to the IMF4 high-frequency mode of the knee joint is φ. k1 φ k2...The instantaneous phase of the angular velocity corresponding to the IMF4 high-frequency mode of the ankle joint is φ a1 φ a2 ... calculate the instantaneous phase difference Δφ for each frame. t =|φ kt -φ at The average value is Q4=0.3π, which has a high correlation; the high frequency mode of IMF5 in the knee joint corresponds to Q5=0.6π, which has a low correlation.

[0111] The correlation degree B is directly used as the weight l of the high-frequency mode, i.e., l = B = (π - Q) / π, to ensure that the higher the correlation degree, i.e., the smaller Q, the larger the weight, which conforms to the logic that the higher the correlation degree, the greater the contribution to muscle compensation. For example, Q4 = 0.3π for IMF4, weight l4 = (π - 0.3π) / π = 0.7; Q5 = 0.6π for IMF5, weight l5 = (π - 0.6π) / π = 0.4; weight l4' = 0.75 for ankle joint IMF4, and weight l5' = 0.35 for IMF5. The energy value of each high-frequency mode is multiplied by the corresponding weight, and all multiplications are summed to obtain the total high-frequency energy. For example, substituting the above values ​​yields... .

[0112] Calculate the total energy E of all modal components (including the 5 modal components obtained from decomposition). 全 That is, the sum of the energy values ​​of all modal components; calculate the total high-frequency energy E. 总 With E 全 The ratio of these values ​​yields the energy percentage η = E. 总 / E 全 This ratio is the quantitative value of the degree of muscle compensation; for example, the total energy E of all modal components of the knee joint. 全k =320 (° / second) 2 The total energy E of all modal components of the ankle joint 全a =290 (° / second) 2 E 全 =E 全k +E 全a =610 (° / second) 2 The energy percentage η = 238.05 / 610 ≈ 0.39, which means the quantitative value of muscle compensation is 0.39, indicating that there is a moderate degree of muscle compensation at present.

[0113] This allows for precise characterization of the muscle compensation intensity corresponding to each high-frequency mode, providing a quantitative basis for subsequent weighted accumulation, accurately reflecting the differences in the contribution of different high-frequency modes to muscle compensation, and making the sum of high-frequency energy more consistent with the actual muscle compensation situation; it also eliminates the influence of absolute energy values ​​caused by individual differences and differences in exercise intensity, achieving standardized quantification of the degree of muscle compensation, and ensuring the comparability of quantification results in different scenarios.

[0114] S4. During the monitoring period, calculate the differential entropy increment of the real-time angle time sequence signal relative to the individual dynamic reference window and the corresponding phase lag of the angular velocity, and use the phase lag as a compensating phase deviation.

[0115] Furthermore, based on the historical angle time-series signal of the knee and ankle joint from the three-dimensional skeletal point sequence, an individual dynamic benchmark window is established and the benchmark differential entropy is determined; real-time angle time-series signal within the monitoring period is obtained, real-time differential entropy is calculated, and the difference between the real-time differential entropy and the benchmark differential entropy is used as the differential entropy increment;

[0116] The angular velocities of the knee and ankle joints are acquired synchronously within the monitoring period. The phase difference between the angular velocities of the knee and ankle joints is calculated by weighting the correlation between the high-frequency mode and the knee-ankle linkage, and the phase lag is used as a compensatory phase deviation.

[0117] Specifically, generating an individual dynamic benchmark window includes: selecting the knee and ankle joint angle time-series signal from the first n frames of the monitoring period as the initial sample, calculating the angle fluctuation variance of the initial sample, and dynamically adjusting the window length according to the angle fluctuation variance to form an individual dynamic benchmark window.

[0118] The frame-by-frame differential entropy is calculated for the time-series angle signal of the knee and ankle joint within the individual dynamic reference window, and the average value of all frame-by-frame differential entropies is taken as the reference differential entropy.

[0119] Calculate the instantaneous phase difference of the angular velocities of the knee and ankle joints in each frame within the monitoring period. Multiply the instantaneous phase difference by the corresponding weight of the correlation degree and sum them up. Take the average of the summed results as the phase lag.

[0120] Significant differences exist in running posture and joint range of motion among individuals. The baseline differential entropy calculated using a fixed baseline window cannot adapt to individual differences, leading to deviations in the calculation of real-time differential entropy increments and failing to accurately reflect abnormal changes in an individual's running posture. Furthermore, an individual's normal running posture may exhibit slight fluctuations during running, which the fixed window cannot dynamically adapt to, easily misjudging normal fluctuations as abnormalities. The individual dynamic baseline window refers to a baseline data window whose length is dynamically adjusted based on the individual's running posture characteristics, used to extract the time-series signal of the knee and ankle joint angles under normal running conditions.

[0121] The baseline differential entropy refers to the average frame-by-frame differential entropy of the knee and ankle joint angle time-series signal within an individual's dynamic baseline window, representing the complexity of the angle time-series signal under normal running conditions. The real-time differential entropy refers to the differential entropy of the knee and ankle joint angle time-series signal in each frame within the monitoring period, representing the complexity of the angle time-series under real-time running conditions. The differential entropy increment refers to the difference between the real-time differential entropy and the baseline differential entropy. The larger the difference, the greater the deviation between the real-time angle time-series complexity and the normal state, and the higher the possibility of abnormal running posture.

[0122] The angle time-series signal of the first n frames of the monitoring period is selected as the initial sample, and the window length is dynamically adjusted by the angle fluctuation variance. The knee and ankle joint angle time-series signal of the first 10 frames of the monitoring period is selected as the initial sample to adapt to the number of frames required for the stability of normal running posture, and to ensure that the initial sample can reflect the normal running state of an individual. The angle fluctuation variance of the initial sample is calculated. The angle fluctuation variance is used to characterize the degree of fluctuation of the angle time-series signal in the initial sample. The smaller the variance, the more stable the posture of the initial sample, and the window length should be appropriately shortened. The larger the variance, the larger the posture fluctuation of the initial sample, and the window length should be extended to ensure the representativeness of the benchmark.

[0123] Angle fluctuation variance σ 2 =Σ(λ t -μ0) 2 / n, where λ t Let μ0 be the joint angle of the t-th frame in the initial sample, μ0 be the mean angle of the initial sample, and n be the number of frames in the initial sample. For example, if the knee joint angle time-series signal of the first 10 frames is selected as the initial sample, with a mean angle μ0 = 175°, the angle fluctuation variance σ is calculated. 2 =4.2 (°) 2 The fluctuation is relatively small, so the initial window length of 10 frames is adjusted to 8 frames; if the initial sample angle fluctuation variance σ 2 =8.5 (°) 2 The fluctuations were significant, so the window length was adjusted to 12 frames.

[0124] After adjusting the window length, an individual dynamic reference window is formed. Every 5 frames, the variance of the angle timing signal within the current reference window is recalculated, and the window length is dynamically adjusted based on the variance, ranging from 6 to 14 frames. This ensures the reference window always closely matches the stable state of the individual's real-time running posture, avoiding distortion of the reference differential entropy due to individual posture fluctuations. For example, at frame 15, the angle variance σ of the current reference window (frames 7-14) is calculated. 2 =3.8 (°) 2 The fluctuations were small, so the window length was adjusted to 7 frames; at frame 20, the variance σ 2 =7.1 (°) 2 Adjusted to 11 frames.

[0125] The knee and ankle joint angle time-series signal within the individual dynamic reference window is processed by frame segmentation, with a frame length of 3 frames to accommodate the fluctuation period of the angle time-series signal and ensure that the frame differential entropy can reflect the local complexity. The Shannon differential entropy algorithm is used to calculate the differential entropy of each frame, i.e., G=-Σp(ζ)log2p(ζ), where p(ζ) is the probability density function of the angle time-series signal. The mean of the differential entropy of all frames is calculated as the reference differential entropy G0. For example, the angle time-series signal of the individual dynamic reference window (8 frames) is divided into 3 frames (frames 1-3, frames 4-5, and frames 6-8). The calculated differential entropies of the 3 frames are 1.2, 1.3, and 1.2, respectively. The mean value G0=1.23 is taken as the reference differential entropy.

[0126] For the knee and ankle joint angle time-series signal of each frame within the monitoring period, the same frame length and Shannon differential entropy algorithm as the reference differential entropy are used to calculate the real-time differential entropy G corresponding to each frame. t The real-time differential entropy for each frame is calculated based on the angle signals of the current frame and the frames before and after it, ensuring real-time performance and accuracy. For example, the knee joint angle signal in the 11th frame of the monitoring cycle is combined with the angle data from the 10th, 11th, and 12th frames to calculate the real-time differential entropy G. t =1.56; G is calculated by combining the data from frames 11, 12, and 13 in frame 12. t =1.62.

[0127] The real-time differential entropy G of each frame t The difference between the differential entropy G0 and the baseline differential entropy G0 is taken as the differential entropy increment ΔG for this frame, i.e., ΔG = G t -G0; for example, the baseline differential entropy G0 = 1.23, and the real-time differential entropy G in frame 11. t =1.56, the differential entropy increment is 1.56-1.23=0.33; the differential entropy increment of the 12th frame is 1.62-1.23=0.39, indicating that the angular timing complexity of the 12th frame is greater than that of the normal state.

[0128] This allows for adaptation to the differences in running posture among different individuals, ensuring that the baseline differential entropy always closely matches the stable state of the individual's real-time running posture, thereby improving the representativeness and accuracy of the baseline. It also ensures that the differential entropy can accurately reflect the local complexity of the angle time-series signal, accurately capture abnormal complexity changes in the individual's running posture, provide quantitative indicators adapted to the individual for subsequent anomaly judgment, and ensure that the quantitative results of different individuals are comparable.

[0129] The movements of the knee and ankle joints are coordinated. Under normal conditions, the angular velocities of the two joints are in phase. When muscles compensate, the knee and ankle linkage becomes uncoordinated, resulting in an abnormal phase difference in the angular velocities of the knee and ankle joints, manifested as phase lag. If all phase differences are treated equally, the differences in the influence of different high-frequency modes on the knee and ankle linkage cannot be reflected, leading to inaccurate calculation of the phase lag and failing to truly reflect the degree of knee and ankle linkage miscoordination caused by muscle compensation. The phase lag is the average value of the instantaneous phase difference between the angular velocities of the knee and ankle joints after weighted summation by the correlation degree of high-frequency modes, used to characterize the degree of knee and ankle linkage miscoordination. The compensatory phase deviation is the phase lag; the larger the deviation, the more severe the knee and ankle linkage miscoordination and the higher the degree of muscle compensation.

[0130] Extract the knee joint angular velocity ω for each frame within the monitoring period from the angular velocity time-series signal. k With ankle joint angular velocity ω a To ensure the temporal synchronization of the two, each frame corresponds to a set of knee and ankle joint angular velocity data, providing a basis for instantaneous phase difference calculation; for example, in the 11th frame of the monitoring cycle, the knee joint angular velocity ω k =-150° / second, ankle joint angular velocity ω a =-145° / sec; Frame 12, ω k =-155° / second, ω a =-140° / second.

[0131] Hilbert transform was used to process the angular velocity time-series signals of the knee and ankle joints respectively, and the instantaneous phase φ of the angular velocity signal in each frame was extracted. kt (Knee joint) and φ at (Ankle joint); Instantaneous phase is used to characterize the phase state of the angular velocity signal at a certain moment, reflecting the temporal coordination of joint movement; the absolute difference of instantaneous phases in the same frame is calculated to obtain the instantaneous phase difference Δφ. t =|φ kt -φ at To avoid the positive and negative phase differences canceling each other out, and to ensure that the phase difference accurately reflects the degree of linkage misalignment; for example, the instantaneous phase φ of the knee joint in frame 11. kt =0.8π, instantaneous phase φ of the ankle joint at =0.7π, instantaneous phase difference Δφ t =|0.8π-0.7π|=0.1π; Frame 12φ kt =0.9π, φ at =0.75π, instantaneous phase difference Δφ t =|0.9π-0.75π|=0.15π.

[0132] The correlation weights are the same as those calculated in the previous quantification of muscle compensation degree for high-frequency modal correlation, i.e., the correlation weight l corresponding to each high-frequency modality. The higher the correlation, the greater the weight. For example, the correlation weight l4 of the knee joint high-frequency modality IMF4 is 0.7, and the weight l5 of IMF5 is 0.4; the weight l4' of the ankle joint high-frequency modality IMF4 is 0.75, and the weight l5' of IMF5 is 0.35. This set of weights is directly used in this calculation.

[0133] The instantaneous phase difference Δφ of each frame t Multiplying the weighted instantaneous phase difference by the correlation weight of the corresponding high-frequency mode yields the weighted instantaneous phase difference for a single frame. Under a single high-frequency mode, the weighted instantaneous phase differences of all frames under that mode are summed and then divided by the total number of frames in the monitoring period to obtain the phase lag φ corresponding to that mode. 滞后 As the compensatory phase deviation in this mode, i.e., φ 滞后 =Σ(Δφ t ×l) / N, where N is the total number of frames in the monitoring period, and l is the correlation weight; for example, if the monitoring period has a total of 30 frames, the weighted sum of the instantaneous phase differences from frame 1 to frame 30 is 4.2π, and the total number of frames N=30, the phase lag φ is calculated. 滞后 =4.2π / 30=0.14π, that is, the compensatory phase deviation is 0.14π, indicating that there is a slight mismatch in the current knee-ankle linkage and slight muscle compensation.

[0134] If multiple high-frequency modes exist within the monitoring period, first calculate the phase lag of each high-frequency mode according to the calculation method for a single high-frequency mode. Then, take the arithmetic mean of the phase lag of all high-frequency modes and use the arithmetic mean as the overall phase lag to ensure that the influence of all high-frequency modes on the knee-ankle linkage is considered. For example, the phase lags of the two high-frequency modes of the knee joint and the two high-frequency modes of the ankle joint are 2.1π, 1.9π, 2.3π, and 2π, respectively. After taking the average, the phase lag is (2.1π+1.9π+2.3π+2π) / 4=2.075π, which is used as the final compensatory phase deviation.

[0135] This ensures the accuracy of instantaneous phase extraction, accurately reflects the phase state of the knee and ankle joint angular velocity, provides a reliable basis for phase difference calculation, and ensures that the phase difference can truly reflect the degree of misalignment of knee and ankle linkage; it highlights the influence of highly correlated high-frequency modes on phase lag, improving the calculation accuracy of phase lag; it eliminates the deviation caused by single-frame phase abrupt changes, ensuring the stability and reliability of compensatory phase deviation; and it adapts to individual muscle compensation characteristics, enabling compensatory phase deviation to accurately reflect knee and ankle linkage abnormalities caused by muscle compensation.

[0136] S5. When the incremental differential entropy is greater than the first threshold and the weighted value of the degree of muscle compensation and the phase deviation of compensation is greater than the second threshold, the running is determined to be abnormal, and corresponding visual guidance information is generated according to the direction of the phase deviation of compensation.

[0137] Specifically, such as Figure 3 As shown, the generated visual guidance information includes:

[0138] When the increment of differential entropy is greater than the first threshold, and the weighted value of the degree of muscle compensation and the phase deviation of compensation is greater than the second threshold, the running is judged to be abnormal.

[0139] The abnormal joint is located based on the direction of the compensatory phase deviation. The guidance level is determined by combining the magnitude of the absolute difference between the differential entropy increment and the first threshold, and the magnitude of the absolute difference between the weighted value and the second threshold.

[0140] The presentation parameters of the guidance signs are adjusted based on the correlation between high-frequency modalities and knee-ankle linkage to generate corresponding visual guidance information.

[0141] Among them, the positive direction of the compensatory phase deviation corresponds to knee joint abnormality and the negative direction corresponds to ankle joint abnormality; the correlation between high-frequency mode and knee-ankle linkage is used as the adjustment coefficient of the flashing frequency of the guide sign; the guide sign is superimposed on the target joint position in the panoramic image corresponding to the panoramic RGB video stream, and the color of the guide sign is set according to the guidance level.

[0142] A single indicator is prone to misjudgment; a slight increase in muscle compensation may simply be an individual's force adjustment and not an abnormality. The first threshold is a critical value used to determine whether the increment of differential entropy is abnormal. It is set by the baseline differential entropy of the individual's dynamic benchmark window and the normal complexity range of running motion, representing the maximum allowable value of the increment of differential entropy under normal running conditions. Exceeding this threshold indicates an abnormal increase in angular temporal complexity and a tendency for abnormal running posture. The second threshold is a critical value used to determine the combined abnormality of muscle compensation and phase deviation. It is set by the normal range of muscle compensation and the normal range of compensation phase deviation, representing the combined allowable upper limit of muscle compensation and phase misalignment. The weighted value is obtained by multiplying the quantified value of muscle compensation and compensation phase deviation by a preset weight, which is used to comprehensively represent the severity of running abnormalities.

[0143] The first threshold is set based on the baseline differential entropy of the individual's dynamic benchmark window. Based on the fluctuation range of differential entropy under normal running conditions, the first threshold T1 = 0.3 is set to effectively distinguish between normal fluctuations and abnormal increases. The second threshold is set based on the normal range of muscle compensation degree and the normal range of compensation phase deviation, with a second threshold T2 = 0.2. Those skilled in the art can adjust the first and second thresholds accordingly in practice. The quantified value of muscle compensation degree is [0, 0.2) for slight compensation, [0.2, 0.4) for moderate compensation, and ≥0.4 for severe compensation. The compensation phase deviation is [0, 0.1π) for normal, [0.1π, 0.3π) for slight imbalance, and ≥0.3π for severe imbalance. The weight of muscle compensation degree is set to 0.6, and the weight of compensation phase deviation is set to 0.4, with the sum of the weights being 1, highlighting the core role of muscle compensation degree. The weighting value is C = η × 0.6 + φ. 滞后 ×(0.4 / π), where η is the quantitative value of muscle compensation, ranging from 0 to 1, and φ 滞后 To compensate for phase deviation, values ​​are taken from 0 to π, and the phase lag is normalized to the range of 0 to 1 to ensure that the weighted value is consistent with the quantification value of muscle compensation. This is used as a reference for judging the degree of abnormality when determining the guidance level later.

[0144] The differential entropy increment ΔG and weighted value C are obtained for each frame within the monitoring period. If a frame simultaneously satisfies ΔG > T1 and C > T2, the running posture in that frame is determined to be abnormal. If only one condition is met, or neither condition is met, the running posture is determined to be normal, and visual guidance is not triggered. For example, if the differential entropy increment ΔG = 0.35 (> 0.3), the quantification value of muscle compensation η = 0.39, and the compensation phase deviation φ is... 滞后 =0.14π, weighted value C=0.39×0.6+0.14π×(0.4 / π)=0.234+0.056=0.29 (>0.2), which satisfies both threshold conditions and is judged as running abnormal; for a certain frame ΔG=0.32 (>0.3) and C=0.18 (≤0.2), which only satisfies one condition and is judged as normal; for a certain frame ΔG=0.28 (≤0.3) and C=0.22 (>0.2), it is also judged as normal to avoid misjudgment.

[0145] To ensure the stability of anomaly detection, a running anomaly is only detected and visual guidance is triggered if the dual threshold conditions are met for three consecutive frames. This avoids invalid guidance caused by sudden changes in indicators in a single frame. For example, if ΔG > 0.3 and C > 0.2 are met for three consecutive frames from frame 11 to 13, a running anomaly is detected and visual guidance is triggered. If only frame 11 meets the condition, but frames 12 and 13 do not, it is detected as a single-frame fluctuation and guidance is not triggered.

[0146] This eliminates misjudgments caused by fluctuations in a single indicator, improving the accuracy and reliability of running anomaly detection; it adapts to individual running characteristics and anomaly detection needs, ensuring the rationality of the judgment criteria; it enhances the stability of anomaly detection, achieving accurate identification of abnormal states and providing a reliable trigger basis for subsequent visual guidance.

[0147] After identifying running anomalies, precise location of the abnormal joint is required for targeted guidance. Different degrees of anomaly correspond to different guidance needs. The direction of compensatory phase deviation refers to the positive or negative direction of the phase lag Δφt. A positive direction corresponds to knee joint anomalies, and a negative direction corresponds to ankle joint anomalies. This limitation is based on the knee and ankle joint angular velocity phase coordination law. When muscle compensation causes knee joint anomalies, the phase difference is positive; when ankle joint anomalies occur, it is negative. Guidance levels, categorized according to the severity of the anomaly, determine the intensity of the guidance indicator. The more severe the anomaly, the higher the guidance level and the stronger the presentation intensity. The magnitude of the absolute difference between the differential entropy increment and the first threshold characterizes the degree of anomaly in angular temporal complexity. The magnitude of the absolute difference between the weighted value and the second threshold characterizes the combined degree of anomaly in muscle compensation and phase misalignment.

[0148] Retrieve the positive and negative directions of the compensated phase deviation calculated above. If the phase deviation is positive (Δφ) t If the phase deviation is >0), then the abnormal joint is the knee joint; if the phase deviation is negative (Δφ), then the abnormal joint is the knee joint. t If <0), then the abnormal joint is located as the ankle joint; for example, the compensatory phase deviation Δφ in a certain frame t =0.14π (positive direction), the abnormal joint is located as the knee joint; Δφ in a certain frame t =-0.12π (negative direction), the abnormal joint is located as the ankle joint; if the phase deviation is 0, it indicates that there is no obvious abnormal joint, only suggesting overall posture adjustment.

[0149] The two difference magnitudes are normalized to the range of 0 to 1, and the average of the two is taken as the comprehensive value of the anomaly degree. Based on the comprehensive value, three guidance levels are divided: Level 1 guidance (minor anomaly), Level 2 guidance (moderate anomaly), and Level 3 guidance (severe anomaly). The higher the guidance level, the stronger the guidance indicator. The normalized difference magnitude is the ratio of the actual difference magnitude to the maximum possible difference magnitude. The maximum possible difference magnitude of the differential entropy increment is set to 0.5, corresponding to severe anomaly, and the maximum possible difference magnitude of the weighted value is set to 0.3, corresponding to severe anomaly.

[0150] For example, if the differential entropy increment in a frame is 0.35, the first threshold is 0.3, and the difference in differential entropy increment is |0.35-0.3|=0.05, which, after normalization, is 0.05 / 0.5=0.1; if the weighted value is 0.29, the second threshold is 0.2, and the difference in weighted value is |0.29-0.2|=0.09, which, after normalization, is 0.09 / 0.3=0.3; the comprehensive value is (0.1+0.3) / 2=0.2, corresponding to level one guidance (minor anomaly); if ΔG=0.45 in a frame, the difference in differential entropy increment is... The degree is 0.15, which is 0.3 after normalization; C=0.38, the weighted difference range is 0.18, which is 0.6 after normalization; the comprehensive value is (0.3+0.6) / 2=0.45, corresponding to level 2 guidance (moderate anomaly); for a certain frame, ΔG=0.55, the differential entropy increment difference range is 0.25, which is 0.5 after normalization; C=0.48, the weighted difference range is 0.28, which is 0.93 after normalization; the comprehensive value is (0.5+0.93) / 2=0.715, corresponding to level 3 guidance (severe anomaly).

[0151] Based on the comprehensive value of the abnormality level, a classification threshold is set: a comprehensive value of [0, 0.3) represents Level 1 guidance, [0.3, 0.6) represents Level 2 guidance, and ≥0.6 represents Level 3 guidance. This classification adapts to the actual severity of running abnormalities, ensuring that the guidance level and the degree of abnormality are accurately matched. This allows for precise identification of the abnormal location. Considering the comprehensive abnormality level of angular and temporal complexity abnormalities, muscle compensation, and phase misalignment, the classification ensures that the guidance level and the severity of the abnormality are accurately matched. The three guidance levels adapt to different levels of abnormality needs, improve the effectiveness of guidance, and provide a clear basis for setting the color of subsequent guidance markers.

[0152] The presentation parameters of the guidance signs refer to their flashing frequency and color. These parameters directly affect the intuitiveness and recognizability of the guidance signs. Guidance signs are visual markers used to indicate abnormal joints and their severity. They are circular markers for easy and rapid identification in panoramic images without obscuring the movement of the target joint. The flashing frequency adjustment coefficient is a multiple of the correlation coefficient, meaning the higher the correlation, the faster the flashing frequency and the stronger the indication. The color of the guidance signs is set according to the guidance level; the visual intensity of the color is positively correlated with the guidance level, ensuring users can quickly distinguish the severity of the abnormality. Specifically, considering the panoramic video frame rate (30 frames / second) and the persistence of human vision, the basic flashing frequency is set to 2Hz (twice per second). This frequency balances visual conspicuousness and viewing comfort, quickly attracting user attention without causing visual fatigue due to excessive flashing, and is suitable for real-time observation of human movement and posture.

[0153] The correlation degree B (value 0~1) of the high-frequency mode is retrieved and directly used as the adjustment coefficient B' of the flashing frequency; the flashing frequency = base flashing frequency × (1+B'). The higher the correlation degree of the knee-ankle linkage, the larger B' is and the faster the flashing frequency; for example, if the correlation degree B of a certain high-frequency mode is 0.7, the flashing frequency = 2×(1+0.7) = 3.4Hz; if the correlation degree B = 0.4, the flashing frequency = 2×(1+0.4) = 2.8Hz. Scenes with higher correlation degrees have faster flashing frequencies and stronger prompts.

[0154] Based on the severity of the abnormality indicated by the guidance level, a color-coding system is established. Level 1 guidance (minor abnormality) uses yellow, with medium visual intensity, indicating minor adjustments; Level 2 guidance (moderate abnormality) uses orange, with strong visual intensity, indicating more serious adjustments; and Level 3 guidance (severe abnormality) uses red, with the strongest visual intensity, indicating urgent adjustments. This color scheme aligns with human visual perception of warning colors, ensuring users can quickly distinguish the severity of the abnormality. For example, Level 1 guidance for knee joint abnormalities uses yellow; Level 2 guidance for ankle joint abnormalities uses orange; and Level 3 guidance for knee joint abnormalities uses red.

[0155] The guidance marker is overlaid on the panoramic image corresponding to the panoramic RGB video stream, with the overlay position being the location of the abnormal joint. The adjusted flashing frequency and color parameters are applied to the guidance marker to form visual guidance information, which is overlaid on the panoramic video stream in real time. For example, if the abnormal joint is located as the knee joint, the guidance level is level one, the correlation normalization is 0.7, the guidance marker is a yellow ring, the flashing frequency is 3.4Hz, and it is overlaid at the center of the knee joint image at (1922, 812) pixels, and is presented in real time in the panoramic video, prompting the user that there is a slight abnormality in the knee joint and that posture adjustment is required. If there are multiple high-frequency modes, the average value of the correlation normalization of all high-frequency modes is taken as the flashing frequency adjustment coefficient to ensure that the influence of all high-frequency modes on the guidance marker is considered, thereby improving the rationality of the flashing frequency adjustment. For example, if the correlation normalization of two high-frequency modes is 0.7 and 0.4 respectively, with an average of 0.55, the flashing frequency = 2 × (1 + 0.55) = 3.1Hz.

[0156] This allows for adaptation to panoramic image scenarios, facilitating rapid user identification. It also ensures that the flashing frequency of the guidance markers is precisely matched to the contribution of high-frequency modes to anomalies, helping users quickly perceive the core cause of the anomaly. Furthermore, it enables precise prompts for abnormal joints, intuitively and accurately converting abstract quantitative indicators of anomalies into perceptible visual cues. This helps users quickly identify abnormal joints and the degree of anomaly, adjust their running posture in a timely manner, restore running behavior to normal, and improve the practicality and effectiveness of the guidance.

[0157] The above description is merely a preferred embodiment of this application. The scope of protection of this application is not limited to the above embodiments. All technical solutions falling within the scope of this application's concept are within the scope of protection of this application. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of this application should also be considered within the scope of protection of this application.

Claims

1. A method for recognizing and analyzing running behavior based on machine vision, characterized in that, include: Acquire panoramic RGB video streams, identify and extract the contour deformation features of moving targets in the spherical projection coordinate system, and lock the target identity after assigning labels; For the continuous video frames corresponding to the identified target, the confidence weight of the two-dimensional joint location is calculated based on the pixel radial distance. The three-dimensional skeleton point sequence under spatiotemporal constraints is reconstructed using the consistency between the confidence weight and the skeleton length as a joint constraint. The angular temporal signal of the knee and ankle joint is extracted from the three-dimensional skeletal point sequence. The angular velocity is calculated and variational mode decomposition is performed to extract high-frequency modes. The energy proportion of high-frequency modes is used to quantify the degree of muscle compensation. During the monitoring period, the differential entropy increment of the real-time angle time sequence signal relative to the individual dynamic reference window and the corresponding phase lag of the angular velocity are calculated, and the phase lag is used as a compensating phase deviation. Differential entropy G = -Σp(ζ)log2p(ζ), where p(ζ) is the probability density function of the angle time series signal; When the increment of differential entropy is greater than the first threshold, and the weighted value of the degree of muscle compensation and the compensation phase deviation is greater than the second threshold, the running is determined to be abnormal, and corresponding visual guidance information is generated according to the direction of the compensation phase deviation.

2. The running behavior recognition and analysis method based on machine vision according to claim 1, characterized in that, The identity of the target being identified includes: The panoramic RGB video stream is mapped to a spherical projection coordinate system. The contour points of the moving target are divided into multiple contour rings according to the radial distance of the spherical surface. The contour stretching and offset of the contour rings relative to the spherical reference in continuous video frames are extracted to form the contour deformation features of the moving target. Based on the contour deformation features between consecutive video frames, association matching is performed. Moving targets with a matching degree greater than the matching threshold are identified as the same moving target and assigned a label, so as to lock the identity of the target after associating consecutive video frames.

3. The running behavior recognition and analysis method based on machine vision according to claim 2, characterized in that, Reconstructing the three-dimensional skeleton point sequence includes: Map the continuous video frames corresponding to the identified analysis target to the image coordinate system and extract the two-dimensional key points; Based on the temporal spacing of two-dimensional joints in consecutive video frames, the allowable deviation range of bone length is updated to determine the constraints on bone length consistency. Using confidence weight and bone length consistency as joint constraints, the positioning deviation of two-dimensional joints relative to the spherical projection contour is weighted and corrected according to the confidence weight. By combining spatiotemporal constraints, the corrected two-dimensional joints are reconstructed into three-dimensional skeletal points. After performing temporal coherence verification on the joint displacement and bone length fluctuation of the three-dimensional skeletal points, a three-dimensional skeletal point sequence is obtained.

4. The running behavior recognition and analysis method based on machine vision according to claim 3, characterized in that, Calculate the pixel radial distance from the two-dimensional joint point to the center point of the panoramic image. Combine the radial distortion curve preset by the panoramic lens to establish the relationship between the pixel radial distance and the distortion error. Based on the reciprocal of the distortion error, normalize to obtain the confidence weight of the two-dimensional joint point localization. The spatiotemporal constraints indicate that the displacement of the three-dimensional skeleton points at the same joint between adjacent video frames is not greater than the displacement threshold, and the inter-frame fluctuation of the skeleton length does not exceed a reasonable deviation range.

5. The running behavior recognition and analysis method based on machine vision according to claim 3, characterized in that, The quantitative characterization of muscle compensation includes: The angular timing signal of the knee and ankle joint is extracted from the three-dimensional skeleton point sequence and the corresponding angular velocity is calculated. Variational mode decomposition is performed on the angular velocity to obtain multiple modal components of different frequencies. High-frequency modes with frequencies in the preset frequency range are selected. The energy values ​​of high-frequency modes are determined, and corresponding weights are assigned according to the correlation between high-frequency modes and knee-ankle linkage. The high-frequency energy is obtained by weighted summation, and the degree of muscle compensation is quantified by the energy ratio of the high-frequency energy sum to the total energy of all modal components.

6. The running behavior recognition and analysis method based on machine vision according to claim 5, characterized in that, The three-dimensional spatial coordinates of the hip, knee and ankle joints in the three-dimensional skeleton point sequence are selected. Based on the spatial vector angle between adjacent skeleton points, the flexion and extension angles of the knee and ankle joints are calculated respectively. The corresponding knee and ankle angle time sequence signals are obtained by arranging them in the time sequence of video frames. Calculate the phase difference of the angular velocities of the knee and ankle joints corresponding to the high-frequency mode, and determine the correlation between the high-frequency mode and the knee-ankle linkage based on the magnitude of the absolute value of the phase difference.

7. The running behavior recognition and analysis method based on machine vision according to claim 5, characterized in that, Based on the historical angle time-series signal of the knee and ankle joint using the three-dimensional skeletal point sequence, an individual dynamic benchmark window is established and the benchmark differential entropy is determined; real-time angle time-series signal within the monitoring period is acquired, real-time differential entropy is calculated, and the difference between the real-time differential entropy and the benchmark differential entropy is used as the differential entropy increment. The angular velocities of the knee and ankle joints are acquired synchronously within the monitoring period. The phase difference between the angular velocities of the knee and ankle joints is calculated by weighting the correlation between the high-frequency mode and the knee-ankle linkage, and the phase lag is used as a compensatory phase deviation.

8. The running behavior recognition and analysis method based on machine vision according to claim 7, characterized in that, Generating the individual dynamic benchmark window includes: selecting the knee and ankle joint angle time-series signal from the first n frames of the monitoring period as the initial sample, calculating the angle fluctuation variance of the initial sample, and dynamically adjusting the window length according to the angle fluctuation variance to form the individual dynamic benchmark window. The frame-by-frame differential entropy is calculated for the time-series angle signal of the knee and ankle joint within the individual dynamic reference window, and the average value of all frame-by-frame differential entropies is taken as the reference differential entropy. Calculate the instantaneous phase difference of the angular velocities of the knee and ankle joints in each frame within the monitoring period. Multiply the instantaneous phase difference by the corresponding weight of the correlation degree and sum them up. Take the average of the summed results as the phase lag.

9. The running behavior recognition and analysis method based on machine vision according to claim 7, characterized in that, Generating the visual guidance information includes: When the increment of differential entropy is greater than the first threshold, and the weighted value of the degree of muscle compensation and the phase deviation of compensation is greater than the second threshold, the running is judged to be abnormal. The abnormal joint is located based on the direction of the compensatory phase deviation, and the guidance level is determined by combining the magnitude of the absolute difference between the differential entropy increment and the first threshold, and the magnitude of the absolute difference between the weighted value and the second threshold. The presentation parameters of the guidance signs are adjusted based on the correlation between high-frequency modalities and knee-ankle linkage to generate corresponding visual guidance information.

10. The running behavior recognition and analysis method based on machine vision according to claim 9, characterized in that, The positive direction of the compensatory phase deviation corresponds to knee joint abnormality, and the negative direction corresponds to ankle joint abnormality; the correlation between the high-frequency mode and the knee-ankle linkage is used as the adjustment coefficient for the flashing frequency of the guide sign; the guide sign is superimposed on the target joint position in the panoramic image corresponding to the panoramic RGB video stream, and the color of the guide sign is set according to the guidance level.

Images