A method for recognizing etiquette posture anomaly based on space-time trajectory features

Abstract

This invention relates to the field of posture recognition technology, specifically a method for identifying abnormal etiquette postures based on spatiotemporal trajectory features. The method includes acquiring walking images and point cloud data; calculating the intersection-union ratio (IU) to lock the reference bounding box and mapping it to generate an initial 3D coordinate sequence; identifying sequence breakpoints; performing bidirectional weighted fusion interpolation based on the instantaneous linear velocity and acceleration before and after the breakpoint to generate a continuous spatiotemporal trajectory; inputting the continuous spatiotemporal trajectory into a Bayesian network to extract uncertainty parameters for dynamic filtering, obtaining a smooth rate sequence; segmenting the stride trajectory into extreme points of lower limb joints; constructing an undirected graph input to an autoencoder to calculate the reconstruction loss value, which is then converted into a quantitative score. This invention effectively repairs trajectory breaks caused by occlusion in dense walking scenes, filters out computational distortion and system noise, and achieves objective, fine-grained quantitative assessment of continuous etiquette postures.

Description

Technical Field

[0001] This invention relates to the field of posture recognition technology, specifically to a method for identifying abnormal etiquette postures based on spatiotemporal trajectory features. Background Technology

[0002] Traditional etiquette posture assessment and correction mainly rely on manual observation and subjective scoring by professionals. This method is not only labor-intensive but also has limitations such as strong subjectivity and difficulty in achieving absolutely unified scoring standards. In recent years, automated motion evaluation systems based on computer vision and human posture estimation algorithms have gradually become a research hotspot in this field. These systems typically use cameras or depth sensors to acquire three-dimensional spatial data of the human body and extract key skeletal points for analysis.

[0003] Existing automated posture assessment technologies face significant technical bottlenecks when dealing with complex real-world scenarios. In dynamic movement scenarios involving dense crowds (such as multi-person processions or collective ceremonial displays), frequent mutual occlusion between targets and significant differences in individual physiological scales make it difficult for existing systems to accurately extract the instantaneous evolution velocity and acceleration abrupt changes of key human skeletal structures in three-dimensional space. This core issue has triggered a series of adverse consequences in practical applications. Due to limitations in viewpoint and overlapping occlusion, visual sensors are prone to losing target tracking or missing information on key parts, directly leading to breaks in the motion trajectory extracted by traditional visual measurements and generating numerous data gaps. When faced with these trajectory gaps, existing processing methods often lack consideration for real dynamic characteristics, resulting in severe distortion in subsequent velocity and acceleration calculations, failing to accurately reflect the physical laws and continuity of human movement. Because the extraction of underlying spatiotemporal trajectory and velocity features suffers from breaks and distortions, it is impossible to accurately capture the deep topological relationships of the human skeleton, ultimately preventing existing assessment methods from achieving objective, refined, and quantitative scoring of continuous ceremonial movements.

[0004] To address this, a method for identifying abnormal etiquette postures based on spatiotemporal trajectory features is proposed. Summary of the Invention

[0005] The purpose of this invention is to provide a method for identifying abnormal etiquette postures based on spatiotemporal trajectory features, so as to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: A method for identifying abnormal etiquette postures based on spatiotemporal trajectory features, comprising: Acquire the traveling image and point cloud data; calculate the intersection-over-union ratio of the detection box and the attitude box in the traveling image to lock the two-dimensional reference box, extract the two-dimensional attitude joints in the two-dimensional reference box and map them to the point cloud data, and generate an initial three-dimensional coordinate sequence in chronological order; Identify breakpoints in the initial three-dimensional coordinate sequence, extract continuous valid sequence segments divided by the breakpoints, calculate the first and second time derivatives to obtain instantaneous linear velocity and motion acceleration; interpolate the missing coordinates based on the instantaneous linear velocity and motion acceleration before and after the breakpoints, complete the instantaneous linear velocity and motion acceleration, and generate a continuous spatiotemporal trajectory. The continuous spatiotemporal trajectory is input into a Bayesian network to extract uncertainty parameters. The completed instantaneous linear velocity and motion acceleration are then filtered according to the uncertainty parameters to obtain a smooth velocity sequence. Using the extreme point where the instantaneous linear velocity of the target joint in the smooth rate sequence is zero as the time boundary, the continuous spatiotemporal trajectory is divided into stride trajectory segments; an undirected graph is constructed based on the stride trajectory segments and input into an autoencoder to calculate the reconstruction loss value, which is then converted into a quantitative score to determine whether the etiquette posture is abnormal.

[0007] Preferably, the detection box is an outer bounding box representing the overall physical contour of the target, which is output by locating the human target in the moving image using a target detection algorithm; The attitude bounding box is the smallest bounding box determined by the set of points formed by all the two-dimensional attitude joints after the target's two-dimensional attitude joints are extracted by processing the traveling image using an attitude estimation algorithm.

[0008] Preferably, the intersection-union ratio (IUGR) of the detection boxes and pose boxes in the moving image is the ratio of the area of ​​the overlapping region of the detection boxes and the pose boxes to the difference between the sum of the areas of the detection boxes and the pose boxes and the area of ​​the overlapping region. The step of calculating the IUGR of the detection boxes and pose boxes in the moving image to lock the two-dimensional reference frame includes: calculating the IUGR between each detection box and each pose box in the moving image; selecting the detection box with the largest IUGR and the pose box as a matching combination, and locking the smallest bounding rectangle containing the matching combination as the two-dimensional reference frame.

[0009] Preferably, the initial three-dimensional coordinate sequence generation process involves projecting the point cloud data onto the traveling image based on the extrinsic parameter matrix, and calculating the Euclidean distance between each point cloud point and each of the two-dimensional attitude joints; Based on the principle of minimum distance, the three-dimensional coordinates of the nearest point cloud point are used as the spatial position of the corresponding two-dimensional attitude joint, and the initial three-dimensional coordinate sequence of the corresponding joint is generated by combining the timestamp.

[0010] Preferably, the continuous spatiotemporal trajectory involves traversing the initial three-dimensional coordinate sequence, identifying positions where the time interval between adjacent coordinate points is greater than a preset frame interval threshold as breakpoints, and extracting the continuous coordinate segments divided by the breakpoints as the continuous valid sequence segments; performing differential calculations on the continuous valid sequence segments to obtain the instantaneous linear velocity and motion acceleration of each joint point as a function of time; A forward dynamics prediction vector is constructed based on the instantaneous linear velocity and acceleration before the breakpoint, and a backward dynamics regression vector is constructed based on the instantaneous linear velocity and acceleration after the breakpoint. The forward dynamics prediction vector and the backward dynamics regression vector are used to perform bidirectional weighted fusion interpolation on the missing coordinates of the breakpoint interval to generate a continuous spatiotemporal trajectory.

[0011] Preferably, the Bayesian network maps the coordinate components of each time point in the continuous spatiotemporal trajectory to observation nodes of a dynamic Bayesian network, and establishes a transition matrix between the motion states at adjacent time points and an observation likelihood function between the motion states and the observation nodes; using a recursive probabilistic inference algorithm, the posterior probability distribution of the motion states at each time point is updated by combining the transition matrix and the observation likelihood function, and the covariance matrix corresponding to the posterior probability distribution is extracted as the uncertainty parameter; The filtering gain corresponding to each moment is determined based on the magnitude of each element in the covariance matrix. The filtering gain is used to perform weighted correction on the completed instantaneous linear velocity and motion acceleration. The weighted correction results are arranged and encapsulated in chronological order along the time axis and output as the smoothed rate sequence.

[0012] Preferably, the process of constructing an undirected graph based on the stride trajectory segment involves extracting the three-dimensional spatial coordinates of each joint point within the stride trajectory segment and performing feature concatenation with the corresponding instantaneous linear velocity in the smooth rate sequence to generate the node state vector of the undirected graph. Spatial adjacency edges are established according to the natural anatomical connections of the human skeleton, and temporal correlation edges between adjacent moments of the same joint are established according to the chronological order to construct a spatiotemporal topology. The relative motion acceleration between the two nodes connected by each edge in the spatiotemporal topology is calculated, and the calculated relative motion acceleration is mapped to the edge feature attributes of each corresponding edge to construct an undirected graph.

[0013] Preferably, the process of inputting the undirected graph into the autoencoder to calculate the reconstruction loss value and converting it into a quantized score is as follows: The undirected graph is input into a graph autoencoder, which uses the internal graph convolutional layer and attention mechanism, and uses the edge feature attributes as weight factors for node feature aggregation to extract the low-dimensional feature representation of the undirected graph in the spatiotemporal dimension. The low-dimensional feature representation is reconstructed using the decoding layer of the graph autoencoder, and a predicted undirected graph containing predicted coordinates, predicted instantaneous linear velocity, and predicted relative acceleration is output. The deviation values ​​between the predicted undirected graph and the original undirected graph in the coordinate dimension, velocity dimension, and acceleration dimension are calculated respectively, and the weighted sum of each deviation value is performed. The total deviation value is confirmed as the reconstruction loss value. The reconstruction loss value is mapped to a preset percentage score range using a mapping function, and a quantitative score is output.

[0014] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention extracts the initial three-dimensional coordinate sequence by acquiring moving images and point cloud data and performing multimodal spatial mapping. For trajectory breakpoints in the extraction process, a dynamic vector is constructed based on the instantaneous linear velocity and motion acceleration before and after the breakpoint to perform bidirectional weighted fusion interpolation on the missing coordinates. This effectively overcomes the problem of key point loss and tracking loss caused by mutual occlusion of targets in dynamic moving scenes with dense crowds. It also repairs motion trajectory breakpoints generated in traditional visual measurement and provides a complete and continuous spatiotemporal trajectory basis for feature analysis.

[0015] 2. This invention extracts the covariance matrix of the posterior probability distribution from the continuous spatiotemporal trajectory input into a Bayesian network as an uncertainty parameter, and performs dynamic weighted filtering correction on the completed instantaneous linear velocity and motion acceleration accordingly. This effectively filters out systematic noise caused by individual physiological scale differences and basic interpolation calculations, fundamentally solving the problem of distorted velocity and motion acceleration calculations in traditional methods. It realistically and smoothly restores the instantaneous evolution velocity and acceleration mutation characteristics of key human joints in three-dimensional space.

[0016] 3. This invention uses the extreme points of instantaneous linear velocity at the lower limb joints as time boundaries to finely segment stride trajectory segments, and combines smooth rate sequences to construct an undirected graph with spatiotemporal correlation edges and relative motion acceleration attributes. Then, it uses a graph autoencoder to calculate and reconstruct the loss value and map it into a quantitative score. This invention deeply explores the implicit topological correlation of the human skeleton in continuous three-dimensional spatiotemporal evolution, breaks through the limitations of traditional coarse-grained or static threshold discrimination, and realizes objective, accurate and fine-grained anomaly identification and quantitative assessment of continuous etiquette postures in dynamic walking scenarios. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of a method for identifying abnormal etiquette postures based on spatiotemporal trajectory features; Figure 2 This is a schematic diagram of the two-dimensional reference frame locking logic of the present invention; Figure 3 This is a schematic diagram of the continuous spatiotemporal trajectory bidirectional interpolation generation process of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] Example 1: Please see Figure 1 This invention provides a method for identifying abnormal etiquette postures based on spatiotemporal trajectory features. The technical solution is as follows: A method for identifying abnormal etiquette postures based on spatiotemporal trajectory features, comprising: Acquire the traveling image and point cloud data; calculate the intersection-over-union ratio of the detection box and the attitude box in the traveling image to lock the two-dimensional reference box, extract the two-dimensional attitude joints in the two-dimensional reference box and map them to the point cloud data, and generate an initial three-dimensional coordinate sequence in chronological order; Identify breakpoints in the initial three-dimensional coordinate sequence, extract continuous valid sequence segments divided by the breakpoints, calculate the first and second time derivatives to obtain instantaneous linear velocity and motion acceleration; based on the instantaneous linear velocity and motion acceleration before and after the breakpoints, interpolate the missing coordinates to generate a continuous spatiotemporal trajectory and simultaneously complete the instantaneous linear velocity and motion acceleration. The continuous spatiotemporal trajectory is input into a Bayesian network to extract uncertainty parameters. The completed instantaneous linear velocity and motion acceleration are then filtered according to the uncertainty parameters to obtain a smooth velocity sequence. Using the extreme point where the instantaneous linear velocity of the lower limb joints in the smooth rate sequence is zero as the time boundary, the continuous spatiotemporal trajectory is divided into stride trajectory segments; an undirected graph is constructed based on the stride trajectory segments and input into an autoencoder to calculate the reconstruction loss value, which is then converted into a quantitative score to determine whether the etiquette posture is abnormal.

[0020] The detection box is the outer bounding box that represents the overall physical contour of the target when the target detection algorithm is used to locate the human target in the moving image. The attitude bounding box is the smallest bounding box determined by the set of points formed by all the two-dimensional attitude joints after the target's two-dimensional attitude joints are extracted by processing the traveling image using an attitude estimation algorithm.

[0021] Specifically, the process of acquiring the moving images and point cloud data involves using a color high-definition camera and a LiDAR sensor deployed directly in front of the test scene as data acquisition devices. The camera continuously captures a two-dimensional color pixel array of the subject moving within the test channel at a preset sampling rate of 30 frames per second, serving as the moving images. Simultaneously, the LiDAR sensor synchronously emits and receives laser beams at a preset scanning frequency of 10Hz, measuring the distance and angle from the subject's surface reflection points to the sensor, generating a dense set of points containing three-dimensional spatial coordinate information, which serves as the point cloud data. During the acquisition process, a unified clock control via hardware trigger signals ensures that the camera and LiDAR record data at the same millisecond-level timestamp, achieving strict spatiotemporal alignment between each frame of the moving images and the point cloud data at the same moment.

[0022] The detection box is the bounding box representing the overall physical contour of the target, output by the target detection algorithm for human target localization in the moving image. In this embodiment, the target detection algorithm adopts a single-stage target detection model based on a deep convolutional neural network. Based on the acquired single frame of the moving image, the human body edge contour and human body texture features are extracted from the image through multi-layer convolution operations. Multiple candidate bounding boxes and the confidence score of the presence of human body in the boxes are predicted on the feature map. The non-maximum suppression mechanism is used to remove overlapping and redundant candidate bounding boxes, and only the final bounding box with a confidence score greater than a preset threshold of 85% is retained to complete the human target localization. The reason for setting the confidence threshold to 85% is that experiments have verified that this value can achieve the best balance between effectively removing redundant false detection boxes and ensuring accurate human body localization. The final bounding box tightly surrounds the physical boundary of the subject from the highest point of the head to the lowest point of the feet, as well as the widest part of the left and right arms, forming the bounding box representing the overall physical contour of the target. The pixel coordinates of the four vertices of the bounding box are recorded as the detection box. The pose estimation algorithm is used to process the moving image to extract the two-dimensional pose joints of the target. In this embodiment, a top-down high-resolution human pose prediction network is used as the pose estimation algorithm. After obtaining the detection box, the local area of ​​the human body in the moving image is cropped according to the pixel coordinate range of the detection box and scaled to a preset standard resolution of 256×192. This is then input into the human pose prediction network. Through feature extraction and cross-resolution feature fusion, a corresponding spatial probability distribution heatmap is output for each human joint. The coordinates of the pixel with the largest probability response value in each heatmap are extracted one by one as the precise position of the corresponding joint in the image. In this embodiment, 17 core joints of the human body are extracted, specifically including the head, both shoulders, both elbows, both wrists, both hips, both knees, and both ankles. The pixel coordinates of these 17 parts together constitute the two-dimensional pose joints of the target.

[0023] The horizontal and vertical pixel coordinates of the 17 extracted two-dimensional attitude joints are summarized to form a point set consisting of 17 two-dimensional coordinate points. All two-dimensional attitude joints in this point set are traversed in computer memory, and the minimum and maximum values ​​of all horizontal coordinates and vertical coordinates in the moving image pixel coordinate system are found sequentially. The minimum horizontal coordinate is used as the left vertical boundary, the maximum horizontal coordinate as the right vertical boundary, the minimum vertical coordinate as the upper horizontal boundary, and the maximum vertical coordinate as the lower horizontal boundary. These four extreme boundary lines intersect perpendicularly in the moving image, forming a compact rectangular closed region, which is the minimum bounding box. The coordinate information of this bounding box is used as the attitude box and output. By combining a detection box that represents the overall macroscopic physical contour with a pose box generated based on the microscopic core skeleton, this invention constructs a human target locking mechanism from the outside in. The detection box is responsible for initially filtering out background interference and defining the effective area, while the pose box further removes the edge expansion effect caused by the subject's loose clothing or personal items, so that the final spatial boundary strictly fits the real human skeleton structure.

[0024] See Figure 2 Calculate the intersection-union ratio (IoU) of the detection box and the pose box in the traveling image to lock the two-dimensional reference box; The intersection-union ratio (IUR) of the detection boxes and pose boxes in the moving image is the ratio of the area of ​​the overlapping region of the detection boxes and pose boxes to the difference between the sum of the areas of the detection boxes and the pose boxes and the area of ​​the overlapping region. The step of calculating the IUR of the detection boxes and pose boxes in the moving image to lock the two-dimensional reference frame includes: calculating the IUR between each detection box and each pose box in the moving image; selecting the detection box with the largest IUR and the pose box as a matching combination, and locking the smallest bounding rectangle containing the matching combination as the two-dimensional reference frame.

[0025] Specifically, the coordinates of the top-left and bottom-right pixels corresponding to each detection box in the target detection algorithm output are read. The absolute value of the difference between the bottom-right x-coordinate and the top-left x-coordinate is used as the horizontal width of the detection box, and the absolute value of the difference between the bottom-right y-coordinate and the top-left y-coordinate is used as the vertical height of the detection box. The horizontal width and the vertical height are multiplied to obtain the area of ​​each detection box. Similarly, the pixel coordinates corresponding to the left, right, top, and bottom boundaries of each pose box generated by the pose estimation algorithm based on the two-dimensional pose joint extreme values ​​are read. The difference between the right and left boundaries is used as the horizontal width, and the difference between the bottom and top boundaries is used as the vertical height. The two are multiplied to calculate the area of ​​each pose box. In this embodiment, the standard resolution of the moving image is preset to 1920×1080 pixels, and all coordinate readings and area calculations are based on this two-dimensional pixel matrix. Iterate through all bounding boxes and pose boxes in the image. For any pair of bounding boxes and pose boxes to be matched, compare their horizontal coordinate ranges. Take the maximum value of the left boundary x-coordinate and the minimum value of the right boundary x-coordinate. If the minimum value is greater than the maximum value, the difference is the horizontal overlap length; otherwise, the overlap length is recorded as 0. Similarly, compare their vertical coordinate ranges. Take the maximum value of the upper boundary y-coordinate and the minimum value of the lower boundary y-coordinate. If the minimum value is greater than the maximum value, the difference is the vertical overlap length; otherwise, the overlap length is recorded as 0. Multiply the horizontal overlap length and the vertical overlap length to obtain the area of ​​the overlapping region of the matching pair. To calculate the intersection-union ratio (IUGR) between each detection box and each pose box in the moving image, first add the area of ​​the detection box to the area of ​​the pose box, then subtract the previously calculated overlapping area. The difference is taken as the union area of ​​the combination. Using the overlapping area as the numerator, and the difference between the sum of the areas of the detection box and the pose box and the overlapping area as the denominator, perform a division operation. The resulting ratio is the IUGR between the detection box and the pose box. After completing the iterative calculation of all bounding box combinations in the image, a set of cross-union ratio (CUI) values ​​corresponding to each detection box and each pose box will be obtained. By comparing the values, this set is retrieved, and the detection box with the largest CUI and the pose box are selected as the matching combination, thereby eliminating non-corresponding bounding boxes caused by background interference or other personnel.

[0026] The smallest bounding rectangle containing the matching combination is locked as the two-dimensional reference box. Specifically, this involves obtaining the coordinates of the left, right, top, and bottom boundaries of the detection box in the matching combination, and simultaneously obtaining the four boundary coordinates of the corresponding pose box. Through numerical comparison, the minimum value of the left boundary x-coordinate is taken as the left boundary of the two-dimensional reference box, the minimum value of the top boundary y-coordinate is taken as the top boundary, the maximum value of the right boundary x-coordinate is taken as the right boundary, and the maximum value of the bottom boundary y-coordinate is taken as the bottom boundary. The rectangular region enclosed by these four extreme boundary lines is locked as the two-dimensional reference box, completing the final anchoring of the target's spatial position in the current frame image.

[0027] By introducing an intersection-union-ratio (IUU) calculation mechanism for global traversal matching, it can effectively eliminate redundant frames generated by background noise or non-test subjects in complex test scenarios with a strict spatial overlap index, achieving a unique and reliable binding between the macroscopic overall outline of the human body and the microscopic core skeleton. At the same time, the minimum bounding rectangle of the matching combination is locked as a two-dimensional reference box, which combines the inclusiveness of the detection box to physical edges with the sensitivity of the pose box to the extreme values ​​of the skeleton, ensuring that the final anchored two-dimensional region neither misses any effective human pose edge information nor excludes irrelevant backgrounds to the greatest extent.

[0028] Extract the two-dimensional attitude joints within the two-dimensional reference frame and map them to the point cloud data, then generate an initial three-dimensional coordinate sequence in chronological order. The initial three-dimensional coordinate sequence generation process involves projecting the point cloud data onto the traveling image based on the extrinsic parameter matrix, and calculating the Euclidean distance between each point cloud point and each of the two-dimensional attitude joints. Based on the principle of minimum distance, the three-dimensional coordinates of the nearest point cloud point are used as the spatial position of the corresponding two-dimensional attitude joint, and the initial three-dimensional coordinate sequence of the corresponding joint is generated by combining the timestamp.

[0029] Specifically, the relative spatial pose relationship between the camera and the lidar is obtained through a joint calibration device. This pose relationship is established and preset as an extrinsic parameter matrix, which consists of three-axis rotation parameters and three-axis translation parameters in three-dimensional space. For each frame of data acquired at the same millisecond-level timestamp, the three-dimensional true coordinates of each reflection point in the point cloud data are extracted. These three-dimensional true coordinates are then transformed using geometric matrix multiplication and addition using the extrinsic parameter matrix, converting them from the lidar physical coordinate system to the camera physical coordinate system. Combining the camera's preset inherent focal length and optical center point, the transformed three-dimensional coordinates are projected perspective onto the two-dimensional pixel plane where the moving image is located, obtaining the two-dimensional projected pixel coordinates of each point cloud point on the moving image. Read the pixel coordinates of each of the two-dimensional pose joints within the two-dimensional reference frame. To reduce invalid calculations, only point cloud points that fall within the pixel range of the two-dimensional reference frame after projection are retained. A two-layer traversal logic is established: for any one of the 17 preset extracted two-dimensional pose joints of the human body, all retained projected point cloud points are traversed. During calculation, the difference between the horizontal projected pixel coordinates of the point cloud point and the horizontal axis pixel coordinates of the joint point are squared, and the difference between the vertical projected pixel coordinates of the point cloud point and the vertical axis pixel coordinates of the joint point are squared. The two squared values ​​are added together and the square root is taken to obtain the Euclidean distance between each projected point cloud point and the two-dimensional pose joint point on the two-dimensional image plane. For each specific two-dimensional attitude joint, the Euclidean distance result set is retrieved. By comparing the numerical values ​​one by one, the feature point with the smallest distance value is found in the set, and this point cloud point is determined to be the reflection point that best fits the joint in physical space. The original LiDAR data record corresponding to the nearest projected point cloud point before the projection transformation is traced back, and its true physical three-dimensional coordinates are extracted. This true three-dimensional coordinate is forcibly assigned to the two-dimensional attitude joint point as the absolute spatial position of the corresponding two-dimensional attitude joint point in the current specific data frame, thus completing the transformation of the target joint point from two-dimensional pixel coordinates to three-dimensional physical coordinates. A time-dimensional data organization logic is introduced to generate the trajectory. During continuous acquisition and testing, the clock tag marked by the hardware unified clock at the time of acquisition for each frame of the traveling image and its synchronous point cloud data is read and used as the absolute timestamp of the frame data. For each independent joint of the human body, the three-dimensional spatial positions of the joint point mapped in multiple consecutive data frames are sequentially arranged according to the absolute time flow order of the timestamps. This ordered data stream, with a single joint point as the core object, timestamp as the progressive index, and the three-dimensional spatial coordinates obtained by mapping as the node elements, is the initial three-dimensional coordinate sequence of the corresponding joint point.

[0030] This application only calculates distances for point clouds falling within a two-dimensional reference frame, directly eliminating interference from background point clouds and significantly reducing unnecessary computations; matching based on the least Euclidean distance principle can accurately map key points on two-dimensional images to real three-dimensional physical coordinates; concatenating these coordinates by absolute timestamps generates continuous three-dimensional spatial trajectories, directly providing the necessary data foundation for subsequent calculations of instantaneous velocity and motion acceleration.

[0031] Identify breakpoints in the initial three-dimensional coordinate sequence, extract continuous valid sequence segments segmented by the breakpoints, calculate the first and second time derivatives to obtain instantaneous linear velocity and motion acceleration; based on the instantaneous linear velocity and motion acceleration before and after the breakpoints, interpolate the missing coordinates to generate a continuous spatiotemporal trajectory. See Figure 3 The continuous spatiotemporal trajectory involves traversing the initial three-dimensional coordinate sequence, identifying positions where the time interval between adjacent coordinate points is greater than a preset frame interval threshold as breakpoints, and extracting the continuous coordinate segments divided by the breakpoints as the continuous valid sequence segments; performing differential calculations on the continuous valid sequence segments to obtain the instantaneous linear velocity and motion acceleration of each joint point as a function of time; A forward dynamics prediction vector is constructed based on the instantaneous linear velocity and acceleration before the breakpoint, and a backward dynamics regression vector is constructed based on the instantaneous linear velocity and acceleration after the breakpoint. The forward dynamics prediction vector and the backward dynamics regression vector are used to perform bidirectional weighted fusion interpolation on the missing coordinates of the breakpoint interval to generate a continuous spatiotemporal trajectory.

[0032] Specifically, the initial three-dimensional coordinate sequence of each key point is traversed sequentially according to the timestamp order. The timestamps corresponding to every two adjacent three-dimensional coordinate points in the sequence are extracted, and the difference between the two is calculated as the time interval. In this embodiment, based on the preset sampling frequency of 30 frames per second of the data acquisition device, the preset frame interval threshold is specifically set to 50ms. Each calculated time interval is compared with the 50ms threshold. If the time interval is greater than 50ms, it is determined that data loss has occurred in that interval due to occlusion or algorithm omission, and the position is locked and identified as the breakpoint. Based on all identified breakpoints, the originally missing sequence is segmented and stripped, and coordinate sequence data distributed between each breakpoint and within which the time interval of all adjacent points is less than or equal to 50ms is extracted and used as the continuous valid sequence segment. For each extracted continuous valid sequence segment, differential calculation is performed. For any three adjacent valid coordinate points within the continuous valid sequence segment, the 3D spatial value of the next coordinate point is extracted and subtracted from the 3D spatial value of the previous coordinate point. The resulting 3D spatial distance difference is divided by the time difference between the two coordinate points. This first-order difference calculation result of spatial position with respect to time is used as the instantaneous linear velocity of the joint point at the corresponding moment. Similarly, the instantaneous linear velocity of the next moment is extracted and subtracted from the instantaneous linear velocity of the previous moment. The resulting velocity change is divided by the corresponding time difference between the two frames. This second-order difference calculation result of velocity with respect to time is used as the motion acceleration of the corresponding joint point. By traversing all valid segments, the velocity and acceleration of the valid data interval are simultaneously measured. For each identified breakpoint interval, the three-dimensional position coordinates of the last valid coordinate point before the breakpoint occurs, along with the calculated instantaneous linear velocity and acceleration at that point, are extracted. These three sets of spatial and kinematic data collectively characterize the initial force exertion and motion state of the human joint before entering the occlusion blind zone, and are packaged into the forward dynamic prediction vector to characterize the motion trend extrapolated from the historical inertia within the missing time period. Simultaneously, the three-dimensional position coordinates of the first recovered valid coordinate point after the breakpoint ends, along with its corresponding instantaneous linear velocity and acceleration, are extracted to characterize the immediate state of the joint when it exits the occlusion blind zone, and are packaged into the backward dynamic regression vector to characterize the motion trajectory extrapolated from the endpoint state within the missing time period. Using the forward dynamics prediction vector and the backward dynamics regression vector, a bidirectional weighted fusion interpolation is performed on the missing coordinates of the breakpoint interval. Specifically, this involves determining the specific timestamp of each missing frame within the breakpoint interval. For a specific missing timestamp, based on the initial velocity and acceleration values ​​in the forward dynamics prediction vector, the predicted three-dimensional coordinates at that moment are calculated forward using the physical rules of uniformly accelerated linear motion; simultaneously, based on the final velocity and acceleration values ​​in the backward dynamics regression vector, the regressed three-dimensional coordinates at that moment are calculated backward using the rules of time inverse evolution. The process involves calculating the ratio of the time length between the current missing timestamp and the start and end points of the breakpoint, and then performing bidirectional weighted fusion interpolation. Specifically, for the identified breakpoint interval, firstly, forward dynamics deduction is performed based on the motion state before the breakpoint to obtain predicted three-dimensional coordinates, and then backward dynamics regression is performed based on the motion state after the breakpoint to obtain regressed three-dimensional coordinates. The timestamps of the start and end points of the breakpoint are extracted to calculate the total duration of the breakpoint. The difference between the end point and the current missing timestamp is divided by the total duration to obtain the forward weight, and the difference between the current missing timestamp and the start point is divided by the total duration to obtain the backward weight, ensuring that the closer the missing moment is to a certain endpoint, the larger the corresponding weight. The predicted coordinates and regressed coordinates are multiplied by their respective weights and summed to obtain the final smoothed coordinates for that moment. These are then used to fill in the gaps, and the instantaneous linear velocity and acceleration within the breakpoint interval are simultaneously filled in using the same distance-weighted logic to generate a continuous spatiotemporal trajectory.

[0033] By extracting the velocity and acceleration features of the actual motion of the joints, a two-way weighted fusion interpolation of forward prediction and backward regression is adopted to take into account both the motion inertia of the joints and the instantaneous state when leaving the blind zone. This not only conforms to the laws of physical motion, but also effectively avoids trajectory jumps and restores the natural and smooth motion process during occlusion to the greatest extent.

[0034] The continuous spatiotemporal trajectory is input into a Bayesian network to extract uncertainty parameters. The completed instantaneous linear velocity and motion acceleration are then filtered according to the uncertainty parameters to obtain a smooth velocity sequence. The Bayesian network maps the coordinate components of each time point in the continuous spatiotemporal trajectory to observation nodes of the dynamic Bayesian network, and establishes the transition matrix between the motion states of adjacent time points and the observation likelihood function between the motion states and the observation nodes; using a recursive probabilistic inference algorithm, the posterior probability distribution of the motion states at each time point is updated by combining the transition matrix and the observation likelihood function, and the covariance matrix corresponding to the posterior probability distribution is extracted as the uncertainty parameter; The filtering gain corresponding to each moment is determined based on the magnitude of each element in the covariance matrix. The filtering gain is used to perform weighted correction on the completed instantaneous linear velocity and motion acceleration. The weighted correction results are arranged and encapsulated in chronological order along the time axis and output as the smoothed rate sequence.

[0035] Specifically, the three-dimensional spatial coordinates of each key point in the continuous spatiotemporal trajectory are extracted sequentially according to the time axis. The values ​​of the three directions corresponding to the three-dimensional spatial coordinates are used as the state input values ​​of the observation node of the dynamic Bayesian network at that moment. The actual three-dimensional coordinates, instantaneous linear velocity, and motion acceleration of each key point are jointly defined as the implicitly evolved motion state. Based on the physical rules of uniformly accelerated linear motion, a transformation matrix is ​​constructed to linearly derive the motion state of the next moment from the motion state of the previous moment as the transition matrix. In this embodiment, the displacement derivation coefficients and velocity derivation coefficients inside the transition matrix are calculated based on a preset 50ms sampling frame interval. At the same time, a transformation matrix representing the mapping relationship between the implicit motion state and the state input values ​​of the observation node is established as the observation likelihood function. In this embodiment, since the observation node only contains three-dimensional spatial coordinate components, the diagonal matrix of the position components extracted from the motion state is used as the observation likelihood function. In addition, the process noise variance and the observation noise variance are initialized in the transition matrix and the observation likelihood function, respectively. In this embodiment, the specific value of the observation noise variance is preset as a constant based on the inherent three-dimensional coordinate measurement error of the lidar hardware. For each current moment on the time axis, firstly, using the updated posterior motion state from the previous moment and the transition matrix, a forward state deduction is performed in the time dimension to calculate the prior motion state and prior error matrix at the current moment. Then, the actual input value of the observed node state at the current moment is extracted, and the deviation between the actual observed value and the predicted value corresponding to the prior motion state is calculated using the observation likelihood function. The calculated deviation is used to perform probability correction on the prior motion state to obtain the optimal estimated distribution of the motion state at the current moment, thus completing the update of the posterior probability distribution of the motion state at the current moment. During this update process, the variance matrix representing the dispersion and confidence level of each dimension of the optimal estimated distribution is extracted. The variance matrix is ​​locked as the covariance matrix corresponding to the posterior probability distribution, and the entire matrix is ​​used as the uncertainty parameter for evaluating the reliability of the data at the current moment. Extract the diagonal elements corresponding to the instantaneous linear velocity and motion acceleration dimensions from the covariance matrix. The larger the value of the diagonal element, the higher the uncertainty of the current interpolated data. Perform matrix operations using the prior error matrix, the observation likelihood function, and the preset observation noise variance to calculate the proportional coefficient matrix used to weight the dynamic equilibrium theoretical motion derivation value and the actual input value. Use this proportional coefficient matrix as the filter gain. For each time point, extract the weight coefficients corresponding to velocity and acceleration from the filter gain to perform weighted correction on the instantaneous linear velocity and motion acceleration. When the covariance matrix indicates that the data uncertainty is high at a certain time, the filter gain is automatically adjusted to reduce the trust weight of the completed data and correspondingly increase the weight of the kinematic transition matrix derivation value, thereby filtering out abnormal jitter and abrupt noise caused by interpolation algorithms or environmental occlusion. Following the sequential direction from the start timestamp to the end timestamp, traverse all time nodes that have completed the above weighted correction calculation; concatenate and package the final instantaneous linear velocity and motion acceleration values ​​output after filtering correction at each time node to form an independent data frame; push all data frames belonging to the same measured joint point into a continuous sequence memory structure according to the order of their timestamps to complete the arrangement and encapsulation operation, generating a smooth rate sequence that has removed observation noise and interpolation system errors; By introducing a dynamic Bayesian network and a recursive probabilistic inference mechanism, the covariance matrix is ​​extracted as a parameter representing the uncertainty of the data, and the dynamic filtering gain is calculated accordingly. It can automatically adjust the trust weights of the physical inference values ​​and interpolated observation values ​​according to the data quality, and perform precise weighted correction on the completed instantaneous linear velocity and motion acceleration. This process effectively filters out the systematic noise caused by hardware measurement errors, environmental occlusion and basic interpolation algorithms, and fundamentally overcomes the problems of abnormal rate jitter and acceleration mutation that traditional algorithms are prone to in complex scenarios.

[0036] Using the extreme point where the instantaneous linear velocity of the target joint in the smooth rate sequence is zero as the time boundary, the continuous spatiotemporal trajectory is divided into stride trajectory segments. The target joint is the bi-ankle node among the 17 core joints. The reason for this selection is that in the gait cycle of normal human walking, the bi-ankle node, as the distal pivot of the lower limb, remains relatively stationary with respect to the ground only when the lower limb is in the support phase (grounding phase), and its instantaneous linear velocity exhibits a significant minimum value approaching zero. The other joints, in order to propel the body's center of gravity forward, maintain a non-zero continuous motion state. Therefore, the bi-ankle node is the only physiological anchor point for locating and accurately segmenting the stride cycle. Specifically, the target joint is extracted from the 17 core joints contained in the continuous spatiotemporal trajectory. In this embodiment, the target joint is explicitly set as the dual-ankle node among the 17 core joints, that is, the spatial coordinate data of the left ankle node and the right ankle node in each frame are extracted respectively, and the smooth rate sequence of the left ankle and the smooth rate sequence of the right ankle are generated respectively.

[0037] For the smoothed velocity sequences of the left and right ankles, all instantaneous linear velocity values ​​are traversed sequentially to independently find the extreme points where the instantaneous linear velocity of the left and right feet is zero. Considering that in the actual collected motion trajectory data, the slight swaying of the human body makes it difficult to directly match absolute zero values, this embodiment presets a velocity judgment threshold close to zero and a local search time window when specifically implementing the search for extreme points. In this embodiment, the velocity judgment threshold is preset to 0.05 meters per second, and the local search time window is preset to 15 frames before and after the current time frame.

[0038] Taking the traversal of the left-side nodes as an example, the instantaneous linear velocity values ​​of the left ankle nodes are compared frame by frame. When it is identified that the instantaneous linear velocity value of a certain current frame is less than the preset rate determination threshold of 0.05 meters per second, and after numerical comparison, the instantaneous linear velocity value of the current frame is the minimum value within the local search time window range of the 15 frames before and after it, then the current frame is confirmed to be a supported static state with the left foot on the ground; the processing logic for the right foot is exactly the same. The data points corresponding to all time frames that meet the above filtering conditions in the smooth rate sequence are determined as the extreme points where the instantaneous linear velocity is zero.

[0039] Extract the timestamps corresponding to each identified extreme point where the instantaneous linear velocity is zero on the global time axis. Merge the extracted timestamps of the left and right extreme points and arrange them in strict chronological order. Use the arranged timestamp data as the time boundary for subsequent division of the motion period.

[0040] Based on all the extracted time boundaries, the complete continuous spatiotemporal trajectory is segmented. Specifically, on the time axis, the first time boundary arranged in chronological order is selected as the starting cutoff time, and the adjacent second time boundary is selected as the ending cutoff time. The data of the continuous spatiotemporal trajectory between these two cutoff times is completely extracted and treated as an independent stride trajectory segment. This trajectory segment contains all the spatial position information of the 17 core joints within the current single step (i.e., from the landing of one foot to the landing of the other foot).

[0041] After the initial segmentation, the second time boundary is used as the new starting truncation time, and the adjacent third time boundary is used as the new ending truncation time. The next independent stride trajectory segment is then extracted from the continuous spatiotemporal trajectory. Following this logic, the ending truncation time of the previous stride trajectory segment is used as the starting truncation time of the next stride trajectory segment. The same truncation and extraction operations are performed on all time boundaries in a loop, thereby sequentially segmenting the complete continuous spatiotemporal trajectory into multiple continuous stride trajectory segments. An undirected graph is constructed based on the stride trajectory segment and input into an autoencoder to calculate the reconstruction loss value, which is then converted into a quantitative score to determine whether the etiquette posture is abnormal. The process of constructing an undirected graph based on the stride trajectory segment involves extracting the three-dimensional spatial coordinates of each joint point within the stride trajectory segment and performing feature concatenation with the corresponding instantaneous linear velocity in the smooth rate sequence to generate the node state vector of the undirected graph. Spatial adjacency edges are established according to the natural anatomical connections of the human skeleton, and temporal correlation edges between adjacent moments of the same joint are established according to the chronological order to construct a spatiotemporal topology. The relative motion acceleration between the two nodes connected by each edge in the spatiotemporal topology is calculated, and the calculated relative motion acceleration is mapped to the edge feature attributes of each corresponding edge to construct an undirected graph.

[0042] Specifically, the segmented stride trajectory is traversed, and the three-dimensional spatial coordinates of each human joint point within the trajectory segment are extracted frame by frame. These three-dimensional spatial coordinates specifically include the lateral, longitudinal, and depth positions of each joint point in the real physical space. From the smoothed velocity sequence, the instantaneous linear velocity values ​​corresponding to each joint point at the exact same time point are precisely matched and extracted synchronously. The three position values ​​of the extracted three-dimensional spatial coordinates and the corresponding instantaneous linear velocity values ​​are concatenated and combined according to a fixed data arrangement order to perform a feature concatenation operation, generating a one-dimensional data queue containing four dimensions for each joint point at each time point as the node state vector of the undirected graph. Establish connections between all node state vectors to construct the spatiotemporal topology. In the spatial dimension at the same moment, establish spatial adjacency edges according to the natural anatomical connections of the human skeleton. In this embodiment, the preset natural anatomical connection rules are specifically set as follows: connect head joints to neck joints, neck joints to left and right shoulder joints, shoulder joints to elbow joints on the same side, elbow joints to wrist joints on the same side, trunk center joints to left and right hip joints, hip joints to knee joints on the same side, and knee joints to ankle joints on the same side, thereby connecting discrete joints within a single frame into a complete human spatial skeleton connection. In the temporal dimension spanning adjacent frames, establish temporal association edges between adjacent moments for the same joint in chronological order. Specifically, this involves directly connecting a specific joint at a previous moment to the same specific joint at a subsequent adjacent moment via a data line. By integrating the spatial adjacency edges within a single frame and the temporal correlation edges across frames, all isolated node state vectors are connected into a spatiotemporal topology structure that is interwoven in a mesh. For each connecting edge in the spatiotemporal topology, whether it is a spatially adjacent edge or a temporally related edge, the state vectors of the two nodes connected to the two ends of the connecting edge are extracted. Since the smoothed rate sequence retains the filtered motion acceleration information, the motion acceleration values ​​of the two nodes in the smoothed rate sequence are obtained. By calculating the absolute value of the difference between the motion acceleration values ​​of the two nodes, the relative motion acceleration between the two nodes connected by each edge in the spatiotemporal topology is obtained. The calculated relative motion acceleration is assigned as a feature weight value and bound to the corresponding connecting edge. When all nodes in the spatiotemporal topology have been assigned the node state vector, and all connecting edges have completed the mapping and numerical assignment of the edge feature attributes, the overall construction of the undirected graph is completed. By cascading the three-dimensional spatial coordinates of the joints with their instantaneous linear velocities, a deep fusion of spatial form and dynamic velocity features is achieved, providing a multi-dimensional representation of node states. Spatiotemporal association edges are constructed based on natural anatomy and time series, fully depicting the physical constraints and dynamic evolution of human etiquette movements and effectively preserving spatiotemporal context information. Furthermore, mapping the relative motion acceleration between nodes to edge feature attributes can keenly capture the mechanical changes during the coordinated movement of adjacent joints.

[0043] The process of inputting the undirected graph into the autoencoder to calculate the reconstruction loss value and converting it into a quantized score is as follows: The undirected graph is input into a graph autoencoder, which uses the internal graph convolutional layer and attention mechanism, and uses the edge feature attributes as weight factors for node feature aggregation to extract the low-dimensional feature representation of the undirected graph in the spatiotemporal dimension. The low-dimensional feature representation is reconstructed using the decoding layer of the graph autoencoder, and a predicted undirected graph containing predicted coordinates, predicted instantaneous linear velocity, and predicted relative acceleration is output. The deviation values ​​between the predicted undirected graph and the original undirected graph in the coordinate dimension, velocity dimension, and acceleration dimension are calculated respectively, and the weighted sum of each deviation value is performed. The total deviation value is confirmed as the reconstruction loss value. The reconstruction loss value is mapped to a preset percentage score range using a mapping function, and a quantitative score is output. Specifically, the undirected graph is input into a pre-constructed graph autoencoder, and the graph convolutional layer and attention mechanism processing module inside the graph autoencoder are activated. During processing, the node state vectors of each node in the undirected graph are extracted as initial node features, and the edge feature attributes corresponding to each connecting edge are also extracted. Using the attention mechanism, the edge feature attributes are directly used as weight factors when aggregating the features of adjacent nodes; the graph convolutional layer performs weighted summation and information transfer on the node features of each central node and its connected neighbor nodes based on this weight factor, and edges with abnormal relative motion acceleration will receive different aggregation attention. After multi-layer processing and layer-by-layer feature dimensionality reduction by the graph convolutional layer, a low-dimensional feature representation of the undirected graph in the spatiotemporal dimension is extracted. This low-dimensional feature representation is a hidden layer state vector containing the comprehensive motion law of etiquette posture. The extracted low-dimensional feature representation is input into the decoding layer of the graph autoencoder. The decoding layer performs a topology mapping operation, the reverse of the feature extraction process, to demodulate the hidden layer state vector and restore its structure, attempting to reconstruct the original input graph network topology and attribute values. After reconstruction by the decoding layer, a complete and corresponding predicted undirected graph is finally output. This predicted undirected graph is structurally identical to the original undirected graph, specifically containing the predicted coordinates of each reconstructed joint, the predicted instantaneous linear velocity of each reconstructed joint, and the predicted relative acceleration on each associated edge.

[0044] The reconstructed data is compared item by item with the original input data, and the deviation values ​​of the predicted undirected graph and the original undirected graph are calculated in three independent dimensions. The predicted coordinates in the predicted undirected graph and the three-dimensional spatial coordinates in the original undirected graph are extracted, and the sum of the squares of the differences between the two in each spatial component is calculated as the deviation value of the coordinate dimension. The predicted instantaneous linear velocity in the predicted undirected graph and the instantaneous linear velocity in the original undirected graph are extracted, and the absolute value of the difference between the two is calculated as the deviation value of the velocity dimension. The predicted relative acceleration in the predicted undirected graph and the relative acceleration recorded in the edge feature attributes of the original undirected graph are extracted, and the absolute value of the difference between the two is calculated as the deviation value of the acceleration dimension. In this embodiment, the weighting rules for each deviation value are specifically preset as follows: the weighting coefficient for the deviation value in the coordinate dimension is 0.3, the weighting coefficient for the deviation value in the velocity dimension is 0.3, and the weighting coefficient for the deviation value in the acceleration dimension is 0.4. The reason for setting the weights is that minor abnormalities such as stiffness, abrupt exertion, or local discontinuity in ceremonial postures are often most directly and sensitively reflected in the unnatural abrupt changes in relative motion acceleration. Therefore, acceleration is given the highest weight to effectively amplify the incoordination features, while coordinates and velocity are given relatively balanced weights as basic spatiotemporal physical constraints. The deviation values ​​of the above three dimensions are multiplied by their corresponding preset weighting coefficients and then summed. The sum of the calculated weighted deviations is confirmed as the final reconstruction loss value.

[0045] The calculated reconstruction loss value is converted and mapped to a preset percentage score range using a pre-defined mapping function, thereby quantifying the anomaly assessment. In this embodiment, the preset percentage score range is specifically set as a standard range from zero to one hundred points. The mapping function is implemented as follows: with 100 as the baseline full score, the product of the reconstruction loss value and a preset amplification factor (in this embodiment, the preset amplification factor is specifically set to 10) is calculated, and the product value is subtracted from the baseline full score. During the calculation process, if the resulting value is less than zero, it is forcibly truncated to 0 points, thus strictly limiting the final calculation result to the preset percentage score range. Finally, the numerical result within this range is output as the quantitative score for judging whether the etiquette posture is abnormal.

[0046] By introducing a graph autoencoder and combining it with an attention mechanism, relative motion acceleration is used as a weighting factor, enabling the model to automatically focus on dynamic anomalies in limb coordination during feature aggregation, thus enhancing its ability to represent complex spatiotemporal motion patterns. By employing synchronous reconstruction and weighted calculation of coordinates, velocity, and acceleration, a comprehensive deviation evaluation model for multiple physical quantities is constructed, which can capture the deviation details of etiquette postures in spatial trajectory and motion patterns from all angles.

[0047] This application addresses the trajectory breakage problem in visual measurement by fusing walking images and point cloud data. It introduces a bidirectional weighted fusion interpolation algorithm based on instantaneous linear velocity and acceleration before and after the breakpoint, thus providing a complete and continuous spatiotemporal data foundation. To mitigate the systematic noise introduced by interpolation and measurement, a Bayesian network is used to extract the covariance matrix of the posterior probability as an uncertainty parameter. Dynamic weighted filtering is then applied to the completed kinematic data, overcoming the distortion in velocity and acceleration calculations and smoothly restoring the physical movement characteristics of the human body. By finely segmenting the stride period using lower limb extreme points, a spatiotemporal undirected graph containing relative motion acceleration attributes is constructed and input into an autoencoder. This deeply mines the implicit topological relationships of the skeleton in continuous motion, transforming the reconstruction loss into a quantitative score, enabling objective and precise anomaly recognition and assessment of etiquette postures in dynamic walking scenarios.

[0048] Example 2: This embodiment applies a method for identifying abnormal etiquette postures based on spatiotemporal trajectory features to a procession etiquette assessment scenario. A color high-definition camera with a sampling rate of 60 frames per second and a lidar sensor with a scanning frequency of 20Hz are deployed directly in front of the assessment channel. The camera continuously acquires a two-dimensional color pixel array of the marching queue as a procession image, with the image resolution set to 1920×1080 pixels. The lidar synchronously acquires a dense set of points containing spatial three-dimensional coordinate information as point cloud data. The camera and lidar achieve strict data alignment at the same timestamp through a hardware trigger signal.

[0049] A single-stage target detection model is used to process the moving image, retaining bounding boxes with a confidence score greater than 90% as detection boxes representing the overall physical contour of the examinee. Simultaneously, a pose estimation algorithm is used to extract 17 core two-dimensional pose joints of the examinee's entire body. The minimum bounding box formed by the extreme boundaries of these 17 joints is used as the pose box. The overlap area between each detection box and each pose box in the horizontal and vertical directions is calculated, and the intersection-union ratio (IU) is further obtained. The matching combination with the largest IU is selected as the two-dimensional reference box. The point cloud data is projected onto the moving image using an extrinsic parameter matrix. For the point cloud falling within the two-dimensional reference box, its Euclidean distance to the two-dimensional pose joint is calculated. The true 3D coordinates of the projected point cloud point with the smallest distance are assigned to the corresponding joint, and these are concatenated in chronological order according to timestamps to generate the initial 3D coordinate sequence of person B.

[0050] The initial 3D coordinate sequence is traversed, and the time interval between adjacent coordinate points is calculated. Positions with time intervals greater than 40ms are identified as breakpoints, and continuous coordinate sequences between breakpoints are extracted as continuous valid sequence segments. For each of three adjacent coordinate points within a continuous valid sequence segment, a first-order spatial position difference operation is performed, divided by the time difference to obtain the instantaneous linear velocity. For the instantaneous linear velocities at adjacent moments, a second-order difference operation is performed, divided by the time difference to obtain the motion acceleration. For the breakpoint interval, the 3D coordinates, instantaneous linear velocity, and motion acceleration of the last valid coordinate point before the breakpoint occur are extracted to construct a forward dynamics prediction vector. The data of the first recovered valid coordinate point after the breakpoint ends are extracted to construct a backward dynamics regression vector. The ratio of the time length between the missing timestamp and the two ends of the breakpoint is calculated to obtain forward and backward weights. These two vectors are used for bidirectional weighted fusion interpolation to complete the coordinates, velocities, and accelerations of the breakpoint interval, generating a continuous spatiotemporal trajectory.

[0051] The coordinate components of each time point in the continuous spatiotemporal trajectory are used as observation nodes of a dynamic Bayesian network. A transition matrix between motion states is constructed based on a sampling interval of 16.6 ms corresponding to 60 frames per second, and an observation likelihood function is established. A recursive probabilistic inference algorithm is used to calculate the prior motion state and prior error matrix at the current time. Probability corrections are performed based on actual observations, updating the posterior probability distribution of the motion state at each time point. The covariance matrix corresponding to this posterior probability distribution is extracted as an uncertainty parameter, and the filtering gain is determined based on the size of the diagonal elements in the covariance matrix. The filtering gain is used to perform weighted corrections on the completed instantaneous linear velocity and motion acceleration, reducing the trust weight of data with high uncertainty. The correction results are arranged and encapsulated along the time axis, outputting a smoothed rate sequence.

[0052] The left and right ankle nodes are extracted as target joints from the smooth velocity sequence, and extreme points where the instantaneous linear velocity is zero are found. The velocity judgment threshold is preset to 0.03 m / s, and the local search time window is preset to 20 frames before and after the current time frame. When the instantaneous linear velocity of the ankle node in a certain frame is less than 0.03 m / s and is the minimum value within the 20 frames before and after, it is identified as an extreme point and its timestamp is extracted. The timestamps of the left and right extreme points are merged and arranged chronologically as time boundaries. The continuous spatiotemporal trajectory is truncated sequentially using these time boundaries to segment multiple stride trajectory segments containing the complete action of a single step.

[0053] The segmented stride trajectory is traversed, and the three-dimensional spatial coordinates (lateral, longitudinal, and depth values) of 17 joints are extracted frame by frame. These coordinates are then concatenated with the corresponding instantaneous linear velocity values ​​from the smoothed rate sequence to generate a one-dimensional data queue containing values ​​in four dimensions, serving as the node state vector of the undirected graph. Spatial adjacency edges are established in the single-frame spatial dimension according to natural anatomical rules (e.g., shoulder to elbow, elbow to wrist). In the cross-frame temporal dimension, temporal association edges are established by directly connecting the same joints at adjacent moments, forming a spatiotemporal topology. The motion acceleration values ​​of the nodes at both ends of each edge in the spatiotemporal topology are extracted from the smoothed rate sequence. The absolute value of the difference between the two values ​​is calculated to obtain the relative motion acceleration, which is then used as a feature weight value and assigned to the corresponding connecting edge, completing the construction of the undirected graph.

[0054] The constructed undirected graph is input into the graph autoencoder. The internal graph convolutional layer utilizes an attention mechanism, directly using edge feature attributes (relative motion acceleration) as weight factors to perform weighted summation on the features of the center node and its neighboring nodes, extracting a low-dimensional feature representation in the spatiotemporal dimension. The decoding layer reconstructs this low-dimensional feature representation, outputting a predicted undirected graph containing predicted coordinates, predicted velocity, and predicted acceleration. The squared difference in the coordinate dimension and the absolute difference in the velocity and acceleration dimensions between the predicted undirected graph and the original undirected graph are calculated respectively. The weight coefficients for the coordinate dimension deviation are set to 0.25, the velocity dimension to 0.25, and the acceleration dimension to 0.50. The three are then weighted and summed to obtain the reconstruction loss value. With 100 as the baseline and a maximum score, the reconstruction loss value is multiplied by a preset amplification factor of 8. This product is subtracted from 100; if it is less than zero, it is truncated to 0. The final output is the quantitative score of person B's etiquette posture for this segment.

[0055] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for recognizing etiquette posture anomaly based on spatiotemporal trajectory features, characterized in that, include: Acquire moving images and point cloud data; The intersection-over-union ratio (IoU) of the detection box and the attitude box in the traveling image is calculated to lock the two-dimensional reference box. The two-dimensional attitude joints in the two-dimensional reference box are extracted and mapped to the point cloud data. An initial three-dimensional coordinate sequence is generated in chronological order. Identify breakpoints in the initial three-dimensional coordinate sequence, extract continuous valid sequence segments divided by the breakpoints, and calculate instantaneous linear velocity and motion acceleration; interpolate the missing coordinates based on the instantaneous linear velocity and motion acceleration before and after the breakpoints, complete the instantaneous linear velocity and motion acceleration, and generate a continuous spatiotemporal trajectory; The continuous spatiotemporal trajectory is input into a Bayesian network to extract uncertainty parameters. The completed instantaneous linear velocity and motion acceleration are then filtered according to the uncertainty parameters to obtain a smooth velocity sequence. Using the extreme point where the instantaneous linear velocity of the target joint in the smooth rate sequence is zero as the time boundary, the continuous spatiotemporal trajectory is divided into stride trajectory segments; an undirected graph is constructed based on the stride trajectory segments and input into an autoencoder to calculate the reconstruction loss value, which is then converted into a quantitative score to determine whether the etiquette posture is abnormal. 2.The etiquette gesture anomaly recognition method based on space-time trajectory features according to claim 1, characterized in that, The detection box is the outer bounding box that represents the overall physical contour of the target when the target detection algorithm is used to locate the human target in the moving image. The attitude bounding box is the smallest bounding box determined by the set of points formed by all the two-dimensional attitude joints after the target's two-dimensional attitude joints are extracted by processing the traveling image using an attitude estimation algorithm.

3. The method of claim 2, wherein, The intersection-union ratio (IUU) of the detection box and the pose box in the moving image is the ratio of the area of ​​the overlapping region of the detection box and the pose box to the difference between the area of ​​the overlapping region and the sum of the areas of the detection box and the pose box. The step of calculating the intersection-union ratio (IUU) of the detection boxes and pose boxes in the moving image to lock the two-dimensional reference box includes: calculating the IUU between each of the detection boxes and each of the pose boxes in the moving image; The detection box with the largest intersection-union ratio (IUU) and the pose box are selected as a matching combination, and the smallest bounding rectangle containing the matching combination is locked as the two-dimensional reference box. 4.The etiquette gesture anomaly recognition method based on spatio-temporal trajectory features according to claim 1, characterized in that, The initial three-dimensional coordinate sequence generation process involves projecting the point cloud data onto the traveling image based on the extrinsic parameter matrix, and calculating the Euclidean distance between each point cloud point and each of the two-dimensional attitude joints. Based on the principle of minimum distance, the three-dimensional coordinates of the nearest point cloud point are used as the spatial position of the corresponding two-dimensional attitude joint, and the initial three-dimensional coordinate sequence of the corresponding joint is generated by combining the timestamp. 5.The etiquette gesture anomaly recognition method based on spatio-temporal trajectory features according to claim 1, characterized in that, The continuous spatiotemporal trajectory involves traversing the initial three-dimensional coordinate sequence, identifying positions where the time interval between adjacent coordinate points exceeds a preset frame interval threshold as breakpoints, and extracting the continuous coordinate segments segmented by the breakpoints as the continuous valid sequence segments; performing differential calculations on the continuous valid sequence segments to obtain the instantaneous linear velocity and motion acceleration of each joint point as a function of time. A forward dynamics prediction vector is constructed based on the instantaneous linear velocity and acceleration before the breakpoint, and a backward dynamics regression vector is constructed based on the instantaneous linear velocity and acceleration after the breakpoint. The forward dynamics prediction vector and the backward dynamics regression vector are used to perform bidirectional weighted fusion interpolation on the missing coordinates of the breakpoint interval to generate a continuous spatiotemporal trajectory.

6. The method for identifying abnormal etiquette postures based on spatiotemporal trajectory features according to claim 1, characterized in that, The Bayesian network maps the coordinate components of each time point in the continuous spatiotemporal trajectory to observation nodes of the dynamic Bayesian network, and establishes the transition matrix between the motion states of adjacent time points and the observation likelihood function between the motion states and the observation nodes; using a recursive probabilistic inference algorithm, the posterior probability distribution of the motion states at each time point is updated by combining the transition matrix and the observation likelihood function, and the covariance matrix corresponding to the posterior probability distribution is extracted as the uncertainty parameter; The filtering gain corresponding to each moment is determined based on the magnitude of each element in the covariance matrix. The filtering gain is used to perform weighted correction on the completed instantaneous linear velocity and motion acceleration. The weighted correction results are arranged and encapsulated in chronological order along the time axis and output as the smoothed rate sequence.

7. The method for identifying abnormal etiquette postures based on spatiotemporal trajectory features according to claim 1, characterized in that, The process of constructing an undirected graph based on the stride trajectory segment involves extracting the three-dimensional spatial coordinates of each joint point within the stride trajectory segment and concatenating them with the corresponding instantaneous linear velocity in the smooth rate sequence to generate the node state vector of the undirected graph. Spatial adjacency edges are established according to the natural anatomical connections of the human skeleton, and temporal correlation edges between adjacent moments of the same joint are established according to the chronological order to construct a spatiotemporal topology. The relative motion acceleration between the two nodes connected by each edge in the spatiotemporal topology is calculated, and the calculated relative motion acceleration is mapped to the edge feature attributes of each corresponding edge to construct an undirected graph.

8. The method for identifying abnormal etiquette postures based on spatiotemporal trajectory features according to claim 1, characterized in that, The process of inputting the undirected graph into the autoencoder to calculate the reconstruction loss value and converting it into a quantized score is as follows: The undirected graph is input into a graph autoencoder, which uses the internal graph convolutional layer and attention mechanism, and uses the edge feature attributes as weight factors for node feature aggregation to extract the low-dimensional feature representation of the undirected graph in the spatiotemporal dimension. The low-dimensional feature representation is reconstructed using the decoding layer of the graph autoencoder, and a predicted undirected graph containing predicted coordinates, predicted instantaneous linear velocity, and predicted relative acceleration is output. The deviation values ​​between the predicted undirected graph and the original undirected graph in the coordinate dimension, velocity dimension, and acceleration dimension are calculated respectively, and the weighted sum of each deviation value is performed. The total deviation value is confirmed as the reconstruction loss value. The reconstruction loss value is mapped to a preset percentage score range using a mapping function, and a quantitative score is output.

Images