Method for evaluating infant motion based on multi-expert model

Abstract

This invention provides a method for infant motion assessment based on a multi-expert model, relating to the technical field of computer vision. First, key points of the infant's two-dimensional skeleton are extracted, a two-dimensional skeleton sequence is constructed and preprocessed to obtain several local segments of the two-dimensional skeleton sequence; abnormal segments are selected and enhanced to construct a training dataset; a hierarchical hybrid expert model is constructed; segments from the training dataset are input into the spatial hybrid expert layer to extract and fuse local motion features, obtaining spatial fusion features; the spatial fusion features are input into the temporal hybrid expert layer to extract segment-level representations of each local segment. The model is trained by inputting local segments of the infant to be assessed into the trained model to extract segment-level representations of each local segment; the segment-level representations are aggregated to obtain a video-level representation; and the infant motion assessment result is output using a classification head. This invention utilizes a hierarchical hybrid expert model to improve the accuracy and stability of infant motion assessment.

Description

Technical Field

[0001] This invention belongs to the technical field of computer vision, and more specifically relates to a method for evaluating infant movements based on a multi-expert model. Background Technology

[0002] In the medical field, infant posture analysis is of great value for the identification, diagnosis, and treatment of motor disorders and developmental delays. Especially in cerebral palsy risk screening, the movement patterns exhibited by infants' spontaneous movements can provide crucial evidence for clinical assessment. However, current clinical screening still heavily relies on manual observation and experience-based judgment, and challenges remain in large-scale screening, long-term follow-up, and consistency of assessment standards.

[0003] With the development of computer vision and action recognition technologies, action assessment methods based on video analysis and skeleton sequence modeling have provided new technical approaches for the early screening of cerebral palsy risk. However, existing infant action assessment methods still have limitations in practical applications. Infant movements themselves are highly irregular and exhibit individual differences, with abnormal manifestations often showing non-uniform changes in different body parts and at different time stages. Traditional unified encoding methods for entire skeleton sequences struggle to simultaneously capture local limb features, temporal dynamic changes, and global temporal relationships, affecting the accuracy of infant action assessment. Furthermore, discriminative information related to infant action abnormalities is not always significant in long-term movements, making it difficult to extract key abnormal segments and reducing the reliability of abnormality identification. Simultaneously, the acquisition and annotation of infant-related data are costly, and the distribution of categories in infant-related data exhibits a certain degree of imbalance, further increasing the training difficulty of infant action assessment models.

[0004] In summary, existing technologies struggle to simultaneously capture infants' local limb features, temporal dynamics, and global temporal relationships, and abnormality discrimination information is not significant in long-term movements, resulting in low accuracy and stability in infant movement assessment. Summary of the Invention

[0005] To address the issues of low accuracy and stability in existing infant motor assessment methods, this invention proposes an infant motor assessment method based on a multi-expert model, which improves the accuracy and stability of infant motor assessment.

[0006] To achieve the above-mentioned technical effects, the technical solution of the present invention is as follows: S1: Based on infant movement videos, extract key points of the infant's two-dimensional skeleton and construct a two-dimensional skeleton sequence; preprocess the two-dimensional skeleton sequence to obtain several local segments of the two-dimensional skeleton sequence. S2: Based on local fragments of the two-dimensional skeleton sequence, select abnormal local fragments of the two-dimensional skeleton sequence for enhancement to obtain local enhanced fragments of the two-dimensional skeleton sequence; construct a training dataset based on the local fragments of the two-dimensional skeleton sequence and the local enhanced fragments of the two-dimensional skeleton sequence. S3: Construct a hierarchical hybrid expert model containing a spatial hybrid expert layer and a temporal hybrid expert layer; input several local segments of two-dimensional skeleton sequences in the training dataset into the spatial hybrid expert layer, extract and fuse the local motion features to obtain spatial fusion features; input the spatial fusion features into the temporal hybrid expert layer to extract the segment-level representation of each local segment; S4: Train the hierarchical hybrid expert model using the training dataset to obtain the trained hierarchical hybrid expert model; S5: Input several local segments of the two-dimensional skeleton sequence of the infant to be evaluated into the trained hierarchical hybrid expert model, extract the segment-level representation of each local segment; aggregate the segment-level representations of each local segment to obtain the video-level representation; based on the video-level representation, use the preset classification head to output the infant's action evaluation results.

[0007] Furthermore, step S1 is as follows: S101: Randomly select some image frames from the baby movement video, and annotate the key points of the baby's two-dimensional skeleton in the selected image frames to construct a baby pose annotation dataset; S102: Train the YOLO11 model using the infant pose annotation dataset, optimize the YOLO11 model parameters, and obtain an optimized YOLO11 model suitable for extracting key points of the infant's two-dimensional skeleton. S103: Using the optimized YOLO11 model, extract key points of the infant's 2D skeleton frame by frame. The expression is:

[0008] In the formula, This indicates the first video of baby movement. t The first frame j The x-coordinates of the key points This indicates the first video of baby movement. t The first frame j The ordinates of the key points This indicates the first video of baby movement. t The first frame j Detection confidence of each key point; S104: Extract key points of the infant's two-dimensional skeleton frame by frame and arrange them in chronological order to construct a two-dimensional skeleton sequence; normalize and downsample the two-dimensional skeleton sequence to obtain a downsampled two-dimensional skeleton sequence. S105: Using the sliding window method, the downsampled two-dimensional skeleton sequence is divided into several local segments of the two-dimensional skeleton sequence, expressed as:

[0009] In the formula, Indicates the first A local fragment of a two-dimensional skeleton sequence This represents the total number of local segments in a two-dimensional skeleton sequence.

[0010] Furthermore, the process of enhancing local fragments of the abnormal two-dimensional skeleton sequence is as follows: Step 1: For the selected local segments of the abnormal 2D skeleton sequence, define a set of skeleton edges based on the human skeletal structure, with each skeleton edge connecting two adjacent keypoints; calculate the bone vector of the skeleton edge in frame t within the local segment of the abnormal 2D skeleton sequence, expressed as:

[0011] In the formula, Indicates from the key point Pointing to key points bone vector, Indicates the first In a local segment of a two-dimensional skeleton sequence of anomalies, the key points are in frame t. x-axis; Indicates the first In a local segment of a two-dimensional skeleton sequence of anomalies, the key points are in frame t. ordinate; Indicates the first In a local segment of a two-dimensional skeleton sequence of anomalies, the key points are in frame t. x-axis; Indicates the first In a local segment of a two-dimensional skeleton sequence of anomalies, the key points are in frame t. ordinate; Step 2: Arrange the bone vectors of all skeleton edges of each local segment of the two-dimensional skeleton sequence of the anomaly class in time order and skeleton edge order to obtain the representation of the local segment of the two-dimensional skeleton sequence of the anomaly class in the bone vector space. Step 3: Take the representation of each local segment of the two-dimensional skeleton sequence of anomaly class in bone vector space as the reference segment, and calculate the bone vector space distance between the local segments of the remaining two-dimensional skeleton sequences of anomaly classes and the reference segment. The expression for calculating the bone vector space distance is:

[0012] In the formula, This represents the representation of the reference segment in the bone vector space. Indicates the first Representation of each exception class in the bone vector space Represents the distance in the bone vector space. Indicates the first A local fragment of a two-dimensional skeleton sequence of anomalies from key points Pointing to key points bone vector, Represents the set of skeleton edges. Indicates the length of a local segment in a two-dimensional skeleton sequence; Step 4: Select the three local fragments of the two-dimensional skeleton sequence of the anomaly class with the smallest spatial distance in the bone vector space as candidate fragments; for each candidate fragment, generate a candidate synthetic anomaly class two-dimensional skeleton sequence local fragment, expressed as:

[0013] In the formula, This represents a local fragment of a two-dimensional skeleton sequence representing a candidate synthesis anomaly. This represents a local segment of a two-dimensional skeleton sequence representing an anomaly class. This represents the gray wolf search coefficient. Indicates the reference segment, This represents element-wise multiplication; Step 5: Perform weighted fusion on local fragments of the candidate synthetic anomaly class two-dimensional skeleton sequence to obtain the synthetic bone vector fragment, expressed as:

[0014] In the formula, This represents a composite bone vector fragment. Indicates the fusion weight coefficient. Indicates the fusion weight coefficient. Indicates the fusion weight coefficient. Indicates the first Local fragments of two-dimensional skeleton sequences of candidate synthetic anomalies Indicates the first Local fragments of two-dimensional skeleton sequences of candidate synthetic anomalies Indicates the first Local fragments of two-dimensional skeleton sequences of candidate synthetic anomalies; Step 6: Assign the two-dimensional coordinates of the root node preset in the reference segment to the root node coordinates of the corresponding frame of the synthesized bone vector segment; based on the skeleton edge set, for each skeleton edge in each frame, use the bone vector in the synthesized bone vector segment to calculate the coordinates of the remaining key points in turn, until the coordinates of all key points in each frame of the synthesized bone vector segment are obtained, and the synthesized coordinate segment is obtained; the detection confidence of each key point is the confidence value of the key point in the corresponding reference segment; Step 7: Based on the synthesized coordinate fragment, calculate the inter-frame displacement of the j-th keypoint between adjacent frames. If the velocity variance of any keypoint exceeds a preset threshold, discard the synthesized bone vector fragment and return to Step 3; otherwise, proceed to Step 8. Step 8: Recombine the remaining synthetic bone vector fragments according to the original time sequence, and perform average fusion on the overlapping areas between adjacent synthetic bone vector fragments to obtain local enhancement fragments of the two-dimensional skeleton sequence.

[0015] Furthermore, the calculation expression for the gray wolf search coefficient is as follows:

[0016] In the formula, Represents a random vector. Indicates the attenuation coefficient. This represents the gray wolf search coefficient.

[0017] Furthermore, the hierarchical hybrid expert model includes: a spatial hybrid expert layer and a temporal hybrid expert layer connected in sequence; The spatial hybrid expert layer includes: a first spatial expert module for extracting local motion features of the upper body region, a second spatial expert module for extracting local motion features of the left lower limb region, and a third spatial expert module for extracting local motion features of the right lower limb region; it also includes: a spatial routing module for fusing local motion features; the outputs of the first spatial expert module, the second spatial expert module, and the third spatial expert module are all connected to the spatial routing module; The first spatial expert module, the second spatial expert module, and the third spatial expert module each include: a first graph convolutional network and a second graph convolutional network connected in sequence; The time-mixing expert layer includes: a first time expert module for extracting time dynamic features of low-speed pause states, a second time expert module for extracting time dynamic features of continuous steady motion states, and a third time expert module for extracting time dynamic features of sudden transition states; it also includes: a time routing module for fusing time dynamic features; the outputs of the first time expert module, the second time expert module, and the third time expert module are all connected to the time routing module.

[0018] Furthermore, in the spatial hybrid expert layer: The local segments of the two-dimensional skeleton sequence are divided into upper body, left lower limb, and right lower limb regions according to a predefined body area, and local motion features are extracted for each region. The expression is as follows:

[0019]

[0020] In the formula, This represents the activation function. Indicates feature splicing, and Let represent the learnable parameters of the m-th spatial expert module. This represents the local motion features extracted by the m-th spatial expert module. This represents the intermediate features extracted by the first graph convolutional network in the m-th spatial expert module. Represents the normalized adjacency matrix. This represents a local segment of the two-dimensional skeleton sequence input by the m-th spatial expert module; Each local segment of the two-dimensional skeleton sequence is input into a pre-defined gating network in the spatial routing module. This gating network uses a global graph convolutional network to extract frame-level features; then, a short-window LSTM network is used to obtain a segment-level gating representation, expressed as:

[0021] In the formula, This indicates a fragment-level gating representation. This indicates a short-window LSTM network. This represents a global graph convolutional network. Represents a local segment of a two-dimensional skeleton sequence; Mapping the fragment-level gating representation to the routing weights of the spatial expert module, the expression is:

[0022] In the formula, Indicates the first Spatial routing weights of each spatial expert module Indicates a fully connected layer; Based on routing weights, the spatial fusion features are obtained by fusing various local motion features, and the expression is as follows:

[0023] In the formula, Indicates spatial integration characteristics, This represents the local motion features extracted by the m-th spatial expert module.

[0024] Furthermore, in the time-mixing expert layer: Based on spatial fusion features, the time dynamic features of low-speed pause state, continuous steady motion state, and sudden transition state are extracted, and the expression is:

[0025] In the formula, This represents the nth time expert module. Indicates spatial integration characteristics, This represents the time dynamic characteristics output by the nth time expert module; Each local segment of the two-dimensional skeleton sequence is input into a pre-defined gating network in the temporal routing module. This gating network uses a global graph convolutional network to extract frame-level features; a short-window LSTM network is then used to obtain segment-level gating representations; these segment-level gating representations are mapped to routing weights for the temporal expert module, expressed as:

[0026] In the formula, Indicates the first The routing weight of a local segment to the nth time expert module. Indicates a fully connected layer. This represents a fragment-level gated representation; Based on routing weights, the time dynamic features of low-speed pause states, continuous steady motion states, and sudden transition states are fused to obtain the segment-level representation of each local segment, expressed as:

[0027] In the formula, Indicates the first Fragment-level representation of a local segment.

[0028] Furthermore, during the training process, spatial routing constraints are introduced into the spatial hybrid expert layer. The specific steps are as follows: For each local segment of the two-dimensional skeleton sequence, calculate the average velocity and regional velocity synchronization of each preset body region; The expression for calculating average velocity is as follows:

[0029] In the formula, L Indicates the length of a local segment. This represents the velocity magnitude of the j-th joint between adjacent frames. This represents the set of key points for the m-th body region. Indicates average speed; The expression for calculating regional speed synchronization is:

[0030] In the formula, Indicates regional speed synchronization. Indicates the velocity correlation coefficient; Using the regional average velocity and regional velocity synchronization, a prior score for the region is constructed; the prior scores of each space expert are normalized to obtain the spatial prior route distribution, expressed as:

[0031] In the formula, Represents the spatial prior route distribution. Indicates the prior score; The difference between the route weights generated by the spatial routing module and the spatial prior route distribution is used as a spatial routing constraint, expressed as:

[0032] In the formula, Indicates the Cauchy-Schwarz divergence. Indicates the route weight. This represents spatial routing constraints.

[0033] Furthermore, during training, a temporal routing constraint loss is introduced into the temporal hybrid expert layer. The specific steps are as follows: For each local segment of the two-dimensional skeleton sequence, calculate the overall velocity of each frame and the change in adjacent frames; The expression for calculating the overall velocity is as follows:

[0034] In the formula, Indicates the number of key points throughout the body. Indicates the first The velocity amplitude of the j-th keypoint in the t-th frame of a segment between adjacent frames. Indicates the first The overall speed of the t-th frame in the segment; The expression for calculating the change between adjacent frames is:

[0035] In the formula, Indicates the amount of change between adjacent frames. Indicates the first The overall speed of the (t-1)th frame in the segment; By normalizing the overall velocity and the changes in adjacent frames, a two-dimensional velocity state for each frame is constructed, expressed as:

[0036] In the formula, Represents two-dimensional velocity state. Indicates the normalized i-th The overall speed of frame t in the segment. Indicates the normalized i-th The change in the adjacent frames of the t-th frame in a segment; Given a pre-defined fixed-speed state prototype, calculate the prior scores of each time-based expert module using the following expression:

[0037] In the formula, Indicates the prior score. This represents a prototype of a fixed-velocity state. Normalizing the prior scores of each time expert yields the time prior route distribution, expressed as:

[0038] In the formula, Indicates the temporal prior route distribution; The difference between the route weights generated by the time-based routing module and the time-prior route distribution is calculated as the time-based routing constraint loss, expressed as:

[0039] In the formula, This represents the time-based routing constraint loss. Indicates the Cauchy-Schwarz divergence. Indicates the first The routing weight of each local segment to the nth time expert module.

[0040] Furthermore, the training process also includes: selective fragment contrastive learning of the fragment-level representations output by the hierarchical hybrid expert model, specifically: For local segments of a two-dimensional skeleton sequence, a hierarchical hybrid expert model is used to extract segment-level representations of each local segment; Using the fragment scoring function, the anomaly score of the fragment-level representation of each local fragment is calculated, as expressed by:

[0041] In the formula, This indicates the b-th baby movement video. Anomaly scores of local segments of a two-dimensional skeleton sequence This represents the segment scoring function. This indicates the first movement video of the b-th baby. Fragment-level representation of local segments of a two-dimensional skeleton sequence; In each infant movement video, select the K highest-scoring two-dimensional skeleton sequence local segments and construct a segment index set, expressed as:

[0042] In the formula, This indicates the preset selection value. Represents a set of fragment indices. This indicates the first movement video of the b-th baby. Anomaly score of local segments of a two-dimensional skeleton sequence; Within the same training batch, a set of comparison segments is created based on the segment indices of each infant's movement video. The expression is:

[0043] In the formula, Represents a set of contrasting segments. Indicates the training batch size; Normalize the fragment-level representation of each local segment of the two-dimensional skeleton sequence in the comparison fragment set to obtain the normalized fragment-level representation of the local segment of the two-dimensional skeleton sequence. Calculate the normalized cosine similarity distribution between the fragment-level representations of the local segments of the two-dimensional skeleton sequences in the comparison fragment set and the fragment-level representations of the other local segments of the two-dimensional skeleton sequences. The expression is as follows:

[0044] In the formula, Represents the normalized cosine similarity distribution. This represents the fragment-level representation of a local segment of the two-dimensional skeleton sequence of the i-th segment after normalization. This represents the fragment-level representation of a local segment of the two-dimensional skeleton sequence of the j-th segment after normalization. The fragment-level representation of a local segment of a two-dimensional skeleton sequence representing the k-th segment after normalization; Based on the tags in the baby's movement videos, construct the target distribution; construct a set of local segments of a two-dimensional skeleton sequence with similar annotations, expressed as:

[0045] In the formula, This represents a set of local segments of a two-dimensional skeleton sequence with similar annotations. Represents the local segment index of the two-dimensional skeleton sequence labeled with the same type as the i-th segment; Align the normalized cosine similarity distribution with a set of local segments of the same labeled 2D skeleton sequence, and construct a segment contrast loss using CS divergence, expressed as:

[0046] In the formula, Indicates fragment contrast loss, This represents the similarity distribution predicted by the model. Indicates the target distribution.

[0047] Compared with existing technologies, the beneficial effects of this method are: This invention provides a method for infant movement assessment based on a multi-expert model. First, based on infant movement videos, key points of a two-dimensional skeleton are extracted and preprocessed to obtain local segments. The original video is then converted into a structured skeleton sequence, preserving key movement dynamics. Abnormal local segments are screened and enhanced to construct a training dataset, effectively alleviating the class imbalance problem caused by insufficient abnormal samples and enhancing the model's learning ability for minority classes. A hierarchical hybrid expert model is constructed, including spatial and temporal hybrid expert layers. Spatial fusion features and segment-level representations are extracted sequentially, and targeted modeling is performed for different body regions and different short-term dynamic patterns, avoiding feature confusion caused by uniform encoding and improving the model's ability to express local limb abnormalities and temporal changes. The model is trained using the training dataset. Local segments of the infant to be assessed are input into the trained model, and the segment-level representations are aggregated to obtain video-level representations, outputting the assessment results. By comprehensively utilizing the spatiotemporal information of the entire segment, stable and accurate infant movement assessment results are output. The hierarchical hybrid expert model constructed and trained using this invention improves the accuracy and stability of infant movement assessment. Attached Figure Description

[0048] Figure 1 A flowchart illustrating the infant movement assessment method based on a multi-expert model proposed in this embodiment of the invention; Figure 2 This diagram illustrates the framework of the infant motor assessment method based on a multi-expert model proposed in this embodiment of the invention. Figure 3 This is a schematic diagram of the infant pose extraction and key point skeleton proposed in the embodiments of the present invention; Figure 4 The graph shows a comparison of ROC curves between the infant movement assessment method based on a multi-expert model proposed in this invention and existing methods. Detailed Implementation

[0049] The accompanying drawings are for illustrative purposes only and should not be construed as limiting the scope of this patent. To better illustrate this embodiment, some parts of the accompanying drawings may be omitted, enlarged, or reduced, and do not represent the actual dimensions; It is understandable to those skilled in the art that some well-known details may be omitted from the accompanying drawings.

[0050] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0051] The positional relationships depicted in the accompanying drawings are for illustrative purposes only and should not be construed as limiting this patent. Example 1 This embodiment proposes an infant motor assessment method based on a multi-expert model, such as... Figure 1The flowchart shown and Figure 2 The framework diagram shown includes the following steps: S1: Based on infant movement videos, extract key points of the infant's two-dimensional skeleton and construct a two-dimensional skeleton sequence; preprocess the two-dimensional skeleton sequence to obtain several local segments of the two-dimensional skeleton sequence. S2: Based on local fragments of the two-dimensional skeleton sequence, select abnormal local fragments of the two-dimensional skeleton sequence for enhancement to obtain local enhanced fragments of the two-dimensional skeleton sequence; construct a training dataset based on the local fragments of the two-dimensional skeleton sequence and the local enhanced fragments of the two-dimensional skeleton sequence. S3: Construct a hierarchical hybrid expert model containing a spatial hybrid expert layer and a temporal hybrid expert layer; input several local segments of two-dimensional skeleton sequences in the training dataset into the spatial hybrid expert layer, extract and fuse the local motion features to obtain spatial fusion features; input the spatial fusion features into the temporal hybrid expert layer to extract the segment-level representation of each local segment; S4: Train the hierarchical hybrid expert model using the training dataset to obtain the trained hierarchical hybrid expert model; S5: Input several local segments of the two-dimensional skeleton sequence of the infant to be evaluated into the trained hierarchical hybrid expert model, extract the segment-level representation of each local segment; aggregate the segment-level representations of each local segment to obtain the video-level representation; based on the video-level representation, use the preset classification head to output the infant's action evaluation results.

[0052] The infant movement assessment method based on a multi-expert model proposed in this embodiment takes a hierarchical hybrid expert structure as its core, performs targeted modeling of different body regions and dynamics at different times, and combines selective segment contrastive learning to strengthen the representation of key abnormal segments; and supplements and optimizes the minority class training distribution by kinematic constraint gray wolf synthesis enhancement in bone vector space, thereby improving the accuracy and stability of infant movement assessment, and can be applied to infant cerebral palsy assisted screening.

[0053] In this embodiment, as Figure 3 The diagram shown illustrates infant pose extraction and keypoint skeleton. Step S1 is as follows: S101: Randomly select some image frames from the baby movement video, and annotate the key points of the baby's two-dimensional skeleton in the selected image frames to construct a baby pose annotation dataset; S102: Train the YOLO11 model using the infant pose annotation dataset, optimize the YOLO11 model parameters, and obtain an optimized YOLO11 model suitable for extracting key points of the infant's two-dimensional skeleton. S103: Using the optimized YOLO11 model, extract key points of the infant's 2D skeleton frame by frame. The expression is:

[0054] In the formula, This indicates the first video of baby movement. t The first frame j The x-coordinates of the key points This indicates the first video of baby movement. t The first frame j The ordinates of the key points This indicates the first video of baby movement. t The first frame j Detection confidence of each key point; S104: Extract key points of the infant's two-dimensional skeleton frame by frame and arrange them in chronological order to construct a two-dimensional skeleton sequence; normalize and downsample the two-dimensional skeleton sequence to obtain a downsampled two-dimensional skeleton sequence. S105: Using the sliding window method, the downsampled two-dimensional skeleton sequence is divided into several local segments of the two-dimensional skeleton sequence, expressed as:

[0055] In the formula, Indicates the first A local fragment of a two-dimensional skeleton sequence This represents the total number of local segments in a two-dimensional skeleton sequence.

[0056] Specifically, in this embodiment, an optimized YOLO11 model is used to extract 21 two-dimensional skeleton key points frame by frame from an infant's motion video. Since infants differ from general human scenes in limb proportions, posture distribution, and occlusion, a general pre-trained model is not used directly. Instead, partial images are extracted from the infant video for manual key point annotation, and YOLO11 is optimized based on this annotation to improve the accuracy and stability of key point extraction in infant scenes.

[0057] Several two-dimensional skeleton sequences consist of local segments that are a set of overlapping, continuous local segments, which are continuous L The system consists of frames, each containing 21 2D skeleton keypoints. Each keypoint includes normalized coordinates and detection confidence. (x,y,c) , as input to the hierarchical hybrid expert model.

[0058] For example, downsampling a two-dimensional skeleton sequence involves downsampling the time dimension by a factor of 2. The sliding window length is set. L Set it to 120, and the sliding step size to 60.

[0059] To alleviate the training bias caused by insufficient outlier samples, this embodiment performs neighborhood augmentation on outlier segments during the training phase. Existing GWO-SMOTE methods can generate richer minority class samples than ordinary SMOTE through the gray wolf search mechanism, but they typically perform neighborhood retrieval and interpolation in the original feature space, making them susceptible to overall positional shifts when directly applied to infant skeletal sequences. To address this issue, this application shifts the synthesis process to the bone vector space, enabling the augmented samples to better reflect local limb structures and short-term motion changes, rather than a simple mixture of original coordinates.

[0060] For anomalous skeleton sequences, first divide them into multiple segments of length [length missing] using the same sliding window strategy as described earlier. L A partial segment, denoted as the first The segment is Local fragments of the anomalous 2D skeleton sequence are subjected to proximity retrieval and bone vector synthesis. After synthesis and velocity filtering, the enhanced fragments are recombined according to the original temporal order to form a new anomalous skeleton sequence. This new sequence is used as the output of Bone Vector Kinematic Constraint Gray Wolf Synthesis Enhancement (KBGWA) and together with the original 2D skeleton sequence local fragments to construct the training dataset.

[0061] In this embodiment, the process of enhancing local fragments of the abnormal two-dimensional skeleton sequence is as follows: Step 1: For the selected abnormal 2D skeleton sequence local fragments, define the skeleton edge set according to the human skeletal structure. E Each skeleton edge connects two adjacent keypoints; calculate the skeleton edges in the t-th frame of a local segment of an anomaly class 2D skeleton sequence. The bone vector is expressed as:

[0062] In the formula, Indicates from the key point Pointing to key points bone vector, Indicates the first In a local segment of a two-dimensional skeleton sequence of anomalies, the key points are in frame t. x-axis; Indicates the first In a local segment of a two-dimensional skeleton sequence of anomalies, the key points are in frame t. ordinate; Indicates the first In a local segment of a two-dimensional skeleton sequence of anomalies, the key points are in frame t. x-axis; Indicates the first In a local segment of a two-dimensional skeleton sequence of anomalies, the key points are in frame t. ordinate; Step 2: For each local segment of the two-dimensional skeleton sequence of anomaly class, arrange the bone vectors of all skeleton edges in chronological order and skeleton edge order to obtain the representation of the local segment of the two-dimensional skeleton sequence of anomaly class in the bone vector space. ; Step 3: Take the representation of each local segment of the two-dimensional skeleton sequence of anomaly class in bone vector space as the reference segment, and calculate the bone vector space distance between the local segments of the remaining two-dimensional skeleton sequences of anomaly classes and the reference segment. The expression for calculating the bone vector space distance is:

[0063] In the formula, This represents the representation of the reference segment in the bone vector space. Indicates the first Representation of each exception class in the bone vector space Represents the distance in the bone vector space. Indicates the first A local fragment of a two-dimensional skeleton sequence of anomalies from key points Pointing to key points bone vector, Represents the set of skeleton edges. Indicates the length of a local segment in a two-dimensional skeleton sequence; Step 4: Select the three local fragments of the two-dimensional skeleton sequence of the anomaly class with the smallest spatial distance in the bone vector space as candidate fragments; for each candidate fragment, generate a candidate synthetic anomaly class two-dimensional skeleton sequence local fragment, expressed as:

[0064] In the formula, This represents a local fragment of a two-dimensional skeleton sequence representing a candidate synthesis anomaly. This represents a local segment of a two-dimensional skeleton sequence representing an anomaly class. This represents the gray wolf search coefficient. Indicates the reference segment, This represents element-wise multiplication; where ; Step 5: Perform weighted fusion on local fragments of the candidate synthetic anomaly class two-dimensional skeleton sequence to obtain the synthetic bone vector fragment, expressed as:

[0065] In the formula, This represents a composite bone vector fragment. Indicates the fusion weight coefficient. Indicates the fusion weight coefficient. Indicates the fusion weight coefficient. Indicates the first Local fragments of two-dimensional skeleton sequences of candidate synthetic anomalies Indicates the first Local fragments of two-dimensional skeleton sequences of candidate synthetic anomalies Indicates the first Local fragments of two-dimensional skeleton sequences of candidate synthetic anomalies; in, .

[0066] Step 6: Assign the two-dimensional coordinates of the root node preset in the reference segment to the root node coordinates of the corresponding frame of the synthesized bone vector segment; based on the skeleton edge set, for each skeleton edge in each frame, use the bone vector in the synthesized bone vector segment to calculate the coordinates of the remaining key points in turn, until the coordinates of all key points in each frame of the synthesized bone vector segment are obtained, and the synthesized coordinate segment is obtained; the detection confidence of each key point is the confidence value of the key point in the corresponding reference segment; Specifically, the root node is the nose keypoint (keypoint index 0) in the infant's 2D skeleton. The 21 skeleton keypoints form a tree structure according to the human body's structure, with the nose as the root node, extending progressively towards the limbs. For any edge in the skeleton edge set E... The end closest to the root node The parent node, the other end For child nodes; the coordinates of the root node are directly inherited from the reference segment and do not participate in the bone vector synthesis; when restoring coordinates, starting from the root node, the synthesized bone vector is used to iterate side by side according to the parent-child relationship, and the coordinates of the child node are obtained by adding the corresponding bone vector to the coordinates of the parent node, until all key point coordinates are restored, and the synthesized coordinate segment is obtained; the detection confidence of each key point is inherited from the confidence value of the corresponding key point in the reference segment; The expression for assigning the two-dimensional coordinates of the preset root node in the reference segment to the root node coordinates of the corresponding frame of the synthesized bone vector segment is as follows:

[0067] In the formula, This represents the root node coordinates of the frame corresponding to the synthesized bone vector fragment. Represents the two-dimensional coordinates of the preset root node in the reference segment; For any edge (o, e) in the skeleton, the end closest to the root node The parent node, the other end If the node is a child node, then the coordinates of the e-th keypoint are recursively obtained from the coordinates of the parent node and the corresponding synthesized bone vector, as expressed in the following expression:

[0068] In the formula, Represents the synthesized bone vector fragment In the t-th frame, the bone vector corresponding to the skeleton edge (o,e).

[0069] The recursive process unfolds sequentially along the skeleton from the root node until all keypoint coordinates in each frame are recovered. Keypoint confidence is not involved in bone vector synthesis but is inherited from the baseline segment. This yields the synthesized coordinate segment. .

[0070] Step 7: Based on the synthesized coordinate fragment, calculate the inter-frame displacement of the j-th keypoint between adjacent frames. If the velocity variance of any keypoint exceeds a preset threshold, discard the synthesized bone vector fragment and return to Step 3; otherwise, proceed to Step 8. To reduce the impact of unreasonable samples on training, a lightweight kinematics screening is performed after coordinate segment reconstruction to filter out abnormal velocity jumps between adjacent frames. The expression for calculating the inter-frame displacement of the j-th keypoint between adjacent frames is:

[0071] In the formula, Indicates the inter-frame offset between adjacent frames. This represents the x-coordinate of the j-th keypoint in the (t+1)-th frame. This represents the x-coordinate of the j-th keypoint in frame t. This represents the ordinate of the j-th keypoint in the (t+1)-th frame. This represents the ordinate of the j-th keypoint in frame t. If the velocity variance of any keypoint within a segment exceeds a threshold, the synthesized segment is considered to have a significant unrealistic motion jump; the expression for the velocity variance of any keypoint exceeding the preset threshold is:

[0072] In the formula, Represents the inter-frame displacement of the synthesized coordinate segment, threshold. The value is taken as the 90th percentile of the velocity variance in the actual segment.

[0073] Step 8: Recombine the remaining synthetic bone vector fragments according to the original time sequence, and perform average fusion on the overlapping areas between adjacent synthetic bone vector fragments to obtain local enhancement fragments of the two-dimensional skeleton sequence.

[0074] In this embodiment, the calculation expression for the gray wolf search coefficient is:

[0075] In the formula, Represents a random vector. Indicates the attenuation coefficient. This represents the gray wolf search coefficient. The value gradually decreases as the synthesis process progresses, and is used to control the search range of the synthesized fragment relative to the candidate fragment.

[0076] Existing methods for infant motion recognition based on skeleton sequences mostly employ unified spatiotemporal encoders, which can extract fragment-level motion representations, but are insufficient in distinguishing abnormal body regions and short-term dynamic states. While existing hybrid expert methods introduce multiple branches, their routing weights primarily rely on classification loss learning, easily biased towards a minority of experts. To address these issues, this application constructs a spatial-temporal cascaded hierarchical hybrid expert model, incorporating velocity priors into both spatial and temporal routing.

[0077] To establish a connection between temporal routing, spatial routing, and motion states in local segments of a two-dimensional skeleton sequence, this paper extracts velocity-related statistics from these local segments. Each infant motion video is divided into multiple local segments of a two-dimensional skeleton sequence. Based on inter-frame displacement, the expression for the velocity amplitude of the j-th two-dimensional skeleton keypoint is defined as follows:

[0078] in, Indicates the first The motion amplitude of the j-th keypoint in a segment between adjacent frames. Indicates the first The x-coordinate of the j-th key point in the (t+1)-th frame of a segment. Indicates the first The x-coordinate of the j-th key point in the t-th frame of a segment. Indicates the first The ordinate of the j-th key point in the (t+1)-th frame of a segment. Indicates the first The ordinate of the j-th key point in the t-th frame of a segment; Based on velocity amplitude, during training, region-level velocity priors and segment-level dynamic velocity priors are constructed respectively to guide the route allocation of spatial and temporal experts.

[0079] In this embodiment, the hierarchical hybrid expert model includes: a spatial hybrid expert layer and a temporal hybrid expert layer connected in sequence; The spatial hybrid expert layer includes: a first spatial expert module for extracting local motion features of the upper body region, a second spatial expert module for extracting local motion features of the left lower limb region, and a third spatial expert module for extracting local motion features of the right lower limb region; it also includes: a spatial routing module for fusing local motion features; the outputs of the first spatial expert module, the second spatial expert module, and the third spatial expert module are all connected to the spatial routing module; The first spatial expert module, the second spatial expert module, and the third spatial expert module each include: a first graph convolutional network and a second graph convolutional network connected in sequence; The time-mixing expert layer includes: a first time expert module for extracting time dynamic features of low-speed pause states, a second time expert module for extracting time dynamic features of continuous steady motion states, and a third time expert module for extracting time dynamic features of sudden transition states; it also includes: a time routing module for fusing time dynamic features; the outputs of the first time expert module, the second time expert module, and the third time expert module are all connected to the time routing module.

[0080] Infant motor assessment results not only focus on the overall strength of movement but also on whether different body parts exhibit relatively independent motor performance. Spontaneous movements in normal infants typically exhibit high diversity, with different limbs displaying relatively independent changes; high-risk infants may show insufficient motor differentiation, manifesting as synchronous movements in local limbs or multiple body areas. If the entire skeletal frame is directly encoded uniformly, the differences in movement between different regions are easily weakened in the overall representation. Therefore, the spatial layer divides the skeletal segment into upper body, left lower limb, and right lower limb regions according to the main anatomical areas, extracts local motor features for each region, and then performs adaptive fusion through spatial routing.

[0081] In this embodiment, in the spatial hybrid expert layer: Let the first m The set of joints corresponding to each space expert is: Joints are the key points of an infant's two-dimensional skeleton. The upper body includes the head, shoulders, elbows, wrists, and fingers; the left lower limb includes the left hip, knee, ankle, and toes; and the right lower limb includes the right hip, knee, ankle, and toes. The node features of a segment in the m-th subgraph are denoted as follows: The normalized adjacency matrix of its subgraph is .

[0082] The local segments of the two-dimensional skeleton sequence are divided into upper body, left lower limb, and right lower limb regions according to a predefined body area. A two-layer graph convolutional network is then used to extract local motion features for each region, as expressed in the following expression:

[0083]

[0084] In the formula, This represents the activation function. Indicates feature splicing, and Let represent the learnable parameters of the m-th spatial expert module. This represents the local motion features extracted by the m-th spatial expert module. This represents the intermediate features extracted by the first graph convolutional network in the m-th spatial expert module. Represents the normalized adjacency matrix. This represents a local segment of the two-dimensional skeleton sequence input by the m-th spatial expert module; Each local segment of the two-dimensional skeleton sequence is input into a pre-defined gating network in the spatial routing module. This gating network uses a global graph convolutional network to extract frame-level features; then, a short-window LSTM network is used to obtain a segment-level gating representation, expressed as:

[0085] In the formula, This indicates a fragment-level gating representation. This indicates a short-window LSTM network. This represents a global graph convolutional network. Represents a local segment of a two-dimensional skeleton sequence; Mapping the fragment-level gating representation to the routing weights of the spatial expert module, the expression is:

[0086] In the formula, Indicates the first Spatial routing weights of each spatial expert module Indicates a fully connected layer; Based on routing weights, the spatial fusion features are obtained by fusing various local motion features, and the expression is as follows:

[0087] In the formula, Indicates spatial integration characteristics, This represents the local motion features extracted by the m-th spatial expert module.

[0088] In this embodiment, the routing weights of the spatial expert module are shared within the segment; that is, all frames within the same local segment use the same set of spatial expert weights, instead of generating routes separately for each frame. This reduces the impact of single-frame keypoint jitter on region allocation and allows the expert weights to reflect the overall motion state of the segment.

[0089] Existing skeleton motion recognition methods typically employ a uniform temporal modeling structure to extract sequence dynamic features, while existing MoE methods are mostly used for the fusion of different skeleton modalities or local region branches. This invention, after spatial experts complete local region modeling, further introduces the short-term dynamic differences within segments into the expert structure, allowing temporal experts to focus on different types of motion states. This avoids all segments sharing the same temporal encoding path, thereby improving the model's ability to express infant short-term movements.

[0090] In this embodiment, in the time-mixing expert layer: Based on spatial fusion features, the time dynamic features of low-speed pause state, continuous steady motion state, and sudden transition state are extracted, and the expression is:

[0091] In the formula, This represents the nth time expert module. Indicates spatial integration characteristics, This represents the time dynamic characteristics output by the nth time expert module; where, .

[0092] Specifically, the extraction of time dynamic features of low-speed pause state, continuous steady motion state, and sudden transition state is implemented based on LSTM.

[0093] Each local segment of the two-dimensional skeleton sequence is input into a pre-defined gating network in the temporal routing module. This gating network uses a global graph convolutional network to extract frame-level features; a short-window LSTM network is then used to obtain segment-level gating representations; these segment-level gating representations are mapped to routing weights for the temporal expert module, expressed as:

[0094] In the formula, Indicates the first The routing weight of a local segment to the nth time expert module. Indicates a fully connected layer. This represents a fragment-level gated representation; Based on routing weights, the time dynamic features of low-speed pause states, continuous steady motion states, and sudden transition states are fused to obtain the segment-level representation of each local segment, expressed as:

[0095] In the formula, Indicates the first Fragment-level representation of a local segment.

[0096] Specifically, the routing weights of the time expert module and the spatial expert module are generated by a shared segment-level gating network.

[0097] In this embodiment, the segment-level representations of each local segment are aggregated to obtain the video-level representation expression as follows:

[0098] In the formula, This indicates a video-level representation. Indicates a fusion operation. Fragment-level representation of a local segment.

[0099] In this embodiment, a preset classification head is used to output the infant's movement assessment result, expressed as:

[0100] In the formula, This indicates the results of the infant's motor assessment. This indicates the preset category header.

[0101] This invention converts infant videos into two-dimensional skeleton sequences, and then analyzes and judges based on the spatiotemporal changes of the skeleton motion. The input video first undergoes pose estimation, extracting key points of the infant frame by frame and constructing the corresponding two-dimensional skeleton sequence. Subsequently, the skeleton sequence undergoes uniform preprocessing, using a sliding window to divide it into a group of continuous local segments, which are then fed into the subsequent recognition model.

[0102] During the training phase, to address the issue of a relatively insufficient number of abnormal samples, kinematically constrained gray wolf synthesis enhancement is introduced in the bone vector space. This generates new enhanced samples while maintaining the continuity of local motion, and unreasonable results are filtered out through velocity mutations. The enhanced segments are fed into the subsequent network training along with the original segments. During the testing phase, this synthesis enhancement is not performed; the skeleton sequence is preprocessed and directly fed into the subsequent cerebral palsy risk recognition model.

[0103] A model is built using a hierarchical hybrid expert framework. First, a spatial hybrid expert is used to model the local motion features of different body regions. Then, a temporal hybrid expert is used to characterize the short-term dynamic changes within a segment, thus obtaining a segment-level representation. Both layers of experts are adaptively fused using a gating mechanism driven by the input sequence, enabling the model to adjust the contributions of different experts according to the segment content. After obtaining all segment representations, an aggregation module is used to form a video-level representation, ultimately outputting the infant movement assessment results.

[0104] This application also introduces selective segment contrastive learning during the training phase, imposing contrastive constraints on segments with strong discriminative power to enhance the distinguishability of key anomalous segments.

[0105] Example 2 In this embodiment, the process of training a hierarchical hybrid expert model using a training dataset in the infant movement assessment method based on a multi-expert model proposed in Embodiment 1 is described in detail.

[0106] In spatial routing, a regional-level multivariate velocity prior constraint (spatial routing constraint) is further introduced. The spatial routing constraint consists of the regional average velocity and the synchronicity of joint velocities within the region, reflecting the local motion intensity and the degree of co-occurrence of multiple joints, respectively. Higher synchronicity may correspond to insufficient motion diversity, which is used as the basis for spatial expert allocation, enabling spatial routing to simultaneously consider local motion amplitude and local linkage patterns.

[0107] In this embodiment, spatial routing constraints are introduced into the spatial hybrid expert layer during training. The specific steps are as follows: For each local segment of the two-dimensional skeleton sequence, calculate the average velocity and regional velocity synchronization of each preset body region; The expression for calculating average velocity is as follows:

[0108] In the formula, L Indicates the length of a local segment. This represents the velocity magnitude of the j-th joint between adjacent frames. This represents the set of key points for the m-th body region. Indicates average velocity; regional average velocity is used to describe the overall motion intensity of that body region in the current segment.

[0109] Where m=1,2,3; corresponding to the upper body, left lower limb, and right lower limb regions processed by the first, second, and third spatial expert modules, respectively.

[0110] For example, 'm' has the same meaning in the m-th spatial expert module and the m-th body region, both referring to the same index: m=1 corresponds to the upper body, m=2 to the left lower limb, and m=3 to the right lower limb. There is a one-to-one correspondence between spatial expert modules and body regions.

[0111] To characterize whether there are synchronous changes in the joints within a region, the expression for calculating the regional velocity synchronization is defined as follows:

[0112] In the formula, Indicates regional speed synchronization. Indicates the velocity correlation coefficient; Among them, for the region For any two joints i and j within the segment, calculate the correlation of the velocity sequences of the two joints within the segment, expressed as:

[0113] In the formula, Indicates the first The motion amplitude of the j-th joint in a segment between adjacent frames. Indicates the first The motion amplitude of the i-th joint in a segment between adjacent frames. Indicates the first The average velocity of the i-th joint in each segment Indicates the first The average velocity of the j-th joint in the segment.

[0114] Construct the prior score for the region using the regional average velocity and regional velocity synchronization. Normalize the prior scores of each space expert to obtain the spatial prior route distribution, expressed as:

[0115] In the formula, Represents the spatial prior route distribution. Represents the prior score; where, .

[0116] The difference between the route weights generated by the spatial routing module and the spatial prior route distribution is used as a spatial routing constraint, expressed as:

[0117] In the formula, Indicates the Cauchy-Schwarz divergence. Indicates the route weight. This represents spatial routing constraints.

[0118] The expression for calculating the Cauchy-Schwarz divergence is:

[0119] In the formula, This represents the first probability distribution, i.e., the spatial routing weights. Q This represents the first probability distribution, which is the spatial prior distribution.

[0120] Through this constraint, spatial routes are still adaptively generated by fragment features, while during training they move closer to the allocation trends indicated by regional motion intensity and regional synchronicity, thereby reducing the expert bias caused by routes relying entirely on classification loss.

[0121] Infant spontaneous movement assessment focuses not only on the presence of movement within a segment but also on the continuity and manner of change of movement over time. High-risk infants may exhibit short-term state differences such as impoverished movement, increased pauses, and sudden movements. This invention does not directly use a single average velocity to construct a temporal prior; instead, it maps the overall velocity of each frame within a segment and its adjacent changes to a two-dimensional velocity state to guide the routing of temporal experts.

[0122] In this embodiment, during the training process, a temporal routing constraint loss is introduced into the temporal hybrid expert layer. The specific steps are as follows: For each local segment of the two-dimensional skeleton sequence, calculate the overall velocity of each frame and the change in adjacent frames; The expression for calculating the overall velocity is as follows:

[0123] In the formula, Indicates the number of key points throughout the body. Indicates the first The velocity amplitude of the j-th joint in the t-th frame of a segment between adjacent frames. Indicates the first The overall speed of the t-th frame in the segment; The expression for calculating the change between adjacent frames is:

[0124] In the formula, Indicates the amount of change between adjacent frames. Indicates the first The overall speed of the (t-1)th frame in the segment; By normalizing the overall velocity and the changes in adjacent frames, a two-dimensional velocity state for each frame is constructed, expressed as:

[0125] In the formula, Represents two-dimensional velocity state. Indicates the normalized i-th The overall speed of frame t in the segment. Indicates the normalized i-th The change in the adjacent frames of the t-th frame in a segment; For example, to preserve the relative speed strengths between different segments, the 90th percentile of the overall speed of the training set is used as the normalization scale. Compressed to the range [0,1], denoted as The changes between adjacent frames were also normalized using the 90th percentile of the velocity changes in the training set, resulting in... .

[0126] Given a pre-defined fixed-speed state prototype, calculate the prior scores of each time-based expert module using the following expression:

[0127] In the formula, Indicates the prior score. This represents a prototype of a fixed-velocity state. The larger the value, the more frames in the segment are close to the nth velocity state.

[0128] Specifically, the expression for the fixed-velocity state prototype is:

[0129]

[0130]

[0131] In the formula, Indicates a slow, paused state. Indicates a state of continuous, steady motion. This indicates a sudden change in state.

[0132] Normalizing the prior scores of each time expert yields the time prior route distribution, expressed as:

[0133] In the formula, Indicates the temporal prior route distribution; The difference between the route weights generated by the time-based routing module and the time-prior route distribution is calculated as the time-based routing constraint loss, expressed as:

[0134] In the formula, This represents the time-based routing constraint loss. Indicates the Cauchy-Schwarz divergence. Indicates the first The routing weight of each local segment to the nth time expert module.

[0135] This constraint allows the routing assignment of time experts to remain consistent with the velocity state distribution within the segment, enabling the three time experts to focus on short-term dynamic modes such as low-speed pauses, continuous smooth motion, and sudden transitions, respectively.

[0136] After spatial and temporal mixing, the b-th video... Each local segment is encoded as a segment-level representation. This representation integrates motion characteristics from different body regions and information from different short-term dynamic states. Since cerebral palsy risk labels correspond to entire video segments, video-level classification loss primarily affects the overall prediction after cross-segment aggregation, easily weakening the supervision signal for a small number of key abnormal segments. To enhance the representation ability of hierarchical hybrid experts on highly discriminative local segments, this paper introduces selective segment contrastive learning into its output segment representation. It selects segments with higher scores from the MoE representation space to construct a contrast set and strengthens the class separation between key abnormal segments and difficult normal segments during the training phase.

[0137] In this embodiment, the training process further includes: selectively comparing segments to learn the segment-level representations output by the hierarchical hybrid expert model, specifically: For local segments of a two-dimensional skeleton sequence, a hierarchical hybrid expert model is used to extract segment-level representations of each local segment; To measure the contribution of different segments to anomaly detection, a segment scoring function is used to calculate the anomaly score of the segment-level representation of each local segment, expressed as:

[0138] In the formula, This indicates the first movement video of the b-th baby. Anomaly scores of local segments of a two-dimensional skeleton sequence This represents the segment scoring function. This indicates the first movement video of the b-th baby. A fragment-level representation of local segments of a two-dimensional skeleton sequence; the scores of the fragment scoring function are only used for fragment anomaly screening and are not directly used as video classification results.

[0139] In each infant movement video, select the K highest-scoring two-dimensional skeleton sequence local segments and construct a segment index set, expressed as:

[0140] In the formula, This indicates the preset selection value. Represents a set of fragment indices. This indicates the first movement video of the b-th baby. Anomaly score of local segments of a two-dimensional skeleton sequence; For anomalous videos, higher-scoring segments are more likely to contain anomalous motion cues; for normal videos, higher-scoring segments can be considered more easily confused segments closer to the anomalous boundary. Therefore, uniformly selecting the top-K segments with the highest scores allows contrastive learning to simultaneously focus on key anomalous segments in anomalous samples and difficult segments in normal samples. Samples generated by the bone vector kinematic constraint gray wolf synthesis augmentation are only used for main classification training and do not participate in segment contrastive learning to avoid affecting the positive and negative sample relationships between real segments.

[0141] Within the same training batch, a set of comparison segments is created based on the segment indices of each infant's movement video. The expression is:

[0142] In the formula, Represents a set of contrasting segments. This indicates the training batch size, i.e., the number of videos in the current batch; For sets Any segment in the text is rewritten as Its corresponding video tag is The fragment-level representation of each local segment of the two-dimensional skeleton sequence in the set of contrasting segments is normalized to obtain the normalized fragment-level representation of the local segment of the two-dimensional skeleton sequence, expressed as:

[0143] In the formula, This represents the fragment-level representation of a local segment of a normalized two-dimensional skeleton sequence.

[0144] The normalized cosine similarity distribution between the fragment-level representations of local fragments of the two-dimensional skeleton sequence in the comparison fragment set and the fragment-level representations of local fragments of the remaining two-dimensional skeleton sequences is calculated as follows:

[0145] In the formula, Represents the normalized cosine similarity distribution. This represents the fragment-level representation of a local segment of the two-dimensional skeleton sequence of the i-th segment after normalization. This represents the fragment-level representation of a local segment of the two-dimensional skeleton sequence of the j-th segment after normalization. The fragment-level representation of a local segment of a two-dimensional skeleton sequence representing the k-th segment after normalization; Indicates fragment i As a reference, the segment j The weights in their similarity distribution. This distribution describes which high-information segments the model currently considers to be more similar to the benchmark segment.

[0146] Based on the tags in the baby's movement videos, construct the target distribution; construct a set of local segments of a two-dimensional skeleton sequence with similar annotations, expressed as:

[0147] In the formula, This represents a set of local segments of a two-dimensional skeleton sequence with similar annotations. Represents the local segment index of the two-dimensional skeleton sequence labeled with the same class as the i-th segment; where, in the target distribution, segments of the same class are considered positive samples and their probabilities are evenly distributed, and the probability of segments of different classes is set to 0.

[0148] The expression for the local segment index of the two-dimensional skeleton sequence labeled with the same type as the i-th segment is:

[0149] Align the normalized cosine similarity distribution with a set of local segments of the same labeled 2D skeleton sequence, and construct a segment contrast loss using CS divergence, expressed as:

[0150] In the formula, Indicates fragment contrast loss, This represents the similarity distribution predicted by the model. Indicates the target distribution.

[0151] In this embodiment, unlike traditional supervised contrastive learning which directly maximizes the normalized similarity of positive sample pairs, the model predicts the similarity distribution... With target distribution Alignment is performed, and fragment contrast loss is constructed using CS divergence.

[0152] Based on temporal routing constraint loss, spatial routing constraint, and segment contrast loss, a total loss function is constructed. The hierarchical hybrid expert model is trained based on the total loss function. When the total loss function converges, the trained hierarchical hybrid expert model is obtained.

[0153] The expression for the total loss function is:

[0154] In the formula, Represents the total loss function. Represents classification loss. Indicates hyperparameters, This represents hyperparameters.

[0155] Example 3 This embodiment experimentally verifies the infant movement assessment method based on a multi-expert model proposed in Embodiment 1.

[0156] In this embodiment, the experiment was conducted based on an infant posture dataset constructed in collaboration with the Children's Hospital Affiliated to Soochow University. Data was collected in real clinical settings, originating from different wards, incubator environments, and backgrounds. All videos were captured from a top-down perspective, although the specific shooting angles varied. During video capture, the aim was to record the infant's natural, continuous movements as completely as possible, ensuring that major body parts such as the head, torso, and limbs were clearly visible. Simultaneously, the impact of occlusion, reflections, and environmental interference on image quality was minimized to ensure the videos could support subsequent posture estimation and motion analysis. The dataset contains videos with a wide range of durations, from 0.4 to 7.6 minutes per segment, with original video frame rates of approximately 25 to 30 fps.

[0157] The constructed dataset contains 606 videos of infants' natural movements, including 383 cases of normal twisting movements and 223 cases of abnormal monotonous movements, with each infant corresponding to a complete video segment. All samples were labeled by senior pediatricians with relevant clinical experience and underwent multiple rounds of cross-validation and correction to ensure the accuracy and clinical consistency of the labels. Subsequently, the YOLO11 pose estimator was used to extract frame-by-frame 2D skeleton keypoint sequences. Given the significant differences between infant pose distribution and general human scene, this paper further fine-tuned the model based on infant pose data to improve the adaptability and robustness of skeleton extraction in the target scene. Before being fed into the recognition model, the skeleton sequence was further downsampled proportionally to the original frame rate to approximately 12.5–15 fps, preserving the main motion dynamics while reducing the computational burden caused by redundant frames. The final skeleton sequence was used as the input to the subsequent recognition model.

[0158] All experiments were conducted on the aforementioned infant dataset. To reduce the impact of randomness from a single data partition and to more objectively evaluate the model's generalization ability, this paper employs a four-fold cross-validation approach based on individual infant partitions for training and testing. All samples were divided into four non-overlapping subsets, with each subset containing an equal proportion of normal and abnormal infants. One subset was used as the test set in each fold, while the remaining subsets were used for training. The model was trained independently for each of the four folds, and the final results are reported as the mean and standard deviation of the four experiments.

[0159] The proposed multi-expert model-based infant motion assessment method is implemented using the PyTorch framework, with a two-dimensional skeleton keypoint sequence as the model input. To reduce spatial scale differences between different samples, all skeleton coordinates are normalized to the [0,1] range before being input into the network. The uniformly preprocessed skeleton sequence is then fed into the network for hierarchical hybrid expert model training and inference. The model is trained for 110 epochs using the Adam optimizer, with an initial learning rate of [value missing]. The learning rate was adjusted using CosineAnnealingLR. To ensure fairness in the comparison, all comparison methods were implemented using their official implementations, retrained on the same dataset, and evaluated using a consistent four-fold partition, input format, and experimental procedure.

[0160] To comprehensively evaluate the model's performance in the infant abnormal movement recognition task, this paper uses AUC, F1, Recall, Precision, and Accuracy as evaluation metrics. AUC and F1 are the primary metrics, used to measure the model's overall discriminative ability and class balance performance, respectively; Recall, Precision, and Accuracy are secondary metrics, used to further characterize the model's ability to detect abnormal samples and its overall classification accuracy. Considering the class imbalance inherent in this task and the fact that medical screening focuses more on the effective identification of abnormal samples, a comprehensive analysis of model performance is conducted by combining AUC and F1.

[0161] To verify the effectiveness of the multi-expert model-based infant movement assessment method proposed in this application, several representative skeletal movement recognition models and related infant movement analysis methods were selected as comparative models.

[0162] The comparison methods mainly fall into two categories: The first category comprises representative skeleton action recognition models, including EfficientGraph Convolution-based Action Recognition (EfficientGCN), DeformableGraph Convolution-based Action Recognition (DeGCN), Cross-Sample Aggregation-based Action Recognition (SkeletonX), Keypoint Representation Enhancement-based Action Recognition (PSE-GCN), Prototype Learning-based Action Recognition (ProtoGCN), and Adaptive Entropy Flow Long-Tail Action Recognition (AEED). These methods are widely used in skeleton sequence modeling tasks and can serve as a general baseline for evaluating the spatiotemporal feature learning ability of the proposed model. The second category includes methods related to cerebral palsy risk screening, including Spatiotemporal Attention-Based Infant Motor Assessment (STAM), Weakly Supervised Infant Motor Assessment (WO-GMA), Quantitative Deep Learning-Based Infant Motor Assessment (MAM), Self-Supervised Spatiotemporal Learning-Based Infant Motor Assessment (STA-GCN), and Uncertainty Decoupling Fusion-Based Infant Motor Assessment (UDF-GMA). These methods are closer to the task scenario presented in this paper and can serve as comparison models with strong task relevance.

[0163] Table 1 shows the quantitative comparison results of each method on the dataset, demonstrating the comparison between the proposed multi-expert model-based infant motion assessment method and existing skeleton motion recognition models and infant motion assessment methods. This application achieves superior performance on key metrics, indicating stronger discriminative ability and class balance in the infant abnormal motion recognition task.

[0164] Table 1

[0165] While general skeleton motion recognition methods can perform spatiotemporal modeling of skeleton sequences, they are primarily geared towards general human motion recognition tasks. Infant natural movements lack clear boundaries between motion categories, and abnormal cues are often concentrated in local limbs or short fragments. Therefore, when general skeleton models are directly applied to this task, they easily overlook fine-grained abnormal information. In contrast, this invention models local differences and short-term dynamic changes, making it more suitable for capturing motion features associated with cerebral palsy risk.

[0166] While existing methods for cerebral palsy risk screening and general skeletal motion recognition can perform spatiotemporal modeling of skeletal sequences, they are primarily geared towards general human motion recognition tasks. Infant natural movements lack clear boundaries between movement categories, and abnormal cues are often concentrated in local limbs or short fragments. Therefore, when general skeletal models are directly applied to this task, they easily overlook fine-grained abnormal information. In contrast, the method presented in this paper models local differences and short-term dynamic changes, making it more suitable for capturing movement features associated with cerebral palsy risk.

[0167] When existing cerebral palsy risk screening methods and this invention are both used for cerebral palsy risk screening, the overall results of existing methods are still lower than those of this application. This indicates that relying solely on conventional spatiotemporal attention or graph convolution modeling is insufficient to adequately handle local differences, dynamic changes, and uneven distribution of key segments in abnormal infant movements. This application, through hierarchical hybrid expert structures, kinematically constrained gray wolf synthesis enhancement in bone vector space, and selective segment contrastive learning, enables the model to pay closer attention to abnormal cues with diagnostic value.

[0168] While some comparative methods demonstrate good ability to distinguish normal samples, their ability to detect abnormal samples is relatively insufficient, reflecting a tendency for the model to favor the normal class under imbalanced conditions. This invention, while maintaining the ability to identify normal samples, improves the sensitivity to identify abnormal samples, thus achieving more stable performance in overall metrics. Figure 4 The ROC curve comparison chart of the existing method and the infant movement assessment method based on the multi-expert model proposed in this application also shows that the present invention maintains good discrimination ability under different discrimination thresholds, further demonstrating that its overall discrimination performance is better than the representative baseline method.

[0169] While the task remains the same, the overall results are still lower than those of our proposed method. This indicates that relying solely on conventional spatiotemporal attention or graph convolutional modeling is insufficient to adequately handle local differences, dynamic changes, and uneven distribution of key fragments in abnormal infant movements. Our proposed method, through hierarchical hybrid expert structures, kinematically constrained gray wolf synthesis enhancement in bone vector space, and selective fragment contrastive learning, enables the model to pay closer attention to diagnostically valuable abnormal cues.

[0170] Furthermore, while some comparison methods demonstrate good ability to distinguish normal samples, their ability to detect abnormal samples is relatively insufficient, reflecting a tendency for the model to favor the normal class under imbalanced conditions. The method presented in this paper maintains the ability to identify normal samples while improving the sensitivity to identify abnormal samples, thus achieving more stable performance on overall metrics. Figure 4 The ROC curves shown also demonstrate that the proposed method maintains good discriminative ability under different discrimination thresholds, further illustrating that its overall discriminative performance is superior to that of representative baseline methods.

[0171] Table 2 shows the ablation experiment results of this application. The complete model achieved the best performance in the three core metrics of AUC, F1, and Recall, indicating a significant synergistic effect between spatial hierarchical modeling, temporal hierarchical modeling, local fragment contrastive learning, and synthetic enhancement. Although the simplified model can approach the complete model in some metrics, its overall performance is still lower.

[0172] Table 2

[0173] The ablation results of the hierarchical hybrid expert structure show that removing the spatial hybrid expert leads to a decrease in model performance, indicating that differentiated modeling by body region helps capture local anomalous movements. Compared to a single whole-body graph network, spatial experts can focus on the movement features of the upper body, left lower limb, and right lower limb separately, reducing the dilution of local anomalous cues in the unified whole-body representation. Removing the temporal hybrid expert also results in a decrease in model performance, indicating that infant movement segments do not conform to a single dynamic pattern. Different segments may exhibit states such as slow pauses, continuous smooth movements, or sudden transitions, and multiple temporal experts can express these short-term dynamic differences in more detail. Further removing the overall hybrid expert structure degenerates the model into a unified graph network structure and a single LSTM, with a more significant decrease in overall performance, indicating that spatial and temporal experts have a synergistic effect in hierarchical modeling.

[0174] The ablation results of gating mechanisms and velocity-guided routing further illustrate that adaptive routing is key to the effectiveness of hierarchical hybrid experts. When gating mechanisms are removed and fixed-average fusion is used, the model cannot dynamically adjust expert contributions based on the motion states of different segments, leading to a significant decrease in F1 and Recall. This indicates that the performance improvement does not come solely from an increase in the number of expert branches, but rather from the input-driven expert selection capability. Further removal of velocity-guided routing, while still retaining data-driven gating, means that spatial routing is no longer guided by regional motion intensity and regional velocity synchronization, and temporal routing no longer utilizes intra-segment velocity state distribution for adjustment. Consequently, the model's ability to distinguish between local coordination and short-term dynamic states decreases. These results demonstrate that incorporating regional coordination and velocity state into expert routing helps to make the allocation of spatial and temporal experts more consistent with the actual motion characteristics of infant motion segments.

[0175] After removing the bone vector kinematic constraint gray wolf synthesis augmentation (KBGWA), the performance drops significantly when using only the original training samples, indicating that the kinematic constraint gray wolf synthesis augmentation in the bone vector space can effectively supplement the distribution of abnormal class samples.

[0176] Ablation analysis of selective segment contrastive learning shows that removing this constraint degrades model performance, indicating that relying solely on video-level classification supervision is insufficient to adequately enhance the discriminative power of key anomaly segments. Our proposed method selects high-information segments for contrastive constraints on the segment representations output by MoE, making similar segments closer together in the representation space and dissimilar segments more distinct, thereby improving the model's sensitivity to local anomaly cues in long sequences.

[0177] The embodiments described are merely examples to clearly illustrate the present invention and are not intended to limit the implementation of the invention. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively describe all possible implementations. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A method for assessing infant motion based on a multi-expert model, characterized in that, Includes the following steps: S1: Based on infant movement videos, extract key points of the infant's two-dimensional skeleton and construct a two-dimensional skeleton sequence; preprocess the two-dimensional skeleton sequence to obtain several local segments of the two-dimensional skeleton sequence. S2: Based on local fragments of the two-dimensional skeleton sequence, select abnormal local fragments of the two-dimensional skeleton sequence for enhancement to obtain local enhanced fragments of the two-dimensional skeleton sequence; construct a training dataset based on the local fragments of the two-dimensional skeleton sequence and the local enhanced fragments of the two-dimensional skeleton sequence. S3: Construct a hierarchical hybrid expert model containing a spatial hybrid expert layer and a temporal hybrid expert layer; input several local segments of two-dimensional skeleton sequences in the training dataset into the spatial hybrid expert layer, extract and fuse the local motion features to obtain spatial fusion features; input the spatial fusion features into the temporal hybrid expert layer to extract the segment-level representation of each local segment; S4: Train the hierarchical hybrid expert model using the training dataset to obtain the trained hierarchical hybrid expert model; S5: Input several local segments of the two-dimensional skeleton sequence of the infant to be evaluated into the trained hierarchical hybrid expert model, extract the segment-level representation of each local segment; aggregate the segment-level representations of each local segment to obtain the video-level representation; based on the video-level representation, use the preset classification head to output the infant's action evaluation results. 2.The infant motion evaluation method based on multi-expert model according to claim 1, wherein, Step S1 is as follows: S101: Randomly select some image frames from the baby movement video, and annotate the key points of the baby's two-dimensional skeleton in the selected image frames to construct a baby pose annotation dataset; S102: Train the YOLO11 model using the infant pose annotation dataset, optimize the YOLO11 model parameters, and obtain an optimized YOLO11 model suitable for extracting key points of the infant's two-dimensional skeleton. S103: Using the optimized YOLO11 model, extract key points of the infant's 2D skeleton frame by frame. The expression is: In the formula, This indicates the first video of baby movement. t The first frame j The x-coordinates of the key points This indicates the first video of baby movement. t The first frame j The ordinates of the key points This indicates the first video of baby movement. t The first frame j Detection confidence of each key point; S104: Extract key points of the infant's two-dimensional skeleton frame by frame and arrange them in chronological order to construct a two-dimensional skeleton sequence; The two-dimensional skeleton sequence is normalized and downsampled to obtain the downsampled two-dimensional skeleton sequence. S105: Using the sliding window method, the downsampled two-dimensional skeleton sequence is divided into several local segments of the two-dimensional skeleton sequence, expressed as: In the formula, Indicates the first A local fragment of a two-dimensional skeleton sequence This represents the total number of local segments in a two-dimensional skeleton sequence.

3. The infant motor assessment method based on a multi-expert model according to claim 1, characterized in that, The process of enhancing local fragments of anomalous two-dimensional skeleton sequences is as follows: Step 1: For the selected local segments of the abnormal 2D skeleton sequence, define a set of skeleton edges based on the human skeletal structure, with each skeleton edge connecting two adjacent keypoints; calculate the bone vector of the skeleton edge in frame t within the local segment of the abnormal 2D skeleton sequence, expressed as: In the formula, Indicates from the key point Pointing to key points bone vector, Indicates the first In a local segment of a two-dimensional skeleton sequence of anomalies, the key points are in frame t. x-axis; Indicates the first In a local segment of a two-dimensional skeleton sequence of anomalies, the key points are in frame t. ordinate; Indicates the first In a local segment of a two-dimensional skeleton sequence of anomalies, the key points are in frame t. x-axis; Indicates the first In a local segment of a two-dimensional skeleton sequence of anomalies, the key points are in frame t. ordinate; Step 2: Arrange the bone vectors of all skeleton edges of each local segment of the two-dimensional skeleton sequence of the anomaly class in time order and skeleton edge order to obtain the representation of the local segment of the two-dimensional skeleton sequence of the anomaly class in the bone vector space. Step 3: Take the representation of each local segment of the two-dimensional skeleton sequence of anomaly class in bone vector space as the reference segment, and calculate the bone vector space distance between the local segments of the remaining two-dimensional skeleton sequences of anomaly classes and the reference segment. The expression for calculating the bone vector space distance is: In the formula, This represents the representation of the reference segment in the bone vector space. Indicates the first Representation of each exception class in the bone vector space Represents the distance in the bone vector space. Indicates the first A local fragment of a two-dimensional skeleton sequence of anomalies from key points Pointing to key points bone vector, Represents the set of skeleton edges. Indicates the length of a local segment in a two-dimensional skeleton sequence; Step 4: Select the three local fragments of the two-dimensional skeleton sequence of the anomaly class with the smallest spatial distance in the bone vector space as candidate fragments; for each candidate fragment, generate a candidate synthetic anomaly class two-dimensional skeleton sequence local fragment, expressed as: In the formula, This represents a local fragment of a two-dimensional skeleton sequence representing a candidate synthesis anomaly. This represents a local segment of a two-dimensional skeleton sequence representing an anomaly class. This represents the gray wolf search coefficient. Indicates the reference segment, This represents element-wise multiplication; Step 5: Perform weighted fusion on local fragments of the candidate synthetic anomaly class two-dimensional skeleton sequence to obtain the synthetic bone vector fragment, expressed as: In the formula, This represents a composite bone vector fragment. Indicates the fusion weight coefficient. Indicates the fusion weighting coefficient. Indicates the fusion weight coefficient. Indicates the first Local fragments of two-dimensional skeleton sequences of candidate synthetic anomalies Indicates the first Local fragments of two-dimensional skeleton sequences of candidate synthetic anomalies Indicates the first Local fragments of two-dimensional skeleton sequences of candidate synthetic anomalies; Step 6: Assign the two-dimensional coordinates of the root node in the reference segment to the root node coordinates of the corresponding frame of the synthesized bone vector segment; Based on the skeleton edge set, for each skeleton edge in each frame, use the bone vector in the synthesized bone vector segment to calculate the coordinates of the remaining key points in turn, until the coordinates of all key points in each frame of the synthesized bone vector segment are obtained, and the synthesized coordinate segment is obtained. The detection confidence level of each key point is the confidence level value of the corresponding key point in the benchmark segment; Step 7: Based on the synthesized coordinate fragment, calculate the inter-frame displacement of the j-th keypoint between adjacent frames. If the velocity variance of any keypoint exceeds a preset threshold, discard the synthesized bone vector fragment and return to Step 3; otherwise, proceed to Step 8. Step 8: Recombine the remaining synthetic bone vector fragments according to the original time sequence, and perform average fusion on the overlapping areas between adjacent synthetic bone vector fragments to obtain local enhancement fragments of the two-dimensional skeleton sequence.

4. The infant motor assessment method based on a multi-expert model according to claim 3, characterized in that, The expression for calculating the gray wolf search coefficient is as follows: In the formula, Represents a random vector. Indicates the attenuation coefficient. This represents the gray wolf search coefficient.

5. The infant motor assessment method based on a multi-expert model according to claim 1, characterized in that, The hierarchical hybrid expert model includes: a spatial hybrid expert layer and a temporal hybrid expert layer connected in sequence; The spatial hybrid expert layer includes: a first spatial expert module for extracting local motion features of the upper body region, a second spatial expert module for extracting local motion features of the left lower limb region, and a third spatial expert module for extracting local motion features of the right lower limb region; it also includes: a spatial routing module for fusing local motion features; the outputs of the first spatial expert module, the second spatial expert module, and the third spatial expert module are all connected to the spatial routing module; The first spatial expert module, the second spatial expert module, and the third spatial expert module each include: a first graph convolutional network and a second graph convolutional network connected in sequence; The time-mixing expert layer includes: a first time expert module for extracting time dynamic features of low-speed pause states, a second time expert module for extracting time dynamic features of continuous steady motion states, and a third time expert module for extracting time dynamic features of sudden transition states; it also includes: a time routing module for fusing time dynamic features; the outputs of the first time expert module, the second time expert module, and the third time expert module are all connected to the time routing module.

6. The infant motor assessment method based on a multi-expert model according to claim 5, characterized in that, In the spatial hybrid expert layer: The local segments of the two-dimensional skeleton sequence are divided into upper body, left lower limb, and right lower limb regions according to a predefined body area, and local motion features are extracted for each region. The expression is as follows: In the formula, This represents the activation function. Indicates feature splicing, and Let represent the learnable parameters of the m-th spatial expert module. This represents the local motion features extracted by the m-th spatial expert module. This represents the intermediate features extracted by the first graph convolutional network in the m-th spatial expert module. Represents the normalized adjacency matrix. This represents a local segment of the two-dimensional skeleton sequence input by the m-th spatial expert module; Each local segment of the two-dimensional skeleton sequence is input into a pre-defined gating network in the spatial routing module. This gating network uses a global graph convolutional network to extract frame-level features; then, a short-window LSTM network is used to obtain a segment-level gating representation, expressed as: In the formula, This indicates a fragment-level gating representation. This indicates a short-window LSTM network. This represents a global graph convolutional network. Represents a local segment of a two-dimensional skeleton sequence; Mapping the fragment-level gating representation to the routing weights of the spatial expert module, the expression is: In the formula, Indicates the first Spatial routing weights of each spatial expert module Indicates a fully connected layer; Based on routing weights, the spatial fusion features are obtained by fusing various local motion features, and the expression is as follows: In the formula, Indicates spatial integration characteristics, This represents the local motion features extracted by the m-th spatial expert module.

7. The infant motor assessment method based on a multi-expert model according to claim 5, characterized in that, In the time-mixing expert layer: Based on spatial fusion features, the time dynamic features of low-speed pause state, continuous steady motion state, and sudden transition state are extracted, and the expression is: In the formula, This represents the nth time expert module. Indicates spatial integration characteristics, This represents the time dynamic characteristics output by the nth time expert module; Each local segment of the two-dimensional skeleton sequence is input into a pre-defined gating network in the temporal routing module. This gating network uses a global graph convolutional network to extract frame-level features; a short-window LSTM network is then used to obtain segment-level gating representations; these segment-level gating representations are mapped to routing weights for the temporal expert module, expressed as: In the formula, Indicates the first The routing weight of a local segment to the nth time expert module. Indicates a fully connected layer. This represents a fragment-level gated representation; Based on routing weights, the time dynamic features of low-speed pause states, continuous steady motion states, and sudden transition states are fused to obtain the segment-level representation of each local segment, expressed as: In the formula, Indicates the first Fragment-level representation of a local segment.

8. The infant motor assessment method based on a multi-expert model according to claim 1, characterized in that, During training, spatial routing constraints are introduced into the spatial hybrid expert layer. The specific steps are as follows: For each local segment of the two-dimensional skeleton sequence, calculate the average velocity and regional velocity synchronization of each preset body region; The expression for calculating average velocity is as follows: In the formula, L Indicates the length of a local segment. This represents the velocity magnitude of the j-th joint between adjacent frames. This represents the set of key points for the m-th body region. Indicates average speed; The expression for calculating regional speed synchronization is: In the formula, Indicates regional speed synchronization. Indicates the velocity correlation coefficient; Using the regional average velocity and regional velocity synchronization, a prior score for the region is constructed; the prior scores of each space expert are normalized to obtain the spatial prior route distribution, expressed as: In the formula, Represents the spatial prior route distribution. Indicates the prior score; The difference between the route weights generated by the spatial routing module and the spatial prior route distribution is used as a spatial routing constraint, expressed as: In the formula, Indicates the Cauchy-Schwarz divergence. Indicates the route weight. This represents spatial routing constraints.

9. The infant motor assessment method based on a multi-expert model according to claim 1, characterized in that, During training, a temporal routing constraint loss is introduced into the temporal hybrid expert layer. The specific steps are as follows: For each local segment of the two-dimensional skeleton sequence, calculate the overall velocity of each frame and the change in adjacent frames; The expression for calculating the overall velocity is as follows: In the formula, Indicates the number of key points throughout the body. Indicates the first The velocity amplitude of the j-th joint in the t-th frame of a segment between adjacent frames. Indicates the first The overall speed of the t-th frame in the segment; The expression for calculating the change between adjacent frames is: In the formula, Indicates the amount of change between adjacent frames. Indicates the first The overall speed of the (t-1)th frame in the segment; By normalizing the overall velocity and the changes in adjacent frames, a two-dimensional velocity state for each frame is constructed, expressed as: In the formula, Represents two-dimensional velocity state. Indicates the normalized i-th The overall speed of frame t in the segment. Indicates the normalized i-th The change in the adjacent frames of the t-th frame in a segment; Given a pre-defined fixed-speed state prototype, calculate the prior scores of each time-based expert module using the following expression: In the formula, Indicates the prior score. This represents a prototype of a fixed-velocity state. Normalizing the prior scores of each time expert yields the time prior route distribution, expressed as: In the formula, Indicates the temporal prior route distribution; The difference between the route weights generated by the time-based routing module and the time-prior route distribution is calculated as the time-based routing constraint loss, expressed as: In the formula, This represents the time-based routing constraint loss. Indicates the Cauchy-Schwarz divergence. Indicates the first The routing weight of each local segment to the nth time expert module.

10. The infant motor assessment method based on a multi-expert model according to claim 1, characterized in that, The training process also includes: selective fragment contrastive learning of the fragment-level representations output by the hierarchical hybrid expert model, specifically: For local segments of a two-dimensional skeleton sequence, a hierarchical hybrid expert model is used to extract segment-level representations of each local segment; Using the fragment scoring function, the anomaly score of the fragment-level representation of each local fragment is calculated, expressed as: In the formula, This indicates the first movement video of the b-th baby. Anomaly scores of local segments of a two-dimensional skeleton sequence This represents the segment scoring function. This indicates the b-th baby movement video. Fragment-level representation of local segments of a two-dimensional skeleton sequence; In each infant movement video, select the K highest-scoring two-dimensional skeleton sequence local segments and construct a segment index set, expressed as: In the formula, This indicates the preset selection value. Represents a set of fragment indices. This indicates the first movement video of the b-th baby. Anomaly score of local segments of a two-dimensional skeleton sequence; Within the same training batch, a set of comparison segments is created based on the segment indices of each infant's movement video. The expression is: In the formula, Represents a set of contrasting segments. Indicates the training batch size; Normalize the fragment-level representation of each local segment of the two-dimensional skeleton sequence in the comparison fragment set to obtain the normalized fragment-level representation of the local segment of the two-dimensional skeleton sequence. Calculate the normalized cosine similarity distribution between the fragment-level representations of the local segments of the two-dimensional skeleton sequences in the comparison fragment set and the fragment-level representations of the other local segments of the two-dimensional skeleton sequences. The expression is as follows: In the formula, Represents the normalized cosine similarity distribution. This represents the fragment-level representation of a local segment of the two-dimensional skeleton sequence of the i-th segment after normalization. This represents the fragment-level representation of a local segment of the two-dimensional skeleton sequence of the j-th segment after normalization. The fragment-level representation of a local segment of a two-dimensional skeleton sequence representing the k-th segment after normalization; Based on the tags in the baby's movement videos, construct the target distribution; construct a set of local segments of a two-dimensional skeleton sequence with similar annotations, expressed as: In the formula, This represents a set of local segments of a two-dimensional skeleton sequence with similar annotations. Represents the local segment index of the two-dimensional skeleton sequence labeled with the same type as the i-th segment; Align the normalized cosine similarity distribution with a set of local segments of the same labeled 2D skeleton sequence, and construct a segment contrast loss using CS divergence, expressed as: In the formula, Indicates fragment contrast loss, This represents the similarity distribution predicted by the model. Indicates the target distribution.

Images