A method and system for assessing the development level of an infant, and a storage medium

Abstract

The application provides a baby development level evaluation method, system and storage medium. The method comprises the following steps: acquiring a multi-modal behavior video stream of a baby, pre-processing the multi-modal behavior video stream to obtain a time-sliced video segment; extracting explicit basic behavior features, hidden micro-behavior features and time sequence correlation features representing the linkage relationship between behaviors and the time sequence distribution rule of abnormal features of the time-sliced video segment through a three-level feature extraction system to construct a baby behavior feature set; inputting the baby behavior feature set into a baby development evaluation vertical large model to generate a preliminary evaluation result through double-library matching of a benchmark behavior feature library and a development abnormal feature library and attention-weighted reasoning of core abnormal features of a targeted disease; inputting the preliminary evaluation result into a month threshold model for secondary verification to determine a development abnormality risk level and output an abnormal feature matching report. The application can quickly and accurately evaluate the development level of a baby.

Description

Technical Field

[0001] This application relates to the field of developmental screening technology, and in particular to a method, system and storage medium for assessing the developmental level of infants and young children. Background Technology

[0002] Infant and toddler developmental assessment is a core component of child health management, especially for neurodevelopmental disorders such as autism spectrum disorder. The period from 0 to 3 years old is considered the golden window for early screening and intervention; timely identification and intervention are crucial for improving the prognosis of affected children. Currently, mainstream infant and toddler developmental assessment methods heavily rely on in-person observation and scale testing by professional physicians.

[0003] However, this traditional manual assessment model faces significant technical bottlenecks in practical application. First, its screening efficiency is low, making large-scale, universal application difficult. Limited resources among professional physicians, especially in primary healthcare institutions, cannot support comprehensive, routine screening of the vast 0-6 year old population, resulting in many potential children with developmental abnormalities missing the optimal intervention period. Second, existing video-based assisted assessment attempts are technically superficial, typically only extracting explicit basic behavioral characteristics of infants and toddlers, such as large limb movements or simple facial expressions. These methods fail to delve into implicit microbehavioral characteristics highly correlated with specific conditions like autism (such as subtle facial expression changes and brief social responses) and the temporal correlations of behavior (such as the duration and frequency of specific abnormal behaviors), thus leading to low assessment accuracy and a high rate of missed and misdiagnosed cases that fails to meet clinical requirements. Summary of the Invention

[0004] To enable rapid and accurate assessment of infant and toddler developmental levels, this application provides a method, system, and storage medium for such assessments.

[0005] Firstly, a method for assessing the developmental level of infants and young children is provided, the method including: Acquire multimodal behavior video streams of infants and young children, and preprocess the multimodal behavior video streams to obtain time-slice video segments; A structured set of infant and toddler behavioral features is constructed by extracting explicit basic behavioral features, implicit micro-behavioral features, and temporal correlation features that characterize the temporal distribution patterns of behavioral linkages and abnormal features from the time-slice video segments using a three-level feature extraction system consisting of convolutional neural networks, optical flow methods and key point detection, and recurrent neural networks. The infant behavioral feature set is input into a large-scale infant development assessment model that integrates prior knowledge from an infant care vertical database. The model generates a preliminary assessment result that includes the matching degree between the developmental level assessment value and the abnormal feature through dual-database matching of the baseline behavioral feature database and the developmental abnormality feature database, and attention-weighted reasoning on the core abnormal features of the targeted disease. The preliminary assessment results are input into a month-age threshold model that is dynamically adjusted based on individual infant development data for secondary verification. By comparing the model with personalized thresholds, a graded risk level of developmental abnormalities is determined, and an abnormal feature matching report for the targeted disease is output.

[0006] In some embodiments, acquiring the multimodal behavioral video stream of infants includes: The first visual behavioral video and the first audio-accompanied video are captured by a home mobile terminal in an unmarked free shooting mode, and / or the second visual behavioral video and the second audio-accompanied video are captured by a fixed camera position at an institutional end in a multi-view high-definition shooting mode. Collect basic data on infants' age in months, gender, and growth and development associated with the video stream as related metadata; The first and / or second visual behavior videos, the first and / or second audio-related videos, and the associated metadata are combined into a multimodal behavior video stream.

[0007] In some embodiments, the preprocessing of the multimodal behavioral video stream to obtain time-slice video segments includes: The multimodal behavioral video stream is denoised and deblurred to obtain an enhanced video stream; The main video stream is obtained by segmenting the infant subject from the enhanced video stream; The effective video stream is obtained by removing blurry frames, occluded frames, and frames without behavior from the main video stream. The effective video stream is divided into several timestamped video segments as time-sliced ​​video segments.

[0008] In some embodiments, the extraction of explicit basic behavioral features, implicit micro-behavioral features, and temporal correlation features representing the temporal distribution patterns of inter-behavioral linkages and abnormal features from the time-slice video segments using a three-level feature extraction system composed of convolutional neural networks, optical flow methods and keypoint detection, and recurrent neural networks to construct a structured infant behavioral feature set includes: The time-slice video is segmented and input into a convolutional neural network to extract explicit basic behavioral features, including head movements, body movements, facial expressions, eye contact, and speech, to construct a subset of basic behavioral features. The time-slice video segments are input into the optical flow method and the key point detection algorithm to extract implicit micro-behavioral features, including facial micro-expressions, body micro-movements and social micro-reactions, under a set effective micro-feature recognition threshold, to construct a subset of micro-behavioral features. The basic behavioral feature subset and the micro-behavioral feature subset are input into a long short-term memory neural network for temporal modeling to mine the temporal correlation of behavioral features and the temporal pattern of missing targeted abnormal features in order to construct a temporal correlation feature subset. The basic behavioral feature subset, micro-behavioral feature subset, and temporal correlation feature subset are integrated to obtain a structured infant and toddler behavioral feature set.

[0009] In some embodiments, the generation of preliminary assessment results, including the matching degree between developmental level assessment values ​​and abnormal features, through dual-library matching of a baseline behavioral feature library and a developmental abnormality feature library, and attention-weighted inference of the core abnormal features of the targeted disease, includes: The infant behavioral feature set is input into the infant development assessment vertical model to calculate the first similarity between it and the benchmark features of the same age and gender in the benchmark behavioral feature library, and the second similarity between it and the features of various diseases in the developmental abnormality feature library to obtain the normal development level assessment value and the initial abnormal feature matching degree. By introducing an attention mechanism, the matching degree corresponding to the core abnormal features of the target disease in the initial abnormal feature matching degree is weighted to strengthen the recognition weight of the core abnormal features and obtain the attention-weighted abnormal feature matching degree. The normal developmental level assessment value and the attention-weighted abnormal feature matching degree are combined to generate a preliminary assessment result that includes the developmental level assessment value and the targeted disease abnormal feature annotation information.

[0010] In some embodiments, the step of inputting the preliminary assessment results into a month-age threshold model dynamically adjusted based on individual infant development data for secondary verification, in order to determine a graded developmental abnormality risk level by comparing it with a personalized threshold and outputting an abnormal feature matching report for the targeted disease, includes: The infant's age in months, gender, and basic growth and development data in the associated metadata are used as dynamic threshold adjustment factors and input into the age threshold model to output the corresponding developmental level assessment threshold and abnormal risk level judgment threshold through multiple regression calculation. The developmental level assessment value in the preliminary assessment results is compared with the developmental level assessment threshold to obtain the first verification result; The second verification result is obtained by comparing the abnormal feature matching degree in the preliminary assessment result with the abnormal risk level determination threshold. The developmental abnormality risk level is determined as low risk, medium risk, or high risk based on the logical combination of the first verification result and the second verification result. For infants and young children identified as medium-risk or high-risk, an abnormal feature matching report is generated based on the abnormal feature annotation information of the targeted disease in the preliminary assessment results. The report includes the abnormal feature name, matching degree and time sequence pattern.

[0011] In some embodiments, the method further includes: The full-process clinical feedback data, including diagnostic results, intervention measures, and intervention effects, is obtained from the hierarchical early warning and full-link linkage triggered by the abnormal feature matching report. After the full-process clinical feedback data is annotated, it is added as an incremental fine-tuning sample to the fine-tuning dataset of the large-scale vertical model of infant and toddler development assessment to incrementally train the large-scale vertical model of infant and toddler development assessment and obtain the iterative large-scale vertical model of infant and toddler development assessment.

[0012] In some embodiments, the method further includes: Based on the diagnostic results in the full-process clinical feedback data, the threshold parameters of the age threshold model are corrected and the weights of the feature dimensions with high misdiagnosis or missed diagnosis rates are adjusted to obtain the iterative age threshold model. The iterative infant development assessment vertical model and the iterative age threshold model are verified through clinical effectiveness verification, and after the verification is passed, they are deployed to replace the model before iteration.

[0013] Secondly, an infant developmental level assessment system is provided, comprising: a preprocessing module, a feature extraction module, a preliminary assessment module, and an assessment adjustment module; wherein... The preprocessing module is used to acquire multimodal behavior video streams of infants and young children, and to preprocess the multimodal behavior video streams to obtain time-slice video segments. The feature extraction module is used to extract explicit basic behavioral features, implicit micro-behavioral features, and temporal correlation features that characterize the linkage between behaviors and the temporal distribution of abnormal features from the time slice video segments through a three-level feature extraction system consisting of convolutional neural network, optical flow method and key point detection and recurrent neural network, so as to construct a structured infant behavior feature set. The preliminary assessment module is used to input the infant behavioral feature set into a large-scale infant development assessment model that integrates prior knowledge from an infant care vertical database. The model generates a preliminary assessment result containing the developmental level assessment value and the degree of matching of abnormal features through dual-database matching of the baseline behavioral feature database and the developmental abnormality feature database, and attention-weighted reasoning on the core abnormal features of the targeted disease. The assessment and adjustment module is used to input the preliminary assessment results into a month-age threshold model that is dynamically adjusted based on individual infant development data for secondary verification. By comparing the model with a personalized threshold, the module determines the graded risk level of developmental abnormalities and outputs an abnormal feature matching report for the targeted disease.

[0014] Thirdly, a computer-readable storage medium is provided having a computer program stored thereon that can run on a processor, wherein the computer program, when executed by the processor, implements a method for assessing the developmental level of infants and young children as described in the first aspect.

[0015] Using the above method, this application obtains multimodal behavioral video streams of infants and young children, preprocesses the multimodal behavioral video streams to obtain time-slice video segments, and uses a three-level feature extraction system composed of convolutional neural networks, optical flow and key point detection, and recurrent neural networks to extract explicit basic behavioral features, implicit micro-behavioral features, and temporal correlation features representing the linkage between behaviors and the temporal distribution patterns of abnormal features from the time-slice video segments to construct a structured infant and young child behavioral feature set. The infant and young child behavioral feature set is input into a large-scale infant and young child development assessment model that integrates prior knowledge from an infant and young child care vertical database. Through dual-database matching of a benchmark behavioral feature library and a developmental abnormality feature library, and attention-weighted inference of core abnormal features of targeted diseases, a preliminary assessment result containing developmental level assessment values ​​and abnormal feature matching degrees is generated. The preliminary assessment result is input into a month-age threshold model dynamically adjusted based on individual infant and young child development data for secondary verification. By comparing with personalized thresholds, a graded developmental abnormality risk level is determined, and an abnormal feature matching report for the targeted disease is output. Thus, the developmental level of infants and young children can be assessed quickly and accurately. Attached Figure Description

[0016] Figure 1 This is a flowchart of a method for assessing the developmental level of infants and young children provided in this application.

[0017] Figure 2 This is a block diagram of a method for preprocessing multimodal behavioral video streams to obtain time-slice video segments, as provided in this application.

[0018] Figure 3 This application provides a method flowchart for constructing a structured set of infant and toddler behavioral features by extracting explicit basic behavioral features, implicit micro-behavioral features, and temporal correlation features that characterize the temporal distribution patterns of behavioral linkages and abnormal features from time-slice video segments using a three-level feature extraction system consisting of convolutional neural networks, optical flow methods and key point detection, and recurrent neural networks.

[0019] Figure 4This application provides a method block diagram for generating preliminary assessment results containing the matching degree between developmental level assessment values ​​and abnormal features by using dual-database matching of a baseline behavioral feature database and a developmental abnormality feature database, and attention-weighted reasoning on the core abnormal features of the target disease.

[0020] Figure 5 This is a schematic diagram of the connection of an infant development level assessment system provided in an embodiment of this application. Detailed Implementation

[0021] To better understand the purpose, technical solutions, and advantages of this application, it has been described and illustrated below with reference to the accompanying drawings and embodiments. However, those skilled in the art should understand that this application can be implemented without these details. It will be apparent to those skilled in the art that various modifications can be made to the embodiments disclosed in this application, and the general principles defined in this application can be applied to other embodiments and application scenarios without departing from the principles and scope of this application. Therefore, this application is not limited to the illustrated embodiments, but is consistent with the broadest scope claimed in this application.

[0022] The embodiments of this application will now be described in further detail with reference to the accompanying drawings.

[0023] Figure 1 This is a flowchart of a method for assessing the developmental level of infants and young children provided in this application. Figure 1 As shown, a method for assessing the developmental level of infants and young children includes the following steps: Step S100: Acquire the multimodal behavior video stream of infants and young children, and preprocess the multimodal behavior video stream to obtain time-slice video segments.

[0024] This application describes the process from the assessment perspective. Acquiring multimodal behavioral video streams of infants and toddlers is the data foundation for achieving accurate developmental assessment. The acquisition of multimodal behavioral video streams of infants and toddlers includes the following steps: Step S101: Collect a first visual behavior video and a first audio-accompanied video in a markerless free shooting mode via a home mobile terminal, and / or collect a second visual behavior video and a second audio-accompanied video in a multi-view high-definition shooting mode via a fixed camera position at the institutional end.

[0025] Step S102: Collect basic data on infants' age in months, gender, and growth and development associated with the video stream as associated metadata.

[0026] Step S103: Combine the collected first visual behavior video and / or second visual behavior video, first audio-accompanying video and / or second audio-accompanying video, and associated metadata into a multimodal behavior video stream.

[0027] Specifically, the assessment system supports two modes: lightweight data collection at home and standardized data collection at institutions, to adapt to application needs in different scenarios. In home mode, parents can use their own mobile devices (such as smartphones or tablets) to collect first-visual behavioral videos and corresponding first-audio videos in everyday care scenarios (such as parent-child interaction and free play) using unmarked, free-form filming. No professional equipment or fixed location is required, significantly lowering the barrier to parental participation and facilitating universal screening. In institution mode, multi-view high-definition cameras at fixed positions are used to simultaneously collect second-visual behavioral videos and second-audio videos in standardized clinical or assessment environments. This ensures the standardization and high fidelity of the video data's perspective, providing a high-quality, labeled data source for clinical review and model training.

[0028] During video capture, the system automatically acquires or guides parents to manually input metadata closely related to the infant, including basic growth and development data such as age in months, gender, height, weight, and gestational age at birth, as well as environmental parameters such as the shooting scene (e.g., feeding, playing) and shooting duration. This metadata is not only a key input for dynamic threshold adjustment and personalized assessment in subsequent steps, but also provides an important reference dimension for hierarchical feature extraction, ensuring that behavioral analysis can be accurately interpreted in conjunction with the individual's developmental background. Next, the collected visual behavioral videos, accompanying audio videos, and corresponding metadata are time-aligned and logically encapsulated, combining them into a structured multimodal behavioral video stream. This video stream serves as the raw input for subsequent preprocessing and feature extraction, preserving the original visual and audio signals while also including crucial individual background information, achieving a high degree of data fusion. This achieves full-scenario coverage and standardized access to multimodal behavioral data, balancing convenience for families with professionalism for institutions, solving the problems of single data sources and high collection thresholds, and providing a data foundation for large-scale screening; on the other hand, the synchronous introduction of associated metadata enables subsequent intelligent assessments to be combined with individual developmental trajectories, reducing one-size-fits-all analyses divorced from real-world contexts, and improving the personalization and clinical reference value of assessment results.

[0029] To transform multimodal behavioral video streams into high-quality data suitable for subsequent accurate analysis, further preprocessing is required. Figure 2 This is a block diagram of the method provided in this application for preprocessing multimodal behavioral video streams to obtain time-slice video segments. For example... Figure 2 As shown, preprocessing a multimodal behavioral video stream to obtain time-slice video segments includes the following steps: Step S104: Denoise and deblur the multimodal behavior video stream to obtain an enhanced video stream.

[0030] Step S105: Segment the infant subject from the enhanced video stream to obtain the main subject video stream.

[0031] Step S106: Remove blurry frames, occluded frames, and frames without behavior from the main video stream to obtain a valid video stream.

[0032] Step S107: Divide the valid video stream into several timestamped video segments as time-sliced ​​video segments.

[0033] Specifically, due to the complexity of the home-based acquisition environment, raw video streams often contain a large amount of noise (such as uneven lighting and sensor noise) and motion blur (caused by the movement of infants or unstable handheld shooting). This application first uses a Gaussian filtering algorithm to smooth and denoise video frames, effectively suppressing Gaussian noise in the image. Subsequently, for blurred frames, a deep learning-based super-resolution reconstruction algorithm is introduced, using a pre-trained model to repair and enhance the texture and edges of the blurred image, thereby improving the overall clarity of the video. Through the above combined processing, an enhanced video stream with high signal-to-noise ratio and rich visual details is obtained, providing a high-quality image source for subsequent accurate target segmentation and feature point detection.

[0034] To eliminate interference from complex and varied backgrounds (such as furniture and toys in a home environment) on subsequent behavioral analysis, a high-performance instance segmentation network, such as Mask R-CNN, is employed to process each frame in the enhanced video stream. This network can accurately identify individual infants in the video frames and generate foreground masks with pixel-level precision. Based on this mask, the infant subject is precisely extracted from the original background, resulting in a subject-only video stream. This minimizes background noise, allowing subsequent feature extraction networks to focus on the infant's behavioral performance.

[0035] Even after the aforementioned processing, low-quality segments that are detrimental to analysis may still exist in the main video stream. Next, a rigorous quality screening process is performed. The first step involves calculating the sharpness score of each frame using an image sharpness detection algorithm (such as the Tenengrad gradient function) and removing blurry frames below a preset threshold. The second step uses an object detection algorithm to determine if the infant is severely occluded (e.g., blocked by a parent's body), removing occluded frames with an occlusion area exceeding a preset percentage. The third step uses a motion detection algorithm to analyze inter-frame differences, removing frames where the infant has been stationary for extended periods or shows no significant movement. Through these layers of screening, only high-quality frame sequences containing valid behavioral information are retained and combined to form a valid video stream, thus preventing misleading subsequent analysis results from low-quality data at the source.

[0036] Finally, to facilitate temporal modeling and analysis of behavior, the continuous effective video stream is sliced ​​along the time dimension. That is, the long video stream is evenly divided into several short video segments according to a fixed time window. Simultaneously, a unique and continuous timestamp is added to each video segment to record its order and position on the original timeline. These timestamped video segments, i.e., time-sliced ​​video segments, constitute the standard input unit for the subsequent three-level feature extraction system. Through slicing, the continuous behavior stream is discretized into temporal units that are easy for the model to process, laying the foundation for subsequent use of recurrent neural networks to mine the temporal correlation patterns of behavior (such as the duration of specific behaviors and the order in which different behaviors occur). This constructs a standardized processing pipeline from raw video to high-quality temporal samples. Firstly, data quality and analytical accuracy are improved. Through denoising, deblurring, subject segmentation, and outlier frame removal, various interferences in the original video are effectively eliminated, ensuring that the data fed into the feature extraction network is clear, pure, and focused on the infant subject, reducing feature extraction errors and evaluation biases caused by low-quality input data. Secondly, it is compatible with video streams of varying quality from both home and institutional sources. A standardized preprocessing workflow eliminates differences at the acquisition end, enabling stable application in diverse real-world scenarios and providing technical support for large-scale, inclusive infant screening. Finally, it achieves a controllable closed-loop system for data quality.

[0037] Step S200: A structured set of infant and toddler behavioral features is constructed by extracting explicit basic behavioral features, implicit micro-behavioral features, and temporal correlation features that characterize the linkage between behaviors and the temporal distribution of abnormal features from time-slice video segments through a three-level feature extraction system consisting of convolutional neural network, optical flow method and key point detection and recurrent neural network.

[0038] After obtaining video segments with precise timestamps, in order to further explore the implicit behavioral cues and temporal patterns that are highly related to specific conditions such as autism, a three-level feature extraction system was further constructed, ranging from coarse to fine and from static to dynamic. Figure 3 This application provides a method flowchart for constructing a structured set of infant and toddler behavioral features by extracting explicit basic behavioral features, implicit micro-behavioral features, and temporal correlation features representing the temporal distribution patterns of inter-behavioral linkages and abnormal features from time-slice video segments using a three-level feature extraction system consisting of convolutional neural networks, optical flow methods and keypoint detection, and recurrent neural networks. Figure 3As shown, a three-level feature extraction system consisting of convolutional neural networks, optical flow and keypoint detection, and recurrent neural networks is used to extract explicit basic behavioral features, implicit micro-behavioral features, and temporal correlation features representing the temporal distribution patterns of behavioral linkages and abnormal features from time-slice video segments to construct a structured set of infant behavioral features. The steps include: Step S201: Input the time-slice video segments into a convolutional neural network to extract explicit basic behavioral features including head movements, body movements, facial expressions, eye contact, and speech to construct a subset of basic behavioral features.

[0039] Step S202: The time segment video is segmented and input into the optical flow method and key point detection algorithm to extract implicit micro-behavioral features, including facial micro-expressions, body micro-movements and social micro-reactions, under the set effective micro-feature recognition threshold, to construct a subset of micro-behavioral features.

[0040] Step S203: Input the basic behavioral feature subset and the micro-behavioral feature subset into the long short-term memory neural network for temporal modeling, and mine the temporal correlation of behavioral features and the temporal pattern of missing targeted abnormal features to construct a temporal correlation feature subset.

[0041] Step S204: Integrate the basic behavioral feature subset, the micro-behavioral feature subset, and the temporal correlation feature subset to obtain a structured infant and toddler behavioral feature set.

[0042] Specifically, the process begins by constructing a subset of basic behavioral features to extract macroscopic and explicit characteristics of infant behavior. This involves inputting each time-slice video segment into a lightweight 3D convolutional neural network (3D-CNN). This network effectively captures short-term spatiotemporal features within the video segments by performing convolution operations simultaneously in both spatial and temporal dimensions. Its output is a high-dimensional feature vector, which is then fed into multiple parallel task-specific classifiers to identify and quantify specific explicit behaviors. For example, a head motion estimator outputs the head's rotation angle and trajectory in 3D space; a limb movement recognizer classifies preset action types such as grasping, crawling, and standing based on key points of the human skeleton and calculates their frequencies; a facial expression analyzer uses a finely tuned facial action unit (AU) detection model to determine the intensity and duration of expressions such as laughing, crying, and surprise; an eye contact detector combines head posture and eye key points to calculate the gaze direction (towards a person, an object, or aimlessly) and its duration; and an audio branch independently processes the decoupled audio stream, extracting features such as vocal frequency, pitch, and syllable clarity. Through the above process, each video segment was quantified into a subset of basic behavioral features, including head movements, body movements, facial expressions, eye contact, and vocalizations. This subset outlines the basic characteristics of infant and toddler behavior, laying the foundation for subsequent in-depth analysis.

[0043] Then, a subset of micro-behavioral features is constructed. Building upon the macro-behavioral features, the algorithm further focuses on latent micro-behavioral features that are extremely short in duration, small in spatial scale, and difficult to detect with the naked eye, but are crucial indicators for diagnosing developmental disorders such as autism. Specifically, for each video segment, a high-resolution network (HRNet) is used for dense human pose keypoint detection to obtain a fine keypoint coordinate sequence including the face, hands, and torso. Simultaneously, optical flow is used to calculate the motion vector field of each pixel between adjacent frames to generate an optical flow map. Subsequently, the keypoint coordinate sequence and optical flow map are input into a specific spatiotemporal graph convolutional network (ST-GCN). This network can simultaneously model the spatial connectivity of keypoints and their temporal evolution trends. To eliminate physiological noise such as muscle tremors, the algorithm sets an effective recognition threshold for micro-features. For example, a micro-behavioral event is only considered valid when the displacement of a keypoint accumulates to more than 0.5 mm within several consecutive frames (e.g., 0.2 seconds) and the motion pattern conforms to a preset micro-movement template (e.g., slight flexion and extension of fingers, slight pursing of the mouth, slight head turn in response to being called). The ST-GCN network ultimately outputs the type, timing, intensity, and duration of various micro-behavioral events in each video segment, forming a subset of micro-behavioral features. This captures fine behavioral signals closely related to neurodevelopmental function that are missed by conventional video analysis.

[0044] Next, a temporal correlation feature subset is constructed. One of the core characteristics of developmental disorders such as autism lies in abnormal behavioral patterns, which are often reflected in the temporal correlation of behaviors rather than isolated behaviors themselves. Further, the linkage and missing patterns of different behaviors in the temporal dimension are explored. Specifically, the basic behavioral feature subsets and micro-behavioral feature subsets obtained above for multiple consecutive time-slice video segments are concatenated into a behavioral feature sequence in chronological order. This sequence is then input into a bidirectional long short-term memory network (Bi-LSTM). Bi-LSTM, with its unique gating structure, can effectively learn long-distance dependencies in sequence data. Two key temporal patterns are modeled: one is the temporal correlation of behavioral features, which calculates the order and co-occurrence probability of certain behaviors occurring within a continuous time period. For example, in social interaction scenarios, the model learns the probability of a complete social response chain: someone calling out—a slight head turn—eye contact. If the probability of this chain is significantly lower than normal, it indicates a risk of missing social responses. Second, it targets the temporal patterns of missing anomalous features. The model automatically calculates and outputs the duration and frequency of persistent absences of targeted behavioral features such as eye contact and social smiling within a specific time window, or the periodic patterns of persistent occurrences of anomalous behaviors such as stereotyped swaying. The output of Bi-LSTM is a fixed-length vector that encodes the aforementioned temporal information, forming a subset of temporally correlated features.

[0045] Finally, the feature set of infants and toddlers is generated by merging the three complementary feature subsets mentioned above. This involves concatenating the three feature vectors—the basic behavioral feature subset, the micro-behavioral feature subset, and the temporal correlation feature subset—at the feature layer to form a higher-dimensional, more information-rich comprehensive feature vector. This vector is the final structured feature set of infants and toddlers. It includes not only what the behavior is (basic features), but also the fine details of how the behavior occurs (micro-features), and reveals the dynamic patterns of how the behavior evolves (temporal features), providing unprecedented, multi-dimensional, and high-fidelity data support for subsequent intelligent assessment models. This achieves a comprehensive digital characterization of infant and toddler behavior from macro to micro, from instantaneous to temporal. Firstly, micro-behavioral feature extraction is introduced to capture subtle behavioral signals strongly correlated with conditions such as autism that are unrecognizable by traditional methods, fundamentally solving the problem of superficial assessment. Secondly, it reveals deep patterns in behavior. Through temporal correlation feature analysis, isolated actions are linked into clinically significant behavioral chains and patterns, enabling precise identification of abnormal patterns such as broken social response chains and the periodic recurrence of stereotyped behaviors. This improves the sensitivity and specificity of identifying developmental disorders, especially autism. The resulting infant behavioral feature set provides a reliable data foundation for subsequent high-precision, personalized intelligent assessments.

[0046] Step S300: Input the infant and toddler behavioral feature set into a large-scale infant and toddler development assessment model that integrates prior knowledge from an infant and toddler care vertical database. This model generates a preliminary assessment result that includes the matching degree between the developmental level assessment value and the abnormal feature through dual-database matching of the baseline behavioral feature database and the developmental abnormality feature database, as well as attention-weighted reasoning on the core abnormal features of the targeted disease.

[0047] After obtaining a structured set of infant and toddler behavioral characteristics, in order to deeply integrate the data-driven characteristics with expert knowledge in the medical field and thus obtain more clinically interpretable preliminary assessment results, this application further generates preliminary assessment results. Figure 4 This application provides a method flowchart for generating preliminary assessment results containing the matching degree between developmental level assessment values ​​and abnormal features through dual-database matching of a baseline behavioral feature database and a developmental abnormality feature database, and attention-weighted inference of core abnormal features of the targeted disease. (See diagram for details.) Figure 4 As shown, the preliminary assessment results, which include the matching degree between developmental level assessment values ​​and abnormal features, are generated through dual-database matching of a baseline behavioral feature database and a developmental abnormality feature database, and attention-weighted inference of the core abnormal features of the targeted disease. The steps include: Step S301: Input the infant and toddler behavioral feature set into the large-scale model of infant and toddler development assessment, and calculate the first similarity with the baseline features of the same age and gender in the baseline behavioral feature library and the second similarity with the features of various diseases in the developmental abnormality feature library to obtain the normal development level assessment value and the initial abnormal feature matching degree.

[0048] Step S302: By introducing an attention mechanism, the matching degree corresponding to the core abnormal features of the target disease in the initial abnormal feature matching degree is weighted to strengthen the recognition weight of the core abnormal features and obtain the attention-weighted abnormal feature matching degree.

[0049] Step S303: Integrate the normal developmental level assessment value with the attention-weighted abnormal feature matching degree to generate a preliminary assessment result containing the developmental level assessment value and the targeted disease abnormal feature annotation information.

[0050] Specifically, this application pre-constructs a large-scale vertical model for infant and toddler development assessment. This model is not a general neural network, but rather a specialized model fine-tuned based on an existing large-scale model, utilizing a vast amount of labeled infant and toddler behavioral video samples and clinical case samples of developmental abnormalities provided by institutions such as the National Health Commission and authoritative children's hospitals. This model possesses a deep understanding of infant and toddler behavioral characteristics. Simultaneously, authoritative clinical guidelines and research data are integrated to construct an infant and toddler care vertical database, which includes two core sub-databases: a baseline behavioral feature database and a developmental abnormality feature database. The baseline behavioral feature database stores benchmark values ​​and reasonable fluctuation ranges of behavioral characteristics of infants and toddlers of different ages and genders under normal developmental conditions. The developmental abnormality feature database stores typical behavioral feature templates for various conditions, including autism and developmental delays. These templates not only include basic behavioral features but also cover the extracted micro-behavioral features and temporal correlation features. During inference, the generated infant and toddler behavioral feature set is input into the vertical model. The core task of the model is to perform dual-database matching between the input features and the two knowledge bases mentioned above. The first step involves retrieving baseline feature groups from the baseline behavioral feature database that match the current infant's age and gender, and calculating the first similarity between the input feature set and these baseline features. This similarity measure quantifies the extent to which the infant's behavior conforms to the general developmental trajectory of children of the same age and gender, ultimately outputting a normal developmental level assessment value, such as a score; a higher score indicates closer proximity to normal development. The second step involves comparing the input feature set with feature templates for various diseases stored in the developmental abnormality feature database, calculating the second similarity with each disease feature, thus obtaining an initial abnormal feature matching vector. This vector reflects the preliminary degree of agreement between the infant's behavior and different disease types.

[0051] In clinical practice, certain specific conditions, especially autism spectrum disorders, possess some highly typical but easily overlooked core abnormal features, such as persistent lack of eye contact, stereotyped and repetitive behaviors, and unresponsiveness to social stimuli. These features should have higher decision weight in condition identification. Therefore, an attention mechanism was introduced into the vertical-category large-scale model. The core idea of ​​this mechanism is to allow the model to focus on core abnormal features with high discriminative power for specific targeted conditions (such as autism). Specifically, the model scans the input set of infant behavioral features to identify whether it contains features that match the core abnormal feature database. Once these core features are identified, such as the absence of eye contact in multiple consecutive video segments in the temporal correlation features, the attention mechanism automatically assigns higher weights to the matching degree corresponding to these features. Through this weighted processing, the model can effectively suppress the interference of background noise and common behavioral biases, focusing the analysis on key behavioral signals strongly correlated with the specific condition, thus obtaining the attention-weighted abnormal feature matching degree. This weighted matching degree more accurately reflects the intensity of the infant's performance on key pathological features.

[0052] Finally, the obtained normal developmental level assessment value is fused with the abnormal feature matching degree after attention weighting. This fusion is not a simple arithmetic average, but a logical integration at the model's output layer to generate a comprehensive preliminary assessment result. This result typically includes two core pieces of information: the developmental level assessment value and the abnormal feature annotation information for the targeted disease. The developmental level assessment value is a quantitative score that intuitively reflects the infant's current overall developmental level. The abnormal feature annotation information for the targeted disease refers to the model's explicit identification of possible abnormal types when the abnormal feature matching degree after attention weighting exceeds a certain threshold. Like highlighting, it marks which specific features led to this judgment in the original behavioral feature set, providing clues for subsequent doctor review. This achieves a deep integration of data-driven and knowledge-driven approaches. On the one hand, the dual-database matching mechanism ensures that the assessment has both a general developmental norm reference and a specific disease template comparison, giving the assessment results both breadth and depth. On the other hand, the introduction of the attention mechanism, especially the weighted enhancement of core abnormal features for targeted diseases such as autism, improves the model's sensitivity and recognition accuracy to key pathological signals, effectively avoiding missed diagnoses due to the omission of subtle but crucial features. In addition, the generated preliminary assessment results containing abnormal feature annotations are no longer just a single black-box score, but a clinically interpretable semi-finished report. It not only quantifies the developmental level, but also indicates the potential abnormal direction and its specific manifestations, providing precise guidance for doctors' subsequent diagnosis and significantly improving the efficiency of the link from screening to diagnosis.

[0053] Step S400: Input the preliminary assessment results into a month-age threshold model that is dynamically adjusted based on individual infant development data for secondary verification. By comparing with personalized thresholds, the graded developmental abnormality risk level is determined and an abnormal feature matching report for the targeted disease is output.

[0054] After obtaining the initial assessment results, to overcome the shortcomings of a uniform assessment threshold, such as poor individual adaptability and high rates of misdiagnosis and missed diagnosis, a second verification is introduced using a dynamically adjusted age-based threshold model based on individual infant development data. This allows for personalized calibration and risk grading of the assessment results. The process of inputting the initial assessment results into a dynamically adjusted age-based threshold model based on individual infant development data for second verification, comparing it with personalized thresholds to determine a graded risk level of developmental abnormalities, and outputting an abnormal feature matching report for the targeted disease includes the following steps: Step S401: The infant's age in months, gender, and basic growth and development data in the associated metadata are used as dynamic threshold adjustment factors and input into the age threshold model to output the corresponding individual infant's developmental level assessment threshold and abnormal risk level determination threshold through multiple regression calculation.

[0055] Step S402: Compare the developmental level assessment value in the preliminary assessment results with the developmental level assessment threshold to obtain the first verification result.

[0056] Step S403: Compare the abnormal feature matching degree in the preliminary assessment results with the abnormal risk level judgment threshold to obtain the second verification result.

[0057] Step S404: Based on the logical combination of the first verification result and the second verification result, determine the developmental abnormality risk level as low risk, medium risk, or high risk.

[0058] Step S405: For infants and young children who are determined to be at medium or high risk, generate an abnormal feature matching report containing the abnormal feature name, matching degree and time sequence pattern based on the target disease abnormal feature annotation information in the preliminary assessment results.

[0059] Specifically, the age-based threshold model does not use a fixed normative cut-off value, but is constructed as a dynamic computational model that takes individual developmental data as input. The model's input consists of the associated metadata collected in step S102, with core input variables including the infant's age in months and sex, while also incorporating basic growth and development data such as height, weight, and gestational age at birth as covariates. Internally, the model uses a pre-trained multiple regression algorithm on a large-scale clinical sample to calculate two key thresholds for that specific individual: a developmental level assessment threshold to measure overall developmental level, reflecting a reasonable lower limit for the degree of matching normal developmental behavioral characteristics under the current age, sex, and growth conditions; and an abnormal risk level determination threshold to define the starting point for exceeding the normal fluctuation range. In this way, the model achieves a shift from a uniform standard to a personalized approach.

[0060] After obtaining the personalized dynamic threshold, a dual verification is performed. The first verification focuses on the overall developmental level, comparing the generated developmental level assessment value (i.e., the first similarity obtained by matching with the baseline behavioral feature library) with the calculated developmental level assessment threshold. If the former is lower than the latter, the first verification result is a warning, indicating that overall development may deviate from the normal track. The second verification focuses on abnormal features, comparing the abnormal feature matching degree after attention weighting in step S303 with the abnormal risk level judgment threshold calculated in step S401. If the former is higher than the latter, the second verification result is abnormal, indicating the existence of specific developmental abnormal features. These two verification results together constitute the quantitative basis for risk judgment.

[0061] The results from the two dimensions are then logically fused to output a clinically significant risk classification. The judgment rules are pre-defined: if both the first and second verification results are normal, the risk level is low; if the first verification result is warning or the second verification result is abnormal, but neither reaches a severity level, the risk level is medium, indicating the need for further review; if the first verification result is significantly below the threshold and the second verification result is significantly above the threshold, especially when the matching degree of core abnormal features enhanced by attention mechanisms exceeds the preset high-risk threshold, the risk level is high. This classification mechanism achieves refined risk stratification, providing a basis for decision-making regarding subsequent differentiated intervention measures.

[0062] For individuals flagged as medium-risk or higher, a structured abnormal feature matching report is automatically generated. This report is not simply a risk warning, but rather a deep integration and visualization of the targeted disease abnormal feature annotation information generated in step S303. The report specifically includes the name of the abnormal feature identified by the model; the degree of matching between this feature and disease templates in the developmental abnormality feature library; and the temporal distribution pattern of this feature, such as the frequency of occurrence in consecutive video segments and the longest duration. This report, along with the preliminary assessment results, is submitted to the reviewing physician or treatment institution in the next stage, providing objective and quantitative data support for clinical diagnosis and effectively shortening the link from screening to clinical decision-making. By constructing a threshold model that is dynamically adjusted based on age, sex, and basic growth and development data, the limitations of a one-size-fits-all norm comparison are overcome, allowing the assessment results to closely reflect the actual developmental trajectory of individual infants and significantly reducing the rate of misdiagnosis and missed diagnosis due to individual differences. Secondly, by introducing a dual verification mechanism and graded risk assessment, the single assessment score is transformed into clinically operable low, medium, and high risk levels, providing precise triggering conditions for the subsequent establishment of a graded early warning and end-to-end linkage mechanism. Finally, the abnormal feature matching report automatically generated for medium- and high-risk individuals transforms the model's black-box judgment into interpretable and traceable clinical evidence, which not only improves the transparency and credibility of the assessment results but also provides doctors with precise diagnostic clues, significantly improving the collaborative efficiency from screening to diagnosis.

[0063] Preferably, this application also acquires full-process clinical feedback data, including diagnostic results, intervention measures, and intervention effects, from the hierarchical early warning and full-link linkage triggered by abnormal feature matching reports. After annotating the full-process clinical feedback data, it is added as incremental fine-tuning samples to the fine-tuning dataset of the large-scale infant development assessment model to incrementally train the large-scale infant development assessment model, resulting in an iterative large-scale infant development assessment model. Based on the diagnostic results in the full-process clinical feedback data, the threshold parameters of the age threshold model are corrected, and the weights of feature dimensions with high misdiagnosis or missed diagnosis rates are adjusted to obtain the iterative age threshold model. The iterative large-scale infant development assessment model and the iterative age threshold model are validated through clinical effectiveness verification, and after successful verification, they are deployed to replace the previous model. This achieves continuous model evolution and clinical adaptation, effectively reducing misdiagnosis and missed diagnosis rates.

[0064] Figure 5 This is a schematic diagram of the connection of an infant development level assessment system provided in an embodiment of this application. Figure 5 As shown, an infant development level assessment system includes: a preprocessing module, a feature extraction module, a preliminary assessment module, and an assessment adjustment module.

[0065] The system comprises several modules: a preprocessing module to acquire multimodal behavioral video streams of infants and toddlers, and a feature extraction module to extract explicit basic behavioral features, implicit micro-behavioral features, and temporal correlation features representing the interrelationships between behaviors and the temporal distribution patterns of abnormal features from the time-slice video segments using a three-level feature extraction system consisting of convolutional neural networks, optical flow methods, keypoint detection, and recurrent neural networks. This constructs a structured set of infant behavioral features. A preliminary assessment module inputs the infant behavioral feature set into a large-scale infant development assessment model that integrates prior knowledge from an infant care database. This model generates a preliminary assessment result containing developmental level assessment values ​​and abnormal feature matching degrees through dual-database matching of a baseline behavioral feature library and a developmental abnormality feature library, and attention-weighted inference of core abnormal features of the target condition. An assessment adjustment module inputs the preliminary assessment result into a month-age threshold model dynamically adjusted based on individual infant developmental data for secondary verification. This model determines the graded developmental abnormality risk level by comparing the result with personalized thresholds and outputs an abnormal feature matching report for the target condition.

[0066] The other functions performed by the preprocessing module, feature extraction module, preliminary assessment module, and assessment adjustment module, as well as the technical details of each function, are the same as or similar to the corresponding features in the infant development level assessment method described above, so they will not be repeated here.

[0067] This application also provides a computer storage medium storing a computer program that, when run on a computer, enables the computer to perform the steps in the aforementioned method for assessing the developmental level of infants and young children.

[0068] It should be understood that although the steps in the flowcharts in the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily performed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order requirement for the execution of these steps, and they can be performed in other orders.

[0069] The above description is only a partial embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for assessing the developmental level of infants and young children, characterized in that, The method includes: Acquire multimodal behavior video streams of infants and young children, and preprocess the multimodal behavior video streams to obtain time-slice video segments; A structured set of infant and toddler behavioral features is constructed by extracting explicit basic behavioral features, implicit micro-behavioral features, and temporal correlation features that characterize the temporal distribution patterns of behavioral linkages and abnormal features from the time-slice video segments using a three-level feature extraction system consisting of convolutional neural networks, optical flow methods and key point detection, and recurrent neural networks. The infant behavioral feature set is input into a large-scale infant development assessment model that integrates prior knowledge from an infant care vertical database. The model generates a preliminary assessment result that includes the matching degree between the developmental level assessment value and the abnormal feature through dual-database matching of the baseline behavioral feature database and the developmental abnormality feature database, and attention-weighted reasoning on the core abnormal features of the targeted disease. The preliminary assessment results are input into a month-age threshold model that is dynamically adjusted based on individual infant development data for secondary verification. By comparing the model with personalized thresholds, a graded risk level of developmental abnormalities is determined, and an abnormal feature matching report for the targeted disease is output.

2. The method according to claim 1, characterized in that, The acquisition of multimodal behavioral video streams of infants and young children includes: The first visual behavioral video and the first audio-accompanied video are captured by a home mobile terminal in an unmarked free shooting mode, and / or the second visual behavioral video and the second audio-accompanied video are captured by a fixed camera position at an institutional end in a multi-view high-definition shooting mode. Collect basic data on infants' age in months, gender, and growth and development associated with the video stream as related metadata; The first and / or second visual behavior videos, the first and / or second audio-related videos, and the associated metadata are combined into a multimodal behavior video stream.

3. The method according to claim 2, characterized in that, The preprocessing of the multimodal behavioral video stream to obtain time-slice video segments includes: The multimodal behavioral video stream is denoised and deblurred to obtain an enhanced video stream; The main video stream is obtained by segmenting the infant subject from the enhanced video stream; The effective video stream is obtained by removing blurry frames, occluded frames, and frames without behavior from the main video stream. The effective video stream is divided into several timestamped video segments as time-sliced ​​video segments.

4. The method according to claim 3, characterized in that, The three-level feature extraction system, consisting of convolutional neural networks, optical flow and keypoint detection, and recurrent neural networks, extracts explicit basic behavioral features, implicit micro-behavioral features, and temporal correlation features representing the temporal distribution patterns of inter-behavioral linkages and abnormal features from the time-slice video segments to construct a structured set of infant and toddler behavioral features. The time-slice video is segmented and input into a convolutional neural network to extract explicit basic behavioral features, including head movements, body movements, facial expressions, eye contact, and speech, to construct a subset of basic behavioral features. The time-slice video segments are input into the optical flow method and the key point detection algorithm to extract implicit micro-behavioral features, including facial micro-expressions, body micro-movements and social micro-reactions, under a set effective micro-feature recognition threshold, to construct a subset of micro-behavioral features. The basic behavioral feature subset and the micro-behavioral feature subset are input into a long short-term memory neural network for temporal modeling to mine the temporal correlation of behavioral features and the temporal pattern of missing targeted abnormal features in order to construct a temporal correlation feature subset. The basic behavioral feature subset, micro-behavioral feature subset, and temporal correlation feature subset are integrated to obtain a structured infant and toddler behavioral feature set.

5. The method according to claim 4, characterized in that, The preliminary assessment results, which generate a matching degree between developmental level assessment values ​​and abnormal features, are generated through dual-database matching of a baseline behavioral feature database and a developmental abnormality feature database, and attention-weighted inference of the core abnormal features of the targeted disease. The infant behavioral feature set is input into the infant development assessment vertical model to calculate the first similarity between it and the benchmark features of the same age and gender in the benchmark behavioral feature library, and the second similarity between it and the features of various diseases in the developmental abnormality feature library to obtain the normal development level assessment value and the initial abnormal feature matching degree. By introducing an attention mechanism, the matching degree corresponding to the core abnormal features of the target disease in the initial abnormal feature matching degree is weighted to strengthen the recognition weight of the core abnormal features and obtain the attention-weighted abnormal feature matching degree. The normal developmental level assessment value and the attention-weighted abnormal feature matching degree are combined to generate a preliminary assessment result that includes the developmental level assessment value and the targeted disease abnormal feature annotation information.

6. The method according to claim 5, characterized in that, The process of inputting the preliminary assessment results into a month-age threshold model dynamically adjusted based on individual infant development data for secondary verification, and determining the graded developmental abnormality risk level by comparing it with personalized thresholds, and outputting an abnormal feature matching report for the targeted disease includes: The infant's age in months, gender, and basic growth and development data in the associated metadata are used as dynamic threshold adjustment factors and input into the age threshold model to output the corresponding developmental level assessment threshold and abnormal risk level judgment threshold through multiple regression calculation. The developmental level assessment value in the preliminary assessment results is compared with the developmental level assessment threshold to obtain the first verification result; The second verification result is obtained by comparing the abnormal feature matching degree in the preliminary assessment result with the abnormal risk level determination threshold. The developmental abnormality risk level is determined as low risk, medium risk, or high risk based on the logical combination of the first verification result and the second verification result. For infants and young children identified as medium-risk or high-risk, an abnormal feature matching report is generated based on the abnormal feature annotation information of the targeted disease in the preliminary assessment results. The report includes the abnormal feature name, matching degree and time sequence pattern.

7. The method according to claim 6, characterized in that, The method further includes: The full-process clinical feedback data, including diagnostic results, intervention measures, and intervention effects, is obtained from the hierarchical early warning and full-link linkage triggered by the abnormal feature matching report. After the full-process clinical feedback data is annotated, it is added as an incremental fine-tuning sample to the fine-tuning dataset of the large-scale vertical model of infant and toddler development assessment to incrementally train the large-scale vertical model of infant and toddler development assessment and obtain the iterative large-scale vertical model of infant and toddler development assessment.

8. The method according to claim 7, characterized in that, The method further includes: Based on the diagnostic results in the full-process clinical feedback data, the threshold parameters of the age threshold model are corrected and the weights of the feature dimensions with high misdiagnosis or missed diagnosis rates are adjusted to obtain the iterative age threshold model. The iterative infant development assessment vertical model and the iterative age threshold model are verified through clinical effectiveness verification, and after the verification is passed, they are deployed to replace the model before iteration.

9. A developmental assessment system for infants and young children, characterized in that, The system includes: a preprocessing module, a feature extraction module, a preliminary evaluation module, and an evaluation adjustment module; wherein, The preprocessing module is used to acquire multimodal behavior video streams of infants and young children, and to preprocess the multimodal behavior video streams to obtain time-slice video segments. The feature extraction module is used to extract explicit basic behavioral features, implicit micro-behavioral features, and temporal correlation features that characterize the linkage between behaviors and the temporal distribution of abnormal features from the time slice video segments through a three-level feature extraction system consisting of convolutional neural network, optical flow method and key point detection and recurrent neural network, so as to construct a structured infant behavior feature set. The preliminary assessment module is used to input the infant behavioral feature set into a large-scale infant development assessment model that integrates prior knowledge from an infant care vertical database. The model generates a preliminary assessment result containing the developmental level assessment value and the degree of matching of abnormal features through dual-database matching of the baseline behavioral feature database and the developmental abnormality feature database, and attention-weighted reasoning on the core abnormal features of the targeted disease. The assessment and adjustment module is used to input the preliminary assessment results into a month-age threshold model that is dynamically adjusted based on individual infant development data for secondary verification. By comparing the model with a personalized threshold, the module determines the graded risk level of developmental abnormalities and outputs an abnormal feature matching report for the targeted disease.

10. A computer-readable storage medium storing a computer program executable on a processor, characterized in that, When the computer program is executed by the processor, it implements a method for assessing the developmental level of infants and young children as described in any one of claims 1 to 8.

Images