A method, system, device and medium for generating an observation record of a child's behavior
By using lightweight neural networks and mapping matching algorithms, we have achieved automated and standardized assessment of children's behavior, which solves the problems of time lag and subjective bias in traditional observation and recording, and provides professional and personalized educational guidance.
Patent Information
- Application Number
- CN202610413682.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-03-31
- Publication Date
- 2026-07-07
AI Technical Summary
Traditional observation and recording of children's behavior suffers from time lag and subjective bias, smart hardware lacks end-to-end collaboration mechanisms, and video analysis technology lacks educational semantic mapping capabilities, resulting in observation results that cannot be standardized and compared for evaluation.
A lightweight neural network is used for multimodal feature extraction. Combined with the "Guidelines for Learning and Development of Children Aged 3-6" standard, a semantic relevance score is calculated through a mapping matching algorithm to generate an observation and assessment text report.
It enables automated and standardized assessment of children's behavior, reduces teachers' workload, provides objective and accurate assessment data, and generates professional and personalized educational guidance reports.
Smart Images

Figure CN122347827A_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of educational informatization and computer vision technology, and in particular relates to a method, system, device and medium for generating observation records of children's behavior. Background Technology
[0002] In the field of preschool education, the observation and assessment of children's behavior is a core component of teachers' daily work and a crucial step in implementing the "Guidelines for Learning and Development of Children Aged 3-6" (hereinafter referred to as the "Guidelines"). Traditional methods of recording children's behavior primarily rely on teachers' manual records, which present the following technical problems: Observation records suffer from serious time lag and subjective bias: Currently, children's behavior records mainly rely on manual completion by teachers. Due to the busy teaching schedule, teachers often record from memory after the fact, which can easily lead to the omission of key information due to memory decay. At the same time, the textual descriptions in manual records are heavily influenced by personal subjectivity and lack objective behavioral data support, making it difficult to form a standardized and comparable assessment basis for the observation results, and making it difficult to accurately match the actual developmental level of children.
[0003] The lack of edge-side collaboration mechanisms for smart hardware hinders real-time on-site processing: Existing smart hardware used in preschool education scenarios (such as fitness trackers, surveillance cameras, and ordinary shooting devices) are isolated data collection systems, lacking a real-time linkage mechanism between the "collection end" and the "computing end." Video, image, and other behavioral data must be manually exported to a PC or uploaded to the cloud for subsequent analysis and processing. This prevents the immediate identification of children, analysis of behavioral characteristics, and feedback of results using portable devices in the kindergarten classroom, resulting in significant delays in behavioral assessment and negating the value of timely educational guidance.
[0004] Video analytics technology lacks the semantic mapping capabilities required by professional educational standards, resulting in a disconnect between technology and educational needs. Existing computer vision and video analytics technologies are largely limited to basic security applications, enabling functions such as fall detection, intrusion detection, and basic facial recognition. They fail to understand the educational semantics of complex behaviors in preschool settings. For typical behaviors such as building with blocks, sharing with peers, and reading picture books, current technologies cannot automatically associate and map them to the professional indicators outlined in the "Guidelines for Learning and Development of Children Aged 3-6," including the five major domains, sub-domains, and specific developmental goals. This disconnect between the application of intelligent technology and professional assessment standards in preschool education prevents the provision of professional technical support for early childhood development assessment.
[0005] In summary, there is an urgent need for a technical solution that can adapt to the preschool education scene, enable real-time data processing on the edge, and accurately map children's behavioral characteristics to professional education standards. This solution would address issues such as subjective bias in traditional observation and recording, lack of edge collaboration in existing intelligent technologies, and insufficient semantic mapping, thereby achieving automation, standardization, and scientification of children's behavioral observation and recording. Summary of the Invention
[0006] The technical problem to be solved by this application is to provide a method, system, device and medium for generating observation records of children's behavior, so as to solve the problems mentioned in the background art.
[0007] To address the aforementioned technical problems, this application provides the following technical solution: Firstly, this application provides a method for generating observation records of children's behavior, including: Real-time behavioral video stream data of a group of young children is acquired, and after preprocessing the real-time behavioral video stream data, a preset lightweight neural network is used to extract multimodal features to obtain the results of multimodal feature extraction. The results of multimodal feature extraction include: action feature vector, emotion feature vector, environment / object feature vector, and child's identity feature vector. Based on the standard indicators for preschool children in the standard document "Guidelines for Learning and Development of Children Aged 3-6", a set of standard indicator vectors in a high-dimensional feature space is generated. A mapping matching algorithm is used to calculate the semantic relevance score between the results of the multimodal feature extraction and each standard indicator vector in the standard indicator vector group, and to form a semantic relevance score dataset. Based on the semantic relevance score dataset, determine the metrics to be evaluated; Based on the indicators to be evaluated, obtain the corresponding indicator template; Based on the indicator template, the extraction results of the multimodal features, and historical assessment data, an observation and assessment text report of the target child is generated.
[0008] Furthermore, the preprocessing includes: video frame extraction, key frame extraction and filtering, and feature region cropping and positioning.
[0009] Further, determining the metrics to be evaluated based on the semantic relevance score dataset includes: The scores in the semantic relevance score dataset are sorted from highest to lowest. Based on the ranking results, the indicators corresponding to the semantic relevance scores of the top N scores are selected as the indicators to be evaluated.
[0010] Furthermore, the feature extraction is performed using the following calculation formula: ; in, For real-time behavioral video stream data; For feature extraction functions; This is the action feature vector; This is a feature vector representing emotions. For environment / object feature vectors; This is a feature vector representing the identity of a young child.
[0011] Furthermore, the child's identity feature vector is constructed using the child's facial information. If the acquisition of the child's facial information fails, the following calculation formula is used for construction: ; in, Features detected in video stream data; For the characteristic template of child i; Weighting coefficients for clothing color features; This is the color feature matching function for clothing.
[0012] Furthermore, the semantic relevance score between the multimodal feature extraction results and the standard indicator vector group of the preschoolers is calculated using a mapping matching algorithm, and the following calculation formula is used: ; in, The semantic correlation score between the multimodal features of young children and the k-th standard indicator vector. Let be the standard feature values of the k-th standard indicator in the dimensions of action, emotion, and environment / object, respectively. The weight coefficients for each dimension. This is the environment context bias function.
[0013] Furthermore, the step of generating an observation and assessment text report for the target child based on the indicator template, the extraction results of the multimodal features, and historical assessment data uses the following calculation formula: ; in, For observation and evaluation of text reports; For generating functions; This is the template for the Kth indicator; Historical assessment data for target preschool children.
[0014] Secondly, this application also provides a system for generating observation records of children's behavior, including: The acquisition module is used to acquire real-time behavioral video stream data of a group of children, and after preprocessing the real-time behavioral video stream data, it uses a preset lightweight neural network to perform multimodal feature extraction to obtain the results of multimodal feature extraction. The results of multimodal feature extraction include: action feature vector, emotion feature vector, environment / object feature vector, and child's identity feature vector. The standard vector generation module is used to generate a set of standard indicator vectors in a high-dimensional feature space based on the content of the standard indicators for preschool children in the standard document "Guidelines for Learning and Development of Children Aged 3-6". The mapping module is used to calculate the semantic relevance score between the results of the multimodal feature extraction and each standard indicator vector in the standard indicator vector group using a mapping matching algorithm, and to form a semantic relevance score dataset. The evaluation metric selection module is used to determine the metrics to be evaluated based on the semantic relevance score dataset. The indicator template selection module is used to obtain the corresponding indicator template based on the indicator to be evaluated. The observation report generation module generates an observation and assessment text report for the target children based on the indicator template, the extraction results of the multimodal features, and historical assessment data.
[0015] Thirdly, this application also provides a computer electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method for generating observation records of children's behavior as described in any of the above.
[0016] Fourthly, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method for generating observation records of children's behavior as described above.
[0017] The method, system, device, and medium for generating observation records of children's behavior provided in this application have the following beneficial effects: 1. Enables seamless, continuous data acquisition and efficient edge processing, significantly reducing teachers' workload. This application relies on edge hardware to capture first-person perspective video streams of children's behavior. It only requires a simple trigger to complete the extraction and preprocessing of video frames within a specified time period, without requiring teachers to interrupt the teaching process for complex operations. Moreover, all feature extraction, computational reasoning, and text generation steps are completed on the handheld terminal without cloud intervention, avoiding the cumbersome process of data export and uploading, greatly improving the efficiency of observation record generation, freeing teachers from tedious manual recording work, and allowing them to focus more on children's teaching and care.
[0018] 2. Accurate multimodal feature extraction + anti-occlusion identity recognition ensures the objectivity and accuracy of behavioral data. This application utilizes a lightweight neural network to achieve parallel extraction of multimodal features including actions, emotions, environment / objects, and identity. This transforms unstructured behavioral videos of young children into structured, quantifiable feature data, eliminating the subjective assumptions inherent in traditional manual recording. Simultaneously, it employs a multi-granularity feature fusion face / body weight recognition algorithm to obtain the child's identity feature vector, effectively solving the recognition challenges in scenarios such as side profiles, back views, and occlusions. This achieves precise binding between the child's identity and behavioral features, providing objective and accurate raw data support for subsequent assessments.
[0019] 3. Achieve quantitative mapping of educational standards, ensuring that observation and assessment are strictly anchored to national professional standards, thereby enhancing the standardization and professionalism of the assessment. This application transforms all specific indicators from the five domains of the "Guidelines for Learning and Development of Children Aged 3-6" into computable high-dimensional feature vector sets. Through a weighted vector space mapping and matching algorithm, it achieves precise semantic association between multimodal features of children's behavior and educational professional indicators. Furthermore, through dual optimization of dynamic weight coefficients and environmental context bias functions, it makes the indicator matching more closely aligned with the assessment needs of different domains and scenarios in preschool education. At the same time, by sorting the indicators to be assessed through score ranking, it ensures that the assessment criteria are highly consistent with children's actual behavior, completely solving the problems of existing technology being disconnected from educational professional standards and traditional records lacking professional anchoring. This guarantees the professional consistency and standardization of assessment results from different teachers and different kindergartens.
[0020] 4. Multi-dimensional data fusion generates professional assessment reports that are both personalized and instructive, empowering precise teaching in preschool education. The method uses professional templates corresponding to the indicators to be evaluated as a framework, integrates multimodal feature extraction results with children's historical assessment data to generate observation and assessment text reports. It can not only provide an objective and detailed description of children's current behavior based on quantitative features, but also complete professional indicator correspondence analysis in conjunction with the "Guidelines" standards, and achieve longitudinal comparative evaluation of children's behavioral development. The generated reports also include targeted educational suggestions, providing a scientific basis for teachers to carry out personalized and precise preschool education and teaching activities, and truly realize a closed-loop empowerment from "behavioral data collection" to "education and teaching guidance".
[0021] 5. The lightweight algorithm design is adapted to edge computing power, combining practicality and scalability, making it easy to implement and promote. All algorithms in the method are designed specifically for edge computing, with lightweight neural networks and simplified quantization models adapted to the computing power and power consumption requirements of handheld terminals, eliminating the need for additional high-end computing equipment. Furthermore, the number of evaluation indicators, dynamic weight coefficients, and environmental bias rules can be flexibly adjusted according to the actual teaching needs of preschool education. The indicator template library can also be updated and expanded according to the implementation requirements of the "Guidelines," making it highly practical and scalable, and easy to implement and promote in various kindergartens and preschool education institutions. Attached Figure Description
[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0023] Figure 1 This is a flowchart illustrating a method for generating observation records of children's behavior in an embodiment of this application; Figure 2 This is a schematic diagram of the structure of a system for generating observation records of children's behavior according to an embodiment of this application; Figure 3 This is a schematic diagram of the structure of a computer electronic device according to an embodiment of this application. Detailed Implementation
[0024] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0025] Please see Figure 1 The present application provides a method for generating observation records of children's behavior, which includes at least the following steps: S10. Acquire real-time behavioral video stream data of the preschool group, and after preprocessing the real-time behavioral video stream data, use a preset lightweight neural network to perform multimodal feature extraction to obtain the results of multimodal feature extraction. The results of multimodal feature extraction include: action feature vector, emotion feature vector, environment / object feature vector, and child's identity feature vector.
[0026] Specifically, in this embodiment, wearable acquisition devices are used to capture real-time first-person perspective behavioral video stream data of preschool children in a preschool education setting. The real-time behavioral video stream data is subjected to edge-side lightweight preprocessing operations to remove invalid data and reduce computational costs while retaining core behavioral information, providing standardized image input for subsequent feature extraction. After preprocessing, a preset lightweight neural network model is called in the handheld computing terminal to perform parallel multimodal feature extraction on the processed video stream data, and finally outputs structured multimodal feature extraction results. The entire process is completed using edge computing power without cloud intervention.
[0027] It should be noted that the wearable acquisition device and the handheld computing terminal use a "low-power dual-link collaborative mechanism". In normal standby mode, BLE (Bluetooth Low Energy) is used to maintain the heartbeat. When an event is triggered (when transmitting video), the Wi-Fi Direct channel is instantly woken up to transmit video at high speed. After the transmission, it returns to sleep mode.
[0028] In one embodiment of this application, the preprocessing includes: video frame capture, keyframe extraction and filtering, and feature region cropping and positioning.
[0029] Specifically, in this embodiment, the preprocessing operations include at least video frame extraction, keyframe extraction and filtering, and feature region cropping and localization. All operations are adapted to the low-power and low-latency requirements of edge computing. Video frame capture: Extract a sequence of frames from a continuous real-time video stream within a specified time period after the wearable device is triggered, accurately capture observational segments of children's behavior, and remove invalid frames without behavioral characteristics; Keyframe extraction and filtering: Based on the frame difference method or optical flow method, the pixel change rate or pixel motion vector of adjacent frames is calculated, and the keyframes with the largest movement amplitude, the clearest facial expressions of children, and the most significant behavioral characteristics are automatically filtered out, converting the continuous video stream into single-frame / few-frame images, which greatly reduces the subsequent computing cost. Feature region cropping and localization: Through visual detection algorithms, core regions such as children's faces, bodies, and body interactions in video frames are automatically identified and cropped. At the same time, related regions such as objects and environmental markers in the scene are located, redundant background pixels are removed, and only effective regions of interest (ROI) are retained, which improves the accuracy and efficiency of subsequent feature extraction.
[0030] In one embodiment of this application, the feature extraction is performed using the following calculation formula: ; in, For real-time behavioral video stream data; For feature extraction functions; This is the action feature vector; This is a feature vector representing emotions. For environment / object feature vectors; This is a feature vector representing the identity of a young child.
[0031] Specifically, the result of multimodal feature extraction is a set of structured feature vectors, which is the core data foundation for subsequent semantic mapping, including: action feature vectors. (Identify and represent specific interactive actions of young children, such as building, delivering, reading, grabbing, sharing, etc.) Emotional feature vector (Based on facial key points of young children, this accurately represents their emotional valence (positive / negative) and arousal (focus, excitement, frustration, calmness, etc.), as well as environmental / object feature vectors.) (Teaching scenarios and objects that represent the occurrence of children's behaviors, such as construction areas, reading areas, building blocks, picture books, drawing tools, etc.), children's identity characteristic vectors (The unique identification information representing the child is extracted through a face / body weight recognition (Re-ID) algorithm.)
[0032] In one embodiment of this application, the child's identity feature vector The system prioritizes constructing the identity feature vector by recognizing the child's facial information. However, since a child's face may be obscured or incompletely recognized during activity, this embodiment addresses the issue of obscured facial features by using a different feature vector. It can also be obtained through a face / body weight recognition (Re-ID) algorithm using multi-granularity feature fusion, effectively solving the recognition failure problem caused by factors such as the child's back view, side profile, or occlusion. The specific calculation formula is as follows: ; In the formula, The current features of the child (including core features such as face and body outline) detected from video stream data. The identity feature template of child i in the child feature database stored locally on the handheld computing terminal; The weighting coefficient for clothing color features is used to balance the recognition weights of facial features and clothing features, adapting to the recognition needs of early childhood education scenarios. The clothing color feature matching function calculates the similarity between the color and texture features of the current child's clothing and the clothing features of child i in the feature library; , These are the modulus lengths of the current detected feature and the feature template of child i, respectively. The maximum similarity is calculated when... ( When the identity matching threshold is reached, the identity matching is considered successful, and the corresponding child's identity feature vector is extracted. .
[0033] S20. Based on the standard indicators for preschool children in the standard document "Guidelines for Learning and Development of Children Aged 3-6", generate a set of standard indicator vectors in a high-dimensional feature space.
[0034] Specifically, based on the national professional standard document for preschool education, "Guidelines for Learning and Development of Children Aged 3-6," the five major domains (health, language, social, science, and art), M sub-domains, and N specific development goals contained therein are structurally decomposed and professionally vectorized. The original textual preschool development standard indicators are transformed into a set of standard indicator vectors in a calculable and matchable high-dimensional feature space, realizing the quantitative expression of educational professional standards. This allows abstract educational assessment indicators to become quantitative data that can be accurately matched with children's behavioral characteristics, laying a core foundation for subsequent semantic mapping.
[0035] The standard index vector group is denoted as ,in Let gk be the standard indicator vector corresponding to the k-th specific development goal, and N be the total number of all specific standard indicators in the Guidelines. Each standard indicator vector gk contains three dimensions of standard feature values corresponding to the multimodal features: action, emotion, and environment / object, denoted as gk, ... This ensures a perfect dimensional match with the multimodal feature vectors of young children, guaranteeing the accuracy of subsequent semantic relevance calculations. This standard indicator vector set is pre-stored in the Guidelines' knowledge graph library on the handheld computing terminal and can be directly accessed for calculation.
[0036] S30. Using a mapping matching algorithm, calculate the semantic relevance score between the results of the multimodal feature extraction and each standard indicator vector in the standard indicator vector group, and form a semantic relevance score dataset.
[0037] Specifically, in handheld computing terminals, a mapping matching algorithm based on a weighted vector space model is adopted. Using the multimodal feature extraction results and the standard indicator vector set in the "Guidelines" as the computational basis, the multimodal feature sets are matched one by one. With standard indicator vector group Each standard indicator vector in Semantic relevance is calculated to obtain the semantic relevance score for each standard indicator. All scores are integrated to form a semantic relevance score dataset, enabling precise quantitative matching between the actual behavioral characteristics of young children and the professional standards of preschool education.
[0038] In one embodiment of this application, the semantic relevance score is calculated using the following formula: ; in, For the multimodal characteristics of young children and the first The semantic relevance score of each standard indicator vector indicates that the higher the score, the better the match between the child's current behavior and that indicator in the "Guidelines". For respectively the first The standard indicator has standard feature values in the dimensions of action, emotion, and environment / object. The dynamic weighting coefficients for the corresponding dimensions can be adaptively adjusted according to the assessment focus of different domains in the Guidelines (e.g., the health and social domains emphasize action interaction, and the weighting of the action dimension is adjusted accordingly). Higher; the language domain emphasizes emotional expression and communication, with the emotion dimension having a higher weight. Higher; in science and art, the focus is on scene and object manipulation, with a higher weight for the environment / object dimension. higher); As an environmental context bias function, it can adjust the basic scores of the standard indicators in the corresponding areas according to the actual teaching scenarios in which the children's behavior occurs (such as construction area, reading area, art area, outdoor play area). For example, it can increase the scores of science and health in the construction area and increase the scores of language in the reading area, thereby further improving the accuracy of matching and the adaptability of the scenario.
[0039] S40. Based on the semantic relevance score dataset, determine the metrics to be evaluated.
[0040] Specifically, in this step, based on the semantic relevance score dataset, the indicators with the highest matching degree with the current behavioral characteristics of children are selected from the standard indicator vector group of the "Guidelines" through the score sorting and filtering rules. These indicators are used as the evaluation indicators for this observation of children's behavior, ensuring that the evaluation indicators are highly consistent with the actual behavior of children.
[0041] In one embodiment of this application, step S40 includes: S401. Sort the score data in the semantic relevance score dataset according to the score from highest to lowest. S402. Based on the ranking results, select the indicators corresponding to the semantic relevance scores of the top N scores as the indicators to be evaluated.
[0042] Specifically, in this embodiment, all score data in the semantic relevance score dataset are first sorted in descending order from high to low to obtain an ordered score sequence; then, based on the actual needs of on-site assessment of preschool education, the standard indicators corresponding to the top N semantic relevance scores in the sorting results are selected and determined as the indicators to be evaluated (N is a positive integer and can be flexibly set, such as selecting the first 1 as the core evaluation indicator, or selecting the first 2-3 as multi-dimensional relevance evaluation indicators).
[0043] The selection logic can be implemented with the help of the optimal solution formula: This means selecting the indicator corresponding to the highest score as the core indicator to be evaluated, which meets the needs of rapid on-site evaluation.
[0044] S50. Obtain the corresponding indicator template based on the indicator to be evaluated.
[0045] Specifically, in this step, based on the indicators to be evaluated determined in step S40, indicator templates corresponding to each indicator to be evaluated are retrieved from the **Guideline Indicator Professional Template Library** pre-set locally on the handheld computing terminal. These indicator templates are standardized text frameworks pre-designed according to the educational connotations, evaluation requirements, and professional expression norms of each specific indicator in the Guidelines. Each template includes core modules such as objective behavioral descriptions, professional evaluation dimensions, analysis of the corresponding Guideline indicators, and educational suggestion directions, ensuring that the subsequently generated observation and evaluation texts align with the professional needs of preschool education and avoiding generalized descriptions without professional basis.
[0046] S60. Based on the indicator template, the extraction results of the multimodal features, and historical evaluation data, generate an observation and evaluation text report of the target child.
[0047] Specifically, using the retrieved evaluation indicator template as the core framework, and integrating the multimodal feature extraction results obtained in step S10 and the target child's historical evaluation data, the system uses a built-in small-parameter, large-language model on a handheld computing terminal to perform text generation. The final output is a standardized and personalized observation and evaluation text report for the target child. The entire generation process relies on edge computing power, eliminating the need for cloud uploads and ensuring data security. This report includes an objective quantitative description of the child's current behavior, a professional development assessment based on the guidelines, and a longitudinal comparison of the child's behavioral development, providing a complete and scientific basis for personalized teaching and periodic assessments in preschool education.
[0048] In one embodiment of this application, the process of generating the observation and evaluation text report is quantified using the following calculation formula:
[0049] In the formula, The final target preschool observation and assessment text report; It is a natural language generation function, implemented by a small-parameter, large-language model built into the handheld computing terminal. It is optimized for edge computing and balances the efficiency, professionalism, and fluency of text generation. For the professional templates corresponding to the indicators to be evaluated retrieved in step S50, define the professional framework and core dimensions of text generation; The multimodal feature extraction results provide objective feature data on children's behavior for text reports, making the descriptions more accurate; The target child's historical assessment data is stored locally on the handheld computing terminal, including the child's past behavioral observation records, development assessment results, changes in behavioral characteristics, etc., to achieve longitudinal comparison and stage evaluation of the child's behavioral development (such as "compared to last week, the child's attention span has significantly improved").
[0050] The method for generating observation records of children's behavior provided in this application has the following beneficial effects: 1. Enables seamless, continuous data acquisition and efficient edge processing, significantly reducing teachers' workload. This application relies on edge hardware to capture first-person perspective video streams of children's behavior. It only requires a simple trigger to complete the extraction and preprocessing of video frames within a specified time period, without requiring teachers to interrupt the teaching process for complex operations. Moreover, all feature extraction, computational reasoning, and text generation steps are completed on the handheld terminal without cloud intervention, avoiding the cumbersome process of data export and uploading, greatly improving the efficiency of observation record generation, freeing teachers from tedious manual recording work, and allowing them to focus more on children's teaching and care.
[0051] 2. Accurate multimodal feature extraction + anti-occlusion identity recognition ensures the objectivity and accuracy of behavioral data. This application utilizes a lightweight neural network to achieve parallel extraction of multimodal features including actions, emotions, environment / objects, and identity. This transforms unstructured behavioral videos of young children into structured, quantifiable feature data, eliminating the subjective assumptions inherent in traditional manual recording. Simultaneously, it employs a multi-granularity feature fusion face / body weight recognition algorithm to obtain the child's identity feature vector, effectively solving the recognition challenges in scenarios such as side profiles, back views, and occlusions. This achieves precise binding between the child's identity and behavioral features, providing objective and accurate raw data support for subsequent assessments.
[0052] 3. Achieve quantitative mapping of educational standards, ensuring that observation and assessment are strictly anchored to national professional standards, thereby enhancing the standardization and professionalism of the assessment. This application transforms all specific indicators from the five domains of the "Guidelines for Learning and Development of Children Aged 3-6" into computable high-dimensional feature vector sets. Through a weighted vector space mapping and matching algorithm, it achieves precise semantic association between multimodal features of children's behavior and educational professional indicators. Furthermore, through dual optimization of dynamic weight coefficients and environmental context bias functions, it makes the indicator matching more closely aligned with the assessment needs of different domains and scenarios in preschool education. At the same time, by sorting the indicators to be assessed through score ranking, it ensures that the assessment criteria are highly consistent with children's actual behavior, completely solving the problems of existing technology being disconnected from educational professional standards and traditional records lacking professional anchoring. This guarantees the professional consistency and standardization of assessment results from different teachers and different kindergartens.
[0053] 4. Multi-dimensional data fusion generates professional assessment reports that are both personalized and instructive, empowering precise teaching in preschool education. The method uses professional templates corresponding to the indicators to be evaluated as a framework, integrates multimodal feature extraction results with children's historical assessment data to generate observation and assessment text reports. It can not only provide an objective and detailed description of children's current behavior based on quantitative features, but also complete professional indicator correspondence analysis in conjunction with the "Guidelines" standards, and achieve longitudinal comparative evaluation of children's behavioral development. The generated reports also include targeted educational suggestions, providing a scientific basis for teachers to carry out personalized and precise preschool education and teaching activities, and truly realize a closed-loop empowerment from "behavioral data collection" to "education and teaching guidance".
[0054] 5. The lightweight algorithm design is adapted to edge computing power, combining practicality and scalability, making it easy to implement and promote. All algorithms in the method are designed specifically for edge computing, with lightweight neural networks and simplified quantization models adapted to the computing power and power consumption requirements of handheld terminals, eliminating the need for additional high-end computing equipment. Furthermore, the number of evaluation indicators, dynamic weight coefficients, and environmental bias rules can be flexibly adjusted according to the actual teaching needs of preschool education. The indicator template library can also be updated and expanded according to the implementation requirements of the "Guidelines," making it highly practical and scalable, and easy to implement and promote in various kindergartens and preschool education institutions.
[0055] Please see Figure 2 This application also provides a system 200 for generating observation records of children's behavior, including: The acquisition module 201 is used to acquire real-time behavioral video stream data of the preschool group, and after preprocessing the real-time behavioral video stream data, it uses a preset lightweight neural network to perform multimodal feature extraction to obtain the results of multimodal feature extraction. The results of multimodal feature extraction include: action feature vector, emotion feature vector, environment / object feature vector, and child's identity feature vector. The standard vector generation module 202 is used to generate a set of standard indicator vectors in a high-dimensional feature space based on the content of the standard indicators for preschool children in the standard document "Guidelines for Learning and Development of Children Aged 3-6". The mapping module 203 is used to calculate the semantic relevance score between the result of the multimodal feature extraction and each standard indicator vector in the standard indicator vector group using a mapping matching algorithm, and to form a semantic relevance score dataset. The evaluation index selection module 204 is used to determine the index to be evaluated based on the semantic relevance score dataset. The indicator template selection module 205 is used to obtain the corresponding indicator template based on the indicator to be evaluated. The observation report generation module 206 generates an observation and assessment text report for the target child based on the indicator template, the extraction results of the multimodal features, and historical assessment data.
[0056] Please see Figure 3 This application also provides a computer electronic device 300, including a memory 503 and a processor 302. The memory 303 stores a computer program, and the processor 302 executes the computer program to implement the steps of the above-described method for generating observation records of children's behavior.
[0057] Specifically, the computer electronic device 300 includes: a transceiver 301, a bus interface, and a processor 302. The processor 302 is used to acquire real-time behavioral video stream data of a group of preschool children, and after preprocessing the real-time behavioral video stream data, to perform multimodal feature extraction using a preset lightweight neural network to obtain the results of multimodal feature extraction. The results of multimodal feature extraction include: action feature vectors, emotion feature vectors, environment / object feature vectors, and child identity feature vectors. Based on the standard indicators for preschool children in the standard document "Guidelines for Learning and Development of Children Aged 3-6", a set of standard indicator vectors in a high-dimensional feature space is generated. A mapping matching algorithm is used to calculate the semantic relevance score between the results of multimodal feature extraction and each standard indicator vector in the set of standard indicator vectors, and a semantic relevance score dataset is formed. Based on the semantic relevance score dataset, the indicators to be evaluated are determined. Based on the indicators to be evaluated, the corresponding indicator templates are obtained. Based on the indicator templates, the extraction results of the multimodal features, and historical evaluation data, an observation and evaluation text report of the target preschool children is generated.
[0058] In this embodiment of the application, the computer electronic device 300 further includes a memory 303. Figure 3 In this context, the bus architecture can include any number of interconnected buses and bridges, specifically linking various circuits of one or more processors 302 (represented by processor 302) and various circuits of memory 303 (represented by memory 303). The bus architecture can also link various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. The bus interface provides an interface. The transceiver 301 can be multiple elements, including transmitters and receivers, providing a unit for communicating with various other devices over a transmission medium. The processor 302 is responsible for managing the bus architecture and general processing, and the memory 303 can store data used by the processor 302 during operation.
[0059] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method for generating observation records of children's behavior as described above.
[0060] In this embodiment, the computer-readable storage medium can be a non-volatile storage medium or a volatile storage medium. For example, the computer storage medium may include, but is not limited to, various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.
[0061] In all examples shown and described herein, any specific values should be interpreted as merely exemplary and not as limitations; therefore, other examples of exemplary embodiments may have different values.
[0062] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.
[0063] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that, in alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.
[0064] In addition, the functional modules or units in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.
[0065] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a terminal device (which may be a smartphone, personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.
[0066] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.
Claims
1. A method for generating observation records of children's behavior, characterized in that, include: Real-time behavioral video stream data of a group of young children is acquired, and after preprocessing the real-time behavioral video stream data, a preset lightweight neural network is used to extract multimodal features to obtain the results of multimodal feature extraction. The results of multimodal feature extraction include: action feature vector, emotion feature vector, environment / object feature vector, and child's identity feature vector. Based on the standard indicators for preschool children in the standard document "Guidelines for Learning and Development of Children Aged 3-6", a set of standard indicator vectors in a high-dimensional feature space is generated. A mapping matching algorithm is used to calculate the semantic relevance score between the results of the multimodal feature extraction and each standard indicator vector in the standard indicator vector group, and to form a semantic relevance score dataset. Based on the semantic relevance score dataset, determine the metrics to be evaluated; Based on the indicators to be evaluated, obtain the corresponding indicator template; Based on the indicator template, the extraction results of the multimodal features, and historical assessment data, an observation and assessment text report of the target child is generated.
2. The observation record generation method according to claim 1, characterized in that, The preprocessing includes: video frame extraction, key frame extraction and filtering, and feature region cropping and positioning.
3. The observation record generation method according to claim 1, characterized in that, The step of determining the metrics to be evaluated based on the semantic relevance score dataset includes: The scores in the semantic relevance score dataset are sorted from highest to lowest. Based on the ranking results, the indicators corresponding to the semantic relevance scores of the top N scores are selected as the indicators to be evaluated.
4. The observation record generation method according to claim 1, characterized in that, The feature extraction uses the following calculation formula: ; in, For real-time behavioral video stream data; For feature extraction functions; This is the action feature vector; This is a feature vector representing emotions. For environment / object feature vectors; This is a feature vector representing the identity of a young child.
5. The observation record generation method according to claim 4, characterized in that, The child's identity feature vector is constructed using the child's facial information. If the child's facial information cannot be obtained, the following calculation formula is used for construction: ; in, Features detected in video stream data; For the characteristic template of child i; Weighting coefficients for clothing color features; This is the color feature matching function for clothing.
6. The observation record generation method according to claim 1, characterized in that, The mapping matching algorithm is used to calculate the semantic relevance score between the multimodal feature extraction results and the standard indicator vector group of children, using the following calculation formula: ; in, The semantic correlation score between the multimodal features of young children and the k-th standard indicator vector is calculated. Let be the standard feature values of the k-th standard indicator in the dimensions of action, emotion, and environment / object, respectively. The weight coefficients for the dimensions. This is the environment context bias function.
7. The observation record generation method according to claim 1, characterized in that, The observation and assessment text report for the target child is generated based on the indicator template, the extraction results of the multimodal features, and historical assessment data, using the following calculation formula: ; in, For observation and evaluation of text reports; For generating functions; This is the template for the Kth indicator; Historical assessment data for target preschool children.
8. A system for generating observation records of children's behavior, characterized in that, include: The acquisition module is used to acquire real-time behavioral video stream data of a group of children, and after preprocessing the real-time behavioral video stream data, it uses a preset lightweight neural network to perform multimodal feature extraction to obtain the results of multimodal feature extraction. The results of multimodal feature extraction include: action feature vector, emotion feature vector, environment / object feature vector, and child's identity feature vector. The standard vector generation module is used to generate a set of standard indicator vectors in a high-dimensional feature space based on the content of the standard indicators for preschool children in the standard document "Guidelines for Learning and Development of Children Aged 3-6". The mapping module is used to calculate the semantic relevance score between the results of the multimodal feature extraction and each standard indicator vector in the standard indicator vector group using a mapping matching algorithm, and to form a semantic relevance score dataset. The evaluation metric selection module is used to determine the metrics to be evaluated based on the semantic relevance score dataset. The indicator template selection module is used to obtain the corresponding indicator template based on the indicator to be evaluated. The observation report generation module generates an observation and assessment text report for the target children based on the indicator template, the extraction results of the multimodal features, and historical assessment data.
9. A computer electronic device, characterized in that, It includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the method for generating observation records of children's behavior as described in any one of claims 1-7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method for generating observation records of children's behavior according to any one of claims 1-7.
