A system and method for assessing social-emotional competence based on sequences of children's behavior

Abstract

The application discloses a kind of social emotional ability evaluation system and method based on child behavior sequence, through the synchronous acquisition and anti-interference packaging of multimodal original signal, the structural segmentation of behavior sequence is carried out to original behavior event stream data packet again Feature quantization, finally multidimensional feature vector is input into trained social emotion learning calculation model, and the evaluation score of each ability dimension is calculated to obtain the evaluation result, realize the continuous operation of child unstructured, into structured behavior sequence data, and then extract quantifiable behavior characteristics, finally mapped to specific social emotional ability evaluation result by calculation model, solve the problem that existing hardware system can only output simple on-off signal, cannot handle high-frequency jitter signal generated in the process of child operation, long-tail distribution pause data and time asynchronous problem of multimodal signal.

Description

Technical Field

[0001] This invention relates to the field of educational big data processing and intelligent assessment technology, and in particular to a social and emotional ability assessment system and method based on children's behavioral sequences. Background Technology

[0002] Scientific assessment of children's social-emotional learning (SEL) is a prerequisite for personalized educational intervention. Currently, assessment methods in this field mainly rely on adult observation scales and affective computing technologies based on multimedia signals, but both of these methods have significant limitations in their applicability to natural play scenarios.

[0003] The assessment process, based on behavioral scales filled out by teachers or parents, relies on manual observation and subjective scoring. This method is not only time-consuming and labor-intensive, but also highly susceptible to the influence of assessor bias and memory lapses. More importantly, manual observation makes it difficult to collect high-frequency, continuous data during children's immersive play, resulting in the loss of a large amount of process data reflecting the dynamic evolution of behavior, thus failing to meet the technical requirements for objective and data-driven assessment.

[0004] Automated assessment technologies based on computer vision or speech recognition attempt to infer emotional states by analyzing facial expressions or tone of voice. However, in typical focused scenarios where children manipulate physical educational toys, their facial expressions tend to be static, and voice interaction is rare. This sparsity of perceptual signals directly leads to feature extraction failure, causing a sharp drop in the recognition accuracy of existing emotion computing models, making it difficult to stably implement technical solutions.

[0005] Meanwhile, most existing intelligent educational toy systems only integrate basic state detection sensors, such as contact switches or RFID tags for judging the assembly result. These systems can only acquire binary information about the operation result ("right / wrong") and cannot perceive the fine-grained behavioral sequence during the operation process, such as the frequency of attempts, operation duration, path correction, and pauses and hesitations. Due to the lack of ability to collect process data, existing technologies cannot transform unstructured physical operation behaviors into structured computer-readable data, thus resulting in a lack of necessary data foundation for the quantitative modeling of socio-emotional abilities.

[0006] Furthermore, existing solutions typically upload raw data to the cloud for centralized processing to complete complex model inference. This architecture, which relies on remote servers, results in high network latency and poses security risks due to the remote transmission of children's private data. Given the increasingly stringent requirements for low latency, high security, and offline availability in educational settings, existing technologies are costly to deploy and lack versatility.

[0007] In summary, existing technologies face insurmountable technical obstacles in terms of data acquisition dimensions, signal perception stability, and computing architecture. There is an urgent need for a technical solution that can deeply integrate multimodal behavior perception with localized lightweight computing to address the core pain points of missing process data and poor real-time evaluation. Summary of the Invention

[0008] The purpose of this invention is to address the problem that existing hardware systems can only output simple switching signals and cannot handle the high-frequency jitter signals, long-tailed pause data, and time asynchrony of multimodal signals generated during children's operation. Therefore, this invention proposes a social-emotional ability assessment system and method based on children's behavioral sequences.

[0009] To achieve the above objectives, the present invention adopts the following technical solution: A method for assessing children's socio-emotional abilities based on behavioral sequences includes: Step 1: Obtain the original behavioral event stream data packet through synchronous acquisition and anti-interference encapsulation of multimodal raw signals; Step 2: Perform structured segmentation and feature quantization on the original behavioral event stream data packets to obtain multidimensional feature vectors; Step 3: Input the multidimensional feature vector into the trained social emotion learning computation model, calculate the evaluation score of each ability dimension, and obtain the evaluation result.

[0010] Furthermore, the specific steps of step 1 include: Step 11: Capture physical interaction signals during the child's operation in real time using a heterogeneous sensor array deployed within the smart base; Step 12: Perform anti-jitter processing on the RFID signal in the physical interaction signal, identify and filter millisecond-level signal fluctuations caused by children's hand tremors or poor contact, and generate a stable object identification stream; Step 13: Perform moving target detection on the visual signals in the interaction signals and extract the time boundary information of hand intervention and withdrawal; Step 14: Clean the stable object identification stream and the time boundary information of hand intervention and withdrawal, and then fuse and encapsulate them into a raw behavior event stream data packet, wherein the raw behavior event stream data packet has a millisecond or microsecond-level timestamp.

[0011] Furthermore, the specific steps of step 2 include: Step 21: Based on the signal timing analysis algorithm, perform sliding window analysis on the original behavioral event stream, and identify boundary points with significant pause characteristics by calculating the variance of the time interval between adjacent events, thereby dividing the continuous stream into independent behavioral sessions; Step 22: Extract the temporal and frequency domain features from the behavioral session, including the duration of the operation, the frequency of the event, and the amplitude of the corrective behavior, and quantize them to generate a multi-dimensional feature vector.

[0012] Furthermore, the behavioral session is a behavioral unit with independent analytical significance, which is divided from the continuous operation event stream based on the completion of a preset task, game reset, or pause time exceeding a threshold.

[0013] Furthermore, the multidimensional feature vector is an N-dimensional numerical array used to characterize the statistical properties of the behavioral sequence in the time and spatial domains, serving as the input parameters for the trained social emotion learning computational model.

[0014] Furthermore, the trained social emotion learning computational model is obtained by a general machine learning model through learning the mapping relationship between multi-dimensional feature vectors and ability levels using historical data.

[0015] Furthermore, the heterogeneous sensor array within the smart base includes sensing slots and cameras arranged in rows or columns on the base surface. An RFID reader is installed at the bottom of the sensing slot to sense the placement, removal, and location information of the building blocks in real time. A pressure sensor is installed on the inner wall of the sensing slot to detect whether the building blocks are pressed and locked.

[0016] Furthermore, a unique RFID tag is embedded in the bottom of the detection block to identify the block's identity.

[0017] The present invention also provides a system for performing the socio-emotional competence assessment method based on children's behavioral sequences as described in any one of claims 1-8, comprising: The data acquisition module is used to execute step 1 and output the raw behavioral event stream; The feature processing module is used to execute step 2, and is communicatively connected to the data acquisition module. It is used to receive the original behavior event stream data packet, perform behavior sequence structuring and feature extraction, and output a multi-dimensional feature vector. The capability calculation module is used to execute step 3, and is communicatively connected to the feature processing module. It is used to receive the multi-dimensional feature vector, input the multi-dimensional feature vector into the trained social emotion learning calculation model to perform social emotion capability calculation mapping, and output the evaluation result. The output module is communicatively connected to the capability calculation module and is used to receive the evaluation results and execute the evaluation result output step. The system is deployed in the main control chip of the smart base or in a cloud server.

[0018] Compared with existing technologies, the advantages of this invention are: This invention achieves this by synchronously acquiring and anti-interference encapsulating multimodal raw signals, then performing structured segmentation and feature quantization of the raw behavioral event stream data packets. The continuous raw event stream is divided into meaningful behavioral sessions based on task logic, and pre-defined quantitative features with developmental psychological interpretability are extracted from these sessions. This transforms children's unstructured continuous operations into structured behavioral sequence data, extracts quantifiable behavioral features, and finally maps these features to specific socio-emotional ability assessment results through a computational model. This invention addresses the problem that existing hardware systems can only output simple switching signals and cannot handle high-frequency jitter signals, long-tailed pause data, and the time asynchrony of multimodal signals generated during children's operations. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the process for a social-emotional competence assessment method based on children's behavioral sequences proposed in this invention.

[0020] Figure 2 This is a system architecture diagram of a social and emotional ability assessment system based on children's behavioral sequences proposed in this invention. Detailed Implementation

[0021] The invention will now be further explained with reference to the accompanying drawings.

[0022] like Figure 1 As shown, this invention provides a method for assessing children's socio-emotional abilities based on behavioral sequences, including: Step 1: Obtain the original behavioral event stream data packet by synchronously acquiring and anti-interference encapsulating the multimodal raw signals.

[0023] Step 2: Perform structured segmentation and feature quantization of the original behavior event stream data packets to obtain multidimensional feature vectors.

[0024] Step 3: Input the multidimensional feature vector into the trained social emotion learning computation model, calculate the evaluation score of each ability dimension, and obtain the evaluation result.

[0025] In step 1, as the child begins to interact, physical events are continuously captured by integrated sensing units. Each event, such as the connection, disconnection, or movement of building blocks, is encapsulated in real time into a standardized data packet. This data packet contains at least the event type, object identifier, and a timestamp with microsecond-level precision, forming a strictly chronological stream of raw events. This step ensures the objectivity and traceability of the assessment basis.

[0026] In step 2, the original behavioral event stream data packets undergo high-level processing. Based on task logic (such as goal achievement) or timing rules (such as long pauses), the continuous stream is segmented into independent behavioral sessions. Each behavioral session is then subjected to in-depth analysis to identify goal-oriented subsequences, count the alternation frequency of failed and successful attempts, calculate the hesitation time before key decision points, and finally calculate the feature vector corresponding to the session based on a pre-set feature definition library. For example, by counting the number of consecutive failures before final success but still continuing to try, feature values ​​for the persistence dimension are generated, and by analyzing the disorder of the operation sequence, feature values ​​related to planning are generated.

[0027] In step 3, the multidimensional feature vectors are input into the trained socio-emotional learning computational model for comprehensive analysis and judgment. This considers both individual feature values ​​and the combinations and interaction patterns between features. The trained socio-emotional learning computational model is calibrated with reference to normative data or theoretical thresholds, mapping behavioral features to socio-emotional competence dimensions such as perseverance, frustration coping, and cooperative tendencies. This results in a structured assessment summary, including quantitative scores, levels, or probability distributions for each dimension, and may include key behavioral evidence supporting the derivation. The trained socio-emotional learning computational model then converts the structured summary into a format that users can directly understand. Output formats include, but are not limited to, visual dashboards (displaying competence profiles with radar charts or bar charts), text analysis reports (converting data conclusions into natural language descriptions), or structured data interfaces for use by third-party educational software. This allows the assessment results to not only provide conclusions but also offer an intuitive and actionable development profile.

[0028] The above steps enable the automated and objective extraction and interpretation of socio-emotional competence signals contained in children's free play behavior, firmly linking abstract competence development with specific and observable behavioral data.

[0029] like Figure 2 As shown, the present invention also provides a system for performing the socio-emotional competence assessment method based on children's behavioral sequences as described in any one of claims 1-8, comprising: The data acquisition module is used to execute step 1 and output the raw behavior event stream.

[0030] The feature processing module is used to execute step 2, and is communicatively connected to the data acquisition module. It is used to receive the original behavior event stream data packet, perform behavior sequence structuring and feature extraction, and output a multi-dimensional feature vector.

[0031] The capability calculation module is used to execute step 3. It is communicatively connected to the feature processing module, receives the multidimensional feature vector, inputs the multidimensional feature vector into the trained social emotion learning calculation model to perform social emotion capability calculation mapping, and outputs the evaluation result.

[0032] The output module is communicatively connected to the capability calculation module and is used to receive the evaluation results and execute the evaluation result output step.

[0033] The system is deployed in the main control chip of the smart base or in a cloud server.

[0034] The system hardware consists of physical interlocking building blocks, a smart base, and an edge computing terminal. Physical educational toy layer: Several interlocking building blocks, each with a unique RFID tag embedded in its bottom to identify the block's identity (ID).

[0035] Intelligent base layer: The heterogeneous sensor array inside the intelligent base includes sensing slots and cameras distributed in rows or columns on the base surface. An RFID reader is set at the bottom of the sensing slot. The RFID reader is used to sense the placement, removal and position information of the building blocks in real time. A pressure sensor is set on the inner wall of the sensing slot. The pressure sensor is used to detect whether the building blocks are pressed and locked.

[0036] Edge computing terminal: An embedded processor deployed inside the base, which runs evaluation algorithms and is responsible for receiving raw data and performing subsequent processing.

[0037] After the system starts up, it enters real-time monitoring mode: First, the system captures the signal. When a child picks up a block, the RFID reader signal disappears, and the system captures the removal signal. When the child places the block on the base, the RFID signal reappears and the pressure sensor value changes, and the system captures the placement signal.

[0038] Further event encapsulation involves the processor encapsulating the aforementioned physical signals into a structured stream of raw behavioral events, where each event unit contains the following fields: Timestamp: The time of an operation, accurate to the millisecond.

[0039] Object ID: The unique RFID number of the currently operated block.

[0040] Event Types: These include Pickup, Place, Connect, and Disconnect.

[0041] Spatial coordinates: The X / Y coordinates of the block on the base.

[0042] The feature processing module receives the raw event stream and executes the following logic: First, the system segments the behavior into sessions. A silence threshold of 5 seconds is set. If the time interval between two consecutive operation events exceeds 5 seconds, the preceding event sequence is divided into an independent behavior session. For example, if child A operates continuously from 0 to 120 seconds and stops operating at 125 seconds, the system segments the data from 0 to 120 seconds into a complete behavior session P1.

[0043] Re-quantization calculation is performed for the behavior session, and the following logic is executed: The system calls the feature definition library to calculate the persistence feature. This process first counts the total number of connection failure events Nfail in the session. If Nfail > 3 times and the final state is a successful connection, the session is determined to have a high persistence feature. In this embodiment, this feature can correspond to the persistence ability in SEL theory, but in the technical solution, it is only treated as a numerical feature. The specific psychological meaning is defined by the user.

[0044] Next, the impulsivity characteristic is calculated. This process first calculates the time difference Δt between the initial pickup and the initial placement. If Δt < 1 second, it is determined to be an impulsive operation characteristic. In this embodiment, this characteristic can correspond to the impulse control ability in SEL theory, but in the technical solution, it is only treated as a numerical characteristic, and the specific psychological meaning is defined by the user.

[0045] Then, the cooperative feature (Planning) is calculated. In this process, the alternating appearance of two educational toy components with different IDs is detected (simulating multi-person collaboration). If the actual number of times the components are placed alternately exceeds a preset threshold (e.g., 20 times), it is determined to be a cooperative operation feature. In this embodiment, this feature can correspond to the cooperative ability in SEL theory, but in the technical solution, it is only treated as a numerical feature. The specific psychological meaning is defined by the user.

[0046] Next, feature vector generation is performed. During this process, the system normalizes the above calculation results to generate multi-dimensional feature vectors. For example, the first three bits of the vector may be represented as [0.85, 0.12, 0.45], which correspond to the quantitative values ​​of persistence, impulsivity, and cooperation, respectively. In particular, in response to the unstable operation caused by the incomplete development of children's hand muscles, this embodiment has specially designed a signal jitter filtering window during feature extraction. For example, if the system detects that the RFID signal has more than 5 on-off alternations within 1 second (which is physically illogical and belongs to noise), the system automatically judges it as contact jitter noise and filters it out, instead of recording it as 5 attempts. This technical means effectively solves the problem of data distortion caused by hardware sensor misjudgment and ensures the accuracy of subsequent evaluation.

[0047] Next, the capability calculation module loads a pre-trained lightweight decision tree model, i.e., a trained social emotion learning calculation model, and performs inference on the feature vectors: Its matching rule is: Rule set R1 (frustration coping ability): IF persistence eigenvalue > 0.8 AND number of failed attempts > 3 THEN output: Task persistence: Excellent.

[0048] Rule set R2 (impulse control ability): IF impulsivity eigenvalue 2s THEN outputs impulse control ability: moderate.

[0049] Rule set R3 (Collaborative Capability): IF collaborative eigenvalue > 20 times THEN output indicates a high level of cooperation tendency.

[0050] Output: The system converts the inference results into a JSON format data package, which includes capability dimension labels and quantified scores. The specific code is as follows: { "Session_ID":"S1_20260120", "SEL_Dimensions":{ "Perseverance":"High", "Impulse_Control":"Medium", "Planning_Skill":"Low" }, "Key_Evidence":["3failedattemptsbeforesuccess","Rapidfirstmove(<1s)"] } Finally, the output module receives JSON data and generates a visual dashboard on the connected tablet. The teacher's interface displays a triangular radar chart, with the vertices representing persistence, impulse control, and cooperation, respectively, intuitively showing the behavioral profile of the puzzle activity.

[0051] The technical solution of this invention realizes the transformation of children's physical actions of manipulating building blocks into a computer-understandable data stream without relying on subjective human intervention, and finally outputs an objective social and emotional ability assessment result. This process is entirely based on local hardware acquisition and calculation, which verifies the feasibility and practicality of the solution.

[0052] As is known from common technical knowledge, this invention can be implemented through other embodiments that do not depart from its spirit or essential characteristics. Therefore, the disclosed embodiments described above are merely illustrative and not exhaustive. All modifications within the scope of this invention or its equivalents are included in this invention.

Claims

1. A method for assessing children's socio-emotional abilities based on behavioral sequences, characterized in that, include: Step 1: Obtain the original behavioral event stream data packet through synchronous acquisition and anti-interference encapsulation of multimodal raw signals; Step 2: Perform structured segmentation and feature quantization on the original behavioral event stream data packets to obtain multidimensional feature vectors; Step 3: Input the multidimensional feature vector into the trained social emotion learning computation model, calculate the evaluation score of each ability dimension, and obtain the evaluation result.

2. The method for assessing children's social-emotional abilities based on behavioral sequences according to claim 1, characterized in that, The specific steps of step 1 include: Step 11: Capture physical interaction signals during the child's operation in real time using a heterogeneous sensor array deployed within the smart base; Step 12: Perform anti-jitter processing on the RFID signal in the physical interaction signal, identify and filter millisecond-level signal fluctuations caused by children's hand tremors or poor contact, and generate a stable object identification stream; Step 13: Perform moving target detection on the visual signals in the interaction signals and extract the time boundary information of hand intervention and withdrawal; Step 14: Clean the stable object identification stream and the time boundary information of hand intervention and withdrawal, and then fuse and encapsulate them into a raw behavior event stream data packet, wherein the raw behavior event stream data packet has a millisecond or microsecond-level timestamp.

3. The method for assessing children's social-emotional abilities based on behavioral sequences according to claim 1, characterized in that, The specific steps of step 2 include: Step 21: Based on the signal timing analysis algorithm, perform sliding window analysis on the original behavioral event stream, and identify boundary points with significant pause characteristics by calculating the variance of the time interval between adjacent events, thereby dividing the continuous stream into independent behavioral sessions; Step 22: Extract the temporal and frequency domain features from the behavioral session, including the duration of the operation, the frequency of the event, and the amplitude of the corrective behavior, and quantize them to generate a multi-dimensional feature vector.

4. The method for assessing children's social-emotional abilities based on behavioral sequences according to claim 3, characterized in that, The behavioral session is a behavioral unit with independent analytical significance, which is divided from the continuous operation event stream based on the completion of a preset task, game reset, or pause time exceeding a threshold.

5. The method for assessing children's social-emotional abilities based on behavioral sequences according to claim 3, characterized in that, The multidimensional feature vector is an N-dimensional numerical array used to characterize the statistical properties of the behavioral sequence in the time and spatial domains, and serves as the input parameter for the trained social emotion learning computational model.

6. The method for assessing children's social-emotional abilities based on behavioral sequences according to claim 1, characterized in that, The trained social emotion learning computational model is obtained by a general machine learning model through learning the mapping relationship between multi-dimensional feature vectors and ability levels using historical data.

7. The method for assessing children's social-emotional abilities based on behavioral sequences according to claim 2, characterized in that, The heterogeneous sensor array inside the smart base includes sensing slots and cameras arranged in rows or columns on the base surface. An RFID reader is installed at the bottom of the sensing slot to sense the placement, removal and position information of the building blocks in real time. A pressure sensor is installed on the inner wall of the sensing slot to detect whether the building blocks are pressed and locked.

8. The method for assessing children's social-emotional abilities based on behavioral sequences according to claim 1, characterized in that, The bottom of each detection block is embedded with a unique RFID tag to identify its identity.

9. A system for performing the socio-emotional competence assessment method based on children's behavioral sequences as described in any one of claims 1-8, characterized in that, include: The data acquisition module is used to execute step 1 and output the raw behavioral event stream; The feature processing module is used to execute step 2, and is communicatively connected to the data acquisition module. It is used to receive the original behavior event stream data packet, perform behavior sequence structuring and feature extraction, and output a multi-dimensional feature vector. The capability calculation module is used to execute step 3, and is communicatively connected to the feature processing module. It is used to receive the multi-dimensional feature vector, input the multi-dimensional feature vector into the trained social emotion learning calculation model to perform social emotion capability calculation mapping, and output the evaluation result. The output module is communicatively connected to the capability calculation module and is used to receive the evaluation results and execute the evaluation result output step. The system is deployed in the main control chip of the smart base or in a cloud server.

Images