Game interaction method oriented to children social behavior guidance

Abstract

The invention relates to the technical field of child social ability cultivation and man-machine interaction, and particularly discloses a game interaction method oriented to child social behavior guidance, which comprises the following steps: collecting child multi-modal behavior data, generating a social behavior event stream through structured processing, and matching with a social behavior atom library to obtain a behavior instance sequence with a label; calculating social activity, reciprocity response rate and emotion synchronization index based on the behavior instance sequence, dynamically judging the ability level of the child by adopting a clustering algorithm, and matching an adaptive task in combination with a social task knowledge graph; interactive guidance is realized through forward feedback and progressive prompt, the long-term ability evolution trend of children is predicted by using a three-layer stacked LSTM model, and initialization parameters are provided for the next round of interaction. According to the method, accurate perception, dynamic adaptation and sustainable development guidance of the social behaviors of the children are realized, and the scientificity and effectiveness of social training are effectively improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of children's social skills training and human-computer interaction, in particular to a game interaction method for guiding children's social behavior. BACKGROUND

[0002] The cultivation of social skills is an important part of early childhood development, especially in interpersonal interaction, emotional understanding and cooperative communication. Traditional social skills training relies on manual intervention, such as therapist-guided role-playing or group activities. This approach has limitations such as high labor costs, low standardization, and difficulty in large-scale implementation. In recent years, with the development of information technology, some social training tools and games based on computers or mobile terminals have emerged, aiming to provide repeatable and controllable interactive experiences through virtual scenarios.

[0003] Existing technologies usually use pre-set scripts and fixed processes to simulate social situations. For example, cartoon characters demonstrate specific social behaviors such as greetings and sharing, and children are asked to observe or make simple choices. However, this approach has significant shortcomings: first, the interaction mode is often single and stereotyped, mainly relying on touch operations such as clicking and dragging, lacking the ability to synchronize and analyze multi-dimensional social behaviors such as eye contact, facial expressions, voice tone, and body posture in a natural state. The system cannot accurately determine whether the child truly understands the social cues and responds appropriately, resulting in superficial training effects.

[0004] Secondly, existing systems generally lack adaptive adjustment mechanisms. Task difficulty and content are usually statically pre-set and cannot be dynamically adjusted based on children's real-time ability performance and emotional state. Children with strong abilities may feel insufficiently challenged, while children with weak abilities may lose interest due to repeated setbacks, severely affecting the personalization and effectiveness of training.

[0005] Furthermore, most systems focus on single or short-term interaction feedback, lacking tracking and modeling of children's long-term social skill development trajectory. The system cannot predict the growth trend of children based on historical data, so it cannot plan a gradual and personalized training path for them, making it difficult to achieve continuous and developmental guidance. SUMMARY

[0006] The purpose of the present application is to provide a game interaction method for guiding children's social behavior to solve the problems raised in the background.

[0007] The purpose of the present application can be achieved by the following technical solution: a game interaction method for guiding children's social behavior, comprising the following steps: S1. Multimodal Behavior Acquisition: Through an interactive terminal device equipped with a front-facing camera, microphone, touch screen and depth sensor, multimodal raw behavior data of the target child in the game environment is collected simultaneously. The multimodal raw behavior data includes: facial expression video stream, voice dialogue audio stream, body posture image sequence and touch operation log. S2. Structured Social Behavior Event Stream Generation: The multimodal raw behavior data is processed in a structured manner to generate a structured social behavior event stream; S3, Social Behavior Atomic Library Matching and Labeling: Matching and labeling the structured social behavior event stream with a predefined social behavior atomic library to obtain a sequence of labeled social behavior instances; S4. Dynamic Difficulty Adjustment: Based on the sequence of social behavior instances, calculate the social ability assessment index of the target child, determine the current ability level of the target child and match social tasks of corresponding difficulty, so as to achieve dynamic adaptation between game difficulty and children's social behavior. S5. Game Scene Rendering and Interactive Guidance: Based on the current session's target task, load task attribute data into the game scene for rendering and interactive guidance; S6. Long-term social skills evolution and prediction: Based on the target child's historical conversation data, predict the appropriate task type and required support strength for their next interaction. The prediction results will be used to initialize the task selection strategy and prompting strategy when the next round of the game starts.

[0008] Furthermore, the generation of structured social behavior event streams includes: S201: Perform frame-level segmentation on the facial expression video stream, decomposing the continuous video into independent image frames. For each frame, call a convolutional neural network model based on an attention mechanism to extract feature vectors in three dimensions: eye gaze direction, mouth corner curvature, and eyebrow elevation. Perform reverse normalization on the eye gaze direction of each frame to obtain the eye contact intensity of each frame, and arrange them in time stamp order to obtain the eye contact intensity sequence. Perform weighted fusion processing on the mouth corner curvature and eyebrow elevation of each frame according to preset weights to obtain the emotion tendency index of each frame, and arrange them in time stamp order to obtain the emotion tendency index sequence. Extract the eye gaze direction, mouth corner curvature, and eyebrow elevation of each frame, calculate the feature changes between adjacent frames, and analyze to obtain the facial activity sequence. S202: The dual-threshold method based on energy and zero-crossing rate is used to detect the audio stream of speech dialogue, separate the effective speech segments and remove silent or background noise segments, transcribe the effective speech segments into text through a pre-trained children's speech recognition model, and call the children's common language corpus for normalization processing to obtain a normalized social sentence sequence. S203: The OpenPose algorithm is used to locate and extract the two-dimensional coordinates of key points of the upper body from the body posture image sequence, calculate the arm spread angle, torso forward tilt amplitude and head deflection angular velocity, and form a non-verbal interaction activity index sequence after moving average filtering. S204: Divide the touch operations recorded by the underlying API of the operating system into three categories: click, swipe, and long press, and extract their features to obtain the touch operation feature sequence; S205: Align all feature sequences generated in S201 to S204 according to timestamps to form a structured social behavior event stream indexed by a timeline, with each time point containing six types of feature fields.

[0009] Furthermore, the social behavior atomic library pre-stores judgment rules for multiple types of social behavior patterns. Each rule is defined by temporal constraints and multimodal feature thresholds. A sliding window mechanism is used to traverse the entire event stream and perform rule matching on the feature combinations within each window. When the data within the window meets all the judgment conditions for a certain type of atomic behavior, the behavior is labeled at the corresponding time point, and the duration and confidence of the behavior are recorded, ultimately resulting in a sequence of labeled social behavior instances.

[0010] Furthermore, the step of calculating the social ability assessment index of the target child based on the social behavior instance sequence, determining the target child's current ability level, and matching social tasks of corresponding difficulty includes: S401: Based on the sequence of social behavior instances, count the frequency of proactive behaviors and compare it with the maximum effective frequency of preset proactive behaviors to obtain a social initiative score; count the number of behaviors initiated by virtual peers and the number of effective responses from children within a preset time, and compare the number of effective responses with the number of behaviors initiated to obtain a reciprocity response rate. The emotional tendency index sequence of the target child and the emotional state sequence of the virtual companion in the game scene were extracted, and the emotional synchronization index of the emotional sequence of the child and the virtual companion was calculated by the Pearson correlation coefficient algorithm. S402: The social initiative score, reciprocity response rate, and emotional synchronicity index are weighted and integrated to obtain the target child's comprehensive behavioral score; S403: Obtain users of the same age as the target child from the individual growth database as labeled children, and obtain the comprehensive historical behavioral scores of all labeled children to form a sample set. Where n is the number of marked children, This represents the overall score of the historical behavior of the f-th marked child; S404: Select a cluster size k=4, corresponding to the same number of capability levels; initialize cluster centers, and randomly select 1 sample as the first cluster center. The shortest distance from each sample to the selected cluster center is calculated using the Euclidean distance algorithm. ; through formula Calculate the selection probability for each sample Select the cluster with the highest probability as the second cluster center; repeat the above steps until all four initial cluster centers have been selected. ; S405: For each sample Calculate its relationship to the four initial cluster centers. The Euclidean distance will Assigned to the nearest cluster, resulting in 4 initial clusters. ; S406: For each cluster Calculate the mean of all samples within a cluster, and use it as the new cluster center. The formula is: ,in Let be the number of samples in the j-th cluster; S407: Repeat steps S405-S406. When the condition is met: the average change in cluster centers < 0.001, stop the iteration and obtain the final 4 clusters. ; S408: Calculate the average composite score for each final cluster. Based on the average comprehensive score of the clusters from low to high, the clusters are mapped to four levels of social competence, from L1 to L4. The median of the means of neighboring clusters is used as the threshold for dividing social competence levels, the threshold between L1 and L2: The threshold between L2 and L3: The threshold for separating L3 and L4: That is, the comprehensive score corresponding to Level L1 is < Level 2 corresponds to ≤Overall Score< Level 3 corresponds to ≤Overall Score< Level L4 corresponding to a comprehensive behavioral score ≥ This allows us to determine the judgment range for each level of social ability. S409: Based on the target child's current social competence level, initiate a query to the social task knowledge graph, retrieve all task nodes with an out-degree of 0 and a level label that matches the current social competence level from the social task knowledge graph, and use them as the target task for the current session.

[0011] Furthermore, the step of loading task attribute data into the game scene for rendering and interaction guidance based on the current session's target task includes: Construct a 3D game scene, instantiate virtual companion characters, arrange the background, generate interactive props, and set up dialogue trigger areas; drive the virtual companion to perform a synchronized sequence of facial expressions, actions, and voices according to the task script, and record the voice output timestamps; The system continuously monitors the sequence of social behaviors of the target child. If a combination of atomic behaviors that meets the task requirements is detected within the preset response window, positive feedback is immediately triggered. Positive feedback includes: animated feedback, voice feedback, and reward feedback. If the internal counter tracks and finds that the child fails to trigger a valid behavior within the response window three times in a row, a progressive prompting mechanism is activated: the first-level prompt is for a virtual companion to repeat key phrases and extend the waiting time; if there is still no response, the second-level prompt is triggered, rendering a bright halo around the interactive props with a slight pulsating animation; if there is still no response, the third-level prompt is triggered, loading and playing a sample animation clip to demonstrate the ideal social behavior corresponding to the task. The triggering time and level of each prompt are recorded and associated with the current task instance. When the session meets the termination conditions, the social behavior instances, task completion status, number of triggers for each level of prompts, and final values ​​of evaluation indicators for this session are integrated into a session data packet, which is then uploaded to the individual growth profile database through an encrypted channel.

[0012] Furthermore, based on the target child's historical conversation data, the system predicts the appropriate task type and required support strength for their next interaction. The prediction results will be used to initialize the task selection strategy and prompting strategy when the next round of the game starts, including: Historical conversation data of target children are regularly obtained from individual growth profile databases to construct a social competence evolution model. The standardized evaluation index sequence, one-hot encoded task completion markers, and task type labels are integrated into the model input tensor. The task type and required support strength most likely to be successfully completed in the next interaction are calculated through forward propagation. The task type most likely to be successfully completed in the next interaction is selected as the initial candidate task for the next round of game session. The current social ability level of the target child is retrieved, and the level label corresponding to the initial candidate task is compared with the current ability level. If the level label of the initial candidate task matches the current ability level, the initial candidate task is directly selected as the target task for the next round of game session. If the level label of the candidate task does not match the current ability level, all task nodes that match the current ability level are retrieved from the social task knowledge graph, and one of the tasks is selected as the target task for the next game session through a random selection algorithm. The required support level for the target child is mapped to the initial prompt level for the next game session. The specific mapping rules are as follows: when the required support level is in the range [0,1), the initial prompt level is set to level 0; when the required support level is in the range [1,2), the initial prompt level is set to level 1; and when the required support level is in the range [2,3], the initial prompt level is set to level 2.

[0013] The beneficial effects of this invention are: This invention collects visual, auditory, and tactile behavioral data of target children in a game environment, and performs structured processing on the behavioral data to generate a structured social behavior event stream. This enables accurate and detailed analysis of the complex social behaviors of target children in the game environment. A social behavior atomic library with predefined multiple core social behavior patterns is used. Each behavior pattern is defined by temporal constraint rules and multimodal feature thresholds. Combined with a sliding window mechanism, it achieves accurate matching and labeling of the structured social behavior event stream. Compared with existing behavior recognition schemes that rely on manual observation or single feature matching, it has higher matching efficiency and recognition accuracy, and can effectively avoid the subjective errors of manual observation and the one-sidedness of single feature matching.

[0014] Based on a sequence of social behavior instances, three evaluation indicators—social initiative score, reciprocity response rate, and emotional synchronization index—are calculated. A clustering algorithm is used to categorize social competence into four levels. Combined with a social task knowledge graph, corresponding social tasks of varying difficulty are matched. This approach accurately identifies the target child's social competence level, achieving dynamic adaptation between game task difficulty and the child's ability. Compared to existing fixed-difficulty training methods, this avoids the problems of frustration caused by excessive difficulty and ineffective training due to insufficient difficulty. Furthermore, task matching based on the progressive dependencies of the social task knowledge graph ensures a gradual progression of training content. The targeted and effective nature of social training helps children gradually improve their social skills through appropriately challenging levels. Through a dual-guidance mechanism of positive feedback and progressive prompts, positive feedback through animation, voice, and virtual rewards reinforces children's active behavior when they complete effective social actions. When children fail to respond continuously, a three-level progressive prompt guides them step-by-step, avoiding both excessive intervention that hinders children's independent exploration and insufficient guidance that leads to training stagnation. Compared to existing singular guidance methods, this approach aligns better with children's cognitive development, effectively increasing their participation in games and their enthusiasm for training, thus improving the efficiency of social behavior guidance.

[0015] Based on historical conversation data, a three-layer stacked LSTM social ability evolution model is constructed to predict the appropriate task type and required support intensity for the next interaction. The prediction results will be used to initialize the task selection strategy and prompt strategy when the next round of the game starts, so as to realize the long-term tracking and prediction of children's social ability development trajectory. Compared with the existing solutions that only focus on the effect of a single training session, it can predict the trend of children's ability development in advance, match appropriate training tasks and guidance intensity in advance, and realize continuous and coherent personalized social guidance. Attached Figure Description

[0016] The invention will now be further described with reference to the accompanying drawings.

[0017] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] Please see Figure 1 As shown, this invention is a game interaction method for guiding children's social behavior, comprising the following steps: S1. Multimodal Behavior Data Acquisition: Through an interactive terminal device equipped with a front-facing camera, microphone, touch screen and depth sensor, multimodal raw behavior data of the target child in the game environment is collected simultaneously. The multimodal raw behavior data includes facial expression video stream, voice dialogue audio stream, body posture image sequence and touch operation log. Specifically, it captures facial expression video streams at a frame rate of no less than 30 frames per second and a resolution of 640×480 using the front-facing camera; records audio streams of voice conversations at a sampling rate of 16kHz using a directional microphone; acquires body pose image sequences using a depth sensor for pose estimation; and acquires touch operation logs recorded by the underlying API of the operating system, which include timestamps, screen coordinates, and gesture types, including clicks, swipes, and long presses.

[0020] S2. Structured Social Behavior Event Stream Generation: The multimodal raw behavior data is processed in a structured manner to generate a structured social behavior event stream; Specifically, frame-level segmentation is performed on the facial expression video stream, breaking down the continuous video into independent image frames. For each image frame, a convolutional neural network model based on an attention mechanism is invoked. This convolutional neural network model is pre-trained on a large-scale labeled dataset containing children's facial expressions and outputs feature vectors in three dimensions: eye gaze direction, mouth corner curvature, and eyebrow elevation. The eye gaze direction is represented by the offset angle of the pupil center relative to the midline of the bridge of the nose; the smaller the offset angle, the more focused the gaze. The mouth corner curvature is calculated by fitting a quadratic curve of the lip contour to determine the opening degree and the degree of upward movement. The opening degree reflects the extent to which the lips are open, and the degree of upward movement reflects whether the corners of the mouth are up or down. The degree of eyebrow elevation is determined by the rate of change of the vertical distance between the center of the eyebrows and the brow peak; the greater the rate of change, the more pronounced the eyebrow elevation. Based on the single-frame feature vectors of the above three dimensions, the eye gaze direction of each frame is reverse normalized to obtain the eye contact intensity of each frame. These are then arranged in timestamp order to obtain the eye contact intensity sequence. The reverse normalization formula is as follows: The maximum allowable offset angle refers to the upper limit of the allowable offset angle range. For example, if the allowable offset angle range is 0°-30°, then the maximum allowable offset angle is 30°. 0° represents complete fixation on the target, and more than 30° represents serious deviation from the target. The higher the eye contact intensity value, the more stable the fixation on the target. The curvature of the corners of the mouth and the degree of eyebrow elevation in each frame are weighted and fused according to preset weights to obtain the emotional tendency index of each frame, which is used to reflect the emotional state of the target child. The larger the emotional tendency index, the higher the level of happiness of the target child; the smaller the emotional tendency index, the higher the level of depression of the target child. The emotional tendency index sequence is obtained by arranging the data in the order of timestamps. Extract the eye gaze direction, mouth corner curvature, and eyebrow elevation for each frame, and then use a formula... Calculate the feature change between two adjacent frames, where i is the index of the current frame. This represents the change in features between the i-th frame and the (i-1)-th frame. These represent the eye gaze direction, mouth corner curvature, and eyebrow elevation in the i-th frame, respectively. These represent the eye gaze direction, mouth corner curvature, and eyebrow elevation in the (i-1)th frame, respectively; a preset micro-expression detection threshold is also provided. ,when When a micro-expression event occurs, it is considered a micro-expression event, and the facial activity level is assigned a value of 1; otherwise, the facial activity level is assigned a value of 0. The facial activity sequence is obtained by iterating through the feature changes between all adjacent frames. It should be noted that the micro-expression determination threshold... This was obtained through statistical analysis of a large-scale children's facial expression dataset; Endpoint detection of voice dialogue audio stream using a double-threshold method based on energy and zero-crossing rate, separating the effective speech segments and removing the silent or background noise segments. Transcribe the effective speech segments into text through a pre-trained children's speech recognition model, and call the children's common corpus for normalization processing to obtain a normalized social sentence sequence. It should be noted that considering the characteristics of children's language, such as simplified vocabulary, loose grammar, and redundancy, call the built-in children's common corpus containing high-frequency social phrases and their variants to perform synonym replacement and sentence simplification on the transcription results. For example, replace "俺" with "我", "咋" with "怎么", and simplify "能不能给我那个红色的球" to "给我红球". Locate and extract the two-dimensional coordinates of the upper body key points from the body pose image sequence through the OpenPose algorithm. The key points include, but are not limited to: shoulders: left acromion, right acromion; elbows: left elbow, right elbow; wrists: left wrist, right wrist; neck: root of the neck; head-related: center of the eyebrows, tip of the nose, left ear, right ear; torso-related: left endpoint of the hip, right endpoint of the hip, etc. Calculate the arm extension angle, torso forward tilt angle, and head yaw angular velocity based on the above two-dimensional coordinates, and form a non-verbal interaction activity index sequence after sliding average filtering. The sliding average filtering eliminates instantaneous fluctuations. The higher the non-verbal interaction activity index value, the stronger the interactivity of the body pose. It should be noted that the arm extension angle is calculated as the sum of the angles between the left and right upper arms and the reference line with the line connecting the two shoulders as the reference. The torso forward tilt amplitude is measured by the vertical projection difference between the neck and hip key points. The head yaw angular velocity is obtained by taking the derivative of the horizontal displacement of the tip of the nose coordinates in consecutive frames and converting it into an angular change rate. Classify the touch operations recorded by the underlying API of the operating system into three categories: click, slide, and long press, and perform feature extraction. For click operations, extract its timestamp, screen normalized coordinates, and the corresponding game interaction areas, such as the prop selection area, dialogue trigger area; for slide operations, extract the start and end timestamps, the direction and distance of the sliding trajectory; for long press operations, extract the start and end timestamps, the duration, and the corresponding game interaction areas; finally, obtain the touch operation feature sequence. Align the eye contact intensity sequence, emotional tendency index sequence, facial activity sequence, normalized social sentences, non-verbal interaction activity index sequence, and touch operation feature sequence according to the timestamp, forming a structured social behavior event stream indexed by the time axis, with six types of feature fields at each time point. This event stream provides the basic data structure for subsequent behavior analysis.

[0021] S3. Matching and annotation with the social behavior atomic library: Match and annotate the structured social behavior event stream with the predefined social behavior atomic library to obtain a sequence of labeled social behavior instances. It should be noted that the social behavior atomic library pre-stores definitions for various social behavior patterns, such as "initiating conversations," "responding to others' questions," "sharing virtual items," "imitating peer actions," and "expressing empathic statements." Each pattern is defined by a set of temporal constraint rules and multimodal feature thresholds. For example, the rules for "initiating conversations" are: there must be a voice start point within any 2-second window, i.e., the start time of the standardized social statement, and there must be no other person's voice event within the 2 seconds before that time point, which is deduced by back-calculating the voice output timestamp in the virtual character's script. At the same time, the eye contact intensity at the corresponding time point must be ≥0.6. The rules for "expressing empathic statements" are: the standardized social statement must contain at least one of the preset keyword lists such as "Are you sad?", "I'm with you," and "Don't cry," and the emotional tendency index at the corresponding time point must be ≤-0.3, indicating a low-pleasure state. The entire event stream is traversed using a sliding window mechanism with a preset sliding window length and step size. For example, the sliding window length is 1.5s and the step size is 0.2s. Rule matching is performed on the feature combinations within each window. When the data within the window meets all the judgment conditions for a certain type of atomic behavior, the behavior is labeled at the corresponding time point, and the duration and confidence of the behavior are recorded, ultimately resulting in a sequence of labeled social behavior instances.

[0022] S4. Dynamic Difficulty Adjustment: Based on the sequence of social behavior instances, calculate the social ability assessment index of the target child, determine the current ability level of the target child and match social tasks of corresponding difficulty, so as to achieve dynamic adaptation between game difficulty and children's social behavior. Specifically, S401: Based on the sequence of social behavior instances, the frequency of proactive behaviors is counted, and the ratio of this frequency to the maximum effective frequency of proactive behaviors is calculated to obtain a social initiative score; the number of behaviors initiated by virtual peers and the number of effective responses from children within a preset time period are counted, and the ratio of the number of effective responses to the number of behaviors initiated is calculated to obtain a reciprocity response rate. Extract the emotional tendency index sequence from the target child's side, denoted as... t represents the number of time points, each The emotional tendency index of the target child at a certain moment, and the emotional state sequence of the virtual peers in the game scene, are denoted as follows: Each Based on the preset emotion values ​​of virtual companions at the same time, the Pearson correlation coefficient algorithm is used. The emotional synchronization index between the target child and the virtual peer's emotional sequence was calculated. ,in, This represents the mean of a series of children's emotional tendency indices. The emotional synchronization index represents the mean of a sequence of emotional states of a virtual companion. The range of values ​​is ,like This indicates that the child's emotions are in positive sync with their virtual peer; for example, when the virtual peer is down, the child also shows signs of being down. The closer to 1, the stronger the synchronization; if This indicates that the child's emotions deviate from those of their virtual peer; for example, the child shows pleasure when the virtual peer is down. This indicates that there is no significant correlation between children's emotions and those of their virtual peers; S402: The social initiative score, reciprocity response rate, and emotional synchronicity index are weighted and integrated to obtain the target child's comprehensive behavioral score; S403: Obtain users of the same age as the target child from the individual growth database as labeled children, and obtain the comprehensive historical behavioral scores of all labeled children to form a sample set. Where n is the number of marked children, This represents the overall score of the historical behavior of the f-th marked child; S404: Select a cluster size k=4, corresponding to the same number of capability levels; initialize cluster centers, and randomly select 1 sample as the first cluster center. The shortest distance from each sample to the selected cluster center is calculated using the Euclidean distance algorithm. ; through formula Calculate the selection probability for each sample Select the cluster with the highest probability as the second cluster center; repeat the above steps until all four initial cluster centers have been selected. ; S405: For each sample Calculate its relationship to the four initial cluster centers. The Euclidean distance will Assigned to the nearest cluster, resulting in 4 initial clusters. The superscript indicates the iteration number; S406: For each cluster Calculate the mean of all samples within a cluster, and use it as the new cluster center. The formula is: ,in Let be the number of samples in the j-th cluster; S407: Repeat steps S405-S406. When the condition is met: the average change in cluster centers < 0.001, stop the iteration and obtain the final 4 clusters. It should be noted that the formula for calculating the average change in cluster centers is: , where t is the current iteration round number; S408: Calculate the mean of the comprehensive behavioral score for each final cluster. Based on the average comprehensive score of the clusters from low to high, the clusters are mapped to four levels of social competence, from L1 to L4. The median of the means of neighboring clusters is used as the threshold for dividing social competence levels, the threshold between L1 and L2: The threshold between L2 and L3: The threshold for separating L3 and L4: That is, the overall score of behavior corresponding to Level L1 is < Level 2 corresponds to ≤Overall Behavioral Score< Level 3 corresponds to ≤Overall Behavioral Score< Level L4 corresponding to a comprehensive behavioral score ≥ This determines the judgment range for each level of social competence. It should be noted that, among the social competence levels, L1 represents the passive response period, L2 represents the active attempt period, L3 represents the cooperative interaction period, and L4 represents the autonomous organization period. The target child's overall behavioral score is matched with the judgment range of each social ability level to obtain the target child's current social ability level; S409: Based on the target child's current social competence level, a query is initiated into the social task knowledge graph. The social task knowledge graph is organized in the form of a directed acyclic graph (DAG). Nodes in the graph represent specific social tasks, such as "inviting a peer to join the game," "comforting a crying friend," and "taking turns using toys." Edges represent the progressive dependencies between tasks. Each node contains the task objective, the required set of social atomic behaviors, the virtual character script, and the reward feedback mechanism attributes. All task nodes with an out-degree of 0 and whose level label matches the current social competence level are retrieved from the social task knowledge graph and used as the target task for the current session. It should be noted that out-degree is a concept in graph theory, referring to the number of edges from a task node to other task nodes. An out-degree of 0 indicates that a task is a task with no subsequent progressive tasks at the current competence level, i.e., the highest difficulty task that can be challenged at the current competence level. By filtering out tasks with an out-degree of 0, the target child is allowed to challenge the highest difficulty task within their current competence range, ensuring that the task is neither too difficult nor too easy, balancing the safety and effectiveness of training, and ensuring that the selected task is within the target child's current social competence development level.

[0023] S5. Game Scene Rendering and Interactive Guidance: Based on the current session's target task, load task attribute data into the game scene for rendering and interactive guidance; Specifically, 3D game scenes are built based on Unity or Unreal Engine, virtual companion characters are instantiated, background scenes are arranged, interactive props are generated, and dialogue triggering areas are set. The task script drives the virtual companion to perform a synchronized sequence of facial expressions, actions, and voices according to timestamps, and records the timestamps of the voice output. The system continuously monitors the sequence of social behaviors of the target child. If a combination of atomic behaviors that meets the task requirements is detected within a preset response window, positive feedback is immediately triggered. Positive feedback includes: animated feedback, voice feedback, and reward feedback. For example, animated feedback loads and plays a celebratory animation, such as fireworks; voice feedback plays a pre-recorded encouraging voice, such as "You're so thoughtful"; and reward feedback displays a virtual reward icon, such as a heartwarming badge, in a designated area of ​​the screen. The system records the response delay and keyword matching degree of behavior completion and associates them with the current task instance. If the internal counter tracks and finds that the child fails to trigger a valid behavior within the response window three times in a row, a progressive prompting mechanism is activated: the first-level prompt is for a virtual companion to repeat key phrases and extend the waiting time; if there is still no response, the second-level prompt is triggered, rendering a bright halo around the interactive props with a slight pulsating animation; if there is still no response, the third-level prompt is triggered, loading and playing a sample animation clip to demonstrate the ideal social behavior corresponding to the task. The triggering time and level of each prompt are recorded and associated with the current task instance. When a session meets the conditions of user-initiated exit or task timeout of 10 minutes, the social behavior instances, task completion status, number of triggers for each level of prompts, and final values ​​of evaluation metrics for this session are integrated into a session data packet. This data packet is then uploaded to the individual growth profile database through an encrypted channel. The database uses the child's unique ID as an index and employs a time-series database to store all session records, supporting multi-dimensional queries.

[0024] S6. Long-term social skills evolution and prediction: Based on the target child's historical conversation data, predict the appropriate task type and required support strength for their next interaction. The prediction results will be used to initialize the task selection strategy and prompting strategy when the next round of the game starts. Specifically, historical conversation data of target children are periodically obtained from the individual growth profile database, including three assessment indicator sequences: social initiative score, reciprocity response rate, and emotional synchronization index; task completion markers, represented in one-hot encoding form, such as "success" encoded as [1,0] and "failure" encoded as [0,1]; task type labels, the target task type of the current conversation, such as "comforting a crying friend" encoded as 001 and "inviting a peer to join the game" encoded as 002, which cover a unique code for all task nodes; A social capability evolution model is constructed, which integrates the standardized evaluation index sequence, one-hot encoded task completion markers, and task type labels into the model input tensor. The task type and required support strength most likely to be successfully completed in the next interaction are calculated through forward propagation. It should be noted that the social ability evolution model is a pre-trained three-layer stacked long short-term memory (LSTM) network model. The number of neurons in each LSTM layer is 128, 64, and 32, respectively. The activation function is tanh. The output layer is divided into two branches: a classification output branch and a regression output branch. The classification output branch uses softmax activation, and the output dimension is the total number of task types. For example, the 8 core tasks correspond to 8-dimensional output, which represents the probability of successfully completing each task in the next step. The regression output branch uses linear activation, and the output dimension is 1, which represents the required support strength and reflects the degree of help needed for the child to complete the task. Its value ranges from [0,3], corresponding to the cue level of 0-2. The task type most likely to be successfully completed in the next interaction is selected as the initial candidate task for the next round of game session. The current social ability level of the target child is retrieved, and the level label corresponding to the initial candidate task is compared with the current ability level. If the level label of the initial candidate task matches the current ability level, the initial candidate task is directly selected as the target task for the next round of game session. If the level label of the candidate task does not match the current ability level, all task nodes that match the current ability level are retrieved from the social task knowledge graph, and one of the tasks is selected as the target task for the next game session through a random selection algorithm. The required support level for the target child is mapped to the initial cue level for the next round of the game session, with the specific mapping rules as follows: When the required support strength is in the range [0,1), the initial prompt level is configured to level 0, that is, the pre-prompt is not enabled by default, allowing the target child to try the task independently; When the required support strength is in the range [1,2), the initial prompt level is configured to level 1, that is, the basic prompt resources of virtual companion repeating key statements are loaded in advance; When the required support strength is in the range [2,3], the initial prompt level is configured to level 2, that is, advanced prompt resources of interactive prop highlighting are loaded in advance.

[0025] It should be noted that prioritizing tasks with high success rates predicted by the model increases the probability of children completing tasks, thereby enhancing their sense of accomplishment in social training to some extent. Simultaneously, hierarchical matching ensures that the task difficulty is appropriate for the child's current social abilities, avoiding frustration caused by overly difficult tasks or inefficient training due to overly easy tasks. Matching the prompting strategy to the child's support needs prevents children from abandoning tasks due to a lack of timely guidance, while reducing unnecessary prompting interventions and optimizing the efficiency of social behavior guidance.

[0026] Through the steps S1 to S6 described above, this invention realizes a complete technical solution from children's social behavior data collection, analysis, and evaluation to task adaptation, interactive guidance, and long-term evolution prediction. It can accurately match children's social ability level, provide personalized game interaction guidance, and effectively help children's social ability improve gradually.

[0027] The above description is merely an example and illustration of the structure of the present invention. Those skilled in the art can make various modifications or additions to the specific embodiments described, or use similar methods to replace them, as long as they do not deviate from the structure of the invention or exceed the scope defined in the claims, all of which should fall within the protection scope of the present invention.

Claims

1. A game interaction method for guiding children's social behavior, characterized in that, Includes the following steps: S1. Collect multimodal raw behavioral data of the target child in the game environment through an interactive terminal device, wherein the multimodal raw behavioral data includes: facial expression video stream, voice dialogue audio stream, body posture image sequence and touch operation log; S2. Perform structured processing on the raw multimodal behavioral data to generate a structured social behavioral event stream; S3. Match and label the structured social behavior event stream with the predefined social behavior atom library to obtain a tagged sequence of social behavior instances; S4. Based on the sequence of social behavior instances, calculate the social ability assessment index of the target child, determine the current ability level of the target child and match social tasks of corresponding difficulty, so as to achieve dynamic adaptation between game difficulty and children's social behavior. S5. Based on the current session's target task, load the task attribute data into the game scene for rendering and interactive guidance; S6. Based on the target child's historical conversation data, predict the appropriate task type and required support strength for their next interaction. The prediction results will be used to initialize the task selection strategy and prompt strategy when the next round of the game starts.

2. The game interaction method for guiding children's social behavior according to claim 1, characterized in that, The generation of structured social behavior event streams includes: S201: Perform frame-level segmentation on the facial expression video stream, decomposing the continuous video into independent image frames. For each frame, call a convolutional neural network model based on an attention mechanism to extract feature vectors in three dimensions: eye gaze direction, mouth corner curvature, and eyebrow elevation. Perform reverse normalization on the eye gaze direction of each frame to obtain the eye contact intensity of each frame, and arrange them in time stamp order to obtain the eye contact intensity sequence. Perform weighted fusion processing on the mouth corner curvature and eyebrow elevation of each frame according to preset weights to obtain the emotion tendency index of each frame, and arrange them in time stamp order to obtain the emotion tendency index sequence. Extract the eye gaze direction, mouth corner curvature, and eyebrow elevation of each frame, calculate the feature changes between adjacent frames, and analyze to obtain the facial activity sequence. S202: The dual-threshold method based on energy and zero-crossing rate is used to detect the audio stream of speech dialogue, separate the effective speech segments and remove silent or background noise segments, transcribe the effective speech segments into text through a pre-trained children's speech recognition model, and call the children's common language corpus for normalization processing to obtain a normalized social sentence sequence. S203: The OpenPose algorithm is used to locate and extract the two-dimensional coordinates of key points of the upper body from the body posture image sequence, calculate the arm spread angle, torso forward tilt amplitude and head deflection angular velocity, and form a non-verbal interaction activity index sequence after moving average filtering. S204: Divide the touch operations recorded by the underlying API of the operating system into three categories: click, swipe, and long press, and extract their features to obtain the touch operation feature sequence; S205: Align all feature sequences generated in S201 to S204 according to timestamps to form a structured social behavior event stream indexed by a timeline, with each time point containing six types of feature fields.

3. The game interaction method for guiding children's social behavior according to claim 1, characterized in that, The social behavior atomic library pre-stores judgment rules for multiple social behavior patterns. Each rule is defined by temporal constraints and multimodal feature thresholds. A sliding window mechanism is used to traverse the entire event stream and perform rule matching on the feature combinations within each window. When the data within the window meets all the judgment conditions for a certain type of atomic behavior, the behavior is labeled at the corresponding time point, and the duration and confidence of the behavior are recorded, ultimately resulting in a sequence of labeled social behavior instances.

4. The game interaction method for guiding children's social behavior according to claim 1, characterized in that, Based on the sequence of social behavior instances, calculate the social ability assessment index of the target child, determine the current ability level of the target child, and match social tasks of corresponding difficulty, including: S401: Based on the sequence of social behavior instances, count the frequency of proactive behaviors and compare it with the maximum effective frequency of preset proactive behaviors to obtain a social initiative score; count the number of behaviors initiated by virtual peers and the number of effective responses from children within a preset time, and compare the number of effective responses with the number of behaviors initiated to obtain a reciprocity response rate. The emotional tendency index sequence of the target child and the emotional state sequence of the virtual companion in the game scene were extracted, and the emotional synchronization index between the child and the virtual companion was calculated by the Pearson correlation coefficient algorithm. S402: The social initiative score, reciprocity response rate, and emotional synchronicity index are weighted and integrated to obtain the target child's comprehensive behavioral score; S403: Obtain users of the same age as the target child from the individual growth database as labeled children, and obtain the comprehensive historical behavioral scores of all labeled children to form a sample set. Where n is the number of marked children, This represents the overall score of the historical behavior of the f-th marked child; S404: Select a cluster size k=4, corresponding to the same number of capability levels; initialize cluster centers, and randomly select 1 sample as the first cluster center. The shortest distance from each sample to the selected cluster center is calculated using the Euclidean distance algorithm. ; through formula Calculate the selection probability for each sample Select the cluster with the highest probability as the second cluster center; repeat the above steps until all four initial cluster centers have been selected. ; S405: For each sample Calculate its relationship to the four initial cluster centers. The Euclidean distance will Assigned to the nearest cluster, resulting in 4 initial clusters. ; S406: For each cluster Calculate the mean of all samples within a cluster, and use it as the new cluster center. The formula is: ,in Let be the number of samples in the j-th cluster; S407: Repeat steps S405-S406. When the condition is met: the average change in cluster centers < 0.001, stop the iteration and obtain the final 4 clusters. ; S408: Calculate the average composite score for each final cluster. Based on the average comprehensive score of the clusters from low to high, the clusters are mapped to four levels of social competence, from L1 to L4. The median of the means of neighboring clusters is used as the threshold for dividing social competence levels, the threshold between L1 and L2: The threshold between L2 and L3: The threshold for separating L3 and L4: That is, the comprehensive score corresponding to Level L1 is < Level 2 corresponds to ≤Overall Score< Level 3 corresponds to ≤Overall Score< Level L4 corresponding to a comprehensive behavioral score ≥ This allows us to determine the judgment range for each level of social ability. S409: Based on the target child's current social competence level, initiate a query to the social task knowledge graph, retrieve all task nodes with an out-degree of 0 and a level label that matches the current social competence level from the social task knowledge graph, and use them as the target task for the current session.

5. The game interaction method for guiding children's social behavior according to claim 1, characterized in that, The step of loading task attribute data into the game scene for rendering and interactive guidance based on the current session's target task includes: Construct a 3D game scene, instantiate virtual companion characters, arrange the background, generate interactive props, and set up dialogue trigger areas; drive the virtual companion to perform a synchronized sequence of facial expressions, actions, and voices according to the task script, and record the voice output timestamps; The system continuously monitors the sequence of social behaviors of the target child. If a combination of atomic behaviors that meets the task requirements is detected within the preset response window, positive feedback is immediately triggered. Positive feedback includes: animated feedback, voice feedback, and reward feedback. If the internal counter tracks and finds that the child fails to trigger a valid behavior within the response window three times in a row, a progressive prompting mechanism is activated: the first-level prompt is for a virtual companion to repeat key phrases and extend the waiting time; if there is still no response, the second-level prompt is triggered, rendering a bright halo around the interactive props with a slight pulsating animation; if there is still no response, the third-level prompt is triggered, loading and playing a sample animation clip to demonstrate the ideal social behavior corresponding to the task. The triggering time and level of each prompt are recorded and associated with the current task instance. When the session meets the termination conditions, the social behavior instances, task completion status, number of triggers for each level of prompts, and final values ​​of evaluation indicators for this session are integrated into a session data packet, which is then uploaded to the individual growth profile database through an encrypted channel.

6. The game interaction method for guiding children's social behavior according to claim 1, characterized in that, Based on the target child's historical conversation data, the appropriate task type and required support strength for their next interaction are predicted. The prediction results will be used to initialize the task selection strategy and prompting strategy when the next round of the game starts, including: Historical conversation data of target children are regularly obtained from individual growth profile databases to construct a social competence evolution model. The standardized evaluation index sequence, one-hot encoded task completion markers, and task type labels are integrated into the model input tensor. The task type and required support strength most likely to be successfully completed in the next interaction are calculated through forward propagation. The task type most likely to be successfully completed in the next interaction is selected as the initial candidate task for the next round of game session. The current social ability level of the target child is retrieved, and the level label corresponding to the initial candidate task is compared with the current ability level. If the level label of the initial candidate task matches the current ability level, the initial candidate task is directly selected as the target task for the next round of game session. If the level label of the candidate task does not match the current ability level, all task nodes that match the current ability level are retrieved from the social task knowledge graph, and one of the tasks is selected as the target task for the next game session through a random selection algorithm. The required support level for the target child is mapped to the initial prompt level for the next game session. The specific mapping rules are as follows: when the required support level is in the range [0,1), the initial prompt level is set to level 0; when the required support level is in the range [1,2), the initial prompt level is set to level 1; and when the required support level is in the range [2,3], the initial prompt level is set to level 2.

Images