A growth understanding and intelligent feedback system and method based on natural interaction of children

Abstract

This invention relates to the field of intelligent interaction for children's growth, and discloses a growth understanding and intelligent feedback system and method based on children's natural interaction. The system includes acquiring multimodal basic data samples and converting them into behavioral feature vectors of a unified dimension; calculating the similarity parameter between the behavioral feature vectors and the basic cognitive schema set; constructing a staged cognitive schema model by combining age-specific weight constraints and time decay variables; extracting the schema feature vectors in the activated state to generate growth analysis results; performing a split mapping on the growth analysis results to convert them into summary information and interaction strategy parameters; and dynamically adjusting the interactive operation logic of the intelligent agent according to the closed-loop interaction strategy parameters. This invention avoids data distortion through a low-intervention data acquisition architecture, improves the accuracy of growth analysis and supports continuous understanding; eliminates exploration noise through staged cognitive schema modeling, improves the stability and interpretability of the results, and enhances the adaptability and continuity of the intelligent agent's interaction.

Description

Technical Field

[0001] This invention relates to the field of intelligent interaction for children's growth, specifically to a growth understanding and intelligent feedback system and method based on children's natural interaction. Background Technology

[0002] With the development of artificial intelligence technology and smart terminal devices, intelligent interactive products targeting children are gradually becoming more widespread. These products attempt to provide companionship or educational services through voice dialogue, content recommendations, or interactive games. Simultaneously, parents and educational settings are increasingly seeking to understand children's growth and development, hoping to use technology to continuously understand their interests, cognitive characteristics, and developmental status. While current smart products have enriched interaction methods to some extent, they still have limitations in deeply understanding natural behavior and continuously assessing cognitive states.

[0003] Existing developmental assessment and interest analysis technologies typically rely on highly interventionist methods such as questionnaires, task tests, or structured questioning, requiring subjects to complete specified behaviors within a given scenario. These methods are easily influenced by context, emotions, and expressive abilities, failing to accurately reflect subjects' natural exploration patterns and leading to distorted data. Furthermore, existing analysis schemes lack long-term, phased modeling mechanisms and intermediate cognitive models, making it difficult to filter out perturbations from low-frequency random actions, resulting in insufficient stability and interpretability of developmental analysis results. Simultaneously, subjects' cognitive abilities and interaction needs vary across different age groups. Existing systems often employ uniform data processing models, failing to translate analysis results into specific control commands encompassing physical actions and visual interfaces. This results in a lack of targeted interactive feedback, hindering the maintenance of adaptability and continuity in agent interaction.

[0004] The aforementioned shortcomings prevent existing systems from accurately extracting cognitive features from multimodal interaction data in real-world scenarios, from meeting the long-term understanding needs of objects during their growth process, and from providing a closed-loop, dynamically adjustable feedback environment based on immediate behavioral performance. Therefore, how to acquire natural behavioral data with low intervention to improve analytical accuracy, how to enhance the stability and interpretability of evaluation results using staged modeling methods and intermediate cognitive models, and how to dynamically adjust the interactive operation logic of the intelligent agent based on the analysis results are problems that need to be solved in this field. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a growth understanding and intelligent feedback system and method based on children's natural interaction. It solves the problems of existing developmental assessment schemes relying on high intervention methods, which leads to distortion of natural behavioral data; lack of stage modeling and intermediate cognitive models, which leads to insufficient stability and interpretability of analysis results; and the inability of unified data processing models to convert analysis results into specific physical control commands, which leads to a lack of adaptability and continuity in intelligent agent interaction.

[0006] To address the above problems, the present invention provides the following technical solution: This invention provides a growth understanding and intelligent feedback system based on children's natural interaction, employing the following technical solution: A growth understanding and intelligent feedback system based on children's natural interaction includes: A multimodal perception and parsing module is used to acquire natural behavior data, extract multimodal features from the natural behavior data, and convert them into behavior feature vectors; The cognitive schema evaluation module is used to calculate the similarity parameter between the behavioral feature vector and the preset basic cognitive schema set, and to generate growth analysis results by combining the similarity parameter with the behavioral statistical features of the natural behavioral data. The intelligent feedback execution module is used to convert the growth analysis results into readable summary information and strategy parameters, output the readable summary information, and adjust the interactive operation logic of the agent based on the strategy parameters. The age-segmentation strategy control module receives the object's age parameter, generates configuration instructions based on the age stage corresponding to the object's age parameter, and uses the configuration instructions to update the operating parameters of the multimodal perception and parsing module, the cognitive graph evaluation module, and the intelligent feedback execution module, respectively.

[0007] By adopting the above technical solutions, the system, through its architecture combining the multimodal perception and analysis module with the cognitive schema evaluation module, can extract natural behavior data from daily interactions and analyze corresponding cognitive states in a non-interventional manner, avoiding data distortion problems caused by preset tasks or standardized questionnaires. By employing the intelligent feedback execution module, the system transforms the growth analysis results into specific strategy parameters, establishing a transformation path from cognitive understanding to hardware execution, and realizing dynamic closed-loop adjustment of the interactive environment. Furthermore, by employing the age-appropriate strategy control module, the system constructs a global operational constraint mechanism at the underlying level, ensuring that perceptual input, computational evaluation, and feedback strategies can adapt to the cognitive development patterns of different age groups, improving the accuracy of growth understanding and the relevance of feedback behavior.

[0008] Furthermore, the natural behavior data includes image capture data, voice interaction data, and function selection data; the multimodal perception and parsing module extracts the subject category features and spatial composition features of the image capture data and merges them into an original image feature vector, extracts the dialogue semantic features and interaction intent features of the voice interaction data and merges them into an original voice feature vector, extracts the operation frequency and trajectory parameters of the function selection data and merges them into an original operation feature vector, and performs normalization processing on the original image feature vector, the original voice feature vector, and the original operation feature vector to transform them into the behavior feature vector of a unified dimension.

[0009] By adopting the above technical solution, the system acquires natural behavior data from multiple dimensions such as visual perception, auditory interaction, and functional operation, ensuring the comprehensiveness of the basic data samples. Through normalization processing, the multi-source heterogeneous raw data is transformed into a unified-dimensional behavioral feature vector, eliminating structural differences between different sensor data types and providing a standardized data input source for subsequent feature matching and model calculation.

[0010] Furthermore, the cognitive schema evaluation module processes the similarity parameter in conjunction with the time decay variable and obtains the schema activation state. The schema activation state below the preset activation judgment threshold is reset to zero and reconstructed into a model activation vector. The preset mapping transition matrix is ​​used to perform a projection operation on the model activation vector to generate a stage growth analysis result vector. The behavioral statistical features are written into the data structure of the stage growth analysis result vector to generate the growth analysis result.

[0011] By adopting the above technical solution, the system incorporates the time decay variable into the calculation, reflecting the dynamic impact of the event's time span on the current cognitive state. Filtering is performed using the activation threshold to eliminate accidental exploration noise and improve the stability of the feature data. Spatial projection operations are performed using the mapping transition matrix to transform the abstract graphical activation state into numerical results reflecting specific dimensions, enhancing the quantifiability of the evaluation metrics.

[0012] Furthermore, the interactive operation logic of the intelligent agent includes response action mode, feedback content display format, and active / passive guidance intensity; the intelligent feedback execution module maps the strategy parameters to physical control variables including response action amplitude variable, feedback content graphic ratio variable, and active prompting frequency variable, adjusts the response action mode based on the response action amplitude variable, adjusts the feedback content display format based on the feedback content graphic ratio variable, and adjusts the active / passive guidance intensity based on the active prompting frequency variable.

[0013] By adopting the above technical solution, the system compiles the strategy parameters into physical control variables, including the response action amplitude variable, the feedback content graphic-text ratio variable, and the proactive prompting frequency variable, thereby achieving multi-dimensional adjustment of the agent's hardware state. This adjustment mechanism alters the agent's performance characteristics, enhances device stickiness during the interaction process, and can specifically guide subsequent cognitive exploration behaviors.

[0014] Furthermore, when the intelligent feedback execution module transforms the growth analysis results into the strategy parameters, it introduces a corresponding age-based constraint bias function in conjunction with the object's age parameter, and uses a preset strategy mapping algorithm and strategy mapping weight matrix to transform the growth analysis results into a strategy parameter vector and package it into the strategy parameters; the intelligent feedback execution module parses the specific dimension values ​​of the strategy parameter vector and maps them to generate the physical control variables.

[0015] By adopting the above technical solution, the system utilizes the policy mapping weight matrix and the age-based constraint bias function to perform calculations, realizing the transformation from the evaluation dimension to the physical execution dimension. Decomposing and mapping the policy parameter vector into physical control variables readable by the agent ensures that policy instructions can be integrated into the underlying hardware control flow.

[0016] Furthermore, the intelligent feedback execution module controls the agent to adjust the range of motion and execution speed parameters of the physical action based on the response action amplitude variable in the physical control variables; the intelligent feedback execution module allocates the proportion of the image display area and the text display area in the display interface based on the feedback content image-text ratio variable in the physical control variables; the intelligent feedback execution module records the no-interaction idle time, and when the no-interaction idle time exceeds the set wake-up waiting threshold, it generates an interaction guidance command based on the active prompting frequency variable in the physical control variables, and controls the agent to actively output an interaction guidance signal.

[0017] By adopting the above technical solution, the system controls the intelligent agent from multiple dimensions, such as the motion parameters of the mechanical drive components, the visual layout ratio of the display interface, and the wake-up command of the system timer, thereby changing the passive waiting response mode and inducing the object to continuously and actively explore.

[0018] Furthermore, the configuration instructions include data acquisition weights, analysis model parameters, and interaction constraints; the age-based strategy control module uses the data acquisition weights to update the computation priority of the multimodal perception and parsing module, uses the analysis model parameters to update the evaluation criteria of the cognitive schema evaluation module, and uses the interaction constraints to update the feedback strategy of the intelligent feedback execution module.

[0019] By adopting the above technical solution, the system outputs corresponding data collection weights, analysis model parameters, and interaction constraints for different age groups, achieving synchronous updates of parameters in each processing stage. This mechanism ensures that the extraction preferences of the underlying data and the feedback performance of the front-end hardware are matched with the physical exploration tendencies or linguistic logic tendencies of the current age group.

[0020] This invention also provides a method for growth understanding and intelligent feedback based on children's natural interaction, employing the following technical solution: A growth understanding and intelligent feedback method based on children's natural interaction includes the following steps: During natural interaction, the multimodal perception and analysis module collects image capture data, voice interaction data, and function selection data of the object, and uses natural interaction behavior as a trigger condition to obtain multimodal natural behavior data. The multimodal perception and parsing module extracts the multimodal features of the natural behavior data and transforms the natural behavior data into a behavior feature vector of a unified dimension. The cognitive schema evaluation module calculates the similarity parameter between the behavioral feature vector and the preset basic cognitive schema set based on a set time window, and calculates the similarity parameter in combination with the age-based weight constraints and time decay variables provided by the age-based strategy control module, to obtain the schema activation state of each basic cognitive schema in the basic cognitive schema set and construct a staged cognitive schema model. The cognitive schema evaluation module extracts the model activation vector from the staged cognitive schema model as input based on the staged cognitive schema model and the behavioral statistical features of the natural behavioral data. It then performs calculations through a preset mapping transition matrix to generate a staged growth analysis result vector and outputs the growth analysis results. The intelligent feedback execution module performs a dual-channel data splitting and mapping operation on the growth analysis results. The first channel converts the growth analysis results into readable summary information in natural language format, and the second channel compiles the growth analysis results into executable strategy parameters. The intelligent feedback execution module dynamically adjusts the closed-loop interaction logic of the agent according to the strategy parameters, adjusts the agent's response action mode, feedback content display format, and active / passive guidance intensity, and uses the updated interaction logic as a new feedback environment to induce subsequent natural interaction behaviors.

[0021] By adopting the above technical solution, the system acquires multimodal natural behavior data through a background monitoring mechanism, constructs a staged cognitive schema model using feature mapping and temporal decay mechanisms, and employs a dual-channel data splitting and mapping operation to meet both the user's reading needs and the execution requirements of the underlying control system. The updated interactive operation logic is used as the output of a new triggering environment, constructing a flow path from data collection and state analysis to policy issuance and the induction of new data, thus achieving a closed-loop update of non-interventional evaluation and interventional guidance.

[0022] Furthermore, the step of extracting the multimodal features from the natural behavior data and transforming the natural behavior data into a behavior feature vector of uniform dimension includes: The multimodal perception and analysis module performs computer vision processing on the image capture data in the natural behavior data, extracting subject category features and spatial composition features, and merging them into an original image feature vector; the multimodal perception and analysis module performs natural language processing on the voice interaction data in the natural behavior data, extracting dialogue semantic features and interaction intent features, and merging them into an original voice feature vector; the multimodal perception and analysis module extracts the operation frequency and trajectory parameters of the function selection data in the natural behavior data, and merges them into an original operation feature vector; the multimodal perception and analysis module uses a feature mapping algorithm to map the original image feature vector, the original voice feature vector, and the original operation feature vector to a unified high-dimensional feature space for normalization processing, transforming them into the behavior feature vector.

[0023] By adopting the above technical solution, the system combines computer vision and natural language processing techniques to extract structured information and retain the core features in multimodal data. Through feature mapping algorithms, it performs projection and normalization processing on the high-dimensional feature space, solving the problem of direct comparison of heterogeneous data structures and providing a standardized numerical basis for similarity calculations of the basic cognitive schema set.

[0024] Furthermore, based on the staged cognitive schema model and the behavioral statistical features of the natural behavioral data, the model activation vector in the staged cognitive schema model is extracted as input, and a staged growth analysis result vector is generated through calculation using a preset mapping transition matrix. The specific steps for outputting the growth analysis results include: The cognitive schema evaluation module compares the schema activation state of each basic cognitive schema in the basic cognitive schema set with a preset activation threshold. Schema activation states below the activation threshold are reset to zero. The reset schema activation states are then sequentially concatenated according to the feature index dimension to reconstruct the model activation vector. The cognitive schema evaluation module uses the model activation vector as a numerical input feature and performs a projection operation with the mapping transition matrix and the basic deviation vector to generate the stage-based growth analysis result vector. The cognitive schema evaluation module writes the behavioral statistical features into the data structure of the stage-based growth analysis result vector to generate the growth analysis result.

[0025] By employing the above technical solution, the system uses the activation determination threshold to perform secondary filtering on the pattern activation state, removing disturbance data generated by low-frequency random actions. The basic deviation vector is introduced to calibrate the developmental baseline constant, ensuring that the stage-based growth analysis result vector generated by the mapping transition matrix accurately reflects the cognitive development level at the current assessment time point.

[0026] This invention provides a growth understanding and intelligent feedback system and method based on children's natural interaction. It has the following beneficial effects: 1. This invention uses a multimodal perception and analysis module to acquire basic data samples by using natural interaction behavior as a trigger condition and transforming them into behavioral feature vectors. It uses a cognitive graph evaluation module to calculate similarity parameters and combines them with behavioral statistical features to generate growth analysis results. This low-intervention data acquisition and analysis architecture avoids the data distortion problem caused by preset tasks, improves the accuracy of growth understanding and feature analysis, and the closed-loop data flow path built based on the intelligent feedback execution module supports the long-term operation of the system, making it suitable for continuous understanding of the object's growth process.

[0027] 2. This invention uses a cognitive schema evaluation module to process similarity parameters by combining age-based weight constraints and time decay variables, and constructs a phased cognitive schema model. By using an activation threshold, the activation state of schemas below the threshold is reset to zero and reconstructed into model activation vectors. This phased modeling method eliminates exploration noise generated by low-frequency random actions, improves the stability of phased growth analysis results, and uses the calculation process of extracting schema feature vectors based on the basic cognitive schema set as an intermediate cognitive model mechanism to enhance the interpretability of the generated growth feature data and analysis results.

[0028] 3. This invention maps interaction strategy parameters into physical control variables, including response action amplitude variables, feedback content graphic-text ratio variables, and proactive prompting frequency variables, through an intelligent feedback execution module. Based on the physical control variables, it dynamically adjusts the agent's interactive operation logic in a closed loop. This strategy feedback mechanism transforms growth analysis results into specific control instructions to adjust the response action mode, feedback content display format, and active / passive guidance intensity. By actively outputting interactive guidance signals as a new feedback environment, it improves the adaptability of agent interaction and the continuity of subsequent interactive behaviors. Attached Figure Description

[0029] Figure 1 This is a structural diagram of a growth understanding and intelligent feedback system based on children's natural interaction, according to an embodiment of the present invention. Figure 2 This is a flowchart of a growth understanding and intelligent feedback method based on children's natural interaction, according to an embodiment of the present invention. Figure 3 This is a comparison chart of the average single interaction dwell time under different interaction strategies according to an embodiment of the present invention; Figure 4 This is a comparison chart of the frequency of active exploration under different interaction strategies in one embodiment of the present invention.

[0030] Among them, 10 is the multimodal perception and parsing module; 20 is the cognitive schema evaluation module; 30 is the intelligent feedback execution module; and 40 is the age-appropriate strategy control module. Detailed Implementation

[0031] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0032] See attached document Figure 1 The present invention provides a growth understanding and intelligent feedback system based on children's natural interaction, including a multimodal perception and analysis module 10, a cognitive schema evaluation module 20, an intelligent feedback execution module 30, and an age-appropriate strategy control module 40.

[0033] The multimodal perception and parsing module 10 is used to acquire natural behavior data during the interaction process and extract features from this data. Natural behavior data includes image capture data, voice interaction data, and function selection data. The multimodal perception and parsing module 10 operates based on daily interaction trigger conditions, acquiring data samples independently of preset test tasks. The multimodal perception and parsing module 10 performs computer vision processing on the image capture data, extracting the subject category features and spatial composition features of the image. The multimodal perception and parsing module 10 performs natural language processing on the voice interaction data, extracting dialogue semantic features and interaction intent features. The multimodal perception and parsing module 10 records the operation frequency and trajectory parameters of the function selection data and transforms the acquired multi-class raw data into a unified-dimensional behavioral feature vector. .

[0034] The cognitive schema evaluation module 20 is communicatively connected to the multimodal perception and parsing module 10. The cognitive schema evaluation module 20 receives behavioral feature vectors and extracts recurring behavioral patterns based on a set time window. The cognitive schema evaluation module 20 internally presets a basic cognitive schema set and calculates the similarity parameter between the behavioral feature vectors and the basic cognitive schema set. Cognitive schema assessment module 20 incorporates time decay variables. Processing similarity parameters The cognitive schema evaluation module 20 outputs the current schema activation state. It aggregates scattered behavioral pattern features into intermediate cognitive model variables. Based on the schema activation state and behavioral statistical features, the cognitive schema evaluation module 20 uses a mapping transition matrix... Generate phased growth characteristic indicators. The cognitive schema assessment module 20 outputs growth analysis results reflecting the current cognitive development dimensions.

[0035] The intelligent feedback execution module 30 is communicatively connected to the cognitive schema evaluation module 20. The intelligent feedback execution module 30 receives the growth analysis results and performs dual-channel data splitting processing. It converts the growth analysis results into readable summary information in natural language format and outputs it to the corresponding display terminal via the first channel. The intelligent feedback execution module 30 compiles the growth analysis results into machine-executable policy parameters and outputs them to the underlying control unit via the second channel. Based on the policy parameters, the intelligent feedback execution module 30 adjusts the interactive operation logic of the intelligent agent. The specific dimensions adjusted by the intelligent feedback execution module 30 include the agent's response action method, the form of feedback content presentation, and the intensity of active and passive guidance.

[0036] The age-segmentation strategy control module 40 is communicatively connected to the multimodal perception and parsing module 10, the cognitive schema evaluation module 20, and the intelligent feedback execution module 30. The age-segmentation strategy control module 40 provides the system's underlying global operational constraint mechanism. The age-segmentation strategy control module 40 receives the object's age parameter. Object age parameter The age segmentation strategy control module 40 obtains the age based on the user's registered profile through a smart terminal, or by performing computer vision age estimation on the user's facial features and combining it with voiceprint features for multimodal dynamic age inference. The age segmentation strategy control module 40 is based on the object's age parameter. The corresponding age group is determined. The age-segmentation strategy control module 40 generates configuration instructions containing data acquisition weights, analysis model parameters, and interaction constraints based on the age group. The age-segmentation strategy control module 40 sends these configuration instructions to the corresponding processing modules. The age-segmentation strategy control module 40 controls the calculation priority of different data input sources, such as images, speech, and operational behaviors, by adjusting the acquisition weights of the multimodal perception and analysis module 10. The age-segmentation strategy control module 40 updates the analysis model parameters of the cognitive schema evaluation module 20 and the response constraints of the intelligent feedback execution module 30 to ensure that the data evaluation standards and actual interaction feedback strategies match the physical exploration tendencies or linguistic logic tendencies of the current age group.

[0037] The various modules of the child-based natural interaction-based growth understanding and intelligent feedback system achieve data flow through an internal data bus or network communication protocol. The system and its modules are deployed on local smart terminal devices, cloud server nodes, or in a distributed computing architecture environment. The local smart terminal device, acting as an intelligent agent, manifests as a robot with a gimbal camera, a smart speaker with a screen, or an early education interactive tablet. The hardware framework of the intelligent agent includes a processor and memory for computation, a microphone array and camera for data acquisition, and a screen display, speaker, and mechanical drive module for executing feedback logic.

[0038] See attached document Figure 2 This invention provides a method for growth understanding and intelligent feedback based on children's natural interaction, comprising the following steps: S10. During the natural interaction process, the image capture data, voice interaction data and function selection data of the object are collected. This step is run by the background trigger mechanism, using natural interaction behavior as the trigger condition, and independently of the preset test task or standardized questionnaire process to obtain multimodal basic data samples. The multimodal perception and analysis module 10 obtains the initial interaction data by recording behavioral events with timestamps. S20. Perform multimodal feature parsing operation on the acquired various behavioral data. The multimodal perception and parsing module 10 performs computer vision processing on the image capture data to extract the subject category and spatial composition features in the image. The multimodal perception and parsing module 10 performs natural language processing on the voice interaction data to extract the structured semantic information and intent features in the interaction process. The multimodal perception and parsing module 10 extracts the operation frequency and trajectory parameters in the function selection data and transforms the collected natural behavioral data into a unified dimension behavioral feature vector. S30. Based on the set time window and behavioral feature vector, identify recurring behavioral patterns and construct a staged cognitive schema model. The cognitive schema evaluation module 20 calculates the similarity parameter between the behavioral feature vector and the preset basic cognitive schema set. The cognitive schema evaluation module 20, in conjunction with the age-based weight constraints and time decay variables issued by the age-based strategy control module 40, calculates the similarity parameter, obtains the activation status of each basic cognitive schema, and abstracts the scattered behavioral data into intermediate cognitive model variables. S40. Based on the stage-based cognitive schema model and related behavioral statistical features, generate stage-based growth analysis results. The cognitive schema evaluation module 20 extracts the schema feature vector that is currently in the active state as input, performs calculations through the preset mapping transition matrix, and generates growth feature data containing multi-dimensional cognitive evaluation indicators to quantify the exploration tendency and state in the current time period. S50 performs a dual-channel split-mapping operation for data execution. The first channel of the intelligent feedback execution module 30 converts the growth analysis results into readable summary information in natural language format, providing phased status text. The second channel compiles the growth analysis results into executable strategy parameters, establishing a transformation path from evaluation results to execution system instructions. S60. The intelligent agent's interactive operation logic is dynamically adjusted in a closed loop according to the interaction strategy parameters. The intelligent feedback execution module 30 reads the interaction strategy parameters to update the original control logic, adjusts the intelligent agent's response action mode, feedback content display form and active and passive guidance intensity. The updated intelligent agent interaction strategy serves as a new feedback environment to induce subsequent interactive behaviors. The newly generated behavioral data is transferred back to step S10 to complete the closed loop flow of data and continuous understanding and updating.

[0039] To enable those skilled in the art to better understand the technical solution and computational logic of the present invention, the core steps in the above method flow will be further elaborated below.

[0040] To achieve accurate acquisition and standardized encapsulation of non-interventional multimodal data, the multimodal perception and parsing module 10 needs to construct an underlying event listening and recording mechanism. In step S10, the specific implementation methods for collecting image capture data, voice interaction data, and function selection data of the acquired object during natural interaction include the following sub-steps: S101. Start the underlying system daemon process to establish a non-interventional data monitoring service. Conventional evaluation systems rely on preset standard test questionnaires or mandatory task guidance interfaces to obtain data. This invention adopts a non-interventional acquisition mechanism, intercepting hardware trigger signals and software interaction events through the operating system's underlying event monitoring interface. Hardware trigger signals include camera activation commands and microphone activation commands. Software interaction events include changes in screen touch coordinates and clicks on interface function keys. The multimodal perception and analysis module 10 defines hardware trigger signals and software interaction events as natural interaction behaviors, and establishes corresponding data capture channels using these natural interaction behaviors as trigger conditions. For the method of calling the underlying event monitoring interface, those skilled in the art can refer to the operating system's general application programming interface specification for configuration, which is well-known in the field and will not be elaborated here.

[0041] S102. Image capture data is captured based on a hardware trigger signal. When a camera call command is detected and the shooting action is completed, the multimodal perception and parsing module 10 obtains the raw still image file output by the image sensor through the hardware abstraction layer. The multimodal perception and parsing module 10 writes the raw still image file as image capture data into the buffer area. The multimodal perception and parsing module 10 assigns a corresponding data type identifier to the image capture data. The multimodal perception parsing module 10 defines the data type identifier for image capture data. This is the first type of identifier.

[0042] S103. Capture voice interaction data based on hardware trigger signals. When a microphone activation command is detected and valid human voice input is detected, the multimodal perception and parsing module 10 starts the audio recording channel. The multimodal perception and parsing module 10 acquires the complete audio stream segment file before the human voice stops. The multimodal perception and parsing module 10 writes the complete audio stream segment file as voice interaction data into the buffer area. The multimodal perception and parsing module 10 assigns a corresponding data type identifier to the voice interaction data. The multimodal perception parsing module 10 defines the data type identifier for voice interaction data. This is the second type of identifier. For the endpoint detection method of valid human voices in the audio recording channel, those skilled in the art use a speech endpoint detection algorithm based on short-time energy and zero-crossing rate. Its feature extraction and boundary determination are well-known techniques in the field and will not be elaborated upon here.

[0043] S104. Capture function selection data based on software interaction events. When a function button click is detected on the interface, the multimodal perception and parsing module 10 records the node hierarchy of the triggering interface and the two-dimensional coordinates of the screen where the click occurred. The multimodal perception and parsing module 10 packages the node hierarchy and the two-dimensional coordinates of the screen into a structured behavior log file. The multimodal perception and parsing module 10 writes the behavior log file as function selection data into the cache area. The multimodal perception and parsing module 10 defines the data type identifier for the function selection data. This is a third type of identifier.

[0044] S105. Construct interactive event entities containing timestamps. This completes the encapsulation of basic data samples. The multimodal perception and parsing module 10 acquires the physical time value output by the system clock in real time and defines the physical time value as the event trigger timestamp. To unify the management of heterogeneous multimodal basic data samples, the multimodal perception and parsing module 10 will assign data type identifiers under the same triggering action. Event trigger timestamp and the original data entities in the cache area Combine them to construct a unified interactive event entity. .

[0045] Interactive event entities The structural definition formula is: ; in, For interactive event entities, A positive integer variable. For the event trigger timestamp, For data type identifiers, This refers to the original data entity.

[0046] Data type identifier The value range includes the first type identifier, the second type identifier, and the third type identifier, when the original data entity When capturing data for an image, the data type identifier The value is the first type identifier, when the original data entity When dealing with voice interaction data, the data type identifier is... The value is the second type identifier, when the original data entity When selecting data for a function, the data type identifier The value is a third-type identifier. This is achieved by constructing an interactive event entity. The multimodal perception and parsing module 10, in an environment completely independent of the preset test task, completed the synchronous acquisition and standardized encapsulation of basic multimodal data samples, providing a standardized data input benchmark for subsequent feature extraction operations.

[0047] After acquiring and encapsulating basic data samples, the multimodal perception and parsing module 10 needs to extract the multi-dimensional features contained therein for subsequent unified recognition and calculation by the model. In step S20, the specific implementation method for performing multimodal feature parsing operations on the acquired various behavioral data includes the following sub-steps: S201, Obtain the interactive event entity According to the interactive event entity Data type identifiers in Perform data distribution processing and establish parallel feature parsing channels to process heterogeneous raw data entities. .

[0048] S202, in data type identifier When the identifier is of type 1, the image feature parsing channel is activated. The image feature parsing channel reads the interaction event entity. The original data entity in The computer vision process is performed. Object detection algorithms are used to extract entities from the original data. The system identifies the subject category and outputs the subject category confidence score. It also extracts the bounding box coordinates of the target subject and calculates the coordinates of the bounding box center point relative to the original data entity. The pixel offset of the center point is used as a spatial composition feature. The subject category confidence and the spatial composition feature are merged into the original image feature vector. For the calculation process of extracting the subject category and bounding box coordinates using the object detection algorithm, those skilled in the art use visual feature extraction and candidate box regression methods based on deep neural networks. The network forward propagation and coordinate decoding operations are well-known technologies in the field and will not be described in detail here.

[0049] S203, in data type identifier When the identifier is of the second type, the speech feature parsing channel is activated. The speech feature parsing channel reads the interaction event entity. The original data entity in Perform natural language processing. Transform the original data entities... The data is converted into a corresponding text sequence. Semantic dependency analysis is performed on the text sequence to extract entity words and part-of-speech tags. The text sequence is then input into an intent classification network, which outputs interaction intent classification features including interrogative, declarative, and imperative intent categories. The word vectors of entity words are merged with the interaction intent classification features to form the original speech feature vector. The speech recognition computation process, which converts speech into a text sequence, is handled by those skilled in the art using Hidden Markov Models or pre-trained acoustic models. The acoustic feature extraction and state decoding operations are well-known techniques in the field and will not be elaborated here.

[0050] S204, in data type identifier When the identifier is of the third type, the behavior trajectory parsing channel is activated. The behavior trajectory parsing channel reads the interaction event entity. The original data entity in Extract the original data entities. The system includes a behavior log file. The frequency of key presses on the user interface recorded in the behavior log file is analyzed. The sequence of two-dimensional screen coordinates recorded in the behavior log file is extracted, and the sum of the Euclidean distances between adjacent two-dimensional screen coordinates is calculated as the touch trajectory length feature. The operation frequency and touch trajectory length feature are merged into a raw operation feature vector.

[0051] S205. Map the extracted heterogeneous original feature vectors to a unified high-dimensional feature space, and output a behavior feature vector of unified dimension. Multi-source heterogeneous raw data entities Since the included feature dimensions differ, a feature mapping algorithm is used to perform dimension normalization processing, transforming the original image feature vector, original speech feature vector, or original operation feature vector into a format that can be uniformly calculated by the subsequent cognitive schema evaluation module 20.

[0052] The feature mapping formula for dimension normalization is: ; in, A unified dimension of behavioral feature vectors; For data type identifiers; For the original feature vector; when When it is a first type identifier, For the original feature vector of the image, when When it is a second type of identifier, For the original feature vector of speech, when When it is a third type of identifier, To manipulate the original feature vector; The feature mapping weight matrix; This is the feature mapping bias vector. The multimodal perception parsing module 10 uses the feature mapping weight matrix... The linear transformation operation transforms the original feature vectors of different dimensions. The input vectors are projected into a unified feature space of the same dimension to establish a standardized computational input source. To reduce the system's hardware storage load and protect interactive privacy, the multimodal perception and parsing module 10 outputs a unified-dimensional behavioral feature vector. Then, the cache cleanup mechanism is triggered, which will clear the original data entities involving audio and images. Physically destroy or overwrite addresses in the cache area while preserving a uniform behavioral feature vector. Participate in subsequent cognitive modeling.

[0053] After completing the parsing and dimensionality normalization of multimodal features, the cognitive schema evaluation module 20 needs to extract behavioral patterns and construct a mathematical model reflecting the current cognitive state by combining time windows. In step S30, the specific implementation method for identifying recurring behavioral patterns and constructing a staged cognitive schema model based on the set time window and behavioral feature vectors includes the following sub-steps: S301. Pre-set a basic cognitive schema set within the cognitive schema evaluation module 20, and configure a corresponding baseline feature vector for each type of basic cognitive schema. The Cognitive Schema Assessment Module 20 divides children's typical exploratory tendencies during development into multiple orthogonal cognitive dimensions, including spatial composition exploration, logical causal testing, and language interaction imitation. The Cognitive Schema Assessment Module 20 defines the set of dimensions as a set of basic cognitive schemas. Each basic cognitive schema in the set of basic cognitive schemas is assigned a high-dimensional mathematical vector as a benchmark feature vector for matching. When the basic cognitive schema is a spatial composition exploration dimension, the corresponding baseline feature vector is... The numerical weights of the spatial pixel offset feature bits are set to be close to one, and the numerical weights of the voice interaction intent bits are set to be close to zero.

[0054] S302. Set a sliding time window to constrain the data extraction range, and extract the total number of valid interaction events within the sliding time window. Cognitive schema assessment module 20 obtains the current assessment timestamp. The effective data calculation interval is determined by combining preset time span parameters. The cognitive schema evaluation module traverses the interactive event entities in the cache area. Extract the event trigger timestamp The total number of valid interaction events is calculated by counting the number of events within the valid data calculation range. .

[0055] S303. Calculate the behavior feature vector of uniform dimension. With reference eigenvectors Similarity parameters between The cognitive schema assessment module 20 extracts uniform-dimensional behavioral feature vectors that fall within the sliding time window. The cosine similarity function is used to calculate its similarity to the baseline feature vector. The distance spatial matching degree is used to output the similarity parameter corresponding to a single event. The calculation of feature vector distance based on cosine similarity is implemented by those skilled in the art using standard inner product space metric methods. The vector dot product and modulus normalization operations are well-known techniques in the field and will not be elaborated here.

[0056] S304, Combining age-based weight constraint parameters With time decay variable Processing similarity parameters Calculate the activation intensity of each basic cognitive schema. As the time of the interaction event increases relative to the current evaluation timestamp... The farther away, the more exponentially the influence of the data on the current cognitive state diminishes. The cognitive schema assessment module 20 introduces the data extraction weights issued by the age-segmentation strategy control module 40 as constraints to ensure that the calculation bias of each modality data at different age stages conforms to the true cognitive tendency.

[0057] Activation strength The specific calculation formula is as follows: ; in, For the intensity of the activation state, The total number of valid interaction events. A positive integer variable. For data type identifiers, For the object's age parameter, These are age-weighted constraint parameters. For a unified dimension of behavioral feature vectors, As the baseline feature vector, For similarity parameters, For time decay variables, This is the current evaluation timestamp. This is the timestamp when the event was triggered.

[0058] S305, Indicate the activation intensity of each basic cognitive schema The system combines and encapsulates data to output a phased cognitive schema model. The cognitive schema evaluation module iterates through each basic cognitive schema in the set of basic cognitive schemas, calculating and obtaining the corresponding activation state strength for each. This abstracts and integrates fragmented, multimodal behavioral data. The cognitive schema evaluation module 20 calculates the intensity of each activation state. Concatenate the sequence to form the model activation vector. Activation vectors through the model To fully quantify the multi-dimensional cognitive development preferences at the current stage, and to establish intermediate cognitive model variables to support subsequent ability indicator assessments.

[0059] After obtaining the intermediate cognitive model variables that quantify current cognitive development preferences, the cognitive schema assessment module 20 needs to further project the abstract model features into specific stage-based growth assessment indicators. In step S40, the specific implementation method for generating stage-based growth analysis results based on the stage-based cognitive schema model and related behavioral statistical characteristics includes the following sub-steps: S401. Construct a mapping and transfer matrix for projecting basic cognitive schema features onto multi-dimensional cognitive growth indicators. The Cognitive Schema Assessment Module 20 includes pre-set multi-dimensional cognitive assessment indicators, encompassing spatial cognition, logical causality, and language expression. Based on theories of children's cognitive development stages, the module configures the correlation weight parameters for each basic cognitive schema with respect to each multi-dimensional cognitive assessment indicator. The module then calculates these correlation weight parameters in a two-dimensional arrangement according to the number of dimensions in both the multi-dimensional cognitive assessment indicators and the basic cognitive schemas, constructing a mapping transition matrix. Based on the theory of children's cognitive development stages, when basic cognitive schemas involve language interaction and imitation, the mapping transition matrix... The correlation weight parameter for the corresponding language expression dimension evaluation index is set to a high weight constant, while the correlation weight parameter for the corresponding logical causality dimension is set to a low weight constant.

[0060] S402. Extracting model activation vectors from staged cognitive schema models. It then performs activation state determination and threshold filtering operations. The cognitive schema evaluation module 20 reads the activation state intensity contained in the staged cognitive schema model. The cognitive schema assessment module has 20 preset activation thresholds, which are used to compare the intensity of activation states. The magnitude of the activation threshold. When the activation state strength... When the activation level is below the activation threshold, the cognitive schema evaluation module 20 classifies it as accidental exploratory noise and adjusts the activation intensity accordingly. Reset to zero. The cognitive schema assessment module 20 will evaluate the intensity of each activation state after judgment and filtering. The sequence is concatenated according to the feature index dimension to reconstruct the model activation vector in column vector form. .

[0061] S403, Combining Model Activation Vectors Mapping transition matrix and the basic deviation vector Perform calculations to generate a vector of phased growth analysis results. The cognitive schema assessment module 20 sets baseline constants for various multi-dimensional cognitive assessment indicators to reflect the developmental baseline of peers, and combines these baseline constants to construct a baseline deviation vector. The cognitive schema assessment module 20 will activate the model's activation vectors. As numerical input features, they are mapped through the transition matrix. Perform spatial projection operations and fuse the fundamental deviation vector. Output comprehensive evaluation indicators.

[0062] The formula for generating the growth analysis results is as follows: ; in, This is a vector representing the results of a phased growth analysis. Let be the mapping transition matrix. For the model activation vector, This is the basic bias vector. Using this calculation formula, the cognitive schema assessment module 20 will use the model activation vector, representing intermediate cognitive model variables. This is transformed into specific numerical values ​​that point to multi-dimensional cognitive assessment indicators. For the multiplication and addition operations of multi-dimensional matrices, those skilled in the art use standard linear algebra libraries for programmed processing; the matrix dot product and dimension alignment operations are well-known techniques in the field and will not be elaborated upon here.

[0063] S404. Obtain relevant behavioral statistical characteristics and vectorize them with the results of the stage-based growth analysis. The system performs joint encapsulation and outputs phased growth analysis results. The cognitive schema assessment module 20 calculates the event trigger frequency, average single interaction time span, and proportional distribution of various natural behavioral data within the valid data calculation interval, defining these statistical items as behavioral statistical features. The cognitive schema assessment module 20 then writes these behavioral statistical features as supplementary parameters into the phased growth analysis result vector. Within the data structure, a phased growth analysis result is generated that fully reflects the exploration tendency and status within the current time period. The cognitive schema evaluation module 20 pushes the generated phased growth analysis result into the shared memory area as a unified input source for subsequent data dual-channel split mapping operations.

[0064] Generate a vector of phased growth analysis results of exploratory tendencies within a quantified time period. Subsequently, the intelligent feedback execution module 30 needs to meet the different needs of front-end user reading and underlying machine execution through a dual-channel mechanism. In step S50, the specific implementation of performing a dual-channel data splitting and mapping operation to perform targeted output processing of the growth analysis results includes the following sub-steps: S501: Monitor the state changes of the shared memory region and extract the phased growth analysis results. The intelligent feedback execution module 30 parses the phased growth analysis result vector contained in the phased growth analysis results. Related behavioral statistical characteristics. The intelligent feedback execution module 30 initializes the first and second processing channels, and vectorizes the phased growth analysis results. The data is synchronously copied to the input buffer queues of the first and second processing channels along with the behavioral statistical features, and an independent dual-channel data splitting and mapping operation is performed.

[0065] S502. Activate the first processing channel to generate user-readable summary information. The first processing channel extracts the vector of phased growth analysis results. The system uses multi-dimensional cognitive assessment indicators. The intelligent feedback execution module 30 internally presets a text evaluation template library corresponding to each multi-dimensional cognitive assessment indicator. The intelligent feedback execution module 30 compares the multi-dimensional cognitive assessment indicator values ​​with the various trigger intervals set in the text evaluation template library, extracting evaluation phrases that match the intervals. Combining the event trigger frequency and average single interaction time span parameters from behavioral statistical features, the intelligent feedback execution module 30 performs semantic combination and feature dimensionality reduction concatenation on the extracted evaluation phrases, outputting readable summary information in natural language format. This provides the guardian's communication device or corresponding associated display terminal with stage-specific status text, helping the guardian intuitively understand the current stage of growth. For the conversion process of generating natural language text based on rule and template matching, those skilled in the art use a string processing algorithm with preset placeholder replacement for configuration. The text concatenation and variable rendering operations are well-known technologies in the field and will not be elaborated upon here.

[0066] S503. Activate the second processing channel and vectorize the phased growth analysis results. Compile into interactive strategy parameters. The second processing channel extracts the vector of phased growth analysis results. and the object's age parameters obtained in real time by the age-segmentation strategy control module 40. The intelligent feedback execution module 30 utilizes an internally preset strategy mapping algorithm to vectorize the phased growth analysis results reflecting the evaluation status. It is transformed into a set of instructions containing multiple underlying control variables, establishing a precise transformation path from evaluation results to execution system instructions.

[0067] S504, The specific calculation of the execution strategy mapping algorithm outputs a feature vector of the control system behavior. The intelligent feedback execution module 30 presets a strategy mapping weight matrix that connects the cognitive evaluation dimension and the interactive execution dimension. Simultaneously, it combines the object's age parameters obtained by the age-segmentation strategy control module 40. Introduce the corresponding age-based constraint bias function Together, we can solve for the interaction strategy parameter vector. .

[0068] The formula for calculating the conversion parameters of the interaction strategy is as follows: ; in, For the interaction strategy parameter vector, The policy mapping weight matrix, This is a vector representing the results of a phased growth analysis. For the object's age parameter, This is the age-based constraint bias function.

[0069] S505, Complete the interaction strategy parameter vector Encapsulation and targeted route distribution. The intelligent feedback execution module 30 parses the interaction strategy parameter vector. The specific dimensional values ​​are mapped to physical control variables for the actual operation of the intelligent agent, including response latency, the ratio of text and images in the feedback content, and the frequency of proactive prompts. The intelligent feedback execution module 30 packages the mapped physical control variables into interactive strategy parameters executable by the underlying machine, and outputs them to the main control bus of the underlying control unit via the second channel, completing the machine-side reconstruction and distribution of data.

[0070] Complete the interaction strategy parameter vector After the underlying machine-side reconstruction and distribution, the intelligent feedback execution module 30 will change the physical behavior of the intelligent agent according to the new control instructions and form a business data closed loop. In step S60, the specific implementation method for the closed-loop dynamic adjustment of the intelligent agent's interaction operation logic according to the interaction strategy parameters includes the following sub-steps: S601, the main control unit of the intelligent agent reads the interaction strategy parameters output by the second channel through the underlying main control bus. The intelligent feedback execution module 30 parses the various physical control variables packaged in the interaction strategy parameters. The intelligent feedback execution module 30 extracts the currently running control state machine in the main control unit. The intelligent feedback execution module 30 uses the physical control variables to replace the default control logic in the control state machine, performing a hot update process for the system control mechanism. For the hot update process of replacing state machine control variables while the device is running, those skilled in the art use memory variable rewriting and multi-threaded safety lock mechanisms for program configuration. The state synchronization and variable overwriting operations are well-known technologies in the field and will not be described further here.

[0071] S602. Based on the updated control state machine, adjust the response action mode of the intelligent agent. The intelligent feedback execution module 30 extracts the response delay duration variable and the response action amplitude variable from the physical control variables. When triggering the next interactive feedback, the intelligent feedback execution module 30 controls the voice output module and the mechanical drive module to execute a delay waiting instruction according to the response delay duration variable. After the delay waiting period ends, the intelligent feedback execution module 30 instructs the mechanical drive module to adjust the motion range value and execution speed parameter of the physical action according to the response action amplitude variable.

[0072] S603. Based on the updated control state machine, adjust the presentation format of the intelligent agent's feedback content. The intelligent feedback execution module 30 extracts the feedback content image-text ratio variable from the physical control variables. When generating the rendering instruction for the next feedback interface, the intelligent feedback execution module 30 allocates the screen ratio of the image display area and the text display area in the multimedia display interface according to the feedback content image-text ratio variable. When the feedback content image-text ratio variable indicates that enhanced visual exploration guidance is needed, the intelligent feedback execution module 30 expands the image display area and proportionally reduces the text display area.

[0073] S604. Based on the updated control state machine, adjust the intensity of active and passive guidance for the intelligent agent. The intelligent feedback execution module 30 extracts the active prompting frequency variable from the physical control variables. The intelligent feedback execution module 30 uses an internal hardware timer to record the idle time without interaction after the interaction stops. When the idle time without interaction exceeds the set wake-up waiting threshold, the intelligent feedback execution module 30 generates corresponding voice prompt wake-up commands and interface pop-up commands based on the active prompting frequency variable, controlling the intelligent agent to change its passive waiting state and actively output interactive guidance signals to the physical environment.

[0074] S605. Establish a closed-loop data flow path for interactive feedback and basic data acquisition. The updated control state machine, combined with the adjusted response action mode, feedback content display format, and active / passive guidance intensity, jointly constructs a new physical performance characteristic of the intelligent agent. The new physical performance characteristic serves as the adjusted feedback environment output to the outside, acting as a new interactive stimulus to induce the object to generate subsequent image capture data, voice interaction data, and function selection data. The multimodal perception and analysis module 10 captures newly generated hardware trigger signals and software interaction events in real time through the underlying non-interventional data monitoring service, and re-transfers the newly generated relevant data to the cache area in step S10 for interactive event entity encapsulation, realizing a closed-loop operation and continuous data update of the entire system from cognitive model understanding to interactive strategy intervention, and then from interactive strategy intervention to induce continuous input of new data samples.

[0075] Specific application examples: To better understand the technical solution of this invention, the invention will be further described in detail below with reference to specific application scenarios and accompanying drawings. This specific application embodiment is built on a growth understanding and intelligent feedback system based on children's natural interaction. This system is deployed in an early education interactive whiteboard device and is responsible for acquiring natural behavioral data during the interaction process and performing intelligent feedback adjustments.

[0076] During the cognitive schema activation state assessment phase, the cognitive schema evaluation module 20 needs to extract events within a sliding time window to calculate the activation state intensity of each basic cognitive schema. The parameters are set as follows: Get the age parameter of the interactive object. The value is 5, and the current evaluation timestamp is [number missing]. The value is 10; extract the total number of valid interaction events within the sliding time window. The event trigger timestamps for 3 events. They are 7, 8, and 9 respectively; set the time decay variable. The value is 0.1; based on the object's age parameter. Define identifiers for each data type. Corresponding age-based weight constraint parameters All values ​​are 1; based on the fundamental cognitive schema of spatial composition dimension, a unified dimension of behavioral feature vector is calculated. With reference eigenvectors Similarity parameters The values ​​are 0.8, 0.4, and 0.9 respectively. The cognitive schema assessment module 20 uses the formula to calculate the activation intensity of the spatial composition dimension. : ; The numerical calculation process is as follows: ; The calculation results show that, for the spatial mapping dimension, the system obtains the activation state strength. The value is 1.73. Similarly, the activation state strength of the language interaction dimension is calculated. The value is 1.07. The Cognitive Schema Assessment Module 20 measures the intensity of various activation states. Concatenate the sequences to construct the model activation vector. for .

[0077] Subsequently, the cognitive schema assessment module 20 performs the step of projecting intermediate cognitive model variables into comprehensive assessment indicators to generate a vector of stage-based growth analysis results. The parameters are set as follows: The preset baseline deviation vector reflects the developmental baseline of peers. for The pre-defined mapping and transfer matrix projects basic cognitive schema features onto multi-dimensional cognitive growth indicators. for The cognitive schema assessment module uses 20 inputs into a formula to calculate the vector of stage-based growth analysis results. : ; The numerical calculation process is as follows: ; Calculation results show that the vector of phased growth analysis results within the current time period The values ​​for the two indicators are 6.60 and 5.14, respectively.

[0078] Next, the intelligent feedback execution module 30 performs the data flow and low-level control instruction compilation steps to output the interaction strategy parameter vector. The parameters are set as follows: Extract the vector of results from the phased growth analysis. for The strategy mapping weight matrix connecting the cognitive evaluation dimension and the interaction execution dimension Set as The corresponding age parameter for a 5-year-old object Age-constrained bias function The value vector is The intelligent feedback execution module 30 substitutes the input into the formula to calculate the interaction strategy parameter vector. : ; The numerical calculation process is as follows: ; The calculation results show that the interaction strategy parameter vector The two feature values ​​are 4.30 and 5.08, respectively. The intelligent feedback execution module 30 reads the interaction strategy parameter vector. Update the ratio of images and text in the feedback content of the early education interactive whiteboard and the response latency, control the intelligent agent to change its passive waiting state and complete the closed-loop dynamic adjustment of the interactive operation logic.

[0079] See attached document Figure 3 , Figure 3 The horizontal axis represents the number of test days, and the direction of the horizontal axis represents the passage of test time. The vertical axis represents the average duration of a single interaction. The dashed line with hollow circles in the figure represents the static interaction strategy, which, due to the lack of a dynamic update mechanism for the feedback environment, exhibits a decreasing average duration of a single interaction over time. The solid line with solid squares in the figure represents the closed-loop dynamic adjustment strategy of this invention, which, due to the continuous generation of interaction parameters matching the current cognitive state, shows a steady upward trend in average duration of a single interaction. This result verifies that the system of this invention has the ability to maintain and improve the focus of the interactive object.

[0080] See attached document Figure 4 , Figure 4 The horizontal axis represents the number of test days, and the direction of the horizontal axis represents the progression of test time. The vertical axis represents the daily frequency of active exploration. The dashed lines with hollow triangles in the figure represent the static interaction strategy, which causes the frequency of active exploration by the object to fluctuate within a basic range. The solid lines with filled rhombuses in the figure represent the closed-loop dynamic adjustment strategy of this invention, which adjusts the intensity of the agent's active and passive guidance in real time, inducing the object's continuously increasing autonomous function selection behavior. This result verifies that this invention has significant technical advantages in improving device stickiness and promoting active exploration by the object.

Claims

1. A growth understanding and intelligent feedback system based on children's natural interaction, characterized in that, include: The multimodal perception and parsing module (10) is used to acquire natural behavior data, extract the multimodal features of the natural behavior data and convert them into behavior feature vectors; The cognitive schema evaluation module (20) is used to calculate the similarity parameter between the behavioral feature vector and the preset basic cognitive schema set, and to generate growth analysis results by combining the similarity parameter with the behavioral statistical features of the natural behavioral data. The intelligent feedback execution module (30) is used to convert the growth analysis results into readable summary information and strategy parameters, output the readable summary information, and adjust the interactive operation logic of the agent based on the strategy parameters. The age-segmentation strategy control module (40) is used to receive the object's age parameters, generate configuration instructions according to the age stage corresponding to the object's age parameters, and use the configuration instructions to update the operating parameters of the multimodal perception and parsing module (10), the cognitive graph evaluation module (20), and the intelligent feedback execution module (30).

2. The growth understanding and intelligent feedback system based on children's natural interaction according to claim 1, characterized in that, The natural behavior data includes image capture data, voice interaction data, and function selection data; The multimodal perception and parsing module (10) extracts the subject category features and spatial composition features of the image capture data and merges them into the original image feature vector. It also extracts the dialogue semantic features and interaction intent features of the voice interaction data and merges them into the original voice feature vector. Furthermore, it extracts the operation frequency and trajectory parameters of the function selection data and merges them into the original operation feature vector. Finally, it performs normalization processing on the original image feature vector, the original voice feature vector, and the original operation feature vector to transform them into the behavior feature vector of a unified dimension.

3. The growth understanding and intelligent feedback system based on children's natural interaction according to claim 1, characterized in that, The cognitive schema evaluation module (20) processes the similarity parameter in conjunction with the time decay variable and obtains the schema activation state. The schema activation state that is lower than the preset activation judgment threshold is reset to zero and reconstructed into a model activation vector. The preset mapping transition matrix is ​​used to perform projection operation on the model activation vector to generate a stage growth analysis result vector. The behavioral statistical features are written into the data structure of the stage growth analysis result vector to generate the growth analysis result.

4. The growth understanding and intelligent feedback system based on children's natural interaction according to claim 1, characterized in that, The interactive operation logic of the intelligent agent includes the response action method, the form of feedback content display, and the intensity of active and passive guidance. The intelligent feedback execution module (30) maps the strategy parameters to physical control variables including response action amplitude variables, feedback content graphic ratio variables, and active prompting frequency variables. Based on the response action amplitude variables, it adjusts the response action mode, based on the feedback content graphic ratio variables, it adjusts the feedback content display format, and based on the active prompting frequency variables, it adjusts the active and passive guidance intensity.

5. A growth understanding and intelligent feedback system based on children's natural interaction according to claim 4, characterized in that, When the intelligent feedback execution module (30) converts the growth analysis results into the strategy parameters, it introduces the corresponding age-based constraint bias function in combination with the object age parameter, and uses the preset strategy mapping algorithm and strategy mapping weight matrix to convert the growth analysis results into a strategy parameter vector and package it into the strategy parameters. The intelligent feedback execution module (30) parses the specific dimension values ​​of the strategy parameter vector and maps them to generate the physical control variables.

6. A growth understanding and intelligent feedback system based on children's natural interaction according to claim 4, characterized in that, The intelligent feedback execution module (30) controls the intelligent agent to adjust the motion range value and execution speed parameter of the physical action based on the response action amplitude variable in the physical control variable; The intelligent feedback execution module (30) allocates the ratio of the image display area to the text display area in the display interface according to the feedback content graphic ratio variable in the physical control variables; The intelligent feedback execution module (30) records the idle time without interaction. When the idle time without interaction exceeds the set wake-up waiting threshold, it generates an interaction guidance instruction based on the active prompting frequency variable in the physical control variable, and controls the intelligent agent to actively output the interaction guidance signal.

7. A growth understanding and intelligent feedback system based on children's natural interaction according to claim 1, characterized in that, The configuration instructions include data acquisition weights, analysis model parameters, and interaction constraints. The age-based strategy control module (40) updates the calculation priority of the multimodal perception and analysis module (10) using the data acquisition weights, updates the evaluation criteria of the cognitive schema evaluation module (20) using the analysis model parameters, and updates the feedback strategy of the intelligent feedback execution module (30) using the interaction constraints.

8. A method for growth understanding and intelligent feedback based on children's natural interaction, characterized in that, The system applied to a child-based natural interaction-based growth understanding and intelligent feedback system as described in any one of claims 1-7 includes the following steps: During the natural interaction process, the multimodal perception and analysis module (10) collects the image capture data, voice interaction data and function selection data of the object, and uses the natural interaction behavior as the trigger condition to obtain multimodal natural behavior data; The multimodal perception and parsing module (10) extracts the multimodal features of the natural behavior data and transforms the natural behavior data into a behavior feature vector of a unified dimension; The cognitive schema evaluation module (20) calculates the similarity parameter between the behavioral feature vector and the preset basic cognitive schema set based on the set time window, and calculates the similarity parameter in combination with the age weight constraint and time decay variable provided by the age strategy control module (40), obtains the schema activation state of each basic cognitive schema in the basic cognitive schema set and constructs a staged cognitive schema model. The cognitive schema evaluation module (20) extracts the model activation vector in the staged cognitive schema model as input based on the staged cognitive schema model and the behavioral statistical features of the natural behavior data, performs calculations through the preset mapping transition matrix to generate the staged growth analysis result vector, and outputs the growth analysis result. The intelligent feedback execution module (30) performs a dual-channel data splitting and mapping operation on the growth analysis results. The first channel converts the growth analysis results into readable summary information in natural language format, and the second channel compiles the growth analysis results into executable strategy parameters. The intelligent feedback execution module (30) dynamically adjusts the closed-loop operation logic of the agent according to the strategy parameters, adjusts the agent's response action mode, feedback content display form and active and passive guidance intensity, and uses the updated interaction operation logic as a new feedback environment to induce the subsequent natural interaction behavior.

9. A method for growth understanding and intelligent feedback based on children's natural interaction according to claim 8, characterized in that, The steps of extracting the multimodal features from the natural behavior data and transforming the natural behavior data into a behavior feature vector of uniform dimension specifically include: The multimodal perception and analysis module (10) performs computer vision processing on the image capture data in the natural behavior data, extracts the subject category features and spatial composition features, and merges them into the original image feature vector; The multimodal perception and parsing module (10) performs natural language processing on the voice interaction data in the natural behavior data, extracts dialogue semantic features and interaction intent features, and merges them into the original voice feature vector; The multimodal perception and parsing module (10) extracts the operation frequency and trajectory parameters of the function selection data in the natural behavior data and merges them into the original operation feature vector; The multimodal perception and parsing module (10) uses a feature mapping algorithm to map the original image feature vector, the original speech feature vector, and the original operation feature vector to a unified high-dimensional feature space for normalization processing, and transforms them into the behavior feature vector.

10. A method for growth understanding and intelligent feedback based on children's natural interaction according to claim 8, characterized in that, Based on the staged cognitive schema model and the behavioral statistical features of the natural behavioral data, the steps of extracting the model activation vector from the staged cognitive schema model as input, performing calculations through a preset mapping transition matrix to generate a staged growth analysis result vector, and outputting the growth analysis results specifically include: The cognitive schema evaluation module (20) compares the schema activation state of each basic cognitive schema in the basic cognitive schema set with the preset activation judgment threshold, resets the schema activation state below the activation judgment threshold to zero, and concatenates each of the reset schema activation states according to the feature index dimension to reconstruct the model activation vector. The cognitive schema evaluation module (20) takes the model activation vector as a numerical input feature, combines the mapping transition matrix and the basic deviation vector to perform projection operation, and generates the stage growth analysis result vector. The cognitive schema assessment module (20) writes the behavioral statistical features into the data structure of the stage growth analysis result vector to generate the growth analysis result.

Images