Companion robot intelligence processing method and device

Abstract

This invention relates to an intelligent processing method and device for companion robots, belonging to the field of data processing. It collects data on the target user's eye movements, facial expressions, body movements, interactive responses, and environmental interference using multimodal sensors, and combines this with received cognitive load data. A first dispersion probability is calculated using a predefined dispersion probability mapping table. Then, by comparing the deviation arrays of environmental interference and cognitive load, the dispersion probability correction vector of historical samples is retrieved for adjustment. Finally, when the adjusted dispersion probability exceeds a threshold, attention correction is performed. This invention solves the technical problem of insufficient multimodal data processing capabilities in companion robots, making it difficult to accurately assess the user's focus state in real time and adaptively adjust interaction strategies, thus affecting the quality of assisted childcare and task execution efficiency. It achieves the technical effect of accurately perceiving changes in a child's state during focused tasks and adjusting interaction strategies in a timely manner to improve the quality of assisted childcare and task execution efficiency.

Description

Technical Field

[0001] This invention relates to the field of data processing, and more particularly to intelligent processing methods and equipment for companion robots. Background Technology

[0002] With the rapid development of artificial intelligence technology, intelligent applications in family childcare scenarios are receiving increasing attention. Child companion robots, as an important auxiliary tool for family childcare, can not only provide educational and entertainment content, but also play a role in safety monitoring and emotional support to a certain extent.

[0003] However, existing child companion robots still have certain limitations in their functionality, particularly in accurately sensing children's states and effectively adjusting companionship strategies to improve the quality of companionship. Further exploration and improvement are needed in these areas. During family childcare, children often need to switch between different tasks, especially when performing tasks requiring high concentration, such as studying, reading, or drawing. Children are prone to distraction and fatigue. While existing child companion robots can provide some entertainment and educational content, they often lack real-time perception and effective intervention of children's states. They suffer from insufficient multimodal data processing capabilities, making it difficult to accurately assess the user's concentration and adaptively adjust interaction strategies, thus impacting the quality of assisted childcare and task execution efficiency. Summary of the Invention

[0004] This invention addresses the technical problem in existing technologies where companion robots lack multimodal data processing capabilities, making it difficult to accurately assess user focus and adaptively adjust interaction strategies in real time, thus affecting the quality of assisted childcare and task execution efficiency. The invention provides an intelligent processing method and device for companion robots to solve this problem.

[0005] The technical solution of the present invention to solve the above-mentioned technical problems is as follows:

[0006] In a first aspect, the present invention provides an intelligent processing method for companion robots, comprising: processing an eye-tracking structured array, an expression structured array, a limb structured array, and an interaction response structured array through a dispersion probability mapping table to obtain a first dispersion probability, wherein the dispersion probability mapping table has an environmental interference structured reference array and a cognitive load structured reference array; comparing the environmental interference structured array and the environmental interference structured reference array to obtain an environmental interference structured deviation array; comparing the cognitive load structured array and the cognitive load structured reference array to obtain a cognitive load structured deviation array; retrieving dispersion probability correction vectors of historical samples that satisfy the dispersion probabilities of the environmental interference structured deviation array and the cognitive load structured deviation array; adjusting the first dispersion probability to obtain a second dispersion probability; and if the second dispersion probability is greater than or equal to a probability threshold, performing attention correction for the robot.

[0007] In a second aspect, the present invention provides an electronic device, comprising: a memory for storing computer software programs; and a processor for reading and executing the computer software programs, thereby realizing the intelligent processing method for companion robots as described in the first aspect.

[0008] The beneficial effects of this invention are as follows: By collecting eye movement, facial expression, body behavior, interaction response, and environmental interference data of the target user through a multimodal sensor, and combining it with the received cognitive load data, a first dispersion probability is calculated using a predefined dispersion probability mapping table. Then, by comparing the deviation array of environmental interference and cognitive load, the dispersion probability correction vector of historical samples is retrieved and adjusted. Finally, when the adjusted dispersion probability exceeds a threshold, attention correction is performed. This achieves the technical effect of accurately perceiving changes in the child's state during focused tasks and adjusting the interaction strategy in a timely manner to improve the quality of assisted childcare and the efficiency of task execution. Attached Figure Description

[0009] Figure 1 This is a flowchart illustrating the intelligent processing method for companion robots provided by the present invention.

[0010] Figure 2 This is a schematic diagram of the structure of the electronic device provided by the present invention.

[0011] Explanation of reference numerals in the attached drawings: Electronic device 500, memory 510, processor 520, computer program 511. Detailed Implementation

[0012] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0013] In the description of this invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0014] In the description of this invention, the term "for example" is used to mean "used as an example, illustration, or description." Any embodiment described as "for example" in this invention is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use the invention. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that the invention can be made without using these specific details. In other instances, well-known structures and processes will not be described in detail to avoid obscuring the description of the invention with unnecessary detail. Therefore, the invention is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed herein. Example

[0015] like Figure 1 As shown, this embodiment of the invention provides an intelligent processing method for companion robots, including:

[0016] S10: By processing the eye-tracking structured array, facial expression structured array, body structured array, and interaction response structured array through the dispersion probability mapping table, a first dispersion probability is obtained, wherein the dispersion probability mapping table has an environmental interference structured reference array and a cognitive load structured reference array.

[0017] For example, in a home-based childcare scenario, when a companion robot provides assisted companionship and monitoring, it processes the collected multimodal data using a dispersion probability mapping table to accurately assess the focus state of the target user (i.e., the child). The dispersion probability mapping table is pre-set based on in-depth research and data analysis of the relationship between children's multimodal behavioral characteristics (such as eye movements, facial expressions, body movements, and interactive responses) and environmental interference and cognitive load during focus tasks. By establishing a correspondence between mapping factor data and dispersion probability calibration values, the table scientifically quantifies the degree of children's distractibility in different states, providing a benchmark reference for the robot to assess children's focus state.

[0018] The dispersion probability map incorporates a structured baseline array of environmental interference and a structured baseline array of cognitive load as reference standards for assessing users' distractibility. The structured baseline array of environmental interference contains baseline data describing the potential impact of environmental factors (such as noise levels, light intensity, temperature, and humidity) on children's concentration; the structured baseline array of cognitive load covers baseline data reflecting children's cognitive resource utilization (such as information processing speed and working memory load) when performing focused tasks. Together, they provide a reference standard for assessing children's concentration status within the dispersion probability map.

[0019] Once the robot collects structured arrays of the target user's eye movements (such as fixation duration and frequency of gaze deviation), facial expressions (such as the frequency of frowning and drooping corners of the mouth), body posture (such as changes in sitting posture and frequency of getting up from the seat), and interaction response (such as command response delay and task error rate) through multimodal sensors (such as cameras and microphones), it will compare and analyze these data with the corresponding benchmark arrays in the dispersion probability mapping table.

[0020] In the specific comparative analysis process, the robot first extracts several mapping factor data and their corresponding dispersion probability calibration values ​​from the dispersion probability mapping table. Then, by traversing these mapping factor data, it compares them one by one with the collected multimodal structured array to find consistent mapping relationships. If a completely consistent mapping factor data is found, its corresponding dispersion probability calibration value is directly extracted as the first dispersion probability. For example, if the eye-tracking structured array shows that the child's gaze frequently deviates from the target area, and the facial expression structured array shows an increased frequency of frowning, while the limb structured array shows frequent changes in sitting posture, and these features match a mapping factor data with a high dispersion probability in the mapping table, the robot will determine that the current first dispersion probability is high, indicating that the child may be in a distracted state.

[0021] If no perfectly consistent mapping factor data is found, the robot will further process the collected multimodal structured array using a dispersion probability mapping model associated with the dispersion probability mapping table to calculate the first dispersion probability. The dispersion probability mapping model is trained on a large amount of multimodal data from children in attentional tasks, and can more accurately reflect the degree of children's distraction in different states. Through this series of processes, the robot can obtain a scientific and objective first dispersion probability value, providing a strong basis for subsequent attention correction strategies.

[0022] S20: Compare the structured array of environmental interference with the structured baseline array of environmental interference to obtain the structured deviation array of environmental interference; compare the structured array of cognitive load with the structured baseline array of cognitive load to obtain the structured deviation array of cognitive load.

[0023] Preferably, to accurately assess the impact of environmental disturbances and cognitive load on the target user's concentration, the robot performs two key comparison operations. First, the robot compares the collected structured array of environmental disturbances with a preset structured baseline array of environmental disturbances. The structured array of environmental disturbances contains various parameters affecting children's concentration in the actual environment, such as noise levels, light intensity, temperature, and humidity, while the structured baseline array represents the corresponding parameter values ​​under ideal or standard conditions. By comparing these two sets of data, the robot can calculate a structured deviation array of environmental disturbances, which reflects the degree of difference between the actual environment and the standard environment. For example, if the actual noise level is much higher than the baseline value, this significant difference will be recorded in the deviation array.

[0024] Simultaneously, the system receives a structured array of the target user's cognitive load. Specifically, this structured array refers to a set of structured data containing two core indicators: information processing speed and working memory occupancy. Information processing speed reflects the efficiency of the target user's brain in receiving, processing, and outputting information while performing the current task. For example, when solving math problems, children may need to quickly understand the problem, perform logical reasoning, and arrive at the answer; the speed of information processing directly affects the efficiency and accuracy of task completion. Working memory occupancy measures the target user's capacity to store and process information related to the current task in their working memory. For example, when performing multi-step calculations, children need to maintain multiple intermediate results in their working memory simultaneously; excessively high working memory occupancy may lead to information omissions or calculation errors.

[0025] By receiving and analyzing structured arrays, companion robots can understand the cognitive state of target users when performing focused tasks in real time. This provides an important basis for subsequent processing of data collected by multimodal sensors through a predefined dispersion probability mapping table and for adjusting companionship strategies accordingly. For example, if the cognitive load structured array shows a significant decrease in information processing speed and an excessively high working memory occupancy rate, the companion robot can infer that the target user may be facing significant cognitive stress and then take corresponding measures, such as adjusting the task difficulty, providing additional prompts, or arranging short breaks, to help the target user recover to their optimal cognitive state.

[0026] The robot then performs a second comparison, comparing the collected structured array of cognitive load with a preset structured benchmark array of cognitive load. The structured array records the child's cognitive resource usage during focused tasks, such as information processing speed and working memory load, while the benchmark array represents the child's cognitive load level under ideal focused conditions. Through this comparison, the robot can derive a structured deviation array of cognitive load, revealing the degree of deviation between the child's current cognitive load and the ideal state. For example, if the child's information processing speed during the task is significantly lower than the benchmark, the deviation array will reflect this excessive cognitive load. These two sets of deviation arrays provide crucial information for the robot to adjust its companionship strategies accordingly.

[0027] S30: Retrieve the dispersion probability correction vector of the historical samples that satisfy the dispersion probability of the environmental interference structured deviation array and the cognitive load structured deviation array, adjust the first dispersion probability to obtain the second dispersion probability, and if it is greater than or equal to the probability threshold, perform robot attention correction.

[0028] In detail, to more accurately assess the focus status of target users, a search is performed in the historical sample database based on the structured bias arrays of environmental interference and cognitive load. The historical sample database stores historical samples of dispersion probabilities and their dispersion probability correction vectors under different combinations of environmental interference and cognitive load biases during past task executions. The dispersion probability correction vector is a set of values ​​reflecting how the initial dispersion probability (i.e., the first dispersion probability) should be adjusted under specific environmental interference and cognitive load bias conditions.

[0029] The robot compares the structured bias arrays of environmental disturbances (such as noise level deviation and light intensity deviation) and cognitive load deviations (such as information processing speed deviation and working memory load deviation) in the current task, and then filters historical samples with matching dispersion probabilities from a historical sample database. It then extracts the dispersion probability correction vectors corresponding to these historical samples and uses these vectors to correct the first dispersion probability. For example, if the environmental disturbance bias in the current task shows that noise is significantly higher than the baseline value, and the cognitive load bias shows that the child's information processing speed has decreased, the robot will retrieve historical samples with similar combinations of biases and apply their dispersion probability correction vectors to adjust the first dispersion probability accordingly. This results in a second dispersion probability that more closely reflects the actual situation, providing a more accurate basis for subsequent attention correction strategies.

[0030] To ensure the target user maintains a high level of focus, the robot compares a calculated second dispersion probability with a preset dispersion probability threshold. The second dispersion probability is the result of adjusting the first dispersion probability after comprehensively considering the structured bias arrays of environmental interference and cognitive load, and retrieving the dispersion probability correction vectors from historical samples of the corresponding dispersion probabilities. It more accurately reflects the degree of attentional distraction the user may experience in the current task environment. When the robot determines that the second dispersion probability is greater than or equal to the preset probability threshold, it indicates that the current task environment or the user's cognitive state has significantly affected focus. At this point, the robot will automatically trigger and execute predefined attention corrections. These attention corrections may include adjusting the task difficulty, providing additional prompts or guidance, changing the interaction method (such as guiding the user to refocus through voice prompts or visual feedback), or even pausing the task to allow the user to rest when necessary. These measures aim to help the user refocus and ensure that the focused task can be completed efficiently and smoothly. For example, if the robot detects that the second dispersion probability exceeds the threshold while monitoring a child's learning task, it may reduce the difficulty of subsequent questions or encourage the child to take a short break before continuing through voice prompts, thereby maintaining the child's focus.

[0031] In a preferred embodiment, a first dispersion probability is obtained by processing the eye-tracking structured array, facial expression structured array, limb structured array, and interaction response structured array through a dispersion probability mapping table. This includes, beforehand, collecting the target user's eye-tracking structured array, facial expression structured array, limb structured array, interaction response structured array, and environmental disturbance structured array through multimodal sensors deployed on the robot when the companionship task belongs to the focus task type.

[0032] Preferably, companionship tasks refer to a series of tasks performed by the companion robot aimed at interacting with the target user (child), providing companionship and support. These tasks are not limited to simple entertainment activities but also cover multiple aspects such as education, emotional communication, and safety monitoring. The goal of the companion robot is to enhance interaction with the user, improve the user experience, and promote the user's growth and development through these tasks. Focused tasks are a specific subset of companionship tasks, referring to tasks that require the target user to concentrate highly and invest significant cognitive resources. These tasks typically have a significant impact on the user's cognitive abilities, learning outcomes, or skill improvement, thus requiring special attention to the user's focus. Focused tasks can include learning activities that require prolonged concentration, such as reading, writing, drawing, and mathematical calculations. When performing these tasks, the companion robot collects user data through multimodal sensors to assess the user's focus and, when necessary, implements attention correction rules to help the user maintain focus.

[0033] When the companionship task is identified as a focused task, the companion robot will activate specific intelligent processing methods to respond. Specifically, it uses camera sensors to capture the target user's eye movement characteristics. By analyzing the duration of gaze fixation, the frequency of gaze deviation from the target area, and the blinking frequency within a preset time, it constructs a structured array of eye movements reflecting the user's visual attention. For example, if a child blinks frequently or their gaze wanders while reading, this data will be recorded and reflected in the structured eye movement array. Furthermore, the camera captures changes in the user's facial expressions, including the frequency of yawning, frowning, downturned mouth, and head turning within a preset time, thus forming a structured array of facial expressions to assess the user's emotional state and fatigue level. For example, continuous yawning may indicate drowsiness. In addition, the robot also monitors the user's body movements through the camera, recording changes in sitting posture, finger playing or leg shaking, and the frequency of getting up from the seat within a preset time, constructing a structured array of body movements to understand the user's physical state and possible signs of distraction. For example, frequent getting up from the seat may indicate a decline in the user's interest in the current task. Simultaneously, the robot analyzes the user's response delay to commands and the frequency of sudden increases in task error rates to form a structured array of interaction responses, used to evaluate the interaction efficiency and focus between the user and the robot. To comprehensively consider the impact of environmental factors on user focus, the robot uses a microphone to collect decibel values ​​of sudden noise, and combines this with blue light intensity and ambient temperature and humidity data collected by temperature and humidity sensors and light sensors to construct a structured array of environmental interference. For example, high noise levels or unsuitable lighting conditions can negatively impact user focus. Through this series of multimodal data collection and structured processing, the robot can comprehensively understand the target user's state during focus tasks, providing a solid data foundation for subsequent analysis and intervention.

[0034] In a preferred embodiment, a multimodal sensor deployed on the robot collects structured arrays of the target user's eye movements, facial expressions, body language, interaction responses, and environmental interference. This includes: using the robot's camera to collect the target user's gaze duration, frequency of gaze deviation from the target area, and blink frequency for a preset duration, constructing a ternary array, which is set as the eye movement structured array; and using the robot's camera to collect the target user's preset duration yawning and / or frowning frequency, preset duration mouth drooping frequency, and preset duration head turning frequency, constructing a ternary array, which is set as the... The system includes: a facial expression structured array; a ternary array (set as the limb structured array) constructed by collecting data from the robot's camera on the target user's preset duration of sitting posture changes, preset duration of finger playing and / or leg shaking, and preset duration of leaving the seat; a binary array (set as the interaction response structured array) constructed by collecting data from the robot's camera on the target user's command response delay duration and task error rate spike frequency; and a quaternary array (set as the environmental interference structured array) constructed by collecting data from the robot's microphone on sudden noise decibel values ​​and from the robot's temperature and humidity sensors and light sensors.

[0035] Optionally, to accurately assess the target user's focus state, the robot utilizes its own multimodal sensors to collect multi-dimensional data. Specifically, through the robot's camera—a visual sensor—the robot can capture various behavioral characteristics of the target user: For eye movements, it collects the duration of gaze fixation (reflecting the user's level of attention to a specific area), the frequency of gaze deviation from the target area (reflecting distraction), and the blinking frequency within a preset time (related to cognitive load or fatigue). These three data points are then constructed into a ternary array as a structured array for eye movements. For facial expressions, it collects the frequency of yawning and / or frowning (indicating drowsiness or confusion), the frequency of downturned corners of the mouth (reflecting negative emotions), and the frequency of head turning (indicating a shift in attention) within a preset time, also constructing a ternary array as a structured array for facial expressions. For body movements, it collects the frequency of posture changes within a preset time (reflecting instability or distraction), the frequency of finger playing and / or leg shaking (indicating anxiety or boredom), and the frequency of getting up from the seat (directly reflecting distraction), constructing a ternary array as a structured array for body movements. In addition, by monitoring the target user's interaction response when performing tasks through the camera, the system collects the instruction response delay duration (reflecting cognitive processing speed) and the frequency of sudden increases in task error rate (indicating an increase in errors caused by inattention), and constructs a binary array as a structured array of interaction responses.

[0036] Meanwhile, the robot also uses other types of sensors to capture environmental interference factors: it collects sudden noise decibel values ​​through a microphone (reflecting noise interference in the environment), collects ambient temperature and humidity through a temperature and humidity sensor (affecting user comfort and potentially concentration), and collects blue light intensity through a light sensor (excessive blue light may interfere with user visual comfort and attention). These three environmental data points, together with the noise decibel values, are constructed into a quaternion array as a structured array of environmental interference.

[0037] By collecting and analyzing these structured arrays, the robot can comprehensively and objectively assess the user's focus, providing data support for subsequent attention correction strategies. For example, if the eye-tracking structured array shows that the user's gaze frequently deviates from the target area, and the facial expression structured array shows an increased frequency of frowning, while the environmental interference structured array shows a high noise level, the robot can comprehensively determine that the user may be distracted by environmental interference and then take corresponding measures to improve the user's focus.

[0038] In a preferred embodiment, the first dispersion probability is obtained by processing the eye-tracking structured array, the facial expression structured array, the body structured array, and the interaction response structured array through a predefined dispersion probability mapping table, including: extracting several mapping factor data and several dispersion probability calibration values ​​from the dispersion probability mapping table; traversing the several mapping factor data and comparing them with the eye-tracking structured array, the facial expression structured array, the body structured array, and the interaction response structured array to obtain a comparison result; when the comparison result has consistent mapping factor data, extracting the corresponding dispersion probability calibration value from the several dispersion probability calibration values ​​and setting it as the first dispersion probability.

[0039] Specifically, in the process of the companion robot assessing the focus state of the target user, in order to quantify the user's current degree of attention distraction, the robot will use a predefined dispersion probability mapping table to process the collected multimodal structured array.

[0040] The robot first extracts several mapping factor data and their corresponding dispersion probability calibration values ​​from the dispersion probability mapping table. The mapping factor data represents the attention distraction patterns that may occur under different combinations of behavioral characteristics. These data are determined after training with a large amount of data and can reflect the correlation between different behavioral characteristics and the degree of attention distraction. The dispersion probability calibration values ​​are pre-set probabilities of these patterns occurring. Next, the robot traverses these mapping factor data and compares them one by one with the collected eye movement structured arrays (such as fixation point dwell time, frequency of gaze deviation, etc.), facial expression structured arrays (such as frowning frequency, frequency of drooping corners of the mouth, etc.), body structured arrays (such as sitting posture change frequency, frequency of getting up from the seat, etc.), and interaction response structured arrays (such as command response delay time, frequency of sudden increase in task error rate, etc.).

[0041] During the comparison process, the robot searches for mapping factor data that matches the collected multimodal structured array. Once a matching mapping factor data is found, the robot extracts the corresponding dispersion probability calibration value from the pre-extracted dispersion probability calibration values ​​and sets it as the current first dispersion probability. For example, if the eye-tracking structured array shows frequent gaze deviations, the facial expression structured array shows increased frowning frequency, the body structured array shows frequent changes in posture, and the interaction response structured array shows increased command response delay, and these features perfectly match a mapping factor data in the dispersion probability mapping table, then the robot will use the dispersion probability calibration value corresponding to that mapping factor data as the first dispersion probability, thereby reflecting the user's current level of attentional distraction.

[0042] In a preferred embodiment, the method further includes: when the comparison result has inconsistent mapping factor data, processing the eye-tracking structured array, the facial expression structured array, the limb structured array, and the interaction response structured array through a dispersion probability mapping model associated with the dispersion probability mapping table to obtain the first dispersion probability; and storing the processed eye-tracking structured array, facial expression structured array, limb structured array, and interaction response structured array as updated mapping factor data, and storing the first dispersion probability as updated dispersion probability; and updating the dispersion probability mapping table based on the updated mapping factor data and the updated dispersion probability.

[0043] Furthermore, if a comparison is performed using a predefined dispersion probability mapping table to examine the multimodal structured arrays (including eye-tracking, facial expression, body language, and interaction response structured arrays) against the mapping factor data, and no consistent mapping factor data is found in the comparison results, it indicates that the current user's behavioral feature combination is not fully covered by the mapping table. In this case, the robot will activate the dispersion probability mapping model associated with the dispersion probability mapping table to process these structured arrays. The dispersion probability mapping model, built on machine learning algorithms, can comprehensively consider multiple factors such as eye movements, facial expressions, body language, and interaction responses based on the input multimodal data, and output a first dispersion probability that matches the current user's state. For example, if the user exhibits frequent eye movement, slight frowning, and a long delay in command response, but these feature combinations do not have directly corresponding mapping factor data in the mapping table, the mapping model will calculate a reasonable first dispersion probability value based on the combined influence of these features.

[0044] Meanwhile, to continuously optimize the accuracy and adaptability of the dispersion probability mapping table, the robot stores the multimodal structured array combination used in this processing as the updated mapping factor data, and records the first dispersion probability output by the mapping model as the corresponding updated dispersion probability. Subsequently, based on these newly collected updated mapping factor data and updated dispersion probabilities, the robot updates the dispersion probability mapping table to incorporate more possible combinations of user behavior features and their corresponding dispersion probabilities, thereby improving the accuracy and reliability of subsequent attention state assessments.

[0045] In a preferred embodiment, the first dispersion probability is obtained by processing the eye-tracking structured array, the facial expression structured array, the body structured array, and the interaction response structured array through a dispersion probability mapping model associated with the dispersion probability mapping table. This includes: collecting the child's eye-tracking structured recording array, facial expression structured recording array, body structured recording array, interaction response structured recording array, and labels identifying dispersion probabilities; using the labels identifying dispersion probabilities as supervision, and the eye-tracking structured recording array, facial expression structured recording array, body structured recording array, and interaction response structured recording array as input, training the dispersion probability mapping model through machine learning, and associating it with the dispersion probability mapping table; and inputting the eye-tracking structured array, facial expression structured array, body structured array, and interaction response structured array into the dispersion probability mapping model to obtain the first dispersion probability.

[0046] For example, in the process of constructing a dispersion probability mapping model associated with a dispersion probability mapping table, in order to ensure that the model can accurately reflect the degree of distraction of children under different behavioral characteristics, it is necessary to collect a series of multimodal structured record arrays and their corresponding dispersion probability labels.

[0047] Specifically, through multimodal sensors deployed on the robot, structured records of children's eye movements (such as fixation point distribution and gaze shift frequency), facial expressions (such as the frequency of smiles and frowns), body movements (such as changes in posture and frequency of small movements), and interactive responses (such as instruction response speed and task completion accuracy) are collected during focused task execution. At the same time, each record array is labeled with a label indicating the probability of distraction. This label is based on expert evaluation or historical data and reflects the actual degree of distraction of the child under that combination of behavioral characteristics.

[0048] Subsequently, using labels indicating dispersion probabilities as supervision signals and structured arrays of eye-tracking, facial expression, body language, and interaction response data as input features, the dispersion probability mapping model is trained using machine learning algorithms (such as neural networks and decision trees). During training, the model continuously adjusts its internal parameters to minimize the difference between the predicted dispersion probabilities and the actual labels, thereby learning the complex mapping relationship between behavioral features and the degree of distraction. After training, the dispersion probability mapping model is associated with a dispersion probability mapping table to form a complete attention state assessment system.

[0049] In practical applications, when it's necessary to assess a child's current level of distraction, simply inputting the real-time collected structured arrays of eye movements, facial expressions, body language, and interaction responses into the model is sufficient. The model can then output a first distraction probability that matches the current behavioral characteristics based on the learned mapping relationships, providing crucial information for subsequent attention correction strategies. For example, if the model detects frequent blinking, wandering gaze, and delayed command responses in a child, it might output a high first distraction probability, prompting the robot to take appropriate measures to improve the child's focus.

[0050] In a preferred embodiment, retrieving the dispersion probability correction vector of historical samples that satisfy the environmental disturbance structured deviation array and the cognitive load structured deviation array includes: retrieving historical samples that satisfy the environmental disturbance structured deviation array and the cognitive load structured deviation array, wherein the historical samples have a predicted dispersion probability evaluated by a dispersion probability mapping table and an actual dispersion probability corrected by the administrator; and calculating the dispersion probability correction vector based on the predicted dispersion probability and the actual dispersion probability.

[0051] In detail, during the process of optimizing the accuracy of the companion robot's focus state assessment, in order to adjust the dispersion probability prediction value in response to current environmental interference and cognitive load, the robot will perform the retrieval and calculation of dispersion probability correction vector.

[0052] Specifically, the robot first retrieves historical samples from the historical sample database that satisfy the current environmental interference structured deviation array and cognitive load structured deviation array, showing historical dispersion probabilities. These historical samples not only record the deviations between environmental interference parameters (such as noise decibel deviation, light intensity deviation, etc.) and cognitive load parameters (such as information processing speed deviation, working memory load deviation, etc.) during past task execution, but also include the predicted dispersion probability obtained through preliminary evaluation using a dispersion probability mapping table, as well as the actual dispersion probability corrected by the administrator based on the actual situation. For example, a historical sample might show a dispersion probability of 30% predicted by the mapping table under specific noise and light conditions, but the administrator corrects it to 40% based on the child's actual performance. Subsequently, the robot calculates a dispersion probability correction vector based on the predicted and actual dispersion probabilities in the retrieved historical samples. This vector reflects the degree and direction of the difference between the predicted and actual dispersion probabilities under specific environmental interference and cognitive load deviation conditions, and is used to correct subsequent dispersion probability predictions.

[0053] Through this process, the robot can continuously learn and adapt to the changing patterns of dispersion probability under different environments and cognitive states, thereby improving the accuracy and reliability of attention state assessment.

[0054] In a preferred embodiment, calculating the dispersion probability correction vector based on the predicted dispersion probability and the actual dispersion probability includes: comparing the predicted dispersion probability and the actual dispersion probability to obtain an initial dispersion probability correction vector; performing box plot analysis on the initial dispersion probability correction vector to obtain a box dispersion probability correction vector; performing central tendency analysis on the box dispersion probability correction vector to obtain a central dispersion probability correction vector; and calculating the mean of the central dispersion probability correction vector, which is then set as the dispersion probability correction vector.

[0055] Specifically, in calculating the dispersion probability correction vector, the predicted dispersion probability obtained through the dispersion probability mapping table is compared with the actual dispersion probability corrected by the administrator to obtain the initial dispersion probability correction vector. This step aims to quantify the difference between the predicted and actual values. For example, if the predicted dispersion probability is 30% and the actual dispersion probability is 40%, the initial dispersion probability correction vector will reflect this 10% positive deviation.

[0056] To eliminate the impact of outliers on the accuracy of the correction vector, the robot performs box plot analysis on the initial dispersion probability correction vector. Box plot analysis can identify and eliminate extreme values ​​in the data, thus obtaining a more robust box dispersion probability correction vector. For example, if there are a few values ​​with large deviations in the initial correction vector, these values ​​will be regarded as outliers and excluded in the box plot analysis, making the box dispersion probability correction vector closer to the actual situation of most samples.

[0057] Next, the robot will perform central tendency analysis on the box dispersion probability correction vector to further refine a representative correction vector. Central tendency analysis can reveal the central location or typical value in the data, such as the median or mean, thus obtaining the central tendency probability correction vector. For example, by analyzing the distribution of the box dispersion probability correction vector, the robot may determine that the median is 5% and use it as the representative value of the central tendency probability correction vector.

[0058] To obtain a single, representative dispersion probability correction vector, the robot calculates the mean of the clustered dispersion probability correction vectors and sets it as the final dispersion probability correction vector. This mean reflects the average difference between the predicted and actual dispersion probabilities under specific environmental disturbances and cognitive load conditions, providing an important basis for subsequent dispersion probability prediction adjustments. For example, if the mean of the clustered dispersion probability correction vector is 7%, the robot will use this value to adjust subsequent dispersion probability predictions to improve the accuracy of attention state assessment.

[0059] In a preferred embodiment, performing attention correction on the robot includes: updating the second distraction probability after initiating a robot voice prompt; and if the updated distraction probability is greater than a probability threshold, pushing an attention distraction warning to the parent's device.

[0060] In detail, the robot's attention correction process also involves the use of the robot's voice reminder function and a warning mechanism based on distraction probability. Specifically, when the robot's voice reminder function is activated, the system updates the second distraction probability. This distraction probability is an important parameter for measuring the degree of distraction of the object the robot is focusing on (the child). It is affected by various factors, such as ambient interference and the child's own shift in interest. For example, in a child's learning scenario, if a loud noise suddenly occurs, it may cause the child's attention to be distracted, and the updated second distraction probability may change. Subsequently, the system compares the updated distraction probability with a preset probability threshold. If the updated distraction probability is greater than the threshold, it indicates that the child's level of distraction has reached a level that requires attention. At this time, an attention distraction warning message will be pushed to the parents so that they can take timely measures to help the child concentrate and ensure the smooth progress of learning or other activities.

[0061] The intelligent processing method for companion robots provided in this embodiment of the invention has at least the following technical effects:

[0062] 1. By deploying multimodal sensors on the robot, comprehensive multi-dimensional data such as eye movements, facial expressions, body movements, interactive responses, and environmental interference of the target user are collected. The data is then processed using a predefined dispersion probability mapping table or associated mapping model to achieve accurate assessment of the user's distracted attention. This multimodal data fusion method can more comprehensively reflect the user's actual state, improve the accuracy and reliability of dispersion probability assessment, and provide strong support for subsequent attention correction.

[0063] 2. A structured baseline array for environmental disturbance and a structured baseline array for cognitive load are introduced. By comparing the deviations of the actual collected structured baseline arrays for environmental disturbance and cognitive load with the baseline arrays, a deviation array is obtained. Based on this, the correction vectors of historical samples of dispersion probability that meet the conditions are retrieved, and the first dispersion probability of the preliminary assessment is dynamically adjusted. This adaptive adjustment mechanism enables the dispersion probability assessment to be flexibly adjusted according to different environmental and cognitive load conditions, thereby improving the adaptability and robustness of the assessment.

[0064] 3. The final dispersion probability correction vector is calculated by taking the mean of the dispersion probability correction vectors in the statistical set. This correction vector is then used to adjust the first dispersion probability. Simultaneously, new data during the processing is used as update mapping factor data and update dispersion probabilities to update the dispersion probability mapping table. This enables continuous optimization and iteration of the model. This correction vector optimization and model update mechanism based on historical data allows the dispersion probability assessment to continuously learn and adapt to new user behaviors and environmental changes, improving the long-term effectiveness and accuracy of the assessment. Example

[0065] Please see Figure 2 , Figure 2 This is a schematic diagram illustrating an embodiment of the electronic device provided in this invention. For example... Figure 2 As shown, an embodiment of the present invention provides an electronic device 500, including a memory 510, a processor 520, and a computer program 511 stored in the memory 510 and executable on the processor 520. When the processor 520 executes the computer program 511, it implements the intelligent processing method for companion robots as described in Embodiment 1.

[0066] It should be noted that the descriptions of each embodiment in the above embodiments have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0067] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects.

[0068] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0069] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0070] Although preferred embodiments of the invention have been described, those skilled in the art, once they have learned the basic inventive concept, can make other changes and modifications to these embodiments.

[0071] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of this invention and its equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for intelligent processing of companion robots, characterized in that, include: By processing the eye-tracking structured array, facial expression structured array, body structured array, and interaction response structured array using a dispersion probability mapping table, a first dispersion probability is obtained. The dispersion probability mapping table includes an environmental interference structured reference array and a cognitive load structured reference array. The process of obtaining the first dispersion probability includes: When the companionship task falls under the category of focused tasks, multimodal sensors deployed on the robot collect structured arrays of the target user's eye movements, facial expressions, body language, interaction responses, and environmental disturbances, including: Using a robot camera, the duration of the target user's gaze point, the frequency of the gaze deviating from the target area, and the blink frequency for a preset duration are collected to construct a ternary array, which is set as the eye-tracking structured array. Using the robot's camera, the frequency of yawning and / or frowning at a preset duration, the frequency of drooping corners of the mouth at a preset duration, and the frequency of head turning at a preset duration of the target user are collected to construct a ternary array, which is set as the structured array of the facial expressions. Using the robot's camera, the frequency of sitting posture changes, the frequency of finger playing and / or leg shaking, and the frequency of leaving the seat for a preset duration of the target user are collected and a ternary array is constructed, which is set as the limb structured array. The robot's camera collects the target user's command response delay duration and the frequency of sudden increases in task error rate, and constructs a binary array, which is set as the interactive response structured array. The robot's microphone collects the decibel value of sudden noise, and the temperature and humidity sensors and light sensors deployed on the robot collect the blue light intensity and ambient temperature and humidity. A four-element array is constructed and set as the structured array of the environmental interference. By comparing the structured array of environmental disturbances with the structured baseline array of environmental disturbances, a structured deviation array of environmental disturbances is obtained. By comparing the structured array of cognitive load with the structured baseline array of cognitive load, a structured deviation array of cognitive load is obtained. Retrieve the dispersion probability correction vector of historical samples that satisfy the dispersion probability of the environmental interference structured deviation array and the cognitive load structured deviation array, adjust the first dispersion probability to obtain the second dispersion probability, and if it is greater than or equal to the probability threshold, perform robot attention correction.

2. The intelligent processing method for companion robots as described in claim 1, characterized in that, By processing the eye-tracking structured array, the facial expression structured array, the body structured array, and the interaction response structured array using a predefined dispersion probability mapping table, a first dispersion probability is obtained, including: Extract several mapping factor data and several dispersion probability calibration values ​​from the dispersion probability mapping table; The mapping factor data is traversed and compared with the eye movement structured array, the facial expression structured array, the body structured array, and the interaction response structured array to obtain the comparison results; When the comparison results have consistent mapping factor data, the corresponding dispersion probability calibration value is extracted from the plurality of dispersion probability calibration values ​​and set as the first dispersion probability.

3. The intelligent processing method for companion robots as described in claim 2, characterized in that, Also includes: When the comparison results have inconsistent mapping factor data, the eye movement structured array, the facial expression structured array, the body structured array, and the interaction response structured array are processed by the dispersion probability mapping model associated with the dispersion probability mapping table to obtain the first dispersion probability; In addition, the dispersed probability mapping model associated with the dispersed probability mapping table processes the eye movement structured array, the facial expression structured array, the limb structured array, and the interaction response structured array and stores them as update mapping factor data, and stores the first dispersed probability as update dispersed probability; The dispersion probability mapping table is updated based on the updated mapping factor data and the updated dispersion probability.

4. The intelligent processing method for companion robots as described in claim 3, characterized in that, The first dispersion probability is obtained by processing the eye-tracking structured array, the facial expression structured array, the body structured array, and the interaction response structured array through a dispersion probability mapping model associated with the dispersion probability mapping table, including: Collect structured recordings of children's eye movements, facial expressions, body language, and interactive responses, along with labels indicating dispersion probability. Using the labels identifying the dispersion probability as supervision, and taking the eye-tracking structured recording array, the facial expression structured recording array, the body structured recording array, and the interaction response structured recording array as input, the dispersion probability mapping model is trained through machine learning and associated with the dispersion probability mapping table; The eye-tracking structured array, the facial expression structured array, the body structured array, and the interaction response structured array are input into the dispersion probability mapping model to obtain the first dispersion probability.

5. The intelligent processing method for companion robots as described in claim 1, characterized in that, Retrieve the dispersion probability correction vector of historical samples that satisfy the dispersion probability of the environmental disturbance structured deviation array and the cognitive load structured deviation array, including: Retrieve historical samples of dispersion probability that satisfy the structured deviation array of environmental disturbance and the structured deviation array of cognitive load, wherein the historical samples of dispersion probability have a predicted dispersion probability evaluated by a dispersion probability mapping table and an actual dispersion probability corrected by the administrator. The dispersion probability correction vector is calculated based on the predicted dispersion probability and the actual dispersion probability.

6. The intelligent processing method for companion robots as described in claim 5, characterized in that, The dispersion probability correction vector is calculated based on the predicted dispersion probability and the actual dispersion probability, including: By comparing the predicted dispersion probability with the actual dispersion probability, an initial dispersion probability correction vector is obtained; Box plot analysis is performed on the initial dispersion probability correction vector to obtain the box dispersion probability correction vector; Central tendency analysis is performed on the dispersion probability correction vector of the box to obtain the central dispersion probability correction vector; The mean of the centralized and dispersed probability correction vectors is calculated and set as the dispersed probability correction vector.

7. The intelligent processing method for companion robots as described in claim 1, characterized in that, The robot's attention correction includes: Update the second dispersion probability after activating the robot's voice prompt; If the probability of distraction is greater than the probability threshold, a distraction warning will be sent to the parents' devices.

8. An electronic device, characterized in that, include: Memory, used to store computer software programs; A processor is used to read and execute the computer software program, thereby implementing the intelligent processing method for companion robots according to any one of claims 1-7.

Images