A local offline AI multi-model collaborative accompanying interaction method for a child intelligent toy

Abstract

The application discloses a local offline AI multi-model cooperative accompanying interaction method for a child intelligent toy, and is applied to an embedded child intelligent toy hardware loaded with an on-chip NPU, a local nonvolatile storage module and a visual acquisition module. The local offline AI multi-model cooperative accompanying interaction method for the child intelligent toy is entirely dependent on the toy end-side hardware to complete global AI reasoning, data analysis and instruction interaction, is completely separated from WiFi, a cellular network and a cloud server, and can normally operate all accompanying interaction functions in a network-free environment, and is suitable for outdoor, vehicle-mounted and home network interruption scenes. Child face and body data are processed locally and are not stored or uploaded, so that the risk of cloud privacy leakage is avoided, and the child electronic product safety standard is adapted.

Description

Technical Field

[0001] This invention relates to the field of smart toy-related products, specifically a local offline AI multi-model collaborative companionship interaction method for children's smart toys. Background Technology

[0002] The market for smart electronic toys for children aged 6-12 is rapidly expanding. Electronic pets and interactive dolls equipped with AI perception and interaction functions are gradually replacing traditional button toys. Existing AI interactive toy technology solutions are mainly divided into two categories: cloud-based online interaction and simple single-model offline interaction. Both types of solutions have technical defects that make it difficult to adapt to home use scenarios for children.

[0003] Cloud-based online AI smart toys are the mainstream solution on the market. The toy itself only has peripherals for data collection such as cameras and microphones. All AI inference operations, including emotion recognition, body visual tracking, and semantic understanding, are deployed on a remote cloud server. The toy uploads the collected images and audio data of the child to the cloud via WiFi and Bluetooth. The cloud then analyzes the data and issues action commands to drive the toy's interactive feedback. This solution has several key shortcomings: First, it heavily relies on wireless networks. In scenarios with no network, weak network, or no WiFi outdoors, the AI ​​smart interaction function is completely ineffective, and only factory-preset fixed actions and voice commands can be triggered, severely limiting its use cases. Second, the continuous uploading of children's facial and body privacy data to the cloud poses a risk of data leakage and does not comply with privacy and security regulations for children's electronic products.

[0004] Simple offline AI smart toys fall into two categories. One type is equipped with only a fixed voice broadcast chip, lacking AI reasoning capabilities and unable to recognize children's emotions or body movements, resulting in a limited form of interaction. The other type is equipped with a single lightweight offline AI model, capable of only single visual or voice recognition functions, and unable to achieve emotion + body linkage interaction. If manufacturers stack multiple types of offline AI models, the existing hardware architecture lacks a unified computing power scheduling unit. All models reside in the main control memory and run serially, directly competing for the limited memory of the embedded MCU and the computing power resources of the NPU, which can easily lead to interactive stuttering, instruction parsing delays, and response timeouts.

[0005] In addition, existing offline AI toys still suffer from several industry pain points: First, the parameters of general open-source AI models are enormous, and they have not been specifically tailored and fine-tuned for the interactive characteristics of children, such as small limb movements, vague actions, and expressive facial emotions. This makes it difficult to deploy low-cost toy hardware and results in low accuracy of interactive recognition. Second, the industry lacks supporting edge power consumption control logic. AI models and image acquisition peripherals operate at high frequency around the clock, resulting in wasted computing power and electricity during idle periods without interaction. The built-in lithium batteries in toys also have short battery life. Third, the multi-model scheduling logic is crude and does not distinguish the priority of children's companionship interactions. The response speed of high-frequency, essential interactions such as playing and feeding is slow, resulting in a poor product user experience.

[0006] In summary, developing an AI companion interaction solution that can run entirely locally and offline, adapts to children's interactive characteristics, has multi-model computing power collaborative scheduling, and low power consumption control is the core direction for solving the pain points of existing children's smart toys and optimizing the product experience. Summary of the Invention

[0007] The purpose of this invention is to provide a local offline AI multi-model collaborative companion interaction method for children's smart toys, so as to solve the problems mentioned in the background art.

[0008] To achieve the above objectives, the present invention provides the following technical solution: a local offline AI multi-model collaborative companionship interaction method for children's smart toys, applied to embedded children's smart toy hardware equipped with an on-chip NPU, a local non-volatile storage module, and a visual acquisition module, the method comprising the following steps:

[0009] S1. Model Pre-adaptation and Deployment: Based on the children's exclusive interactive dataset, the pre-trained original AI model is lightweighted and compressed iteratively to obtain a lightweight emotion recognition model, target visual tracking model, and target visual recognition model that are adapted to the toy's NPU computing power and memory resources. The weight files of the three types of lightweight models are then solidified and written into the toy's local non-volatile storage module.

[0010] S2. End-side non-intrusive data acquisition: The vision acquisition module collects raw image data of children's facial and limb interactions in real time. All image data is only cached in the temporary memory of the toy's main controller and is stored locally in a closed loop throughout the process. It is not transmitted to the outside world through the network or uploaded to the cloud server.

[0011] S3. Dynamic Coordinated Scheduling of Hardware Computing Power: The main controller has a built-in multi-model computing power scheduling unit that identifies the current image data features in real time, matches the interactive scenario, dynamically allocates on-chip NPU computing power, and creates model running threads; for AI models that do not need to be called in the current interactive scenario, it performs thread suspension, computing power cut-off, and resource release operations.

[0012] S4. Layered offline inference instruction parsing: Call the mounted AI model, rely on the on-chip NPU to complete local offline layered inference, retrieve the preset child interaction feature judgment threshold, and parse the child's waving, physical touch, and facial emotion change behavior into three types of standardized interactive control instructions: toy feeding, interactive play, and voice chat.

[0013] S5. Peripheral closed-loop interaction response: Standardized interactive control commands are sent to the toy peripheral drive unit to drive the toy to perform corresponding interactive behaviors such as limb movements, sound and light feedback, and form switching, thus completing a single companion interaction closed loop.

[0014] S6. Adaptive Sleep and Wake-up Control: The computing power scheduling unit monitors the hardware level signal of the visual acquisition module in real time. After determining that the idle interaction conditions are met, it shuts down the high-frequency sampling of visual acquisition, suspends all AI model threads, and cuts off the power supply to the model computing power. After detecting a valid child interaction image signal, it instantly wakes up the corresponding AI model and acquisition peripherals and restores the offline interaction function.

[0015] As a preferred embodiment of the present invention, the AI ​​model lightweight compression method in step S1 is selected from any combination of the following: model parameter distillation compression, neural network structured pruning, and 8-bit quantization compression; for fuzzy limb movements and facial micro-expression samples of children aged 6-12 years, the model parameters are fine-tuned to optimize the accuracy of interactive recognition.

[0016] As a preferred embodiment of the present invention, the computing power scheduling mode in step S3 includes a scenario-adaptive scheduling mode and a priority scheduling mode.

[0017] The scene adaptation scheduling mode is as follows: when only static human body images are collected, the target visual recognition model is attached separately; when dynamic limb movements are collected, the target visual tracking model is attached in conjunction with the scene; when a child's facial image is recognized, the emotion recognition model is attached in conjunction with the scene.

[0018] The priority scheduling mode is as follows: prioritize interaction commands, with feeding and play-related high-priority interaction commands taking up all available NPU computing power for inference; and chat-related low-priority interaction commands using computing power for inference in the background at low frequency, without competing for core computing power resources.

[0019] As a preferred embodiment of the present invention, the criteria for determining the idle interaction condition in step S6 are as follows: if the visual acquisition module fails to acquire a valid human interaction image for 2.5 min to 3.5 min continuously, it is determined that the device is in an idle state; in the idle state, the sampling frequency of the visual acquisition module is reduced by 60% to 70%.

[0020] As a preferred embodiment of the present invention, a voice-linked wake-up step is added: the toy pre-stores a dedicated children's wake-up voice library locally, completes the comparison of collected audio features offline, and only starts the visual acquisition and AI model inference function after matching the preset wake-up word locally, and keeps low power consumption standby during other times.

[0021] As a preferred embodiment of the present invention, the three types of lightweight model storage deployment methods include two types: First, the model weight file is permanently stored in the Flash memory inside the toy chip; Second, the model weight file is externally stored in a pluggable TF memory card, and the corresponding model is dynamically loaded into memory for operation as needed after the device is powered on.

[0022] A local offline AI-powered smart companion toy device for children that implements the aforementioned interaction method includes:

[0023] The embedded main control module has a built-in on-chip micro NPU computing unit, which is used to support offline inference of AI models and full-domain hardware timing control.

[0024] The multimodal acquisition module is electrically connected to the embedded main control module and includes at least a high-definition visual camera and a microphone, which are used to acquire images of children's limbs and wake-up voice signals, respectively.

[0025] The hierarchical storage module is electrically connected to the embedded main control module, including on-chip non-volatile Flash and an external pluggable TF card slot, for storing lightweight AI models, child interaction judgment thresholds, and wake-up voice sample libraries;

[0026] The multi-model collaborative computing power scheduling unit, embedded in the embedded main control module, is used to realize the start and stop of AI model threads, time-sharing allocation of NPU computing power, sleep and wake-up of idle models, and peripheral sampling frequency adjustment.

[0027] The AI ​​hierarchical inference unit is connected to the computing power scheduling unit and includes an emotion recognition inference submodule, a target visual tracking submodule, and a target visual recognition submodule, which can independently complete offline inference operations for a single model.

[0028] The peripheral interaction driving unit is electrically connected to the AI ​​hierarchical inference unit and includes servo motor motion driving, sound and light playback driving, and indicator light driving, which are used to execute toy interactive feedback actions.

[0029] The entire data processing, model reasoning, and instruction parsing process of the device are all completed on the edge hardware, without any cloud data interaction link.

[0030] As a preferred embodiment of the present invention, the embedded main control is a 32-bit industrial-grade MCU, equipped with an on-chip NPU with 8 TOPS computing power; the on-chip Flash storage capacity of the hierarchical storage module is not less than 16MB, and after lightweighting, the size of the emotion recognition model is ≤2.5MB, the size of the target visual tracking model is ≤200KB, and the size of the target visual recognition model is ≤3.5MB.

[0031] As a preferred embodiment of the present invention, the multi-model collaborative computing power scheduling unit has a built-in power consumption control subroutine, which reduces the idle power consumption of the whole machine by more than 40% when the device is idle or in hibernation state.

[0032] Compared with the prior art, the beneficial effects of the present invention are:

[0033] This local offline AI multi-model collaborative companionship interaction method for children's smart toys relies entirely on the toy's hardware for AI inference, data analysis, and command interaction, completely independent of WiFi, cellular networks, and cloud servers. All companionship interaction functions operate normally in offline environments, adapting to all scenarios of offline use, including outdoor, in-vehicle, and home environments. Children's facial and body data are processed locally without being stored or uploaded, avoiding the risk of privacy leaks from the cloud and complying with children's electronic product safety standards. Attached Figure Description

[0034] Figure 1 This is a hardware architecture block diagram of a local offline AI multi-model collaborative companionship interaction method for children's smart toys according to the present invention.

[0035] Figure 2 This is a sequence diagram of the interaction flow of a local offline AI multi-model collaborative companion interaction method for children's smart toys according to the present invention.

[0036] In the diagram: 1. Embedded main control module; 2. Multimodal acquisition module; 3. Hierarchical storage module; 4. Multi-model collaborative computing power scheduling unit; 5. AI hierarchical inference unit; 6. Peripheral interaction driving unit. Detailed Implementation

[0037] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0038] In the description of this invention, it should be noted that the terms "upper," "lower," "inner," "outer," "front end," "rear end," "both ends," "one end," and "the other end," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0039] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installed," "equipped with," "connected," etc., should be interpreted broadly. For example, "connection" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium; it can be a connection within two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0040] Please see Figure 1-2 The present invention provides an embodiment of a local offline AI multi-model collaborative companionship interaction method for children's smart toys, applied to embedded children's smart toy hardware equipped with an on-chip NPU, a local non-volatile storage module, and a visual acquisition module. The method includes the following steps:

[0041] S1. Model Pre-adaptation and Deployment: Based on a dedicated limb and facial interaction dataset for children aged 6-12, the pre-trained original AI model is subjected to lightweight compression iteration. The compression method can be any one or a combination of parameter distillation, structured pruning, and 8-bit quantization to obtain a lightweight emotion recognition model, target visual tracking model, and target visual recognition model adapted to the toy's NPU computing power and memory resources. The weight files of the three types of lightweight models are then solidified and written into the toy's local non-volatile storage module. The storage format supports two modes: on-chip Flash solidification and external TF card dynamic loading.

[0042] S2. End-side non-intrusive data acquisition: The visual acquisition module collects raw image data of children's facial and limb interactions in real time, and the microphone simultaneously collects environmental voice wake-up signals. All image and audio data are only cached in the toy's main control temporary memory. The cache is cleared immediately after processing. The entire process is local closed-loop storage and is not transmitted to the outside through the network or uploaded to the cloud server.

[0043] S3. Dynamic Cooperative Scheduling of Hardware Computing Power: The main controller has a built-in multi-model computing power scheduling unit that identifies the current image data features in real time and supports dual-mode computing power scheduling of scene adaptation and priority. In scene mode, the corresponding model is mounted on demand and idle models are suspended to release computing power. In priority mode, high-priority interactions such as playing and feeding occupy priority computing power, while low-priority interactions such as chatting run in the background at low frequency.

[0044] S4. Layered offline inference instruction parsing: Call the mounted AI model, rely on the on-chip NPU to complete local offline layered inference, retrieve the preset child interaction feature judgment threshold, and accurately parse the child's waving, physical touch, and facial emotional changes of joy, anger, sorrow and happiness into three types of standardized interactive control instructions: toy feeding, interactive play, and voice chat.

[0045] S5. Peripheral closed-loop interactive response: Standardized interactive control commands are sent to the toy peripheral drive unit to drive the toy servo motor limbs to rotate, switch colored lights and sounds, and broadcast built-in voice, completing a one-to-one companion interactive closed-loop feedback.

[0046] S6. Adaptive Sleep and Wake-up Control: The computing power scheduling unit monitors the hardware level signal of the visual acquisition module in real time. If there is no effective human interaction image for 3 consecutive minutes, it is determined to be idle, and the camera sampling frequency is reduced, all AI models are suspended, and the power supply to the model computing power is cut off. After capturing the image of a child's body again and matching the local wake-up voice, the model and peripherals are instantly woken up, and the interactive function is quickly restored.

[0047] Along with this interactive method, the present invention also provides a local offline AI smart companion toy device for children. The hardware architecture is fully adapted to the above method process. The hardware components include an embedded main control module 1, a multimodal acquisition module 2, a hierarchical storage module 3, a multi-model collaborative computing power scheduling unit 4, an AI hierarchical inference unit 5, and a peripheral interaction driving unit 6. The modules are electrically interconnected, and the computing power scheduling unit uniformly manages the allocation of computing power, thread start and stop, and power consumption control. There is no cloud data transmission link at all, and all AI calculations are completed on the terminal side.

[0048] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. A local offline AI multi-model collaborative companionship interaction method for children's smart toys, characterized in that, The method for embedded children's smart toy hardware, which incorporates an on-chip NPU, a local non-volatile memory module, and a vision acquisition module, includes the following steps: S1. Model Pre-adaptation and Deployment: Based on the children's exclusive interactive dataset, the pre-trained original AI model is lightweighted and compressed iteratively to obtain a lightweight emotion recognition model, target visual tracking model, and target visual recognition model that are adapted to the toy's NPU computing power and memory resources. The weight files of the three types of lightweight models are then solidified and written into the toy's local non-volatile storage module. S2. End-side non-intrusive data acquisition: The vision acquisition module collects raw image data of children's facial and limb interactions in real time. All image data is only cached in the temporary memory of the toy's main controller and is stored locally in a closed loop throughout the process. It is not transmitted to the outside world through the network or uploaded to the cloud server. S3. Dynamic Coordinated Scheduling of Hardware Computing Power: The main controller has a built-in multi-model computing power scheduling unit that identifies the current image data features in real time, matches the interactive scenario, dynamically allocates on-chip NPU computing power, and creates model running threads; for AI models that do not need to be called in the current interactive scenario, it performs thread suspension, computing power cut-off, and resource release operations. S4. Layered offline inference instruction parsing: Call the mounted AI model, rely on the on-chip NPU to complete local offline layered inference, retrieve the preset child interaction feature judgment threshold, and parse the child's waving, physical touch, and facial emotion change behavior into three types of standardized interactive control instructions: toy feeding, interactive play, and voice chat. S5. Peripheral closed-loop interaction response: Standardized interactive control commands are sent to the toy peripheral drive unit to drive the toy to perform corresponding interactive behaviors such as limb movements, sound and light feedback, and form switching, thus completing a single companion interaction closed loop. S6. Adaptive Sleep and Wake-up Control: The computing power scheduling unit monitors the hardware level signal of the visual acquisition module in real time. After determining that the idle interaction conditions are met, it shuts down the high-frequency sampling of visual acquisition, suspends all AI model threads, and cuts off the power supply to the model computing power. After detecting a valid child interaction image signal, it instantly wakes up the corresponding AI model and acquisition peripherals and restores the offline interaction function.

2. The local offline AI multi-model collaborative companionship interaction method for children's smart toys according to claim 1, characterized in that, In step S1, the AI ​​model lightweight compression method is selected from any combination of the following: model parameter distillation compression, neural network structured pruning, and 8-bit quantization compression; for fuzzy limb movements and facial micro-expression samples of children aged 6-12, the model parameters are fine-tuned to optimize the accuracy of interactive recognition.

3. The local offline AI multi-model collaborative companionship interaction method for children's smart toys according to claim 1, characterized in that, Step S3 computing power scheduling modes include scenario-adaptive scheduling mode and priority scheduling mode; The scene adaptation scheduling mode is as follows: when only static human body images are collected, the target visual recognition model is attached separately; when dynamic limb movements are collected, the target visual tracking model is attached in conjunction with the scene; when a child's facial image is recognized, the emotion recognition model is attached in conjunction with the scene. The priority scheduling mode is as follows: prioritize interaction commands, with feeding and play-related high-priority interaction commands taking up all available NPU computing power for inference; and chat-related low-priority interaction commands using computing power for inference in the background at low frequency, without competing for core computing power resources.

4. The local offline AI multi-model collaborative companionship interaction method for children's smart toys according to claim 1, characterized in that, The criteria for determining the idle interaction condition in step S6 are as follows: if the visual acquisition module fails to acquire a valid human interaction image for 2.5 min to 3.5 min continuously, it is determined that the device is in an idle state; in the idle state, the sampling frequency of the visual acquisition module is reduced by 60%-70%.

5. The local offline AI multi-model collaborative companionship interaction method for children's smart toys according to claim 1, characterized in that, It also adds a voice-linked wake-up step: the toy has a dedicated children's wake-up voice library pre-stored locally, and the audio feature comparison is completed offline. Only after the local comparison matches the preset wake-up word will the visual acquisition and AI model inference functions be activated, and the toy will remain in low-power standby mode at other times.

6. The local offline AI multi-model collaborative companionship interaction method for children's smart toys according to claim 1, characterized in that, The three types of lightweight model storage deployment methods include two types: First, the model weight file is permanently stored in the Flash memory inside the toy chip; second, the model weight file is externally stored on a pluggable TF memory card, and the corresponding model is dynamically loaded into memory for operation as needed after the device is powered on.

7. A local offline AI children's intelligent companion toy device implementing the interaction method of any one of claims 1-6, characterized in that, include: Embedded main control module (1) has a built-in on-chip micro NPU computing unit, which is used to carry out AI model offline inference and full-domain hardware timing control; The multimodal acquisition module (2) is electrically connected to the embedded main control module (1) and includes at least a high-definition visual camera and a microphone, which are used to acquire images of children's limbs and wake-up voice signals, respectively. The hierarchical storage module (3) is electrically connected to the embedded main control module (1), including on-chip non-volatile Flash and external pluggable TF card slot, for storing lightweight AI models, child interaction judgment thresholds, and wake-up voice sample library; The multi-model collaborative computing power scheduling unit (4) is embedded in the embedded main control module (1) and is used to realize the start and stop of AI model threads, time-sharing allocation of NPU computing power, sleep and wake-up of idle models, and peripheral sampling frequency adjustment. AI hierarchical reasoning unit (5) is connected to computing power scheduling unit, including emotion recognition reasoning submodule, target visual tracking submodule and target visual recognition submodule, and independently completes single-model offline reasoning operation; The peripheral interaction driving unit (6) is electrically connected to the AI ​​hierarchical reasoning unit (5), which includes servo motor action driving, sound and light playback driving, and indicator light driving, and is used to perform toy interactive feedback actions. The entire data processing, model reasoning, and instruction parsing process of the device are all completed on the edge hardware, without any cloud data interaction link.

8. The local offline AI children's intelligent companion toy device according to claim 7, characterized in that, The embedded main controller is a 32-bit industrial-grade MCU, equipped with an on-chip NPU with 8 TOPS computing power; the hierarchical storage module (3) has an on-chip Flash storage capacity of not less than 16MB, and after weight reduction, the size of the emotion recognition model is ≤2.5MB, the size of the target visual tracking model is ≤200KB, and the size of the target visual recognition model is ≤3.5MB.

9. The local offline AI children's intelligent companion toy device according to claim 7, characterized in that, The multi-model collaborative computing power scheduling unit (4) has a built-in power consumption control subroutine. When the device is idle or in hibernation mode, the idle power consumption of the whole machine is reduced by more than 40%.

Images