An AI toy agent scheduling method and system

Abstract

This invention relates to the field of artificial intelligence terminal interaction technology, and discloses an AI toy intelligent agent scheduling method and system, including: acquiring interactive input generated by an AI toy terminal or an external business system; performing source identification and content normalization processing on the interactive input to generate a unified task object; determining the task execution method based on the unified task object and device status information; filtering candidate capability services and determining the target capability service based on the unified task object, device status information, and capability registry; calling the target capability service to generate structured intermediate results; generating target response text based on the structured intermediate results, role configuration, and session context; synchronously generating the toy's broadcast voice and the external terminal's displayed text based on the target response text, and associating them with the same response number; and performing fault rollback when the target capability service call is abnormal. This invention can realize unified scheduling, capability expansion, same-source output, and abnormal rollback of multi-source interactive tasks of AI toys.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence terminal interaction technology, specifically to an AI toy intelligent agent scheduling method and system. Background Technology

[0002] With the development of speech recognition, natural language processing, large language models, and IoT communication technologies, children's companion smart toys, story machines, educational toys, and companion robots are gradually acquiring capabilities such as online Q&A, content playback, character voice broadcasting, parent interaction, and background message push. Existing AI toys typically collect user voice data via microphone, then complete the interaction through cloud-based speech recognition, semantic understanding, text generation, and speech synthesis. Some AI toys can also receive messages from parent applications or operational backends, and, when equipped with cameras, achieve a certain level of image recognition or scene understanding. Therefore, AI toys have gradually evolved from simple voice Q&A devices into multi-source interactive terminals that include voice input, visual input, business events, background messages, and external display.

[0003] However, in existing AI toy systems, voice Q&A, visual understanding, parental messages, background push notifications, and mini-program development events are typically handled separately by different business chains, lacking a unified task object and device state constraint mechanism. When interactive inputs from multiple sources arrive simultaneously, the system struggles to uniformly schedule them based on the current playback status of the AI ​​toy terminal, the task source, and the task priority. This easily leads to problems such as low-priority messages interrupting real-time dialogue, high-priority reminders failing to be broadcast in a timely manner, inconsistent contexts between different input sources, and disordered execution order of multiple tasks on the same device. Because multi-source interactive inputs are not uniformly incorporated into the same scheduling chain, subsequent capability service calls, response generation, voice broadcasting, and external display can also easily result in fragmented processing outcomes, affecting the continuity, controllability, and traceability of the AI ​​toy interaction process. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides an AI toy intelligent agent scheduling method and system, which solves the problems in existing AI toy systems where voice input, visual input, parent messages, background push notifications, and business events are processed separately, lacking unified task objects and device state constraints, resulting in chaotic execution order of multi-source interactive tasks, inconsistent contexts, and low-priority tasks easily interrupting real-time interaction.

[0005] To achieve the above objectives, the present invention provides the following technical solution: an AI toy agent scheduling method, comprising: S1. Obtain interactive input and simultaneously obtain device status information of the AI ​​toy terminal. The interactive input comes from the AI ​​toy terminal or an external business system. S2. Perform source identification and content normalization processing on the interactive input to generate a unified task object. The unified task object includes device identifier, task source, session number, role identifier, language identifier, task content, and task priority. S3. Determine the task execution method based on the unified task object and the device status information, and determine the time order in which the unified task object enters the capability routing process according to the task execution method; S4. Filter candidate capability services based on the unified task object, the device status information and the capability registry, and determine the target capability service from the candidate capability services; S5. Invoke the target capability service to process the unified task object and generate structured intermediate results; S6. Generate the target response text based on the structured intermediate results, role configuration, and session context; S7. Based on the target reply text, simultaneously generate the toy-side broadcast voice and the external terminal display text, and associate the target reply text, the toy-side broadcast voice, and the external terminal display text with the same reply number; S8. When the target capability service call is abnormal, switch to the rollback capability service or local fallback content according to the fault rollback policy in the capability registry, and keep the role identifier and language identifier in the unified task object unchanged.

[0006] Preferably, the device status information includes at least one of the following: current role, current language, current playback status, online status, camera availability status, network connection status, and recent interaction time; the current playback status includes at least one of the following: idle status, listening status, broadcasting status, and dialogue status.

[0007] Preferably, the task execution mode includes immediate execution, queuing, delayed broadcasting, merged broadcasting, or preemptive broadcasting; the step of determining the task execution mode includes: identifying the task category corresponding to the unified task object according to the task source; determining the task priority according to the task category and the task content; and determining the task execution mode according to the task priority and the current playback status.

[0008] Preferably, the capability registry includes the service identifier, service address, authentication information, start / stop status, supported task types, supported languages, supported roles, supported scenarios, service priority, timeout threshold, and fault rollback strategy for the capability service.

[0009] Preferably, the step of determining the target capability service from the candidate capability services includes: eliminating capability services that do not match the task category, the language identifier, the role identifier, the device status information, or the start / stop status to obtain the candidate capability services; calculating the service matching score of the candidate capability services based on the degree of matching of task type, device capability, role, language, service health status, and service response latency; and selecting the candidate capability service with the highest service matching score that meets the timeout threshold as the target capability service.

[0010] Preferably, when the unified task object corresponds to user voice input, the user voice input is converted into user text; when the user text hits a visual trigger word and the device status information indicates that the AI ​​toy terminal has visual acquisition capabilities, the AI ​​toy terminal is controlled to perform a single capture or a single frame capture to obtain visual input data; the visual input data is input into a visual understanding service to output structured visual results, and the structured visual results are used as a component of the structured intermediate results.

[0011] Preferably, the structured intermediate result includes result type, source identifier, result content, confidence identifier, output constraints, and task time; the step of generating the target response text includes: combining the structured intermediate result with the session context to form a response generation input; determining response style constraints based on the role configuration and the language identifier; and generating the target response text based on the response generation input and the response style constraints.

[0012] Preferably, the step of simultaneously generating the toy-side broadcast voice and the external-side display text based on the target reply text includes: generating the reply number for the target reply text; inputting the target reply text into a speech synthesis service to generate the toy-side broadcast voice; sending the target reply text as the external-side display text to the parent app, the operation backend, or the mini-program development system; and writing the device identifier, the session number, the reply number, the toy-side broadcast voice, and the external-side display text into the session record.

[0013] Preferably, the abnormal call to the target capability service includes call failure, response timeout, authentication failure, service shutdown, or empty return content; the step of switching to the fallback capability service or local fallback content according to the fault fallback strategy includes: reading the fault fallback strategy corresponding to the target capability service; determining the fallback capability service according to the fault fallback strategy; calling the local fallback content when the fallback capability service is unavailable; and associating the call exception information and the fallback processing result with the session record.

[0014] The present invention also provides an AI toy intelligent agent scheduling system, comprising: The AI ​​toy terminal is used to collect user voice, play the voice broadcast from the toy, and perform a single capture or a single frame capture when it receives a visual acquisition command. The device status management module is used to maintain the device status information of the AI ​​toy terminal; The entry parsing module is used to acquire interactive input, and to perform source identification and content normalization processing on the interactive input to generate a unified task object; The event and message scheduling module is used to determine the task execution method based on the unified task object and the device status information, and to determine the time order in which the unified task object enters the capability routing process according to the task execution method. The capability registration module is used to maintain the capability registry. The capability routing module is used to filter candidate capability services based on the unified task object, the device status information, and the capability registry, and to determine the target capability service from the candidate capability services. The multimodal result generation module is used to call the target capability service to process the unified task object and generate structured intermediate results; The response generation module is used to generate target response text based on the structured intermediate results, role configuration, and session context. The same-source output module is used to synchronously generate the toy-side broadcast voice and the external terminal display text based on the target reply text, and associate the target reply text, the toy-side broadcast voice, and the external terminal display text with the same reply number; The fault rollback module is used to switch to the rollback capability service or local fallback content according to the fault rollback policy in the capability registry when the target capability service call is abnormal.

[0015] This invention provides an AI toy intelligent agent scheduling method and system. It has the following beneficial effects: 1. This invention identifies the source and normalizes the content of interactive inputs generated by AI toy terminals and external business systems, generating a unified task object that includes device identifier, task source, session number, role identifier, language identifier, task content, and task priority. Combined with device status information, it determines task execution methods such as immediate execution, queuing, delayed broadcasting, merged broadcasting, or preemptive broadcasting, enabling voice input, visual input, parental messages, background push notifications, and mini-program development events to enter the same scheduling chain. Therefore, the system can rationally determine the time order in which tasks enter the capability routing process based on the current playback status and task priority, avoiding low-priority tasks interrupting real-time dialogue and preventing high-priority tasks from failing to be processed in a timely manner, thus improving the orderliness and continuity of multi-source interactive task orchestration for AI toys.

[0016] 2. This invention records the service identifier, start / stop status, supported task types, supported languages, supported roles, supported scenarios, service priority, timeout threshold, and fault rollback strategy of capability services through a capability registry. Based on a unified task object, device status information, and the capability registry, it filters candidate capability services and determines the target capability service. This eliminates the need to modify the main process logic when adding visual understanding, brand knowledge, online search, language capabilities, or business linkage services. Therefore, it reduces the cost of expanding and maintaining AI toy capabilities and improves the system's adaptability to different device types, role configurations, and language scenarios.

[0017] 3. This invention converts the processing results returned by the target capability service into structured intermediate results, and then combines role configuration and session context to generate the target response text. This allows the outputs of different capability services to enter the response generation process with a unified data structure. Therefore, it avoids the inconsistencies in format and discontinuities in context caused by different capability services directly outputting their final content, and helps ensure the consistency of role style, language style, and session semantics in response content from different task sources.

[0018] 4. This invention generates the toy's voice broadcast and the external display text simultaneously through the target reply text, and associates the target reply text, the toy's voice broadcast, and the external display text with the same reply number. This ensures that the actual broadcast content of the AI ​​toy terminal and the content displayed in the parent's app, the operation backend, or the mini-program development system share the same text source. This avoids semantic discrepancies caused by generating display text separately after voice broadcast, improving the traceability of interaction records and the credibility of content viewed on external devices.

[0019] 5. This invention only controls the AI ​​toy terminal to perform a single snapshot or single frame capture when the user's voice input hits a visual trigger word and the device status information indicates that the AI ​​toy terminal has visual acquisition capabilities. The structured visual results output by the visual understanding service are then incorporated into the structured intermediate results. Therefore, visual understanding does not require continuously keeping the camera on, reducing power consumption and privacy risks associated with invalid image acquisition. Simultaneously, the structured visual results ensure that the visually understood content can stably enter the response generation process.

[0020] 6. When the target capability service experiences call anomalies such as call failure, response timeout, authentication failure, service shutdown, or empty return content, this invention switches to the fallback capability service or local fallback content according to the fault fallback strategy in the capability registry, while maintaining the role identifier and language identifier in the unified task object unchanged. Therefore, even if some capability services are unavailable, the dialogue link can be maintained continuously, avoiding problems such as the AI ​​toy terminal becoming silent, directly reporting errors, or experiencing sudden changes in character language. Attached Figure Description

[0021] Figure 1 This is an architecture diagram of an AI toy intelligent agent scheduling system provided in an embodiment of the present invention; Figure 2 This is a flowchart of the AI ​​toy agent scheduling method provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the capability registry and capability routing process provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the event and message scheduling process provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of a user-initiated visual understanding process provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of the target response text homology output process provided in an embodiment of the present invention. Detailed Implementation

[0022] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0023] This invention provides an AI toy intelligent agent scheduling method and system. The method is applicable to AI toy terminals with voice acquisition, voice playback, and network communication capabilities, and also to AI toy terminals further equipped with cameras, displays, buttons, touch sensors, or motion sensors. The AI ​​toy terminal can communicate with a cloud-based intelligent agent platform, which in turn can communicate with an external capability service cluster and external business systems. External business systems include at least one of a parent app, an operations backend, and a mini-program development system. The external capability service cluster includes at least one of voice recognition services, visual understanding services, online search services, brand knowledge services, encyclopedic knowledge services, language ability services, business linkage services, speech synthesis services, and local backup content services.

[0024] In this embodiment, the AI ​​toy terminal can be a children's companion toy, story machine, smart doll, educational toy, companion robot, or other smart terminal capable of voice interaction. The cloud-based intelligent agent platform is used to uniformly receive interactive inputs generated by the AI ​​toy terminal or external business systems, and to uniformly incorporate interactive inputs from different sources, in different formats, and with different priorities into the same task scheduling link. This solves the problems of context fragmentation, inconsistent output, difficulty in capability expansion, and discontinuous abnormal rollback that exist when voice Q&A, visual understanding, background messages, mini-program development events, and parent messages are processed independently.

[0025] like Figure 1 As shown, the AI ​​toy intelligent agent scheduling system includes an AI toy terminal, a device status management module, an entry parsing module, an event and message scheduling module, a capability registration module, a capability routing module, a multimodal result generation module, a response generation module, a homogeneous output module, and a fault rollback module.

[0026] The AI ​​toy terminal is used to collect user voice input, play voice messages from the toy, and perform single-capture or single-frame extraction when a visual acquisition command is received. The AI ​​toy terminal may include a microphone, speaker, communication module, camera, local cache unit, and local playback control unit. The microphone is used to collect user voice input; the speaker is used to play voice messages from the toy; the communication module is used to interact with the cloud-based intelligent agent platform; the camera is used to perform single-capture or single-frame extraction when visual trigger conditions are met; the local cache unit is used to cache some local fallback content; and the local playback control unit is used to perform playback control actions such as play, pause, stop, preemption, and resumption.

[0027] The device status management module maintains the device status information of the AI ​​toy terminal. This information includes at least one of the following: device identifier, current role, current language, current playback status, online status, camera availability status, network connection status, and recent interaction time. The current playback status includes at least one of the following: idle status, listening status, broadcasting status, and dialogue status. The device status management module can update the device status information in real time based on the status feedback uploaded by the AI ​​toy terminal, playback completion feedback, network heartbeat, camera status feedback, and task processing results from the cloud-based intelligent agent platform.

[0028] The entry point parsing module is used to acquire interactive input and perform source identification and content normalization processing on the interactive input to generate a unified task object. Interactive input comes from the AI ​​toy terminal or external business systems. Interactive input from the AI ​​toy terminal includes at least one of user voice input, visual trigger input, button trigger input, and local wake-up input; interactive input from external business systems includes at least one of parent app messages, operation backend push messages, mini-program development events, and scheduled tasks.

[0029] A unified task object is a standardized data object that enters the subsequent scheduling and capability routing process. A unified task object includes a device identifier, task source, session number, role identifier, language identifier, task content, and task priority. In a preferred embodiment, the unified task object also includes task category, creation time, expiration time, playback constraints, whether interruption is allowed, whether merging is allowed, whether a receipt is required, and source service identifier. Through the unified task object, the system can transform voice input, visual triggers, parental messages, background push notifications, and developmental events into the same data structure, avoiding contextual inconsistencies caused by different input sources entering different service chains.

[0030] The event and message scheduling module determines the task execution method based on the unified task object and device status information, and determines the time order in which the unified task object enters the capability routing process according to the task execution method. Task execution methods include immediate execution, queuing, delayed broadcasting, merged broadcasting, or preemptive broadcasting. The event and message scheduling module can determine the specific scheduling result based on the task source, task category, task content, task priority, expiration time, whether interruption is allowed, whether merging is allowed, and the current playback status.

[0031] The capability registration module maintains the capability registry. The capability registry includes the service identifier, service address, authentication information, start / stop status, supported task types, supported languages, supported roles, supported scenarios, service priority, timeout threshold, and fault rollback strategy for each capability service. Capability services in the registry can be added, disabled, or adjusted through configuration without modifying the main flow logic of the entry parsing module, event and message scheduling module, and response generation module. This reduces development costs when subsequently expanding to include network search, brand knowledge, visual understanding, bilingual question answering, or other capability services.

[0032] The capability routing module is used to filter candidate capability services based on a unified task object, device status information, and a capability registry, and then determine the target capability service from among the candidate services. When filtering candidate capability services, the module performs filtering based at least on task category, language identifier, role identifier, device status information, and start / stop status. When determining the target capability service, it can further sort it based on task type matching degree, device capability matching degree, role matching degree, language matching degree, service health status, and service response latency.

[0033] The multimodal result generation module invokes the target capability service to process a unified task object and converts the raw processing results returned by the target capability service into structured intermediate results. The structured intermediate results include result type, source identifier, result content, confidence flag, output constraints, and task time. These structured intermediate results mask the differences in return formats between different capability services, enabling the response generation module to generate the target response text based on a unified structure.

[0034] The response generation module generates the target response text based on structured intermediate results, role configuration, and conversation context. Role configuration includes at least one of the following: role name, role settings, tone of voice, prohibited words, applicable age group, language style, and tone identifier. Conversation context includes user history input, system history responses, current role, current language, most recent task category, and most recent structured intermediate result under the same conversation ID. When the historical records under the same conversation ID exceed a preset length, the cloud-based intelligent agent platform retains the most recent rounds of user input and system responses, and summarizes earlier records as part of the conversation context to prevent the conversation context from growing indefinitely.

[0035] The same-source output module is used to synchronously generate the toy's voice broadcast and the external display text based on the target response text, and associates the target response text, the toy's voice broadcast, and the external display text with the same response number. The target response text is the same text source used for speech synthesis and external display in this interaction. The toy's voice broadcast is generated by the voice synthesis service based on the target response text input, and the external display text comes directly from the target response text. By binding the target response text, the toy's voice broadcast, and the external display text with the response number, it is possible to ensure that the actual broadcast content of the AI ​​toy terminal is consistent with the content displayed in the parent app, the operation backend, or the mini-program development system.

[0036] The fault rollback module is used to switch to the rollback service or local fallback content when the target capability service call fails, based on the fault rollback policy in the capability registry. Target capability service call failures include call failure, response timeout, authentication failure, service shutdown, or empty return content. Response timeout is determined by comparing the actual response time of the capability service with the timeout threshold in the capability registry; service shutdown is determined by the start / stop status in the capability registry; empty return content is determined by checking whether the target capability service's return result contains valid content. Local fallback content consists of pre-configured fallback text, fallback audio, or fallback templates in the AI ​​toy terminal or cloud-based intelligent agent platform, categorized and stored according to role and language identifiers.

[0037] like Figure 2 As shown, the AI ​​toy agent scheduling method provided by the present invention includes the following steps.

[0038] S1. Obtain interactive input and simultaneously obtain the device status information of the AI ​​toy terminal.

[0039] Specifically, when a user issues a voice command through the AI ​​toy terminal, the terminal captures the user's voice input via its microphone and uploads the user's voice input, device identifier, and capture time to the cloud-based intelligent agent platform via its communication module. Upon receiving the user's voice input, the cloud-based intelligent agent platform's device status management module reads the corresponding AI toy terminal's device status information based on its device identifier.

[0040] When the parent app, operations backend, or mini-program nurturing system generates messages or events, the external business system sends these messages or events to the cloud-based intelligent agent platform. The cloud-based intelligent agent platform reads the device status information of the corresponding AI toy terminal based on the device identifier carried in the message or event. Therefore, when processing each interactive input, the cloud-based intelligent agent platform can simultaneously determine whether the AI ​​toy terminal is currently in idle, listening, broadcasting, or conversing state, and can also determine whether the AI ​​toy terminal is online, whether its camera is available, and the current role and language used.

[0041] In one implementation, the device status management module maintains a device status table indexed by the device identifier. Each record in the device status table includes the device identifier, current role, current language, current playback status, online status, camera availability status, network connection status, most recent interaction time, and most recent playback completion time. After each voice playback, the AI ​​toy terminal sends a playback completion receipt back to the cloud-based intelligent agent platform. Based on the playback completion receipt, the device status management module updates the current playback status to idle or listening status. Each time the AI ​​toy terminal enters broadcast mode, the device status management module updates the current playback status to broadcast mode. Each time the AI ​​toy terminal enters continuous dialogue mode, the device status management module updates the current playback status to dialogue mode.

[0042] S2. Perform source identification and content normalization processing on the interactive input to generate a unified task object.

[0043] Specifically, the entry point parsing module first identifies the task source of the interactive input. If the interactive input comes from user voice input on the AI ​​toy terminal, the task source is marked as voice dialogue; if the interactive input comes from visual trigger input on the AI ​​toy terminal, the task source is marked as visual understanding; if the interactive input comes from the parent app, the task source is marked as parent message; if the interactive input comes from the operations backend, the task source is marked as backend push; if the interactive input comes from the mini-program development system, the task source is marked as development event; and if the interactive input comes from a scheduled task, the task source is marked as a scheduled task.

[0044] For user voice input, the entry parsing module calls the speech recognition service to convert the user's voice input into user text, which is then used as the task content. The recognized text, recognition confidence score, and language recognition results returned by the speech recognition service can be used as the basis for subsequent language identification and task category determination. For text messages or event content sent by external business systems, the entry parsing module parses the message fields, normalizing the title, body, event type, target device, playback constraints, and effective time into task content. For visually triggered input, the entry parsing module normalizes the trigger text, trigger time, and visual acquisition conditions into task content.

[0045] In one implementation, the entry parsing module assigns a session number to each unified task object. If the interactive input belongs to the current continuous dialogue, the existing session number is reused; if the interval between the interactive input and the most recent interaction time exceeds a preset session interval, or if the interactive input originates from a new external business system event, a new session number is generated. Through the session number, user input, capability service return results, structured intermediate results, target response text, toy-side broadcast voice, and external-side displayed text from the same round or group of continuous interactions can be associated and saved.

[0046] After the unified task object is generated, the entry parsing module initially determines the task category based on the task source and content. For example, user voice input containing question-and-answer intent is identified as real-time dialogue; user voice input containing visual trigger words such as "look," "identify," and "what is this?" is identified as visual understanding; text messages from the operations backend that do not require immediacy are identified as operations push; events from the mini-program's nurturing system that require guiding users to complete virtual nurturing operations are identified as nurturing events; and inputs containing safety reminders, device malfunction reminders, or emergency messages from parents are identified as safety reminders.

[0047] S3. Determine the task execution method based on the unified task object and device status information, and determine the time sequence of the unified task object entering the capability routing process according to the task execution method.

[0048] Specifically, the event and message scheduling module identifies the task category corresponding to the unified task object based on the task source, and determines the task priority based on the task category and task content. Task execution methods include immediate execution, queuing, delayed broadcasting, merged broadcasting, or preemptive broadcasting.

[0049] In one implementation, task categories include real-time dialogue, visual understanding, safety reminders, parental messages, child development events, operational push notifications, and scheduled tasks. Task priority decreases in the following order: safety reminders, real-time dialogue, visual understanding, parental messages, child development events, scheduled tasks, and operational push notifications. For multiple identical task objects within the same task category, the event and message scheduling module performs secondary sorting based on creation time, expiration time, whether interruption is allowed, and whether a receipt is required.

[0050] To further improve scheduling stability, in a preferred embodiment, the event and message scheduling module calculates a scheduling score for a unified task object. The scheduling score is determined as follows: ; Where Q represents the scheduling score, P represents the basic priority corresponding to the task category, T represents the timeliness factor corresponding to the remaining effective time of the task, U represents the urgency factor corresponding to the task content, B represents the interruption cost corresponding to the current playback state, and α, β, γ, and δ represent preset weights. Each factor can be pre-normalized to the same value range, and the weights can be configured according to business strategies. The higher the scheduling score, the higher the priority of the unified task object in the capability routing process. In other implementations, rule tables, state machines, or decision trees can also be used to achieve equivalent task execution method determination.

[0051] When the AI ​​toy terminal is idle, the event and message scheduling module determines that unified task objects with a scheduling score higher than the preset execution threshold should be executed immediately. When the AI ​​toy terminal is in broadcast mode, if a newly arrived unified task object is a safety reminder or real-time dialogue type, and the unified task object allows interruption, it is determined to preempt broadcasting; if the newly arrived unified task object is an operation push or scheduled task type, it is determined to queue or delay broadcasting. When multiple development events arrive consecutively within a preset merging time window, and the task content belongs to the same development theme and merging is allowed, the event and message scheduling module merges the multiple unified task objects into a new unified task object and determines the task execution method as merged broadcasting.

[0052] Through the above scheduling process, we can avoid the AI ​​toy terminal being interrupted by low-priority operation push messages while it is having a real-time conversation, and we can also avoid the failure to broadcast high-priority safety reminders or emergency messages to parents in a timely manner.

[0053] S4. Filter candidate capability services based on the unified task object, device status information and capability registry, and determine the target capability service from the candidate capability services.

[0054] like Figure 3As shown, the capability registration module maintains the capability registry. The capability registry includes the service identifier, service address, authentication information, start / stop status, supported task types, supported languages, supported roles, supported scenarios, service priority, timeout threshold, and fallback policy for each capability service. Capability services can be dynamically added, deleted, enabled, disabled, or have their configurations updated according to business needs. For example, when adding a picture book recognition service, simply configuring the corresponding service identifier, service address, supported task types, supported languages, supported roles, supported scenarios, timeout threshold, and fallback policy in the capability registry will allow it to be included in the candidate capability service selection range by the capability routing module.

[0055] The capability routing module first filters the capability registry based on the task category, language identifier, role identifier, and device status information in the unified task object, eliminating capability services that do not support the corresponding task category, do not support the corresponding language identifier, do not support the corresponding role identifier, do not match the current device status information, or are disabled, thus obtaining candidate capability services.

[0056] For example, if a unified task object corresponds to a visual understanding task, but the device status information indicates that the camera is unavailable, the capability routing module will not select the visual understanding service and will instead redirect the unified task object to the "unable to collect visual data" prompt link. Another example is if the unified task object's language identifier is English, but a certain brand's knowledge service only supports Chinese, the capability routing module will remove that brand's knowledge service from the candidate capability services. Yet another example is if the unified task object corresponds to a background push task, and a certain online search service does not support background push tasks, then that online search service will not be selected as a candidate capability service.

[0057] After obtaining candidate capability services, the capability routing module calculates the service matching score of the candidate capability services based on the degree of matching of task type, device capability, role, language, service health status and service response latency, and selects the candidate capability service with the highest service matching score and that meets the timeout threshold as the target capability service.

[0058] In a preferred embodiment, the service matching score is determined as follows: ; Where M represents the service matching score, A represents the task type matching degree, D represents the device capability matching degree, R represents the role matching degree, L represents the language matching degree, H represents the service health status, and E represents the service response latency. , , , , and This indicates a preset weight. The degree of matching for task type, device capability, role, and language can be determined by the supported task type, supported scenario, supported role, and supported language fields in the capability registry. Service health status can be determined by the call success rate, number of timeouts, and number of exceptions in a recent period. Service response latency can be determined by the average response time of the most recent calls. All of the above factors can be normalized before being used in the calculation. In other implementations, preset priority rules or multi-condition sorting rules can also be used to determine the target capability service.

[0059] In this way, the system no longer simply calls capability services in a fixed service order, but dynamically determines the target capability service by combining the current device status of the AI ​​toy terminal, the character language configuration, and the running status of the capability service, thereby improving the adaptability and stability of capability calls.

[0060] S5. Call the target capability service to process the unified task object and generate structured intermediate results.

[0061] Specifically, the capability routing module sends a unified task object or necessary fields from the unified task object to the target capability service. After processing, the target capability service returns the original processing result. The multimodal result generation module performs format conversion and field normalization on the original processing result to generate a structured intermediate result.

[0062] Structured intermediate results include result type, source identifier, result content, confidence flag, output constraints, and task time. Result type identifies whether the structured intermediate result originates from visual understanding, knowledge retrieval, business events, parental messages, background push notifications, or fallback content. Source identifier identifies the service that generated the result. Result content records structured information that can be used by the response generation module. Confidence flag indicates the reliability of the structured intermediate result. Output constraints limit the output scope of the response generation module, such as whether it must be broadcast, whether it needs to be displayed to the parent app, or whether it is only for internal recording. Task time records the processing time corresponding to the structured intermediate result.

[0063] For example, when the target capability service is brand knowledge service, the structured intermediate results include the knowledge entry title, knowledge text, applicable age group, prohibited expressions, and recommended response direction. When the target capability service is a business linkage service corresponding to a mini-program development system, the structured intermediate results include the event name, event status, reward content, next step guidance, and whether a broadcast is required. When the target capability service is visual understanding service, the structured intermediate results include image description, main object, object attributes, scene tags, risk tags, and visual confidence level. When the target capability service is online search service, the structured intermediate results include the search question, search summary, citation source identifier, and timeliness identifier.

[0064] In one implementation, the multimodal result generation module also performs output security checks on the structured intermediate results. Output security checks may include age-appropriate filtering, sensitive content filtering, dangerous behavior warning filtering, and role-restricted word filtering. When a structured intermediate result fails the output security check, the multimodal result generation module marks the result type as a restricted result and sets the output constraints to prohibit direct broadcasting or require replacement with safety warning content.

[0065] S6. Generate the target response text based on the structured intermediate results, role configuration, and session context.

[0066] Specifically, the response generation module combines structured intermediate results with the session context to form the response generation input. The session context includes user history input, system history responses, current role, current language, most recent task category, and most recent structured intermediate result under the same session ID. Role configuration includes at least one of the following: role name, role settings, tone of voice, prohibited words, applicable age group, language style, and tone identifier.

[0067] The response generation module determines response style constraints based on role configuration and language identifiers, and generates target response text based on the response generation input and response style constraints. The target response text is the sole text source used for subsequent speech synthesis and external display in this interaction. Instead of generating separate text for the toy's broadcast and external display, the system first generates the target response text, which then synchronously drives the generation of the toy's broadcast speech and the external display text.

[0068] In one implementation, when the unified task object corresponds to the Chinese mode, the response generation module generates the target Chinese response text according to the Chinese response style constraints; when the unified task object corresponds to the English mode, the response generation module generates the target English response text according to the English response style constraints. The response style constraints for different languages ​​are all mapped from the same role configuration to ensure that the expression style of the same role remains consistent across different languages.

[0069] In another implementation, when the structured intermediate result contains output constraints, the response generation module prioritizes satisfying the output constraints. For example, when the output constraint representation requires a brief broadcast, the response generation module controls the length of the target response text; when the output constraint representation is only used for external display, the response generation module generates target response text that does not trigger broadcasting on the AI ​​toy terminal; when the output constraint representation prohibits direct broadcasting, the response generation module calls a safety prompt template to generate target response text suitable for children's scenarios.

[0070] S7. Based on the target reply text, simultaneously generate the toy-side broadcast voice and the external terminal display text, and associate the target reply text, the toy-side broadcast voice, and the external terminal display text with the same reply number.

[0071] like Figure 6 As shown, the same-source output module generates a response number for the target response text and inputs the target response text into the speech synthesis service to generate the toy's broadcast voice. Simultaneously, the same-source output module sends the target response text as the external display text to the parent's app, the operations backend, or the mini-program development system. The conversation log saves the device identifier, conversation number, response number, target response text, audio address of the toy's broadcast voice, external display text, task source, and generation time.

[0072] In one implementation, the response number is generated based on the device identifier, session number, timestamp, and incrementing sequence number. The speech synthesis service generates the toy's broadcast voice based on the timbre and language identifiers in the role configuration. If the speech synthesis service returns an audio file address, the same-source output module sends the audio file address to the AI ​​toy terminal; if the speech synthesis service returns an audio stream, the same-source output module forwards the audio stream to the AI ​​toy terminal. After playing the toy's broadcast voice, the AI ​​toy terminal sends a playback completion status back to the cloud-based intelligent agent platform. The device status management module updates the current playback status to idle or listening status based on the playback completion status.

[0073] Since the text displayed on the external end comes directly from the target response text, and the voice broadcast on the toy end is also synthesized from the target response text, the text content seen in the parent app, the operation backend, or the mini-program development system has the same text source as the actual broadcast content of the AI ​​toy terminal, which can reduce semantic inconsistency issues and facilitate subsequent traceability.

[0074] S8. When the target capability service call fails, switch to the rollback capability service or local fallback content according to the fault rollback policy in the capability registry, and keep the role identifier and language identifier in the unified task object unchanged.

[0075] Specifically, exceptions to the target capability service call include call failure, response timeout, authentication failure, service shutdown, or empty return content. When the capability routing module detects an exception to the target capability service call, the fault rollback module reads the fault rollback policy corresponding to the target capability service from the capability registry. The fault rollback policy includes at least one of the following: retry, switching to a similar backup service, switching to a general Q&A service, switching to the default knowledge base service, or calling local fallback content.

[0076] Response timeout is determined by comparing the actual response time of the capability service with the timeout threshold in the capability registry. Authentication failure is determined by the authentication error status returned by the target capability service. Service shutdown is determined by the start / stop status in the capability registry. Empty return content is determined by checking whether the result returned by the target capability service contains valid content.

[0077] In one implementation, if the visual understanding service times out, the fallback module does not continue to trigger camera capture. Instead, it uses local fallback content to generate prompt content consistent with the role and language identifiers. If the brand knowledge service is unavailable, the fallback module switches to the default knowledge base service or the general question-and-answer service, and passes the role and language identifiers from the unified task object to the fallback capability service, ensuring that the fallback-generated content retains the same role and language. If the speech synthesis service call fails, the same-source output module retains the target response text and displays the text normally to the external terminal, while the AI ​​toy terminal plays a preset short prompt tone or local fallback voice.

[0078] The fault rollback module associates call exception information and rollback processing results with the session log. Call exception information includes the exception type, exception service identifier, exception time, and exception description. Rollback processing results include the rollback capability service identifier, local fallback content identifier, and final output status. By recording the above information, the cloud-based intelligent agent platform can statistically analyze the frequency of capability service exceptions during subsequent operations and maintenance, and adjust the service health status in the capability registry based on the exception frequency.

[0079] Example 2 This embodiment describes the specific process by which user voice input triggers visual understanding.

[0080] When the AI ​​toy terminal is in listening mode, the user utters the voice command, "Help me see what this is." The AI ​​toy terminal collects the user's voice input and uploads it to the cloud-based intelligent agent platform. The entry point parsing module calls the speech recognition service to convert the user's voice input into text. The entry point parsing module identifies the visual trigger word "look" in the user's text and reads the camera availability status from the device status information.

[0081] If the camera's availability indicates that the AI ​​toy terminal possesses visual acquisition capabilities, the entry parsing module generates a unified task object categorized as visual understanding. The event and message scheduling module determines whether this unified task object can be executed immediately based on the current playback status. When the AI ​​toy terminal is in an idle or listening state, the event and message scheduling module determines the task execution method as immediate execution; when the AI ​​toy terminal is playing a low-priority background push message, the event and message scheduling module determines the task execution method as preemptive playback and sends a control command to the AI ​​toy terminal to stop the current playback.

[0082] Subsequently, the capability routing module, based on the task category, language identifier, role identifier, and device status information in the unified task object, filters visual understanding services from the capability registry that support visual understanding tasks, support the current language, support the current role, and are in an enabled state, thus determining the target capability service. After receiving the visual acquisition command, the AI ​​toy terminal performs a single snapshot or single frame capture and uploads the visual input data to the cloud-based intelligent agent platform. The visual understanding service processes the visual input data and outputs raw processing results such as image description, main objects, object attributes, and visual confidence scores. The multimodal result generation module converts the raw processing results into structured visual results and uses these structured visual results as components of the structured intermediate results.

[0083] The response generation module generates target response text based on structured visual results, user text, role configuration, and conversation context. For example, in the child companion role, the target response text can describe the identified object in a tone suitable for children's understanding and guide the user to continue interacting. The source output module generates a response number for the target response text, inputs the target response text into a speech synthesis service to generate the toy's broadcast voice, and simultaneously sends the target response text as external display text to the parent's app. Through this process, visual interaction is only executed when actively triggered by the user and the device has visual acquisition capabilities, avoiding increased power consumption and privacy risks caused by continuous camera acquisition.

[0084] If the camera's availability indicates that the AI ​​toy terminal lacks visual acquisition capabilities, or if the camera is off, the entry parsing module still generates a unified task object. However, the capability routing module does not select the visual understanding service; instead, it redirects the unified task object to the prompting link. The response generation module generates a target response text based on the role configuration and language identifier to inform the user that visual recognition is currently unavailable. This target response text is still used by the same-source output module to generate the toy's broadcast voice and the external display text, and is associated with the same response number.

[0085] Example 3 This embodiment describes the scheduling process when external business system messages and real-time dialogues occur concurrently.

[0086] When the AI ​​toy terminal is engaged in real-time dialogue with the user, the mini-program nurturing system sends a nurturing event to the cloud-based intelligent agent platform, while the operations backend simultaneously sends an operations push message. The entry parsing module normalizes the nurturing event and the operations push message into two unified task objects. The event and message scheduling module reads the device status information to determine that the AI ​​toy terminal is currently in a dialogue state.

[0087] For development events, the event and message scheduling module reads the expiration time and whether a receipt is required from the unified task object. If a development event needs to be broadcast within a short period of time, the event and message scheduling module adds it to a medium-priority queue and broadcasts it after the user's current round of dialogue ends. If development events can be merged, and there are multiple similar development events within a preset merging time window, the event and message scheduling module merges multiple development events into a unified task object to reduce duplicate broadcasts.

[0088] For operational push messages, the event and message scheduling module identifies the task category as a low-priority operational push and determines that the current playback state is a dialogue state. Therefore, it determines the task execution method as delayed playback or queuing. Only when the AI ​​toy terminal enters an idle state and the operational push message has not yet expired will the event and message scheduling module allow it to enter the capability routing process.

[0089] Through the above scheduling method, real-time dialogue will not be interrupted by low-priority operational push messages, while high-priority or time-sensitive development events can be broadcast at appropriate times, thereby achieving orderly arrangement among multi-source interactive tasks.

[0090] Example 4 This embodiment describes the specific application process of the capability registry and the fault rollback strategy.

[0091] The capability registry records brand knowledge services, encyclopedia knowledge services, online search services, language capability services, and local fallback content services. Each capability service corresponds to a service identifier, service address, authentication information, start / stop status, supported task types, supported languages, supported roles, supported scenarios, service priority, timeout threshold, and fault rollback strategy.

[0092] When a user asks a question related to the brand story, the capability routing module prioritizes the brand knowledge service based on the task category and content of the unified task object. If the brand knowledge service returns a normal result within the timeout threshold, the multimodal result generation module generates a structured intermediate result based on the returned result, and the response generation module generates the target response text. If the brand knowledge service response times out, the fault rollback module reads the fault rollback strategy corresponding to the brand knowledge service. If the fault rollback strategy is configured to first switch the default knowledge base service, then switch the general Q&A service, and finally call the local fallback content, the fault rollback module sequentially checks whether the default knowledge base service and the general Q&A service are available.

[0093] When the default knowledge base service is available, the fallback module sends a unified task object to it, along with the role and language identifiers, ensuring that the content returned by the default knowledge base service remains suitable for the current role and language. When the default knowledge base service is also unavailable, the fallback module switches to the general question-and-answer service. When all fallback services are unavailable, the fallback module uses local fallback content. This local fallback content is pre-configured according to the role and language identifiers and outputs concise, stable, and child-friendly prompts.

[0094] Regardless of whether a rollback service or local fallback content is used, the response generation module uses the rollback result as a structured intermediate result or a component thereof to continue generating the target response text. The same-source output module still generates a response number for the target response text and simultaneously generates the toy's broadcast voice and the external display text based on the target response text. Therefore, even if some service capabilities malfunction, the system can maintain the continuity of the dialogue and avoid the AI ​​toy terminal becoming silent or directly reporting an error.

[0095] Example 5 This embodiment describes the recording method for output links from the same source.

[0096] After each response is generated, the same-source output module first generates a response number for the target response text. The response number can be generated based on the device identifier, session number, timestamp, and incrementing sequence number, or it can be uniformly assigned by the number generator of the cloud-based intelligent agent platform. The response number is used to uniquely identify each output result.

[0097] The same-source output module sends the target reply text to the speech synthesis service, which generates the toy's broadcast voice based on the timbre and language identifiers in the role configuration. The same-source output module also sends the target reply text as the external display text to the external business system. The session record includes the device identifier, session number, reply number, target reply text, toy's broadcast voice, and external display text in the same record item.

[0098] When the parent app needs to view the most recent broadcast content from the AI ​​toy terminal, it reads the text displayed on the external device using the reply number. When the operations backend needs to trace a specific broadcast, it reads the target reply text, the audio address of the toy's broadcast voice, and the task source using the same reply number. Since the target reply text is the common source of both the toy's broadcast voice and the text displayed on the external device, semantic discrepancies caused by generating separate display text after the voice broadcast is completed can be avoided.

[0099] Example 6 This embodiment describes the collaborative working process of system modules.

[0100] The AI ​​toy terminal collects user voice input via a microphone and uploads it to the cloud-based intelligent agent platform via a communication module. The device status management module reads the device status information based on the device identifier. The entry parsing module identifies the source of the user voice input and normalizes its content to generate a unified task object. The event and message scheduling module determines the task execution method based on the unified task object and device status information, and sends the unified task object into the capability routing process when the execution conditions are met.

[0101] The capability registration module provides a capability registry to the capability routing module. The capability routing module filters candidate capability services based on the unified task object, device status information, and the capability registry, and determines the target capability service. The multimodal result generation module calls the target capability service and converts the processing result into a structured intermediate result. The response generation module generates the target response text based on the structured intermediate result, role configuration, and session context. The same-source output module synchronously generates the toy-side broadcast voice and the external-side display text based on the target response text, and associates them with the same response number. When the target capability service call fails, the fault rollback module switches to the rollback capability service or local fallback content according to the fault rollback strategy.

[0102] In the aforementioned collaborative process, all interactive inputs are first transformed into a unified task object before entering the event and message scheduling module and the capability routing module; all raw processing results returned by capability services are first transformed into structured intermediate results before entering the response generation module; all toy-side broadcast voices and external-side displayed texts originate from the target response text and are associated through response numbers. Therefore, this invention can establish a unified, scalable, and traceable scheduling link between multi-source inputs, multi-capability calls, multi-language character outputs, and anomaly rollbacks in AI toy terminals.

[0103] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for scheduling AI toy intelligent agents, characterized in that, include: S1. Obtain interactive input and simultaneously obtain device status information of the AI ​​toy terminal. The interactive input comes from the AI ​​toy terminal or an external business system. S2. Perform source identification and content normalization processing on the interactive input to generate a unified task object. The unified task object includes device identifier, task source, session number, role identifier, language identifier, task content, and task priority. S3. Determine the task execution method based on the unified task object and the device status information, and determine the time order in which the unified task object enters the capability routing process according to the task execution method; S4. Filter candidate capability services based on the unified task object, the device status information and the capability registry, and determine the target capability service from the candidate capability services; S5. Invoke the target capability service to process the unified task object and generate structured intermediate results; S6. Generate the target response text based on the structured intermediate results, role configuration, and session context; S7. Based on the target reply text, simultaneously generate the toy-side broadcast voice and the external terminal display text, and associate the target reply text, the toy-side broadcast voice, and the external terminal display text with the same reply number; S8. When the target capability service call is abnormal, switch to the rollback capability service or local fallback content according to the fault rollback policy in the capability registry, and keep the role identifier and language identifier in the unified task object unchanged.

2. The AI ​​toy agent scheduling method according to claim 1, characterized in that, The device status information includes at least one of the following: current role, current language, current playback status, online status, camera availability status, network connection status, and recent interaction time; the current playback status includes at least one of the following: idle status, listening status, broadcasting status, and dialogue status.

3. The AI ​​toy agent scheduling method according to claim 2, characterized in that, The task execution methods include immediate execution, queuing, delayed broadcasting, merged broadcasting, or preemptive broadcasting; the steps for determining the task execution method include: Identify the task category corresponding to the unified task object based on the task source; The task priority is determined based on the task category and the task content; The task execution method is determined based on the task priority and the current playback status.

4. The AI ​​toy agent scheduling method according to claim 1, characterized in that, The capability registry includes the service identifier, service address, authentication information, start / stop status, supported task types, supported languages, supported roles, supported scenarios, service priority, timeout threshold, and fault rollback strategy for the capability service.

5. The AI ​​toy agent scheduling method according to claim 4, characterized in that, The steps for determining the target capability service from the candidate capability services include: Eliminate capability services that do not match the task category, language identifier, role identifier, device status information, or start / stop status to obtain the candidate capability services; The service matching score of the candidate capability service is calculated based on the degree of matching of task type, device capability, role, language, service health status, and service response latency. The candidate capability service with the highest service matching score and that meets the timeout threshold is selected as the target capability service.

6. The AI ​​toy agent scheduling method according to claim 1, characterized in that, When the unified task object corresponds to user voice input, the user voice input is converted into user text; when the user text hits a visual trigger word and the device status information indicates that the AI ​​toy terminal has visual acquisition capabilities, the AI ​​toy terminal is controlled to perform a single capture or a single frame capture to obtain visual input data; the visual input data is input into the visual understanding service, a structured visual result is output, and the structured visual result is used as a component of the structured intermediate result.

7. The AI ​​toy agent scheduling method according to claim 1, characterized in that, The structured intermediate results include result type, source identifier, result content, confidence identifier, output constraints, and task time; The steps for generating the target response text include: The structured intermediate results are combined with the session context to generate a response input; The response style constraints are determined based on the role configuration and the language identifier; The target response text is generated based on the response generation input and the response style constraints.

8. The AI ​​toy agent scheduling method according to claim 1, characterized in that, The steps for synchronously generating the toy's broadcast voice and the external display text based on the target reply text include: Generate the response number for the target response text; The target response text is input into a speech synthesis service to generate the broadcast voice from the toy. The target response text will be sent as the external display text to the parent app, the operations backend, or the mini-program development system. Write the device identifier, the session number, the reply number, the voice broadcast from the toy terminal, and the text displayed on the external terminal into the session record.

9. The AI ​​toy agent scheduling method according to claim 1, characterized in that, The target capability service call anomalies include call failure, response timeout, authentication failure, service shutdown, or empty return content; the steps to switch to the rollback capability service or local fallback content according to the fault rollback strategy include: Read the fault rollback policy corresponding to the target capability service; The rollback capability service is determined according to the fault rollback strategy; When the rollback capability service is unavailable, the local fallback content is invoked; The call exception information and rollback processing results are associated with the session record.

10. An AI toy intelligent agent scheduling system, characterized in that, include: The AI ​​toy terminal is used to collect user voice, play the voice broadcast from the toy, and perform a single capture or a single frame capture when it receives a visual acquisition command. The device status management module is used to maintain the device status information of the AI ​​toy terminal; The entry parsing module is used to acquire interactive input, and to perform source identification and content normalization processing on the interactive input to generate a unified task object; The event and message scheduling module is used to determine the task execution method based on the unified task object and the device status information, and to determine the time order in which the unified task object enters the capability routing process according to the task execution method. The capability registration module is used to maintain the capability registry. The capability routing module is used to filter candidate capability services based on the unified task object, the device status information, and the capability registry, and to determine the target capability service from the candidate capability services. The multimodal result generation module is used to call the target capability service to process the unified task object and generate structured intermediate results; The response generation module is used to generate target response text based on the structured intermediate results, role configuration, and session context. The same-source output module is used to synchronously generate the toy-side broadcast voice and the external terminal display text based on the target reply text, and associate the target reply text, the toy-side broadcast voice, and the external terminal display text with the same reply number; The fault rollback module is used to switch to the rollback capability service or local fallback content according to the fault rollback policy in the capability registry when the target capability service call is abnormal.

Images