Control method of embodied intelligent robot based on hierarchical hybrid model

Abstract

The application provides a control method of an embodied intelligent robot based on a hierarchical hybrid model, and relates to the field of embodied intelligence. The application is based on a collaborative hybrid model architecture, dynamically allocates processing paths according to the task complexity corresponding to the ASR text, allocates simple tasks to low-delay local text small models for processing, and allocates complex tasks to remote multi-modal large models for processing. The application significantly improves the performance of the embodied intelligent robot in terms of interactive experience, response speed, task adaptability, etc., while reducing the computing and communication overhead. In addition, the dual evaluation confidence method based on audio and text further ensures the accuracy and robustness of speech recognition.

Description

Technical Field

[0001] This invention relates to the field of embodied intelligence, and more specifically to a control method for an embodied intelligent robot based on a hierarchical hybrid model. Background Technology

[0002] Embodied Intelligence (EI) is a cutting-edge field at the intersection of artificial intelligence and robotics. It emphasizes that intelligent agents achieve autonomous learning and evolution through dynamic interaction between their bodies and the environment. Its core lies in the deep integration of perception, action, and cognition.

[0003] In related technologies, the development of embodied intelligent interactive scenarios generally adopts a system architecture based on a large language model (LLM) to achieve a closed loop of "perception-decision-execution" in complex environments. For example, the paper (Gao Chao, Yang Ying, Chen Shichao, et al. A review of research on embodied intelligence driven by multimodal models [J / OL]. Intelligent Sensing Engineering, 1-12 [2025-06-23].) constructs an embodied intelligent system with voice as the interaction medium. Its core is to obtain user instructions through automatic speech recognition (ASR), perform environmental perception, task planning and decision generation by a multimodal large model, and finally output the results through text-to-speech (TTS) or robot action execution.

[0004] However, the long response time of large models and the fact that traditional TTS requires waiting for the large model to output complete text before synthesizing speech always cause interaction delays and seriously affect the user experience. Summary of the Invention

[0005] (a) Technical problems to be solved

[0006] To address the shortcomings of existing technologies, this invention provides a control method for an embodied intelligent robot based on a hierarchical hybrid model, which solves the technical problem of how to optimize interaction latency.

[0007] (II) Technical Solution

[0008] To achieve the above objectives, the present invention provides the following technical solution:

[0009] A control method for an embodied intelligent robot based on a hierarchical hybrid model, which is based on a cooperative hybrid model architecture, including a local text-based small model and a remote multimodal large model; the method includes:

[0010] It receives voice input from the target user in real time and activates the remote ASR model when a wake word is detected;

[0011] Based on the audio features of the input speech, the activated remote ASR model is used to identify the user's voiceprint and speech content, and the voiceprint recognition result and ASR text are obtained respectively.

[0012] Select the local text mini-model and call the local knowledge base to attempt to analyze the ASR text. If it does not exceed the model's understanding range, obtain the first text rationality feature; otherwise, select the remote multimodal large model and call the local knowledge base to analyze the ASR text and obtain the second text rationality feature.

[0013] Based on the voiceprint recognition result and the first text rationality feature or the second text rationality feature, a confidence level is calculated, and when the confidence level exceeds a preset threshold, embodied intelligent voice feedback and action execution are triggered.

[0014] Preferably, the input speech is noise-reduced before real-time detection of the wake word.

[0015] Preferably, after the input speech is denoised, a lightweight model continuously monitors the system and activates the remote ASR model only when the wake word is matched; and if no new speech is detected within a preset time after activation, the system automatically exits the ASR session.

[0016] Preferably, the voiceprint recognition result is defined as f1, the first text rationality feature or the second text rationality feature is defined as f2, and the confidence level is represented as:

[0017]

[0018] Where α is the confidence level, Normalize(∙) is the normalization calculation, and w is the weight.

[0019] Preferably, the embodied intelligent voice feedback includes:

[0020] If the local text small model is selected, the analysis result of the ASR text will be used as the first text; if the remote multimodal large model is selected, the ASR text will be split and reorganized into several second texts.

[0021] Real-time streaming output of the first or second text based on the WebSocket protocol;

[0022] Synchronously scan the real-time streaming text output and truncate it using Chinese punctuation. Inject the complete sentence before the punctuation into the audio queue and continue to cache the remaining text to wait for the next punctuation trigger.

[0023] The text in the audio queue is sequentially processed by speech synthesis to output an audio stream through the embodied intelligent robot.

[0024] Preferably, the action includes:

[0025] If the local text small model is selected, the ASR text is understood to obtain the first action sequence; if the remote multimodal large model is selected, the ASR text is subjected to complex task decomposition and recombination, and the second action sequence is understood to obtain the second action sequence.

[0026] By querying a preset action list through a function call mechanism, the embodied intelligent robot is controlled to execute the first action sequence or the second action sequence in sequence.

[0027] Preferably, during the process of the embodied intelligent robot executing the second action sequence in sequence, the environmental state is perceived in real time and fed back to the remote multimodal large model in order to dynamically adjust the execution strategy or parameters of the remaining action sequence.

[0028] A control system for an embodied intelligent robot based on a hierarchical hybrid model, wherein the system is based on a cooperative hybrid model architecture, the cooperative hybrid model architecture including a local text-based small model and a remote multimodal large model; the system includes:

[0029] The wake-up module is used to receive the target user's voice input in real time and activate the remote ASR model when a wake word is detected;

[0030] The recognition module is used to identify the user's voiceprint and speech content based on the audio features of the input speech using an activated remote ASR model, and to obtain the voiceprint recognition result and ASR text respectively.

[0031] The selection and analysis module is used to select the local text small model and call the local knowledge base to attempt to analyze the ASR text. If it does not exceed the model's understanding range, the first text rationality feature is obtained; otherwise, the remote multimodal large model is selected and the local knowledge base is called to analyze the ASR text to obtain the second text rationality feature.

[0032] The decision-making and execution module is used to calculate the confidence level based on the voiceprint recognition result and the first text rationality feature or the second text rationality feature, and to trigger embodied intelligent voice feedback and action execution when the confidence level exceeds a preset threshold.

[0033] A storage medium storing a computer program for controlling an embodied intelligent robot based on a hierarchical hybrid model, wherein the computer program causes a computer to perform the control method described above.

[0034] An electronic device, comprising:

[0035] One or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the programs including control methods as described above.

[0036] (III) Beneficial Effects

[0037] This invention provides a control method for an embodied intelligent robot based on a hierarchical hybrid model. Compared with existing technologies, it has the following advantages:

[0038] This invention is based on a collaborative hybrid model architecture, dynamically allocating processing paths according to the task complexity corresponding to the ASR text. Simple tasks are assigned to low-latency local text small models for processing, while complex tasks are assigned to high-precision remote multimodal large models for processing. This significantly improves the performance of embodied intelligent robots in terms of interactive experience, response speed, and task adaptability, while reducing computational and communication overhead. Furthermore, a dual confidence assessment method based on both audio and text further ensures the accuracy and robustness of speech recognition. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 A block diagram illustrating a control method for an embodied intelligent robot based on a hierarchical hybrid model, provided in an embodiment of the present invention.

[0041] Figure 2 A flowchart illustrating a control method for an embodied intelligent robot based on a hierarchical hybrid model, provided as an embodiment of the present invention. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention are described clearly and completely. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0043] This application provides a control method for an embodied intelligent robot based on a hierarchical hybrid model, which solves the technical problem of how to optimize interaction latency.

[0044] The technical solution in this application is to solve the above-mentioned technical problems, and the general idea is as follows:

[0045] First, the designed collaborative hybrid model architecture significantly improves the optimization of interaction latency. Second, a two-stage optimization mechanism is constructed for the remote ASR model, including front-end noise reduction and back-end recognition. Voiceprint recognition is added during the back-end recognition process to determine user identity, thereby calling the corresponding local knowledge base and enhancing the model's contextual understanding ability. Third, dual confidence assessment ensures the accuracy and robustness of speech recognition. Finally, a streaming processing mechanism is adopted in the TTS (Text-to-Speech) synthesis stage to further alleviate latency issues; and during action execution, multimodal data, including environmental feedback, collaboratively provides precise action adjustments to the embodied intelligent robot, ensuring environmental adaptability.

[0046] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0047] Example 1:

[0048] like Figure 1 As shown, this embodiment of the invention provides a control method for an embodied intelligent robot based on a hierarchical hybrid model. The method is based on a collaborative hybrid model architecture, which includes a local text-based small model and a remote multimodal large model. The method includes:

[0049] S1. Receive the target user's voice input in real time and activate the remote ASR model when a wake word is detected;

[0050] S2. Based on the audio features of the input speech, the activated remote ASR model is used to identify the user's voiceprint and speech content, and the voiceprint recognition result and ASR text are obtained respectively.

[0051] S3. Select the local text small model and call the local knowledge base to try to analyze the ASR text. If it does not exceed the model's understanding range, obtain the first text rationality feature; otherwise, select the remote multimodal large model and call the local knowledge base to analyze the ASR text and obtain the second text rationality feature.

[0052] S4. Based on the voiceprint recognition result and the first text rationality feature or the second text rationality feature, calculate the confidence level, and when the confidence level exceeds a preset threshold, trigger embodied intelligent voice feedback and action execution.

[0053] This invention dynamically allocates processing paths based on the task complexity corresponding to the ASR text. Simple tasks are assigned to low-latency local text small models for processing, while complex tasks are assigned to high-precision remote multimodal large models for processing. This significantly improves the performance of embodied intelligent robots in terms of interactive experience, response speed, and task adaptability, while reducing computational and communication overhead. Furthermore, the dual confidence assessment method based on audio and text further ensures the accuracy and robustness of speech recognition.

[0054] like Figure 2 As shown, Figure 2 A flowchart of a control method for an embodied intelligent robot based on a hierarchical hybrid model is disclosed.

[0055] Next, we will combine Figure 2 The steps of the above scheme are described in detail:

[0056] In step S1, the input voice of the target user is received in real time, and the remote ASR model is activated when a wake word is detected.

[0057] To reduce the impact of the environment, especially in noisy or multi-person talking environments, on the quality of the input speech, this step also performs noise reduction processing on the input speech before real-time detection of the wake word.

[0058] For example, the wake word mentioned above could be "Hi, Robot" or any other phrase / sentence. See also... Figure 2 This step specifically uses a local denoising mini-model to perform the denoising operation.

[0059] After denoising the input speech, this step simultaneously uses a lightweight model for continuous monitoring (low-power monitoring), activating the remote ASR model only when the wake word is matched. Furthermore, if no new speech is detected within a preset time after activation, the ASR session is automatically terminated (timeout truncation). This timeout truncation design improves system stability and enhances user experience.

[0060] For example, the preset duration can be set to 10 seconds. See also... Figure 2 In this step, the remote wake-up model is specifically used to activate the remote ASR model.

[0061] In step S2, based on the audio features of the input speech, the activated remote ASR model is used to identify the user's voiceprint and speech content, and the voiceprint recognition result and ASR text are obtained respectively.

[0062] Voiceprint recognition (VR) is a biometric identification technology that identifies a target by analyzing their voice characteristics. In this step, an activated remote ASR model is used to identify the user's voiceprint to determine the target user's identity. Specifically, the audio features of the input speech are compared with multiple pre-trained voiceprint models to find the model with the highest matching degree (i.e., the most similar), thereby determining the target user's identity.

[0063] It should be noted that the above voiceprint recognition result refers to the degree of matching between the audio features of the input speech and the most similar voiceprint model. For example, this step specifically uses the matching degree at each time step to take the log-likelihood probability, then calculates the average value, and finally determines the degree of matching between the audio features of the input speech and the most similar voiceprint model.

[0064] At the same time, this step utilizes the activated remote ASR model to recognize speech content and obtain the corresponding ASR text.

[0065] In step S3, the local text mini-model is selected and the local knowledge base is called to attempt to analyze the ASR text. If it does not exceed the model's understanding range, the first text rationality feature is obtained; otherwise, the remote multimodal large model is selected and the local knowledge base is called to analyze the ASR text and obtain the second text rationality feature.

[0066] This invention presents a collaborative hybrid model architecture that optimizes interaction latency and improves user experience through a collaborative division of labor mechanism between a local text small model (low latency) and a remote multimodal large model (high precision).

[0067] In an optional implementation, the aforementioned local knowledge base corresponds to a preset user and is used to record the ASR text corresponding to the preset user. In this step, calling the local knowledge base specifically means that after determining the identity of the target user based on the voiceprint matching result, the local knowledge base of the target user is called; if it is a new identity, the local knowledge base of the target user is created and the calling process ends.

[0068] Building upon this, in this step, the ASR text obtained in step S2 is passed to the local text mini-model, and the local knowledge base corresponding to the target user is invoked to attempt semantic understanding of the ASR text. First, it is determined whether the ASR text exceeds the model's understanding range. If not, the first text rationality feature is analyzed and output; if so, the model switches to the remote multimodal large model, similarly invoking the local knowledge base corresponding to the target user to perform semantic understanding of the ASR text, analyze and output the second text rationality feature. Clearly, the aforementioned dynamically allocated processing path balances low latency and high accuracy.

[0069] For example, the aforementioned first or second text rationality features can be quantified based on retrieval enhancement generation technology and through a corresponding model. The core is to calculate the "naturalness" of the text sequence by using the probability of the current word appearing in the preceding context (including the current ASR text and historical ASR texts called from the local knowledge base), and substituting it into the model to obtain a rationality value, and then calculating the average value.

[0070] In step S4, based on the voiceprint recognition result and the first text rationality feature or the second text rationality feature, a confidence level is calculated, and when the confidence level exceeds a preset threshold, embodied intelligent voice feedback and action execution are triggered.

[0071] It should be noted that the embodiments of the present invention employ a dual confidence assessment. If the above example is continued, the confidence calculation process in this step can be quantified as follows:

[0072] Define the voiceprint recognition result (i.e., the degree of matching between the audio features of the input speech and the most similar voiceprint model) as f1, the first text plausibility feature or the second text plausibility feature as f2, and the confidence level as:

[0073]

[0074] Where α is the confidence level, Normalize(∙) is the normalization calculation, and w is the weight, which can be defined by a heuristic method.

[0075] Next, this step compares the calculated confidence level with a preset threshold. If the confidence level exceeds the preset threshold, embodied intelligent voice feedback and action execution are triggered; otherwise, the user is prompted to re-enter the voice.

[0076] For example, see Figure 2 The preset threshold mentioned above is set to 0.9 here.

[0077] The specific details regarding the aforementioned embodied intelligent voice feedback and action execution are as follows:

[0078] (1) Embodied intelligent voice feedback

[0079] This invention provides a flow processing pipeline mechanism, and the specific steps are as follows:

[0080] First, if the local text small model is selected, the analysis result of the ASR text is used as the first text; if the remote multimodal large model is selected, the ASR text is split and recombined in a complex task to obtain several second texts.

[0081] Secondly, the first or second text is streamed in real time based on the WebSocket protocol. WebSocket is a network transport protocol that allows full-duplex communication over a single Transmission Control Protocol (TCP) connection, and is used here to implement the streaming text output.

[0082] Next, the text output in real time is scanned synchronously and truncated using Chinese punctuation. The complete sentence before the punctuation is injected into the audio queue, and the remaining text is cached to wait for the next punctuation trigger, so as to ensure the continuity of the output sequence.

[0083] Finally, the text in the audio queue is sequentially processed by speech synthesis to output an audio stream through the embodied intelligent robot.

[0084] (2) Action execution

[0085] This invention proposes a minimum action unit scheduling mechanism, and the specific steps are as follows:

[0086] First, if the local text small model is selected, the ASR text is understood to obtain the first action sequence; if the remote multimodal large model is selected, the ASR text is subjected to complex task decomposition and recombination, and the second action sequence is understood to obtain the second action sequence.

[0087] Secondly, through function calls (i.e. Figure 2 The 'function call' mechanism queries a preset action list and controls the embodied intelligent robot to execute the first action sequence or the second action sequence in sequence. The preset action list stores multiple predefined basic actions.

[0088] Specifically, see Figure 2 For complex tasks, this embodiment of the invention also designs a closed-loop optimization process of perception-decision-execution-feedback: during the process of the embodied intelligent robot executing the second action sequence in sequence, the environmental state is perceived in real time and fed back to the remote multimodal large model, so as to dynamically adjust the execution strategy or parameters of the remaining action sequence and improve environmental adaptability.

[0089] Thus, this embodiment of the invention completes the entire process of the control method for an embodied intelligent robot based on a hierarchical hybrid model.

[0090] Example 2:

[0091] This invention provides a control system for an embodied intelligent robot based on a hierarchical hybrid model. The system is based on a collaborative hybrid model architecture, which includes a local text-based small model and a remote multimodal large model. The system includes:

[0092] The wake-up module is used to receive the target user's voice input in real time and activate the remote ASR model when a wake word is detected;

[0093] The recognition module is used to identify the user's voiceprint and speech content based on the audio features of the input speech using an activated remote ASR model, and to obtain the voiceprint recognition result and ASR text respectively.

[0094] The selection and analysis module is used to select the local text small model and call the local knowledge base to attempt to analyze the ASR text. If it does not exceed the model's understanding range, the first text rationality feature is obtained; otherwise, the remote multimodal large model is selected and the local knowledge base is called to analyze the ASR text to obtain the second text rationality feature.

[0095] The decision-making and execution module is used to calculate the confidence level based on the voiceprint recognition result and the first text rationality feature or the second text rationality feature, and to trigger embodied intelligent voice feedback and action execution when the confidence level exceeds a preset threshold.

[0096] Example 3:

[0097] This invention provides a storage medium storing a computer program for controlling an embodied intelligent robot based on a hierarchical hybrid model, wherein the computer program causes a computer to execute the control method as described in Embodiment 1.

[0098] Example 4:

[0099] An electronic device according to an embodiment of the present invention includes:

[0100] One or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the programs including methods for performing control methods as described in Embodiment 1.

[0101] It is understood that the control system, storage medium, and electronic device of the embodied intelligent robot based on the hierarchical hybrid model provided in the embodiments of the present invention correspond to the control method of the embodied intelligent robot based on the hierarchical hybrid model provided in the embodiments of the present invention. The explanation, examples, and beneficial effects of the relevant contents can be referred to the corresponding parts of the control method, and will not be repeated here.

[0102] In summary, compared with existing technologies, it has the following beneficial effects:

[0103] 1. This invention, based on a collaborative hybrid model architecture, dynamically allocates processing paths according to the task complexity corresponding to the ASR text. Simple tasks are assigned to low-latency local text small models for processing, while complex tasks are assigned to high-precision remote multimodal large models for processing. This significantly improves the performance of embodied intelligent robots in terms of interactive experience, response speed, and task adaptability, while reducing computational and communication overhead. Furthermore, the dual confidence assessment method based on audio and text further ensures the accuracy and robustness of speech recognition.

[0104] 2. To reduce the impact of the environment, especially noisy or multi-person talking environments, on the quality of input speech in this embodiment of the invention, noise reduction processing is performed on the input speech before real-time detection of the wake word. Furthermore, the timeout truncation design of the remote ASR model can improve system stability and enhance user experience.

[0105] 3. The embodiments of the present invention construct a two-stage optimization mechanism for remote ASR models, including front-end noise reduction and back-end recognition. Voiceprint recognition is added during the back-end recognition process to determine the user's identity, and then the corresponding local knowledge base is called to enhance the model's contextual understanding ability.

[0106] 4. This invention adopts a streaming processing pipeline mechanism in the speech synthesis TTS stage, which further alleviates the latency problem compared to traditional TTS, which requires waiting for a large model to output complete text before synthesizing speech.

[0107] 5. In the action execution stage, this embodiment of the invention designs a closed-loop optimization process of perception-decision-execution-feedback for complex tasks: During the process of the embodied intelligent robot executing the second action sequence in sequence, the environmental state is perceived in real time and fed back to the remote multimodal large model to dynamically adjust the execution strategy or parameters of the remaining action sequence and improve environmental adaptability.

[0108] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0109] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A control method for an embodied intelligent robot based on a hierarchical hybrid model, characterized in that, Based on a collaborative hybrid model architecture, the collaborative hybrid model architecture includes a local text-based small model and a remote multimodal large model; the method includes: It receives voice input from the target user in real time and activates the remote ASR model when a wake word is detected; Based on the audio features of the input speech, the activated remote ASR model is used to identify the user's voiceprint and speech content, and the voiceprint recognition result and ASR text are obtained respectively. The local text mini-model is selected and the local knowledge base is invoked to attempt to analyze the ASR text. If the analysis is within the model's comprehension range, the first text rationality feature is obtained; otherwise, the remote multimodal large model is selected and the local knowledge base is invoked to analyze the ASR text and obtain the second text rationality feature. The ASR text is quantified using a retrieval enhancement generation technique and a corresponding model to obtain either the first or second text rationality feature. The local knowledge base corresponds to a preset user and is used to record historical ASR texts corresponding to that user. Based on the voiceprint recognition result and the first text rationality feature or the second text rationality feature, the confidence level is calculated, and when the confidence level exceeds a preset threshold, the embodied intelligent voice feedback and action execution are triggered. The embodied intelligent voice feedback includes: If the local text small model is selected, the analysis result of the ASR text will be used as the first text; if the remote multimodal large model is selected, the ASR text will be split and reorganized into several second texts. Real-time streaming output of the first or second text based on the WebSocket protocol; Synchronously scan the real-time streaming text output and truncate it using Chinese punctuation. Inject the complete sentence before the punctuation into the audio queue and continue to cache the remaining text to wait for the next punctuation trigger. The text in the audio queue is sequentially processed by speech synthesis to output an audio stream through the embodied intelligent robot.

2. The control method as described in claim 1, characterized in that, The input speech is denoised before the wake word is detected in real time.

3. The control method as described in claim 2, characterized in that, After noise reduction processing of the input speech, a lightweight model continuously monitors the system and activates the remote ASR model only when the wake word is matched. If no new speech is detected within a preset time after activation, the system automatically exits the ASR session.

4. The control method as described in claim 1, characterized in that, Define the voiceprint recognition result as f1, the first text plausibility feature or the second text plausibility feature as f2, and the confidence level as: Where α is the confidence level. For normalized calculations, w represents the weight.

5. The control method as described in claim 1, characterized in that, The execution of the action includes: If the local text small model is selected, the ASR text is understood to obtain the first action sequence; if the remote multimodal large model is selected, the ASR text is subjected to complex task decomposition and recombination, and the second action sequence is understood to obtain the second action sequence. By querying a preset action list through a function call mechanism, the embodied intelligent robot is controlled to execute the first action sequence or the second action sequence in sequence.

6. The control method as described in claim 5, characterized in that, During the process of the embodied intelligent robot executing the second action sequence in sequence, the environmental state is perceived in real time and fed back to the remote multimodal large model in order to dynamically adjust the execution strategy or parameters of the remaining action sequence.

7. A control system for an embodied intelligent robot based on a hierarchical hybrid model, characterized in that, Based on a collaborative hybrid model architecture, the collaborative hybrid model architecture includes a local text small model and a remote multimodal large model; The system is used to execute the control method as described in claim 1, comprising: The wake-up module is used to receive the target user's voice input in real time and activate the remote ASR model when a wake word is detected; The recognition module is used to identify the user's voiceprint and speech content based on the audio features of the input speech using an activated remote ASR model, and to obtain the voiceprint recognition result and ASR text respectively. The selection and analysis module is used to select the local text small model and call the local knowledge base to attempt to analyze the ASR text. If it does not exceed the model's understanding range, the first text rationality feature is obtained; otherwise, the remote multimodal large model is selected and the local knowledge base is called to analyze the ASR text to obtain the second text rationality feature. The decision-making and execution module is used to calculate the confidence level based on the voiceprint recognition result and the first text rationality feature or the second text rationality feature, and to trigger embodied intelligent voice feedback and action execution when the confidence level exceeds a preset threshold.

8. A storage medium, characterized in that, It stores a computer program for controlling an embodied intelligent robot based on a hierarchical hybrid model, wherein the computer program causes the computer to perform the control method as described in any one of claims 1 to 6.

9. An electronic device, characterized in that, include: One or more processors; Memory; And one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the programs including methods for performing the control method as described in any one of claims 1 to 6.

Images