Dual-agent based robot xiangsheng script generation method and performance system

Abstract

The application discloses a kind of based on double intelligent agent's robot cross talk script generation method and performance system, belong to man-machine interaction and robot control technical field, using double intelligent agent cooperation, and with the aid of search enhancement generation technique call cross talk knowledge base, generate professional and humorous script;System generates script in cloud, local hub parses script and separates action ID and pure text lines, reorganizes as structured instruction array according to time sequence, uses voice synthesis model on voice to carry out zero sample multi-emotion voice cloning, handles the control demand of multi-sound character pronunciation and timbre emotion change;Action builds semantic-driven action mapping mechanism, converts high-level performance instruction into bottom layer hardware instruction, and controls through the voice action of multi-process separation cooperation. The application can simulate the tone of different roles, generate humorous dialogue in accordance with cross talk context, and synthesize the timbre of a specific role, while coordinating the anthropomorphic body movements, trying to present cross talk performance in a new way.

Description

Technical Field

[0001] This invention belongs to the field of human-computer interaction and robot control technology, and relates to a robot crosstalk script generation method and performance system based on dual intelligent agents. Background Technology

[0002] Traditional robot performances rely on pre-recorded, fixed scripts, mechanically reproducing predetermined content. This results in rigid performance content, limited subject matter, and an inability to improvise based on audience needs or live themes. Even with the introduction of general-purpose large models, a lack of understanding of the characteristics of traditional Chinese crosstalk, such as its "three repetitions and four punchlines" and "smooth and even delivery," leads to loosely structured scripts with stiff punchlines, failing to meet the professional requirements of crosstalk performances.

[0003] Traditional crosstalk requires high-frequency and precise verbal and physical coordination between the straight man and the comedian. Existing systems use a simple "question and answer" model, which cannot generate the unique character interaction logic of crosstalk, nor can they automatically translate the theory of traditional performing arts into performance instructions that match the character's personality. They also lack a multimodal collaborative mechanism for the interaction between the two main characters. Summary of the Invention

[0004] The purpose of this invention is to provide a robot-based method and performance system for generating crosstalk scripts based on dual intelligent agents. Using an embodied robot as a carrier, a crosstalk performance system that integrates dialogue generation, speech synthesis and emotional expression, and action coordination is built. The system can simulate the tone of different characters, generate humorous dialogues that conform to the crosstalk context, synthesize the timbre of specific characters, and combine it with anthropomorphic body movements to attempt to present crosstalk performances in a new way.

[0005] This invention discloses a robot crosstalk script generation method based on dual intelligent agents. By having a screenwriter intelligent agent and a director intelligent agent work together and using retrieval-enhanced generation technology to call upon a crosstalk knowledge base, a professional and humorous script is generated. The crosstalk knowledge base built on the Dify platform segments crosstalk professional theoretical knowledge and imports it into a vector database to form a vectorized vertical knowledge base containing crosstalk performance forms, punchline techniques, and performance skills. The scriptwriting AI includes a lightweight scriptwriting idea model and a scriptwriting draft model deployed locally in the cloud. The scriptwriting draft model is a large language model called in the cloud. After the user inputs the theme of the crosstalk performance, the scriptwriting AI first uses the scriptwriting idea model to break down the ideas and generate punchline ideas, and then lets the large language model create content based on the punchline ideas to generate a script draft. The director's AI agent reviews and iterates the generated script, calling upon a crosstalk appreciation model to extract keywords related to the script's performance style and core techniques based on the crosstalk theory system. These keywords are then used to retrieve corresponding appreciation standards from the crosstalk knowledge base. The model quantitatively scores the script from multiple dimensions, including structure and basic principles, language and technique, theme and style, and characters and tension, generating a review report citing original sources. If the score does not meet the threshold, the system automatically extracts major flaws and modification suggestions from the review report and converts these into constraints added to the scriptwriter's AI agent's prompts, driving targeted modifications. This process can be iterated continuously until the script is deemed acceptable or the maximum number of iterations is reached.

[0006] During the theme decomposition phase, the prompts for the screenwriter's thought process model are designed using a four-layer constraint structure: "role—task—workflow—output format." The role is set as a long-time resident of the city, the task requires extracting the ridiculous logic behind the theme without generating specific plots or dialogue, and the workflow is forced to follow a three-step process of "finding pain points, defining fallacies, and pointing the way." Using a structured thought chain, the life pain points and absurd logic behind the theme are deduced. Finally, through a fixed output format, the results are unified into three standardized modules for easy direct parsing and use in the content generation phase.

[0007] During the content generation stage, the initial draft model guides the larger model through designed prompts. The input prompts are embedded with the following constraints: structurally, they are forced to follow the three-part structure of "introductory remarks - main act - closing remarks", explicitly using crosstalk performance techniques, pre-setting the division of labor and interaction rules between the straight man and the funny man, and forcing each line of dialogue to be accompanied by a predefined action ID.

[0008] The crosstalk appreciation model scans the script generated by the scriptwriter's initial draft model. The prompts in this model are designed according to the crosstalk theory system, and the analysis is limited to professional terminology. The model then determines the performance style of the script, identifies the core punchline techniques used in the script, and selects the keywords that best represent the characteristics of the script.

[0009] Furthermore, this invention discloses a robot crosstalk performance system based on dual intelligent agents, which adopts a cloud-based collaborative architecture of cloud generation, local central processing and robot body execution. This architecture is divided from top to bottom into an interaction layer, a task scheduling layer, an algorithm service layer with hybrid edge-cloud deployment, and a hardware execution layer. The interaction layer is used to receive the crosstalk performance theme input by the user and to set the roles. The interaction layer also undertakes the management of the audio playback queue. After receiving the structured script data returned by the backend, it requests the audio synthesized by speech from the TTS service sentence by sentence and stores the returned audio stream into the playback queue for reading and playback. The task scheduling layer, as the central unit of the system, is built on the FastAPI framework. It is responsible for receiving routing requests from the front-end interaction layer, distributing tasks and verifying parameters, and coordinating and integrating the calling process of various algorithm services. The task scheduling layer uses regular expressions to perform structured processing on the script generated by the robot crosstalk script generation method based on any one of claims 1 to 4, extracts action tags, and splits the script into alternating action nodes and dialogue nodes. Each node carries a preset execution duration or playback duration parameter. The action nodes and dialogue nodes are included in an ordered task queue, and finally an ordered structured instruction array is output. The instruction array encapsulates the action instructions and voice content in a unified data format for subsequent front-end rendering or hardware scheduling modules to read and use. The task execution adopts a strategy of sequential traversal plus blocking wait: when a dialogue node is encountered, the corresponding action is triggered first, then the audio is played and the audio playback status is detected by Pygame to wait for the audio to finish playing in a loop, and the process of "triggering action → playing audio → resetting action" is executed in sequence; when an action node is encountered, the underlying action control script is called in a multi-process non-blocking manner to make the robot perform the corresponding action. A fixed waiting time is used to ensure that the action is completed before moving to the next node. The entire performance proceeds according to the rhythm set by the script until the end. The edge-cloud hybrid deployment algorithm service layer distributes different computing loads to the most suitable computing power nodes. In the cloud, the system accesses the large language model and crosstalk knowledge base by calling the Dify platform API, deploys a dual intelligent agent of screenwriter and director, and uses retrieval-enhanced generation technology to complete the automatic generation, review and iterative optimization of the script. Locally, the system deploys the IndexTTS2 model for speech synthesis, takes the script text and reference audio of the specified character as input, performs zero-sample speech cloning, and outputs an audio file with target timbre and emotional features. The hardware execution layer, as the physical output of the system, consists of pre-compiled motion control scripts and underlying motors. After the task scheduling layer issues motion commands, the hardware execution layer independently starts the corresponding EXE control program through a subprocess. Based on the motion ID mapping table, it converts the motion IDs in the script into limb commands for the robot and drives the motors sequentially to complete the specified actions.

[0010] The system deploys the front-end interaction layer, cloud algorithm calls, and local hardware control as independent logical nodes. The nodes communicate with each other through HTTP requests and asynchronous task queues. The back-end leverages the asynchronous features of the FastAPI framework and uses Python's async / await coroutine syntax to handle concurrent requests. When the main program initiates a network request or file read / write operation, it does not wait for the operation to complete but immediately returns control to the event loop to handle other ready tasks. After the I / O operation is completed, the system retrieves the result through a callback or polling mechanism and continues to execute subsequent logic.

[0011] The reference audio for the designated role serves as a fixed acoustic benchmark for the straight man and the funny man. Classic crosstalk performance videos are selected as the original data source, from which pure human voice segments that can be used for zero-sample cloning are extracted. The reference audio retains natural sentences containing typical tone expressions.

[0012] The action ID mapping table is established as follows: referencing the performance postures of real crosstalk performers, action data is collected by teaching the robotic arm of the robot to be manually guided, and trajectory recording and key frame extraction are performed. These are then compiled into action control scripts for commonly used crosstalk postures. The action identifiers output by the large model are matched with the action control scripts to form an action ID mapping table, and the action IDs are converted into the robot's limb commands.

[0013] By adopting the above technical solution, the present invention has the following technical effects: (1) Script generation through dual-agent collaboration. The system has two agents, "writer" and "director," which work together and use retrieval-enhanced generation technology to call upon the crosstalk knowledge base to generate professional and humorous scripts; (2) Multi-emotion speech synthesis. Zero-sample speech cloning is performed using speech synthesis models (such as IndexTTS2) so that the robot can produce voices with different timbres and corresponding emotions, thereby adapting to the changing roles and contexts in crosstalk performances.

[0014] (3) Semantic-driven action mapping and control. When the script is running, the action IDs in the script are read, and the action IDs are converted into robot limb instructions through the action ID mapping table, so that the robot's limb movements can occur naturally in conjunction with the lines; (4) The performance system is fully integrated. The overall system architecture is divided into three layers: cloud generation, local hub, and lower-level machine (robot body) execution. The script is generated in the cloud, the local hub parses the script and separates the action ID and plain text lines, and reassembles them into a structured instruction array according to the time sequence. In terms of speech, the speech synthesis model (IndexTTS2) is used to clone zero-sample multi-emotion speech and handle the control requirements of polyphonic pronunciation and timbre emotion changes. In terms of action, a semantic-driven action mapping mechanism is constructed to convert high-level performance instructions into low-level hardware instructions. Instructions are sent to the robot body in a fixed time sequence. Through multi-process separation of voice and action collaborative control, a strong synchronous scheduling of "action triggering - audio playback - breath pause" is completed. Attached Figure Description

[0015] Figure 1 This is a system architecture diagram of the present invention; Figure 2 This is an internal logic block diagram of the screenwriter intelligent agent in this invention; Figure 3 This is a schematic diagram of the prompt word design for the scriptwriting logic model of the scriptwriting agent in this invention; Figure 4 This is a schematic diagram of the prompt word design for the initial draft model of the screenwriter intelligent agent in this invention; Figure 5 This is a block diagram of the loop iteration logic of the director agent in this invention; Figure 6 This is a schematic diagram of the prompt word design for the script review model of the director's intelligent agent in this invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0018] In this embodiment of the invention, the terms "exemplary" or "for example" are used to indicate that they are examples, illustrations, or descriptions. Any embodiment or scheme described as "exemplary" or "for example" in this embodiment of the invention should not be construed as being more preferred or advantageous than other embodiments or schemes. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner. Explanation of the principle behind the technical implementation of this invention: I. On the Art of Crosstalk The art of crosstalk relies primarily on language, and can be divided into three categories: solo, duo, and group performances. Performers use four techniques—speaking, imitating, teasing, and singing—to create a humorous effect on stage.

[0019] From a structural perspective, a complete crosstalk performance consists of three parts: the opening remarks / setup, the main story, and the closing punchline.

[0020] In terms of performance techniques, actors use traditional methods such as "three repetitions and four shakes" to lay the groundwork before delivering the punchline. They also flexibly switch between narrator and specific character, forming a unique "jumping in and out" artistic style. Furthermore, crosstalk performers often express their thoughts on social phenomena, so crosstalk possesses the characteristics of being humorous yet philosophical.

[0021] Based on the theory of crosstalk, the following examples illustrate the performance forms and core punchlines of crosstalk to facilitate understanding of the technical solution of this invention.

[0022] A. Crosstalk performance style: One side is dominant: A mainly narrates, while B provides little support; A mother-son pun: A and B are arguing, neither willing to give in, and their roles are equal; Tongue-twister: long, rhythmic recitations (such as reciting the names of dishes); Dialectal imitation (speechlessness); Liu Huo: Learning to sing opera / folk arts; Group performance: More than three people, with a role to fill in the gaps.

[0023] B. Core gag techniques: 1. Three-layer buildup and sudden twist: Three layers of buildup, followed by a sudden twist. 2. Praise first and then criticize: Praise highly and then criticize harshly. 3. Characteristic language: The words match the character's personality (e.g., verbose, impatient). 4. Violate the norm: Absurdity that goes against common sense (e.g., the vehicle makes noise everywhere except for the bell). 5. Mistaken identity: Two people talk at cross purposes, each talking about their own thing.

[0024] 6. Deliberately create mystery: The conversation suddenly turns. 7. Word meaning illusion: Mishearing or ambiguity (e.g., "pat the bubble"). 8. Absurd exaggeration: Extreme exaggeration in logic (e.g., the three-year plan was "executed" for three years). 9. Self-contradiction: The words before and after conflict with each other. [[ID=2,4]]10. Reverse and misstate: Reverse the word order (pointing at wherever one wants). 11. Use homophonic words: Misunderstanding due to homophonic words (sixty-five vs. Liu Shiwu). 12. Flatter and fawn: Take the ugly as beautiful and flatter. 13. Misunderstand and distort: Deliberately distort (e.g., mistake an inspection tour for a monkey show). 14. Misuse words: Misuse idioms (e.g., "warmly eternal"). 15. Extend and expand: Make associations and expansions based on a single point. 16. Argue speciously: Argue without reason. 17. Distort and sing in a distorted way: Distort poems or songs. 18. Intentionally self-deprecate: Self-deprecate. C. Cross-talk performance skills: Timing, rhythm, and intonation: Control the rhythm, be fast when it should be fast and slow when it should be slow. Lay a solid foundation: Whether the logical buildup before the gag is sufficient. Improvise: Make an impromptu performance based on the on-site situation.

[0025] II. Retrieval-Augmented Generation Principle The training data for large language models is fixed. However, in vertical fields like crosstalk (xiangsheng), where publicly available corpora are scarce, the model is prone to "illusion" problems such as outdated professional knowledge or fabricated facts. This invention employs Retrieval Augmentation Generation (RAG) technology to address these issues. The RAG workflow consists of three stages: indexing, retrieval, and generation. In the indexing stage, when a user initiates a script generation request, the system first converts the user input into a vector using the same embedding model. Then, it compares the vectors with the vectors of various knowledge fragments in the crosstalk knowledge base, retrieving the most relevant theoretical knowledge. Subsequently, these knowledge fragments are used as contextual constraints and concatenated with the user's original request to form new prompts, which are then input into the large language model. By supplementing external knowledge before inference, the large model has a professional theoretical basis when generating dialogue, thus compensating for the lack of knowledge in vertical fields inherent in general models.

[0026] III. Zero-shot speech synthesis model Crosstalk uses a lot of colloquial vocabulary, and the pronunciation of many words depends on the context. If a word is mispronounced, the punchline will be difficult to deliver. To address this need in Chinese-speaking contexts, IndexTTS employs a hybrid modeling approach combining Chinese characters and pinyin. This allows for the assignment of pinyin to uncommon or polyphonic characters, thus enabling control over pronunciation. In the acoustic feature quantization stage, IndexTTS2 uses finite scalar quantization technology to improve the utilization rate of the acoustic discrete feature codebook. In the final stage of decoding and waveform reconstruction, the system utilizes a Conformer-based acoustic conditional encoder in conjunction with a BigVGAN2 vocoder to enhance sound quality and content stability, reducing omissions and swallowed sounds.

[0027] In crosstalk performances, even with significant emotional fluctuations, the speaker's timbre must remain unchanged. Traditional models typically compress speaker identity and emotional information into the same latent space. When a reference audio with strong emotion is input, the synthesized result is prone to timbre drift. IndexTTS2 employs a dual-path network structure to clearly separate these two aspects. One path extracts global and stable static timbre features, while the other captures prosodic features. During actual synthesis, the timbre vector remains fixed while emotional conditions are injected, thus controlling the emotional tone of the speech without altering the timbre.

[0028] IV. Asynchronous Communication and Concurrency Scheduling In edge-cloud collaborative robot systems, both script generation via cloud-based large models and local synthesis of long audio segments are time-consuming I / O-intensive operations. Taking a complete performance flow as an example, the system needs to sequentially send requests to the cloud API, wait for the large model's inference, receive the returned script data, and then call the local TTS service line by line to synthesize the audio. If a traditional synchronous blocking I / O method is used, the main program will be suspended while waiting for each network response or file read / write operation. Delay at any point in the call chain will directly block all subsequent tasks, causing the front-end interface to become unresponsive and resulting in a poor user experience.

[0029] This system employs an asynchronous, non-blocking I / O mechanism based on an event loop. Specifically, the backend leverages the asynchronous features of the FastAPI framework and utilizes Python's async / await coroutine syntax to handle concurrent requests. When the main program initiates a network request or file read / write operation, it does not wait for the operation to complete but immediately relinquishes control to the event loop to handle other ready tasks. After the I / O operation is completed, the system retrieves the result through a callback or polling mechanism and continues executing subsequent logic. In this mode, a single thread can manage multiple concurrent tasks simultaneously, avoiding the overhead of multi-threaded context switching.

[0030] The system deploys front-end interaction, cloud algorithm calls, and local hardware control as independent logical nodes, which communicate with each other via HTTP requests and asynchronous task queues. This decoupled design allows each module to be independently expanded and replaced, while also improving the system's throughput and response speed in high-concurrency scenarios, providing a communication foundation for the stable scheduling of subsequent multimodal performances.

[0031] V. Multi-process control and audiovisual synchronization control After script parsing and speech synthesis are completed at the software level, the system still needs to send action commands to the underlying hardware for execution. However, there is a significant difference between the computational logic of the upper-layer application software and the response speed of the underlying hardware drivers: the processing time of the upper-layer algorithm is usually in the millisecond range, while it often takes several seconds for a motor to execute a complete action. If the hardware control script is called synchronously directly in the main process, the entire software flow will be blocked during the motor's movement, which will not only prevent subsequent lines from playing in time, but also cause a timing misalignment between speech and action.

[0032] To address this issue, the system employs a multi-process scheduling strategy, placing time-consuming low-level hardware driver logic into independent subprocesses for execution. When the task scheduling layer parses an action command, it uses Python's `subprocess` module to launch a pre-compiled hardware control script in a non-blocking manner, sending control commands to the lower layer. The main process immediately returns to continue processing subsequent tasks without waiting for the subprocess to finish. This decouples the hardware driver execution process from the main flow, ensuring decoupling between software computation and hardware control. The `subprocess` module can create new subprocesses and establish pipe connections with their input / output streams, enabling upper-layer applications to call the underlying executable files. The system pre-compiles different action trajectories into independent EXE control scripts. When the scheduling layer parses an action command, it launches the corresponding script in a non-blocking manner via `subprocess`, driving the motor to complete the limb movement according to the predetermined trajectory. This subprocess calling method decouples the hardware driver logic from the main program, avoiding the blocking impact of hardware execution time on the upper-layer flow.

[0033] Building upon this foundation, the system further designed a voice-action collaborative control algorithm for crosstalk performances. Crosstalk performances have strict requirements for the timing of actions and speech; for example, while the lead comedian delivers his punchline, the supporting comedian should simultaneously make a coordinated gesture such as waving his hand or leaning back. This system integrates action nodes and dialogue nodes from the structured script into an ordered task queue, with each node carrying preset execution or playback duration parameters. Task execution employs a strategy of sequential traversal combined with blocking and waiting: when encountering a dialogue node, the corresponding action is triggered first, then the audio is played, and the audio playback status is checked using Pygame, looping and waiting for the audio to finish playing, sequentially executing the process of "triggering action → playing audio → resetting action"; when encountering an action node, a multi-process non-blocking method is used to call the underlying script, causing the robot to perform the corresponding action, and a fixed waiting duration ensures that the action is completed before proceeding to the next node. The entire performance progresses according to the rhythm set in the script until the end. Through this "trigger-wait-progress" sequential control mode, the system effectively avoids the mutual preemption of multiple commands on the timeline, ensuring the basic synchronization of voice and physical movements in crosstalk performances, and making multimodal performances smooth and controllable.

[0034] Example 1 Embodiment 1 of the present invention discloses a robot crosstalk script generation method based on dual intelligent agents. By having a screenwriter intelligent agent and a director intelligent agent work together and using retrieval enhancement generation technology to call a crosstalk knowledge base, a professional and humorous script is generated. The crosstalk knowledge base is built on the Dify platform. It extracts and cleans authoritative theoretical works in the field of crosstalk, such as "A Casual Talk on the Art of Crosstalk", through OCR and multimodal large model. After being structurally segmented by chapter, it is imported into a vector database to form a vectorized vertical knowledge base containing knowledge of crosstalk performance forms, punchline techniques, performance skills and other knowledge. The scriptwriting AI includes a lightweight scriptwriting thought model and a scriptwriting draft model deployed locally in the cloud. The lightweight scriptwriting thought model is responsible for breaking down the theme of the crosstalk performance and generating comedic moments. The scriptwriting draft model is a large language model invoked in the cloud, responsible for generating a script draft, such as... Figure 2 As shown, after the user inputs the theme of the crosstalk performance, the internal process is divided into two stages: theme decomposition and content generation. The scriptwriting AI first uses a lightweight scriptwriting idea model to decompose the ideas, and then lets the large language model create the content. The division of labor of "small model determines the ideas, large model creates" avoids the problems of preachy and detached from life that are easy to occur when the general large model directly creates crosstalk, and also takes into account the generation efficiency and content quality.

[0035] In the theme deconstruction stage, qwen3:30b was chosen as a lightweight screenwriting model. The prompts were set from the perspective of a "long-time resident of the neighborhood." This stage didn't involve writing complete dialogue directly, but rather using a structured thought process to deduce the underlying pain points and absurd logic of life, laying a realistic foundation for subsequent creation. For example... Figure 3 As shown, the prompts employ a four-layer constraint structure: "role—task—workflow—output format." The role deliberately avoids "professional screenwriter," using the persona of an "old neighbor" to simplify abstract social issues into the everyday logic of ordinary people, preventing large models from outputting empty theoretical preaching. The task only requires extracting the "ridiculous logic" behind the theme, without generating specific plots or dialogue. This reduces the processing difficulty of the lightweight model and leaves ample creative space for subsequent models. The workflow is forced to follow a three-step process: "find the pain point, define the fallacy, and provide guidance." The core punchline, the "fallacy," is written using a fixed formula: "In order to…, A decides to firmly believe in a set of fallacies." This formula ensures that the output sounds plausible, but upon closer inspection, it's all nonsense, fitting the comedic characteristics of crosstalk. Finally, through a fixed output format, the results are unified into three standardized modules: ["The Plain Language of This Matter," [The Fallacy (Personality Core)], and [Guidance for Screenwriters]," facilitating direct parsing and retrieval during the content generation stage.

[0036] During the content generation stage, the initial script model guides the larger Gemini-3 model (which performs well in Chinese long text creation and format adherence) through designed prompts. The input prompts are embedded with the following constraints: structurally, they enforce a three-part structure of "introductory remarks - main act - closing remarks," explicitly using techniques such as "three repetitions and four shakes"; in terms of roles, they pre-define the division of labor and interaction rules between the straight man and the comedian; and in terms of output format, they force each line of dialogue to be preceded by a predefined action ID, forming a structured text flow. For example... Figure 4 As shown, the prompts in the screenwriter's first draft model mainly cover three aspects: First, regarding the professional knowledge of crosstalk, it clearly stipulates the specific usage of traditional techniques such as "three repetitions and four shakes" and "laying out the foundation." It also requires scripts to follow a three-part structure of "entrance, main act, and conclusion," and mandates that the lead comedian use exaggeration and voice imitation to enhance the comedic effect. Based on the theoretical system of crosstalk, it transforms abstract creative rules into rigid requirements that the model can execute, solving the problem of general models "lacking the flavor of crosstalk" due to a lack of understanding of the professional content of crosstalk.

[0037] Secondly, a fixed role division between the master and apprentice is established. Role A is the straight man / master, responsible for "measuring the material," using guiding phrases like "How do you say it?" and "Is that amazing?" to deliver and interrupt lines. Role B is the lead man / apprentice, responsible for "performing the material," using twisted logic to drive the plot forward. Each role also has pre-defined action preferences and language styles; for example, the straight man uses more listening and guiding actions, while the lead man uses more narrating and rebuttal actions. This role setting avoids arbitrary switching of character relationships within a large model, ensuring clear boundaries between the straight man and lead man. Furthermore, character personalities are directly linked to a library of commonly used crosstalk actions, providing a semantic basis for the automatic generation of subsequent action commands and making the matching of actions and lines more reasonable.

[0038] Third, the scripts output by the model must include action IDs. Each line of dialogue must begin with "Character:[Action ID]" and can only use predefined action IDs, such as basic standing posture, one hand on chest, clasped hands, one hand gesture of invitation, waving dismissively, stroking chin in thought, etc. This ensures that subsequent hardware control can utilize them. We also strictly limited the format, prohibiting any regular parentheses, action descriptions, or explanatory text. This step transformed the originally somewhat irregular natural language into semi-structured machine-readable instructions, significantly reducing the difficulty and error probability of subsequent regular expression parsing. Furthermore, limiting the range of action IDs ensures that the robot can execute all action instructions generated by the model, preventing system errors caused by the model creating a non-existent action.

[0039] The aforementioned director's AI agent transforms professional theories of crosstalk into actionable review rules, and then judges the professionalism of the script according to these rules, forming a closed loop of "creation—review—revision," such as...Figure 5 As shown, the system automatically reviews the script's quality and provides suggestions for improvement.

[0040] The director's AI agent reviews and iterates the generated script, invoking a crosstalk appreciation model to extract keywords related to the script's performance style and core techniques based on crosstalk theory. These keywords are then used to retrieve corresponding appreciation standards from the crosstalk knowledge base. The model quantitatively scores the script across dimensions such as structure and structure, language and technique, theme and style, and characters and tension, generating a review report citing original sources. If the score falls below a threshold, the system automatically extracts defects and modification suggestions, converting them into constraints added to the scriptwriter's AI agent's prompts to drive targeted modifications. This process can iterate continuously until the script is deemed acceptable or the maximum number of iterations is reached. The system first calls a crosstalk appreciation model deployed based on the Qwen3:30b lightweight model to scan the script generated by the initial draft model. The model's prompts are designed according to the crosstalk theoretical system, limiting analysis to professional terminology and prohibiting general evaluation terms like "funny" or "comical." Its tasks are threefold: first, to determine the script's performance style, such as whether it's a one-sided or two-part style; second, to identify the core comedic techniques used in the script, focusing on typical techniques like "three rounds of punchlines" and "forced arguments"; and third, to select 3 to 5 keywords that best represent the script's characteristics. This approach avoids the arbitrariness of direct scoring and provides a basis for quantitative evaluation.

[0041] Next, the system uses the keywords as search terms to find the corresponding appreciation standards in the crosstalk knowledge base, and then uses the search results as embedded review prompts for the next stage. This is done to ensure that every score has a theoretical basis and to prevent inconsistencies in standards due to the model's subjective judgment.

[0042] After theoretical matching is completed, the system invokes the script review module to score the script from four dimensions: structure and basic structure, language and technique, theme and style, and character and tension. Each dimension corresponds to the creative requirements of traditional crosstalk: structure and basic structure assesses the completeness of the "introduction—main act—concluding act" framework; language and technique assess whether the dialogue has a "crosstalk flavor" and how well the techniques are used; theme and style assess the comedic effect and depth of content; and character and tension assess the naturalness of the interaction between the straight man and the comedian. For example... Figure 6 As shown, output the review report cited in the original text and explain the reasons for the scoring to ensure that the scoring is verifiable.

[0043] Scripts that meet the scoring criteria pass review; those that don't are intercepted. The review report extracts key flaws and suggested revisions, which are then incorporated into the scriptwriter's AI agent as prompts to drive targeted modifications. This process iterates continuously until the script is deemed acceptable or the maximum number of revisions is reached. The director's AI agent automates and standardizes script quality control, forming a complete creative loop with the scriptwriter's AI agent. By combining traditional performing arts theory with the reasoning capabilities of a large-scale model, the system ensures both the professionalism and consistency of generated scripts while significantly enhancing automation, completing the entire process from theme input to a qualified script output without human intervention.

[0044] Example 2 Embodiment 2 of the present invention discloses a robot crosstalk performance system based on dual intelligent agents, such as... Figure 1 As shown, a cloud-based collaborative architecture of "cloud generation - local hub - robot body execution" is adopted. This architecture is divided into an interaction layer, a task scheduling layer, an algorithm service layer with hybrid edge-cloud deployment, and a hardware execution layer from top to bottom. Each layer has a clear responsibility, and the layers communicate with each other through standardized interfaces, which facilitates modular development and later maintenance. The system deploys the front-end interaction layer, cloud algorithm calls, and local hardware control as independent logical nodes. The nodes communicate with each other through HTTP requests and asynchronous task queues. The back-end leverages the asynchronous features of the FastAPI framework and uses Python's async / await coroutine syntax to handle concurrent requests. When the main program initiates a network request or file read / write operation, it does not wait for the operation to complete but immediately returns control to the event loop to handle other ready tasks. After the I / O operation is completed, the system retrieves the result through a callback or polling mechanism and continues to execute subsequent logic. In this mode, a single thread can manage multiple concurrent tasks simultaneously, avoiding the overhead of multi-threaded context switching. The interaction layer is a local front-end page directly facing the user. It is used to receive the user's input of the crosstalk performance theme and to set the roles. The interaction layer serializes the input content into a JSON format request body and sends an asynchronous request to the backend via the HTTP protocol, while triggering a loading status prompt. In addition, the interaction layer is also responsible for managing the audio playback queue. After receiving the structured script data returned by the backend, it requests the speech-synthesized audio from the TTS service sentence by sentence, stores the returned binary audio stream into the playback queue, and provides real-time feedback on the system's running status, so that users can intuitively understand the progress of the performance preparation and execution. The task scheduling layer, as the central unit of the system, is built on the FastAPI framework. It is responsible for receiving routing requests from the front-end interaction layer, distributing tasks and validating parameters, and coordinating and integrating the call flow of various algorithm services. When parsing the script generated in Implementation Example 1, the task scheduling layer internally uses regular expressions to structure the original script returned from the cloud, extracting action tags and splitting the original script into alternating action nodes and dialogue nodes. Each node carries a preset execution duration or playback duration parameter. The action nodes and dialogue nodes are incorporated into an ordered task queue, ultimately outputting an ordered structured instruction array. This instruction array encapsulates action instructions and voice content in a unified data format for subsequent front-end rendering or hardware scheduling. The module reads and uses a strategy of sequential traversal plus blocking wait during task execution: when encountering a dialogue node, the corresponding action is triggered first, then the audio is played and the audio playback status is detected by Pygame to wait for the audio to finish playing in a loop, and the process of "triggering action → playing audio → resetting action" is executed in sequence; when encountering an action node, the underlying script is called in a multi-process non-blocking manner to make the robot perform the corresponding action. A fixed waiting time is used to ensure that the action is completed before moving on to the next node. The entire performance proceeds according to the rhythm set in the script until the end; through this sequential control mode of "trigger-wait-progress", the system effectively avoids the mutual preemption of multiple instructions on the timeline, ensures the basic synchronization of voice and body movements in the crosstalk performance, and makes the multimodal performance smooth and controllable; The edge-cloud hybrid deployment algorithm service layer distributes different computing loads to the most suitable computing nodes. In the cloud, the system accesses the large language model and crosstalk knowledge base by calling the Dify platform API, deploys a "writer-director" dual agent, and uses retrieval-enhanced generation technology to complete the automatic generation, review, and iterative optimization of the script. Locally, the system deploys the IndexTTS2 model for speech synthesis, which is encapsulated as an inference service interface through Grado. It takes the script text and the reference audio of the specified character as input, performs zero-sample speech cloning, and outputs an audio file with the target timbre and emotional features. This allows the computationally intensive inference task to be completed in the cloud, while the latency-sensitive speech synthesis task is processed locally in real time, balancing computational efficiency and response speed. The hardware execution layer, as the physical output of the system, consists of pre-compiled motion control scripts and underlying motors. After the task scheduling layer issues motion commands, the hardware execution layer independently starts the corresponding EXE control program through subprocesses. Based on the motion ID mapping table, the motion IDs in the script are converted into robot limb commands, which drive the motors to complete the specified actions in sequence. The independent operation mechanism of the subprocesses ensures that the hardware execution process will not interrupt the continued operation of the upper-layer software, providing underlying support for the synchronous coordination of voice playback and motion execution.

[0045] The reference audio for the designated role, serving as a fixed acoustic benchmark for the straight man and the comedian, refers to selecting classic crosstalk performance videos as the original data source and extracting pure human voice segments that can be used for zero-sample cloning. This reference audio retains natural sentences containing typical tone expressions to enhance the amount of reference information during subsequent emotional cloning. In multi-round speech synthesis, the same reference audio is always called for the same role to ensure the consistency and recognizability of the timbre throughout the entire script.

[0046] The speech synthesis inference service interface receives dialogue text and character identifiers as input parameters, automatically associates the corresponding character's reference audio internally, completes the full-link inference from text to audio, and writes the synthesized WAV audio file to the local cache playback queue for reading and playback.

[0047] Action labels generated by the Large Language Model (LLM) are high-level semantic symbols that need to be converted into low-level hardware control signals. To make the movements of the crosstalk robot more consistent with the script context, the robot's movements are collected by manually guiding and recording, referencing the performance postures of real crosstalk actors. Relying on the intuitive posture perception of the human body, the robot is guided to form physical expressions with a crosstalk style. Specifically, the robot's robotic arm is manually guided to teach common crosstalk postures such as "bowing hands," "measuring the work," and "stroking the chin in contemplation." Trajectory recording and keyframe extraction are performed to obtain the action data of common crosstalk postures. These are then compiled into independent executable motion control scripts. An action ID mapping table is built at the task scheduling layer to match the action labels output by the large model with the independent executable motion control scripts, thereby converting the action IDs into the robot's physical commands.

[0048] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in this invention can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0049] Specifically, the steps of the method embodiments in this invention can be implemented by integrated logic circuits in the processor hardware and / or instructions in software form. The steps of the method disclosed in this invention can be directly implemented by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. Optionally, the software modules can reside in mature storage media in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, and registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps in the above method embodiments.

[0050] In some embodiments of this application, the computer program may be divided into one or more modules, which are stored in the memory and executed by the processor to complete the method of Embodiment 1 provided in this application. The one or more modules may be a series of computer program instruction segments capable of performing a specific function, and these instruction segments are used to describe the execution process of the computer program.

[0051] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for generating robot crosstalk scripts based on dual-agent systems, characterized in that, By having the screenwriter and director agents work together and using retrieval-enhanced generation technology to access the crosstalk knowledge base, a professional and humorous script is generated. The crosstalk knowledge base built on the Dify platform segments crosstalk professional theoretical knowledge and imports it into a vector database to form a vectorized vertical knowledge base containing crosstalk performance forms, punchline techniques, and performance skills. The scriptwriting AI includes a lightweight scriptwriting idea model and a scriptwriting draft model deployed locally in the cloud. The scriptwriting draft model is a large language model called in the cloud. After the user inputs the theme of the crosstalk performance, the scriptwriting AI first uses the scriptwriting idea model to break down the ideas and generate punchline ideas, and then lets the large language model create content based on the punchline ideas to generate a script draft. The director's AI agent reviews and iterates the generated script, calling upon a crosstalk appreciation model to extract keywords related to the script's performance style and core techniques based on the crosstalk theory system. These keywords are then used to retrieve corresponding appreciation standards from the crosstalk knowledge base. The model quantitatively scores the script from multiple dimensions, including structure and basic principles, language and technique, theme and style, and characters and tension, generating a review report citing original sources. If the score does not meet the threshold, the system automatically extracts major flaws and modification suggestions from the review report and converts these into constraints added to the scriptwriter's AI agent's prompts, driving targeted modifications. This process can be iterated continuously until the script is deemed acceptable or the maximum number of iterations is reached.

2. The robot crosstalk script generation method based on dual-agent system according to claim 1, characterized in that, During the theme decomposition phase, the design prompts for the screenwriter's thought process model employ a four-layer constraint structure: "role—task—workflow—output format." The role is set as a long-time resident of the city, the task requires extracting the absurd logic behind the theme without generating specific plots or dialogue, and the workflow is forced to follow a three-step process of "finding pain points, defining fallacies, and pointing the way." This structured thought chain deduces the life pain points and absurd logic behind the theme. Finally, through a fixed output format, the results are unified into three standardized modules for easy direct parsing and retrieval during the content generation phase.

3. The robot crosstalk script generation method based on dual-agent system according to claim 2, characterized in that, During the content generation stage, the initial draft model guides the larger model through designed prompts. The input prompts are embedded with the following constraints: structurally, they are forced to follow the three-part structure of "introductory remarks - main act - closing remarks", explicitly using crosstalk performance techniques, pre-setting the division of labor and interaction rules between the straight man and the funny man, and forcing each line of dialogue to be accompanied by a predefined action ID.

4. The robot crosstalk script generation method based on dual-agent system according to claim 1, characterized in that, The crosstalk appreciation model scans the script generated by the scriptwriter's initial draft model. The prompts in this model are designed according to the crosstalk theory system, and the analysis is limited to professional terminology. The model then determines the performance style of the script, identifies the core punchline techniques used in the script, and selects the keywords that best represent the characteristics of the script.

5. A robot crosstalk performance system based on dual intelligent agents, characterized in that, The cloud-based collaborative architecture, which is generated in the cloud, executed by the local hub and the robot itself, is divided into an interaction layer, a task scheduling layer, an algorithm service layer with hybrid edge-cloud deployment, and a hardware execution layer from top to bottom. The interaction layer is used to receive the crosstalk performance theme input by the user and to set the roles. The interaction layer also undertakes the management of the audio playback queue. After receiving the structured script data returned by the backend, it requests the audio synthesized by speech from the TTS service sentence by sentence and stores the returned audio stream into the playback queue for reading and playback. The task scheduling layer, as the central unit of the system, is built on the FastAPI framework. It is responsible for receiving routing requests from the front-end interaction layer, distributing tasks and verifying parameters, and coordinating and integrating the calling process of various algorithm services. The task scheduling layer uses regular expressions to perform structured processing on the script generated by the robot crosstalk script generation method based on any one of claims 1 to 4, extracts action tags, and splits the script into alternating action nodes and dialogue nodes. Each node carries a preset execution duration or playback duration parameter. The action nodes and dialogue nodes are included in an ordered task queue, and finally an ordered structured instruction array is output. The instruction array encapsulates the action instructions and voice content in a unified data format for subsequent front-end rendering or hardware scheduling modules to read and use. The task execution adopts a strategy of sequential traversal plus blocking wait: when a dialogue node is encountered, the corresponding action is triggered first, then the audio is played and the audio playback status is detected by Pygame to wait for the audio to finish playing in a loop, and the process of "triggering action → playing audio → resetting action" is executed in sequence; when an action node is encountered, the underlying action control script is called in a multi-process non-blocking manner to make the robot perform the corresponding action. A fixed waiting time is used to ensure that the action is completed before moving to the next node. The entire performance proceeds according to the rhythm set by the script until the end. The edge-cloud hybrid deployment algorithm service layer distributes different computing loads to the most suitable computing power nodes. In the cloud, the system accesses the large language model and crosstalk knowledge base by calling the Dify platform API, deploys a dual intelligent agent of screenwriter and director, and uses retrieval-enhanced generation technology to complete the automatic generation, review and iterative optimization of the script. Locally, the system deploys the IndexTTS2 model for speech synthesis, takes the script text and reference audio of the specified character as input, performs zero-sample speech cloning, and outputs an audio file with target timbre and emotional features. The hardware execution layer, as the physical output of the system, consists of pre-compiled motion control scripts and underlying motors. After the task scheduling layer issues motion commands, the hardware execution layer independently starts the corresponding EXE control program through a subprocess. Based on the motion ID mapping table, it converts the motion IDs in the script into limb commands for the robot and drives the motors sequentially to complete the specified actions.

6. The robot crosstalk performance system based on dual intelligent agents according to claim 5, characterized in that, The system deploys the front-end interaction layer, cloud algorithm calls, and local hardware control as independent logical nodes. The nodes communicate with each other through HTTP requests and asynchronous task queues. The back-end uses the asynchronous features of the FastAPI framework and Python's async / await coroutine syntax to handle concurrent requests. When the main program initiates a network request or file read / write operation, it does not wait for the operation to complete, but immediately returns control to the event loop to handle other ready tasks. After the I / O operation is completed, the system retrieves the result through a callback or polling mechanism and continues to execute subsequent logic.

7. The robot crosstalk performance system based on dual intelligent agents according to claim 5, characterized in that, The reference audio for the designated role serves as a fixed acoustic benchmark for the straight man and the funny man. Classic crosstalk performance videos are selected as the original data source, from which pure human voice segments that can be used for zero-sample cloning are extracted. The reference audio retains natural sentences containing typical tone expressions.

8. The robot crosstalk performance system based on dual intelligent agents according to claim 5, characterized in that, The action ID mapping table is established as follows: referencing the performance postures of real crosstalk performers, action data is collected by teaching the robotic arm of the robot to be manually guided, and trajectory recording and key frame extraction are performed. These are then compiled into action control scripts for commonly used crosstalk postures. The action identifiers output by the large model are matched with the action control scripts to form an action ID mapping table, and the action IDs are converted into the robot's limb commands.

Images