Traffic element universe virtual sales agent real-time driving method and system

Abstract

The invention relates to a traffic element universe virtual sales agent real-time driving method and system, and belongs to the technical field of artificial intelligence and traffic element universe crossing. According to the method, aiming at the problems of insufficient interaction fidelity, poor real-time performance, weak knowledge service capability and the like of an existing virtual sales system, a three-layer architecture system based on a presentation interaction layer, a control algorithm layer and a question and answer model layer and a real-time driving method are constructed. Wherein the presentation interaction layer realizes construction and multi-modal interaction of a high-fidelity virtual salesperson image and an automobile sales scene; the control algorithm layer ensures high real-time performance of the system through a real-time communication algorithm, a behavior control algorithm and a voice processing algorithm; and the question and answer model layer deeply fuses a large language model and an automobile domain knowledge base, and provides professional and accurate intelligent question and answer services. According to the invention, high-fidelity, strong-real-time and knowledge-rich interactive experience of the virtual sales agent can be realized, and the reality sense, stability and intelligent level of the virtual automobile sales service are remarkably improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the cross field of artificial intelligence and traffic meta-universe, and relates to a traffic meta-universe virtual sales intelligent agent real-time driving method and system. BACKGROUND

[0002] With the rise of the meta-universe concept and the rapid development of virtual reality technology, virtual digital humans, as an important carrier of human-computer interaction, have shown great potential in many fields. In particular, in the automobile sales industry, the traditional online sales model has the pain points of single display effect and insufficient interactive experience, and urgently needs a new generation of intelligent sales solution. However, the application of existing virtual automobile sales intelligent agents still has several outstanding technical bottlenecks.

[0003] Firstly, the existing system performs poorly in high-fidelity interaction. Most virtual salespersons use pre-recorded animations, resulting in a serious mismatch between mouth movements and speech, stiff body language lacking naturalness, and inability to adaptively express emotions based on vehicle characteristics. This technical limitation seriously affects the realism of user experience.

[0004] Secondly, the system stability and real-time performance have significant defects. Application Programming Interface (API) call failures often cause interaction interruptions, network delays cause animation stuttering problems, and system performance decreases dramatically in high-concurrency application scenarios. The existing architecture lacks effective fault tolerance mechanisms and priority scheduling strategies.

[0005] Thirdly, the knowledge service capability has obvious limitations. Traditional question-and-answer systems rely on fixed scripts and are difficult to meet the personalized consultation needs of customers, and the explanation of professional parameters is not clear enough, and after multiple dialogues, it is easy to deviate from the core sales scene. The knowledge update mechanism lags behind and cannot adapt to rapidly changing market information.

[0006] Finally, the existing technology lacks diversified interaction methods. Most current automobile sales intelligent agents only support text interaction methods and do not have more natural interaction modalities such as voice, limiting the use range and experience depth of user groups. Multi-modal collaborative technology has not yet been maturely applied.

[0007] In summary, the current virtual sales system has technical shortcomings in terms of fidelity, real-time performance, knowledge, and interactive diversity. Therefore, there is an urgent need for a new intelligent agent driving solution that integrates large language models, real-time dynamic content matching, and multi-modal collaborative optimization to achieve comprehensive improvement in fidelity and practicality. SUMMARY

[0008] Therefore, the purpose of the present application is to provide a traffic meta-universe virtual sales intelligent agent real-time driving method and system.

[0009] To achieve the above objectives, the present invention provides the following technical solution: A real-time driving method for a virtual sales agent in a transportation metaverse includes the following steps: Construct a performance and interaction layer that includes a virtual salesperson behavior module and a car display scene module; Construct a control algorithm layer that includes real-time communication algorithms, behavior control algorithms, and speech processing algorithms; Construct a question-answering model layer that includes a large language model interface module and an automotive domain knowledge base module; Through the collaborative work of the performance interaction layer, control algorithm layer, and question-answering model layer, highly realistic, real-time, and knowledge-intensive multimodal interaction of the virtual sales agent is achieved.

[0010] Furthermore, the step of constructing the performance interaction layer includes: A digital human model was built using the Unity Humanoid Avatar system, and eight basic Visem parameters were integrated to achieve facial expression control. A virtual sales showroom environment is built based on Unity's general rendering pipeline, and a 360-degree all-round display of vehicles is achieved through an optimized rotation control algorithm.

[0011] Furthermore, the step of constructing the control algorithm layer includes: employing a real-time communication algorithm based on an exponential backoff retry algorithm and a weighted round-robin strategy, wherein the delay time calculation formula for the exponential backoff retry algorithm is:

[0012] A speech activity detection algorithm based on energy entropy dual thresholds is adopted. The multi-feature weighted scoring calculation formula of the algorithm is as follows: in For rating, E n For short-term energy, T voice For voice judgment threshold, For spectral entropy, For the proportion of harmonics, , and As weight.

[0013] Furthermore, the steps for constructing the question-answering model layer include: It adopts a three-level knowledge system architecture, including a basic layer with hard-coded system prompts, a dynamic layer connected through a real-time application programming interface, and a strategy layer that integrates sales script templates; A knowledge update mechanism based on differential algorithm and Faiss vector database is adopted.

[0014] Furthermore, the multimodal interaction includes: supporting users to query via text or voice input; the virtual sales assistant's response integrates voice output and visual presentation; and real-time generation of synchronized, accurate lip movements and semantically appropriate facial expressions and gestures.

[0015] A real-time driven system for a virtual sales agent in a transportation metaverse includes: a presentation and interaction layer for providing highly realistic virtual salesperson avatars and car sales scene displays; a control algorithm layer connected to the presentation and interaction layer for enabling real-time communication, behavior control, and voice processing; and a question-and-answer model layer connected to the control algorithm layer for providing intelligent question-and-answer services based on a large language model; wherein the presentation and interaction layer, the control algorithm layer, and the question-and-answer model layer constitute a three-layer architecture.

[0016] Furthermore, the presentation interaction layer includes: The virtual salesperson behavior module uses skeletal binding and muscle simulation to create a highly realistic digital human model, and includes a parameter emphasis intelligent algorithm to automatically trigger emphasis gestures; The car display scene module uses a lightweight rendering architecture to build a virtual sales showroom environment and integrates an intelligent multi-camera switching system.

[0017] Furthermore, the control algorithm layer includes: The real-time communication algorithm module adopts a layered architecture design and implements a standardized expressive state transition application programming interface based on Unity Web Request. The behavior control algorithm module adopts a hierarchical state machine architecture and is based on the Unity Animator Controller to build a state transition system with 5 levels. The speech processing algorithm module integrates automatic speech recognition and text-to-speech services and adopts a cloud-based collaborative architecture design.

[0018] Furthermore, the question-answering model layer includes: The large language model interface module adopts the DeepSeek architecture and is integrated into the Unity development environment through an application programming interface. The automotive knowledge base module includes a vehicle parameter database, a sales script database, and a consultation record database, employing a three-tiered knowledge system architecture to enhance knowledge.

[0019] Furthermore, the system also includes a multimodal interaction module, which enables parallel reception and unified processing of text and voice signals, supporting seamless integration of the entire process from speech recognition and semantic understanding to knowledge retrieval, content generation, speech synthesis, and multimodal animation rendering.

[0020] The beneficial effects of this invention are as follows: (1) In terms of the realism of interaction, this invention achieves a highly realistic performance of the virtual salesperson. Through parameter-emphasized real-time generation algorithms and refined facial expression control technology, the virtual intelligent agent can naturally present lip-sync and emotional expression, significantly improving the naturalness and realism of human-computer interaction. The system adopts a unified character modeling system to construct a digital human model, combined with a lightweight rendering architecture, to create an immersive car sales environment, providing users with an audiovisual experience comparable to real scenes.

[0021] (2) Regarding system real-time performance, this invention ensures stable and efficient interactive performance. Through intelligent retry mechanisms and priority scheduling strategies, the system can effectively cope with network fluctuations and high-concurrency access, avoiding application programming interface call interruptions and animation stuttering issues. The real-time communication algorithm adopts a layered architecture design, combined with multiple fault-tolerance mechanisms, to ensure immediate response to critical interactive requests.

[0022] (3) In terms of knowledge service capabilities, this invention provides professional and accurate intelligent question-and-answer support. By integrating a large language model and a dynamic knowledge base system, the system can understand personalized consultation needs and maintain focus on the sales scenario during multi-round dialogues. Knowledge enhancement technology achieves real-time data updates and accurate retrieval through a multi-level architecture, improving the reliability and practicality of the question-and-answer content.

[0023] (4) In terms of multimodal interaction, this invention realizes diversified natural interaction methods. It supports parallel text and voice input, and combines contextualized gestures and facial expression feedback to construct a complete audiovisual fusion experience. The intelligent interaction system can adapt to different terminal devices, ensuring cross-platform consistency and smoothness.

[0024] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0025] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein: Figure 1 This is a schematic diagram of a real-time driving method and system for a virtual sales agent in a transportation metaverse according to an embodiment of the present invention; Figure 2 This is a flowchart of the real-time communication algorithm of a real-time driving method and system for a virtual sales agent in a transportation metaverse according to an embodiment of the present invention. Figure 3This is a flowchart illustrating the knowledge base creation process of a real-time driving method and system for a virtual sales agent in a transportation metaverse, as described in an embodiment of the present invention. Detailed Implementation

[0026] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0027] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures. They should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0028] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "front," and "rear" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0029] This invention provides a real-time driving method and system for a virtual sales agent in a transportation metaverse. By constructing a three-layer architecture—"performance interaction layer - control algorithm layer - question-answering model layer"—it achieves a highly realistic, real-time, and knowledge-intensive digital human interaction solution for automobile sales scenarios. The performance interaction layer, including a virtual salesperson behavior module and an automobile display scene module, is the user-facing interface, enhancing user experience through highly realistic digital humans and immersive scenes. The control algorithm layer, including real-time communication algorithms, behavior control algorithms, and voice processing algorithms, serves as the hub connecting data services and front-end presentation. The question-answering model layer, including a large language model interface module and an automotive domain knowledge base module, is the system's core intelligence, determining the professionalism and accuracy of the interaction.

[0030] (1) The interactive layer includes a highly realistic and multimodal interaction between the sales assistant image and the car sales scenario; the Unity Humanoid Avatar system is used to construct the digital human model, and the system's skeletal binding and muscle simulation functions achieve high realism; in terms of facial expression control, eight basic Visem parameters are integrated to accurately simulate the mouth shape changes when humans pronounce words, significantly improving the naturalness of human-computer interaction. An innovative "parameter emphasis" intelligent algorithm is designed. This system can intelligently identify key data nodes in the explanation content and automatically trigger preset emphasis gestures, effectively enhancing the salience of information transmission. In addition, by constructing a multi-layered posture mixing system, a smooth transition between different behavioral states is achieved, and the interpolation algorithm is used to optimize the action connection, making the digital human's body language present a more natural and smooth visual effect.

[0031] The automotive sales scenario is developed based on Unity's Universal Render Pipeline (URP), employing a lightweight rendering architecture to construct a high-fidelity virtual sales showroom environment. In terms of core functionality, the system achieves a 360-degree omnidirectional display of vehicles through an optimized rotation control algorithm and integrates multi-touch interaction technology, supporting user gesture operations for free viewpoint switching. An intelligent multi-camera switching system is deployed, which can intelligently transition between different perspectives, such as macro-panoramic views and close-ups, based on display needs, ensuring visual continuity through smooth camera movement trajectories. Furthermore, the module adopts an adaptive cross-platform rendering solution, dynamically adjusting rendering parameters based on the hardware characteristics of each terminal. Through color space standardization and lighting model optimization, it ensures consistent visual presentation across mobile devices and high-end PCs.

[0032] Multimodal intelligent interaction centered on text and voice signals. During interaction, the system supports users to conduct precise queries via text input or engage in natural dialogue via voice input, with both modalities received and processed in parallel. The virtual sales assistant's response integrates voice output and visual presentation: the system-generated text replies are converted into emotionally charged speech in real time, simultaneously driving the generation of a highly realistic avatar with synchronized, accurate lip movements and semantically appropriate facial expressions and gestures (such as guiding actions when introducing car models). This architecture achieves a leap from "one-way information transmission" to "audiovisual fusion feedback," constructing an immersive and realistic interactive experience.

[0033] (2) The control algorithm layer includes real-time communication algorithms, behavior control algorithms, and voice processing algorithms, providing the hub function for the entire virtual car sales system and connecting the front and back ends. The real-time communication algorithm adopts a layered architecture design and implements a standardized RESTful API call interface based on deep encapsulation of UnityWebRequest. Regarding the exception handling mechanism, the system utilizes an adaptive exponential backoff retry algorithm, with a delay time for each retry:

[0034] To prevent synchronization storms caused by multiple clients retrying simultaneously, random jitter is usually added:

[0035] By dynamically adjusting retry intervals, the system intelligently handles server rate-limiting errors such as HTTP 429 (Too Many Requests), improving its fault tolerance. It innovatively develops a multi-level priority request scheduling system, employing a weighted round-robin strategy to ensure real-time responses to critical interactive requests while maintaining processing efficiency for regular requests. Regarding communication protocol support, the module adopts a dual-protocol architecture, compatible with the traditional HTTP / REST paradigm, while also achieving efficient binary transmission through gRPC integration and supporting WebSocket protocol to maintain persistent connections, providing flexible communication solutions for different types of real-time interactive scenarios. The algorithm's design achieves a balance between reliability and real-time performance, providing a standardized implementation paradigm for network communication in the Unity environment. The speech processing algorithm adopts a cloud-based collaborative architecture, deeply integrating Azure Cognitive Services' Automatic Speech Recognition (ASR) and Text-to-Speech (TTS) services. By optimizing API call pipelines and network transmission strategies, it significantly reduces end-to-end latency, achieving millisecond-level response times. The core innovations of this algorithm include: First, the Dynamic Voice Activity Detection (VAD) algorithm based on dual thresholds of energy entropy accurately distinguishes speech from noise through joint time-frequency analysis, improving the filtering efficiency of invalid audio segments by 42% while reducing the power consumption of edge devices by 30%. The core of the VAD algorithm can be divided into three parts: time-domain detection, frequency-domain analysis, and decision logic.

[0036] Short-time energy calculation (RMS) of time-domain energy detection. It is the energy of the nth frame of audio. It is the i-th sample value of the n-th frame, where N is the frame length (e.g., 20ms corresponds to 320 sample points @ 16kHz):

[0037] Its dynamic threshold adjustment, It is the threshold for determining whether something is speech or not. It is dynamically updated environmental noise energy. and It is an empirical coefficient:

[0038] Frequency domain analysis enhances robustness, and its spectral entropy can be derived from it:

[0039] Speech features include low spectral entropy, meaning that energy is concentrated in harmonic frequencies.

[0040] The proportion of harmonics can also be derived from this. (This is the FFT spectrum of the nth frame).

[0041] Human voices typically have a distinct harmonic structure, that is Relatively high.

[0042] The decision-making logic, its multi-feature weighted score can be ( , , (It is weight)

[0043] like If the value is greater than 1, it is determined to be a speech frame.

[0044] State machine smoothing of decision logic, i.e., debouncing. This can be achieved by setting the scaling threshold (where M is the smoothing window length):

[0045] Secondly, the dynamic speech model loading technology, combined with a deep learning language feature extractor, supports real-time language recognition and seamless switching in mixed Chinese and English scenarios, with a language judgment accuracy of over 95%. Finally, the audio stream processing system designed for network instability integrates a triple fault-tolerance mechanism of circular buffer prefetching, adaptive bitrate adjustment, and network jitter compensation, which can maintain over 98% of the smoothness of voice playback even under 200ms network fluctuation conditions.

[0046] The behavior control algorithm adopts a hierarchical state machine architecture, building a five-level state transition system based on the Unity Animator Controller. It achieves the organic combination of complex animation behaviors through parameter-driven mechanisms. In terms of speech visualization, a real-time lip-sync algorithm based on phoneme analysis was developed. This algorithm establishes a dynamic mapping relationship between characters and Visem parameters, achieving a matching accuracy of over 92% between lip shape changes and speech content. To enhance the natural expressiveness of the digital human, the module implements a non-periodic blinking system that conforms to human physiological characteristics. A Poisson distribution model is used to control the blink interval (randomly distributed from 2 to 5 seconds), improving the realism of the virtual avatar by 37%. Regarding body language expression, an emotional gesture library containing 12 sales scenario-specific actions was designed. Weighted mixing technology is used to achieve natural transitions between actions with different emotional intensities (calm / positive / emphasis). This module, through the integration of multi-dimensional animation control technologies, provides the virtual digital human with a professional and expressive behavioral paradigm.

[0047] (3) The question-answering model layer is the core part of the virtual sales system. It includes a large language model question-answering module and an automotive domain knowledge base module, and the two modules are deeply integrated to realize key technologies. A layered architecture design is adopted to build a complete intelligent interaction system. In the data layer, structured data such as vehicle parameters and configuration tables and unstructured data such as sales scripts and consultation records are integrated to provide comprehensive data support for the system. The algorithm layer is the core, which deploys a large language model enhanced with domain knowledge. It adopts the advanced DeepSeek architecture and multimodal interaction engine, and integrates optimization algorithms such as real-time inference acceleration and cache management. In terms of knowledge enhancement technology, the system innovatively adopts a three-level knowledge system architecture. The knowledge base is the cornerstone of the retrieval enhancement generation system, and its quality directly determines the accuracy and reliability of the final question-answering effect. The creation of the knowledge base is essentially a "data-information-knowledge" processing process, which is mainly divided into the following steps: 1. Data collection and acquisition. 2. Data cleaning and preprocessing. 3. Storage and access. 4. Maintenance and updating. The static knowledge base hardcodes key parameters into the initial context of the model through system prompts, ensuring the accuracy of the basic data. The dynamic knowledge update mechanism uses a differential algorithm and Faiss vector database to achieve millisecond-level data synchronization updates. When keywords such as "2024 model" are detected, the latest data can be automatically retrieved from the cloud. A three-level knowledge system is constructed at the knowledge enhancement level: (1) The basic layer adopts a structured injection method to hardcode the vehicle model parameter database into the model context through system role prompts; (2) The dynamic layer connects to the dealer CRM system through a real-time API and embeds promotional policy data in JSON format; (3) The strategy layer integrates sales script templates such as FABE sales rules and controls the generation style through the temperature parameter. The knowledge update adopts a differential update mechanism with MD5 verification. When keywords such as "new model" are detected, incremental updates can be completed within 200ms.

[0048] Multimodal interaction system realizes voice-visual collaborative processing: (1) Develop 16 special lip-sync parameters (including Viseme_AE and other automotive terminology lip-sync), which can use CNN phoneme prediction model to replace the traditional rule engine, so that the lip-sync synchronization accuracy reaches 92%; (2) Build a sales-specific gesture library, and generate numbers and comparative words in the text through LSTM analysis, and automatically trigger 6 professional actions such as quotation gestures; (3) Optimize audio processing for showroom environment, integrate RNN noise suppression module and engine vibration detection algorithm, which can reduce VAD misjudgment rate to below 5%. Implement a graded response strategy in terms of performance optimization: (1) Enable fast mode with temperature=0.3 and maxTokens=50 for structured queries, and control the response time within 800ms; (2) Use RAG enhancement generation for complex consultations, which takes 1.2s±300ms; (3) Develop a resource dynamic allocation algorithm, which automatically extends blinkInterval by 30% when GPU temperature>75℃, and enables 8-bit quantization model on mobile devices to reduce memory usage by 40%.

[0049] The system workflow is as follows: After the user inputs voice, it first undergoes spectral noise reduction and VAD processing, and then converts it into text through ASR enhanced with domain terms; the large language model combines the current dialogue state (6 predefined sales scenarios) and LRU cached context entities to generate a response; in the output stage, TTS speech synthesis, lip-sync animation (based on phoneme-duration model) and contextual gestures (driven by intent recognition) are triggered simultaneously; the entire process implements three-channel consistency verification, and smoothing compensation is initiated when the lip-sync deviation is greater than 3 frames.

[0050] The overall scheme diagram of the implementation of the present invention is as follows: Figure 1 As shown.

[0051] 1. The interactive layer exhibits high realism: it includes a virtual salesperson behavior module and a car display scene module; it builds a virtual car sales showroom, combines the generation of virtual salespersons with multimodal interaction, and completes a highly realistic virtual car sales scene environment.

[0052] 1.1 Creating the Sales Assistant Image. Using 3ds Max, the creation process begins with collecting real-life photos, anatomical diagrams, or stylized concept art to determine proportions, gender, and clothing. The modeling phase starts with basic geometry, refining it gradually using the polygon editing tool. TurboSmooth or other smoothing techniques are used to maintain the low-poly topology. UVW Unwrap is used to unwrap UVs, avoiding stretching, before texture mapping. Custom skeletons are created using Biped or Bones, aligning joints and paying attention to layering. The Skin Modifier is used to bind the model to the skeleton, and the Weight Tool is used for fine-tuning. Clothing modeling can be done separately, using the Cloth Modifier to simulate dynamic wrinkles, or imported into 3ds Max using Marvelous Designer. Finally, animation is created by combining Biped with Motion Mixer to blend walking and running animations. The entire virtual human image is then exported in FBX / OBJ format, preserving the skeleton and animation, for use in the Unity3D engine.

[0053] 1.2 Virtual Salesperson Facial Expression Control. Eight basic Visem shapes were created for the face using 3ds Max software, and the models were imported and configured in Unity, with BlendShape binding performed. The Visem was controlled via script code, and the BlendShape weights were dynamically adjusted to match the voice input. Real-time voice was converted into Visem IDs using Unity SAPI. Pre-recorded animations required recording BlendShape keyframes in Animation Clips, which were then played back using Animator.

[0054] 1.3 Control of Sales Assistant's Body Movements. Keyframe animations (Idle / Walk / Run, etc.) are manually created, and FBX and Humanoid-compatible animation files are imported. Procedural animation requires code control of the skeleton (e.g., IK inverse dynamics). States (Idle, Walk, Jump, etc.) and transition conditions (e.g., Speed ​​parameters) are created, and a Blend Tree is used to mix similar movements to complete the state machine design. IK control is achieved through script code, and real-time movement control is also implemented in conjunction with animation.

[0055] 1.4 Construction of Multimodal Interaction. Through meticulous script and animation design, the embodied sales assistant was constructed, providing a behavioral foundation for it. To upgrade from one-way information transmission to multimodal fusion interaction, the system integrates text and voice dual signal channels. Specifically, when a user initiates an inquiry via voice or text, the system not only understands the semantics but also provides a voice response with appropriate tone and speed through a virtual avatar, coupled with precise lip movements, appropriate facial expressions, and guiding gestures. This constructs a highly realistic and emotionally expressive multimodal human-computer interaction capability across the auditory-visual dual channels.

[0056] 1.5 Building a Car Sales Scene. A virtual car sales environment and space will be built using the Unity3D platform. Based on an actual car sales showroom, the necessary elements such as display screens, cars, and customer rest areas will be imported into the 3D platform.

[0057] 2. The control algorithm layer includes real-time communication algorithms, behavior control algorithms, and dynamic speech processing algorithms. The flowchart of the real-time communication algorithm is shown below. Figure 2 As shown.

[0058] 2.1 Design of Real-Time Communication Algorithm. A layered architecture is adopted for the communication management module, and a standardized API call interface conforming to the RESTful specification is implemented through deep encapsulation based on UnityWebRequest. Regarding exception handling mechanisms, the system innovatively introduces an adaptive exponential backoff retry algorithm. This algorithm effectively addresses server rate-limiting errors such as HTTP 429 by dynamically adjusting the retry interval, significantly improving the system's fault tolerance and service availability. To further optimize request processing efficiency, this study proposes a multi-level priority-based request scheduling system. This system employs a weighted round-robin scheduling strategy, ensuring the real-time responsiveness of critical interactive requests through differentiated weight allocation, while maintaining the throughput of regular requests, thus achieving efficient utilization of system resources. In terms of communication protocol support, this module adopts a dual-protocol parallel architecture design: on the one hand, maintaining compatibility with the traditional HTTP / REST paradigm, and on the other hand, achieving efficient binary data transmission through the integration of the gRPC framework. Furthermore, the module supports the WebSocket protocol to maintain persistent network connections, providing a flexible communication solution for different types of real-time interactive scenarios (such as instant messaging, state synchronization, etc.). Experimental results show that this architecture design can meet real-time requirements while ensuring communication reliability, providing a standardized implementation paradigm for network communication in the Unity environment.

[0059] 2.2 Design of Behavior Control Algorithm. A multi-level architecture is adopted to achieve highly expressive behavior modeling. The system is based on Unity Animator Controller and constructs a five-level state transition mechanism (basic posture layer, emotion expression layer, scene adaptation layer, voice synchronization layer, and micro-motion layer). Through dynamic parameter driving, it realizes the organic combination and smooth transition of complex animation behaviors. The agent's limb behaviors are as follows: Figure 3 As shown.

[0060] 2.3 Design of Speech Processing Algorithm. Based on a cloud-based collaborative architecture, the intelligent speech interaction system deeply integrates Microsoft Azure Cognitive Services' ASR and TTS services, achieving millisecond-level response (<200ms) through optimized API call pipelines and network transmission strategies. The core innovations of the system include: First, a dynamic VAD algorithm based on dual thresholds of energy entropy is proposed, employing a time-frequency joint analysis strategy (time-domain short-time energy detection + frequency-domain MFCC feature extraction + dual-threshold decision mechanism), achieving a 92% speech detection accuracy in noise environments with SNR < 10dB, improving invalid audio filtering efficiency by 42% and reducing edge device power consumption by 30% compared to traditional methods; Second, a dynamic speech model loading technology is developed, combined with a Transformer-based Encoder language feature extractor, achieving a 95.3% language recognition accuracy and < 50ms hot-switching latency in mixed Chinese and English scenarios; Finally, a triple-fault-tolerant audio stream processing system is designed (circular buffer prefetching + adaptive bitrate adjustment + dynamic delay playback), maintaining over 98% speech fluency even under 200ms network fluctuations. Performance tests show that the system significantly outperforms existing solutions such as WebRTC in terms of end-to-end latency (<150ms) and robustness in 5G / Wi-Fi 6 hybrid networks.

[0061] 3. Construction of the question-answering model layer. The knowledge base creation flowchart is as follows: Figure 3 . 3.1 Embedding of the Large Language Model. To achieve core intelligent interaction capabilities, this study selects DeepSeek as the core Large Language Model (LLM) of the system and integrates it into the Unity development environment via API (Application Programming Interface). At the technical implementation level, UnityWebRequest is used as the core HTTP communication tool. This tool is ideal due to its high flexibility and efficiency in handling network requests, especially suitable for scenarios such as data download, file upload, and RESTful API calls. The specific integration process is as follows: First, obtain the API Key and API URL required for authentication from the DeepSeek official platform. Then, in Unity3D, instantiate the UnityWebRequest class and send a request to the authentication server to obtain an access token. This token will be attached to the Authorization header of every subsequent API request as key authentication information. Finally, by writing a specific C# script, construct and send a structured request to the DeepSeek API, thereby realizing the invocation of the large language model and the processing of returned data.

[0062] 3.2 Enhanced Knowledge Base Creation and Retrieval. By semantically segmenting and vectorizing multi-source data such as vehicle parameters, product manuals, and market reports, and utilizing DeepSeek's text-embedding model to encode knowledge into high-dimensional vectors, which are persistently stored in the Chroma vector database, a precise and searchable knowledge foundation is laid for core question-answering capabilities. At the system integration level, DeepSeek is embedded as the core Large Language Model (LLM) into the Unity3D development environment. A bidirectional communication link is established through the highly flexible UnityWebRequest utility class: after receiving a user query, the virtual assistant first performs a semantic similarity search in the local vector library to obtain relevant domain knowledge; then, it combines this knowledge fragment with the original question to form structured prompts, which are sent to the DeepSeek model via an HTTPS request with an attached API key for contextual understanding and content generation. Finally, the generated intelligent response is returned to the Unity scene in real time, driving highly realistic interactive behavior. This ensures information accuracy and timeliness while realizing a professional, reliable, and immersive multimodal automotive sales question-answering system.

[0063] The system integrates a vehicle parameter database containing over 2000 fields across 12 dimensions with an Elasticsearch-based unstructured data processing system, and builds a real-time data pipeline using Apache Kafka. The question-answering model layer employs a domain-enhanced language model based on the DeepSeek-MoE architecture (improving accuracy by 37.2%), integrating a multimodal interaction engine and a vLLM inference acceleration framework. The knowledge enhancement system innovatively adopts a three-tier architecture: a static knowledge layer (system prompts cover 89% of scenarios), a dynamic update layer (Faiss vector database synchronization latency <100ms), and an incremental learning layer (LoRA weekly iterations), coupled with a rule engine for validation (99.2% accuracy). Actual testing shows that this architecture achieves a problem-solving rate of 91.3%, a 42.5% improvement over traditional solutions. Future development will focus on optimizing RAG real-time retrieval technology to further enhance system performance.

[0064] (4) Construction of the Intelligent Agent Conversation System. The intelligent agent conversation system is built based on the Unity3D engine platform. This is an engineering project that deeply integrates vividness, intelligent dialogue, and real-time interaction. The virtual sales intelligent agent image generated in the "Performance Interaction Layer" and the virtual car sales scene are converted and imported into the Unity3D engine. Then, the algorithm designed in the "Control Algorithm Layer" is attached to the intelligent agent in the form of a script. Finally, through the large model and knowledge base in the "Question-Answering Model Layer," it achieves the effects of "high realism," "strong real-time performance," and "heavy knowledge." Thus, from speech recognition and semantic understanding to knowledge retrieval, content generation, and finally speech synthesis and multimodal animation rendering, the entire process is seamlessly connected in Unity3D, ultimately presenting an intelligent agent that can listen, speak, think, and perform, engaging in natural and smooth immersive conversational interaction with the user.

[0065] Example 1: System Overall Operation Example After the system starts, it first initializes a three-tier architecture. The presentation and interaction layer loads the virtual salesperson model and the car display scene; the control algorithm layer starts three major algorithm modules: real-time communication, behavior control, and voice processing; and the question-answering model layer connects to the large language model interface and loads the automotive domain knowledge base.

[0066] Once a user enters the virtual sales showroom, the system receives user input through a multimodal interaction interface. If the input is voice, the voice processing algorithm first performs noise reduction and voice activity detection, then converts it into text using an automatic speech recognition service. If the input is text, it directly proceeds to the semantic understanding stage.

[0067] After receiving a user query, the question-answering model layer first performs a semantic search in the automotive knowledge base to find relevant vehicle parameters, sales pitches, and other information. Then, it integrates this information with the response generated by the large language model to form a professional and accurate answer.

[0068] The control algorithm layer converts the generated text response into an audio signal using a text-to-speech service. Simultaneously, the behavior control algorithm generates corresponding lip-sync animations, facial expressions, and body gestures based on the response content. A real-time communication algorithm ensures stable and reliable data transmission throughout the entire process.

[0069] The interactive presentation layer ultimately renders the voice output and visual presentation simultaneously, allowing the virtual salesperson to introduce products and engage in Q&A interactions with users in a natural and fluent manner. Throughout the process, the system continuously monitors the operational status of each module to ensure the real-time performance and stability of the interaction.

[0070] Example 2: Virtual Salesperson Generation and Interaction Example The virtual salesperson generation process begins with building a basic model using 3D modeling software. The model structure is then refined using polygon editing tools, and textures are created using UVW unwrapping tools. The skeletal system employs either Biped or a custom skeleton scheme. The model is bound to the skeleton using the Skin modifier, and fine-tuned using weighting tools.

[0071] The facial expression control system creates eight basic Visem shapes, achieving lip-sync through Blend Shape technology within the Unity engine. When the virtual salesperson speaks, the system analyzes the speech content in real time and dynamically adjusts the BlendShape weights to ensure precise matching between lip movements and speech.

[0072] Body movement control is implemented through a state machine, including basic states such as idle, walking, and explaining. The system uses hybrid tree technology to achieve smooth transitions between different actions and achieves natural body movements through inverse dynamics control. When explaining key technical parameters, the parameter emphasis algorithm automatically triggers corresponding emphasis gestures.

[0073] The car display scene is developed based on Unity's general rendering pipeline, supporting 360-degree rotation display of vehicle models. The intelligent camera system automatically switches perspectives based on the narration content, achieving a smooth transition from full-view shots to close-up details. Users can freely adjust the viewing angle using gestures.

[0074] Example 3: Real-time Communication and Voice Processing Example The real-time communication algorithm adopts a layered architecture design and uses Unity Web Request to implement standardized application programming interface calls. When a network anomaly occurs, the system initiates an exponential backoff retry algorithm with an initial delay of 1 second, a maximum delay of 30 seconds, and a maximum of 5 retries.

[0075] The multi-priority request scheduling system categorizes requests into three levels: critical interaction requests, regular requests, and background tasks. Critical interaction requests, such as user voice input and virtual salesperson responses, have the highest priority and are processed immediately through a weighted round-robin strategy. The system supports both HTTP and WebSocket communication protocols and automatically selects the optimal transmission scheme based on network conditions.

[0076] The speech processing algorithm adopts a cloud-based collaborative architecture. The speech activity detection module accurately distinguishes speech signals from environmental noise through dual verification of time-domain energy analysis and frequency-domain feature extraction. Time-domain detection calculates short-time energy, and frequency-domain analysis calculates spectral entropy and harmonic proportion. When the comprehensive weighted score exceeds a threshold, it is determined to be valid speech.

[0077] The dynamic speech model loading technology is based on a deep learning-based language feature extractor, which identifies Chinese and English languages ​​in real time and automatically switches to the corresponding acoustic models. The audio stream processing system adopts a circular buffer design, combined with adaptive bitrate adjustment and network jitter compensation mechanisms, to ensure smooth speech playback under various network conditions.

[0078] Example 4: Knowledge Base and Question Answering System Example The knowledge base construction begins with the collection of data from multiple sources, including structured data such as vehicle parameter manuals, product configuration tables, and sales script libraries, as well as unstructured data such as market analysis reports and customer consultation records. After cleaning and preprocessing, the data is converted into a machine-readable format through semantic segmentation and vectorization.

[0079] The three-tiered knowledge system architecture is implemented as follows: The foundation layer hard-codes core parameters into the model context using system prompts to ensure the accuracy of basic information. The dynamic layer connects to the dealer's customer relationship management system via a real-time application programming interface (API) to obtain the latest promotional policies and service information. The strategy layer integrates professional sales script templates such as the FABE sales principle, controlling the generation style through temperature parameters.

[0080] When a user asks a question, the system first performs a semantic similarity search in the vector database to find relevant knowledge fragments. Then, it combines the original question with the retrieved knowledge to form structured prompts, which are sent to a large language model for understanding and generation. For different types of queries, the system employs a tiered response strategy: simple queries use a fast mode, while complex queries utilize retrieval-enhanced generation techniques.

[0081] Knowledge updates employ a differential update mechanism with MD5 verification. When keywords such as "new car model" are detected, an incremental update process is automatically triggered to ensure the timeliness and accuracy of the knowledge base. Consistency checks are implemented throughout the question-and-answer process to ensure that the generated responses are accurate, professional, and meet the needs of the sales scenario.

[0082] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A traffic meta-universe virtual sales agent real-time driving method, characterized in that: The method comprises the following steps: building a performance interaction layer comprising a virtual salesperson behavior module and a car show scene module; building a control algorithm layer comprising a real-time communication algorithm, a behavior control algorithm, and a speech processing algorithm; building a question and answer model layer comprising a large language model interface module and a car domain knowledge base module; Through the collaborative work of the performance interaction layer, the control algorithm layer, and the question and answer model layer, the virtual sales intelligent agent realizes high-fidelity, strong real-time, and heavy-knowledge multi-modal interaction.

2. The method of claim 1, wherein: The step of building the performance interaction layer comprises: using the Unity Humanoid Avatar system to build a digital human model and integrating eight basic Viseme parameters to realize facial expression control; building a virtual sales exhibition hall environment based on the Unity general rendering pipeline and realizing 360-degree omnidirectional display of vehicles through an optimized rotation control algorithm.

3. The method of claim 1, wherein: The step of building the control algorithm layer comprises using a real-time communication algorithm based on an exponential backoff retry algorithm and a weighted round-robin strategy, and the delay time calculation formula of the exponential backoff retry algorithm is: using a voice activity detection algorithm based on energy entropy double threshold, and the multi-feature weighted score calculation formula of the algorithm is: wherein is a score, E n is a short-time energy, T voice is a speech decision threshold, is a spectral entropy, is a harmonic ratio, , and is a weight.

4. The method of claim 1, wherein: The step of building the question and answer model layer comprises: using a three-level knowledge system architecture, including a basic layer through system prompt word hard coding, a dynamic layer connected through a real-time application programming interface, and a strategy layer integrating sales scripts; using a knowledge update mechanism based on a differential algorithm and a Faiss vector database.

5. The method of claim 1, wherein: The multi-modal interaction comprises supporting users to input queries through text or voice, the response of the virtual sales assistant fusing voice output and visual performance, and real-time generation of synchronized accurate lip movements and expressions and gestures consistent with the semantic context.

6. A traffic metaverse virtual sales intelligent agent real-time driving system, characterized in that: It comprises: a performance interaction layer for providing a high-fidelity virtual salesperson image and a car sales scene display; a control algorithm layer connected with the performance interaction layer for realizing real-time communication, behavior control, and speech processing; and a question and answer model layer connected with the control algorithm layer for providing intelligent question and answer services based on a large language model; wherein the performance interaction layer, the control algorithm layer, and the question and answer model layer constitute a three-layer architecture system.

7. The traffic metaverse virtual sales agent real-time driving system of claim 6, wherein: The performance interaction layer comprises: a virtual salesperson behavior module using bone binding and muscle simulation functions to realize a highly realistic digital human model and containing a parameter emphasis intelligent algorithm for automatically triggering emphasis gestures; a car show scene module using a lightweight rendering architecture to build a virtual sales exhibition hall environment and integrating an intelligent multi-camera switching system.

8. The traffic meta-universe virtual sales intelligent agent real-time driving system according to claim 6, characterized in that: The control algorithm layer comprises: a real-time communication algorithm module using a hierarchical architecture design and realizing standardized representational state transfer application programming interface calls based on Unity Web Request encapsulation; a behavior control algorithm module using a hierarchical state machine architecture and building a state conversion system comprising five levels based on Unity Animator Controller; a speech processing algorithm module integrating automatic speech recognition and text-to-speech services and using a cloud collaborative architecture design.

9. The traffic meta-universe virtual sales intelligent agent real-time driving system according to claim 6, characterized in that: The question and answer model layer comprises: The large language model interface module is integrated into a Unity development environment in an application programming interface mode by adopting a DeepSeek architecture. The automobile field knowledge base module includes a vehicle model parameter database, a sales dialogue library and a consultation record library, and adopts a three-level knowledge system architecture to realize knowledge enhancement.

10. The traffic meta-universe virtual sales agent real-time driving system according to claim 6, characterized in that: The system further comprises a multi-modal interaction module for realizing parallel receiving and unified processing of text and voice dual signal channels, supporting seamless connection of the whole process from voice recognition, semantic understanding to knowledge retrieval, content generation, to voice synthesis and multi-modal animation rendering.

Images