A desktop floating window digital human system of an open protocol

Abstract

This invention relates to the field of digital human technology and provides an open protocol-based desktop floating window digital human system. The system includes: a service abstraction interface layer, defining a unified call interface independent of AI service providers; a business application layer, which initiates AI calls and receives results through this interface; a configuration management layer, storing configuration data containing the current AI service provider's identifier; an abstract factory layer, dynamically creating corresponding adapter instances based on the configuration; an adapter implementation layer, containing multiple adapters, each corresponding to an AI service provider and implementing the unified call interface, used for protocol conversion with external service interfaces; the adapter implementation layer also includes a dynamic loading module, used to scan the adapter directory, identify and register available adapters, and load corresponding adapter instances on demand according to the abstract factory call request, caching and reusing them. This invention achieves decoupling of business logic from AI service providers through a five-layer architecture, solving the problems of high coupling and low scalability in multi-service provider integration.

Description

Technical Field

[0001] This invention relates to the fields of digital humans, artificial intelligence, and other technologies, and in particular to a desktop floating window digital human system with an open protocol. Background Technology

[0002] Existing desktop digital human systems typically employ a closed architecture, deeply tied to specific AI service providers (such as speech recognition, large language models, and speech synthesis services). This traditional approach suffers from the following technical drawbacks.

[0003] First, it suffers from high coupling and poor scalability. Business logic code directly calls specific vendors' SDKs or APIs, resulting in strong coupling between the system and vendor services. When integrating new AI service providers, core business code must be modified, leading to long development and testing cycles and extremely low scalability.

[0004] Second, vendor lock-in leads to high switching costs. With the system locked into a single vendor, changing AI service providers involves significant code modifications and regression testing, resulting in substantial costs. Users cannot flexibly switch services based on cost, performance, or stability requirements.

[0005] Third, maintenance is difficult and fault tolerance is poor. Upgrades or changes to vendor APIs directly affect multiple core code areas of the system, resulting in a large maintenance workload. If a single vendor's service fails, the corresponding system functions will become unavailable. The lack of effective service degradation and fault tolerance mechanisms leads to low overall system reliability.

[0006] Fourth, code reuse is low, leading to low development efficiency. Repeatedly writing similar call, data processing, and error handling code for each AI service provider results in a large amount of repetitive work, low development efficiency, and unusable test cases.

[0007] In summary, existing digital human system technologies struggle to integrate and switch between multiple AI capabilities at low cost and high efficiency, failing to meet the demands of desktop digital humans for flexibility, reliability, and scalability. Summary of the Invention

[0008] To address the shortcomings of the existing technologies, this invention provides an open protocol-based desktop floating window digital human system to meet the needs of desktop digital humans for flexibility, reliability, and scalability.

[0009] The open protocol-based desktop floating window digital human system provided by this invention includes: The service abstraction interface layer is used to define a unified calling interface that is independent of specific AI service providers; The business application layer, coupled with the service abstraction interface layer, is used to initiate AI capability call requests through the unified call interface and receive return results. A configuration management layer is used to store and manage configuration data, which includes at least the identification information of the currently selected AI service provider; The abstract factory layer is coupled to the service abstract interface layer and the configuration management layer, respectively, and is used to dynamically determine and create the corresponding adapter instance based on the configuration data. The adapter implementation layer contains multiple adapters, each corresponding to an AI service provider and implementing the unified call interface, and is used for protocol conversion with the external service interface of the corresponding AI service provider; the adapter implementation layer includes a dynamic loading module, which is used to scan a preset adapter directory when the system starts, identify and register available adapters, load the corresponding adapter instance on demand according to the call request of the abstract factory layer when the system runs, and cache the loaded adapter instance for reuse; The business application layer obtains the adapter instance corresponding to the current configuration through the abstract factory layer and communicates with the adapter instance through the unified call interface. The adapter instance converts the received call request conforming to the unified call interface into the protocol format required by the corresponding AI service provider's external service interface and forwards it. It also converts the response received from the external service interface back into the unified call interface format and returns it to the business application layer.

[0010] Compared with the prior art, the beneficial effects of this invention are as follows: The open protocol-based desktop floating window digital human system provided by this invention includes: a service abstraction interface layer, used to define a unified call interface independent of specific AI service providers; a business application layer, coupled to the service abstraction interface layer, used to initiate AI capability call requests through the unified call interface and receive return results; a configuration management layer, used to store and manage configuration data, the configuration data including at least the identification information of the currently selected AI service provider; an abstract factory layer, coupled to both the service abstraction interface layer and the configuration management layer, used to dynamically determine and create corresponding adapter instances based on the configuration data; and an adapter implementation layer, containing multiple adapters, each adapter corresponding to an AI service provider and implementing the unified call interface, used to communicate with external systems of the corresponding AI service provider. The service interface performs protocol conversion; the adapter implementation layer includes a dynamic loading module, which scans a preset adapter directory at system startup, identifies and registers available adapters, loads corresponding adapter instances on demand according to the call requests of the abstract factory layer during system runtime, and caches loaded adapter instances for reuse; the business application layer obtains the adapter instance corresponding to the current configuration through the abstract factory layer and communicates with the adapter instance through the unified call interface; the adapter instance converts the received call requests conforming to the unified call interface into the protocol format required by the corresponding AI service provider's external service interface before forwarding them, and converts the responses received from the external service interface back into the unified call interface format before returning them to the business application layer. The system provided in this embodiment of the invention, through the organic coordination of five functional layers, achieves technical decoupling between business logic and specific AI service providers, solving technical problems such as high coupling, low scalability, and difficult maintenance in the integration of multiple AI service providers. Attached Figure Description

[0011] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. Some specific embodiments of the invention will be described in detail below with reference to the accompanying drawings in an exemplary and non-limiting manner. The same reference numerals in the drawings designate the same or similar parts or components. It should be understood by those skilled in the art that these drawings are not necessarily drawn to scale. In the drawings: Figure 1 This is a schematic diagram of an architecture of a desktop floating window digital human system based on an open protocol according to an embodiment of the present invention. Detailed Implementation

[0012] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are merely some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0013] See Figure 1 This invention provides an open protocol-based desktop floating window digital human system, comprising: The service abstraction interface layer is used to define a unified calling interface that is independent of specific AI service providers; The business application layer, coupled with the service abstraction interface layer, is used to initiate AI capability call requests through the unified call interface and receive return results. A configuration management layer is used to store and manage configuration data, which includes at least the identification information of the currently selected AI service provider; The abstract factory layer is coupled to the service abstract interface layer and the configuration management layer, respectively, and is used to dynamically determine and create the corresponding adapter instance based on the configuration data. The adapter implementation layer contains multiple adapters, each corresponding to an AI service provider and implementing the unified call interface, and is used for protocol conversion with the external service interface of the corresponding AI service provider; the adapter implementation layer includes a dynamic loading module, which is used to scan a preset adapter directory when the system starts, identify and register available adapters, load the corresponding adapter instance on demand according to the call request of the abstract factory layer when the system runs, and cache the loaded adapter instance for reuse; The business application layer obtains the adapter instance corresponding to the current configuration through the abstract factory layer and communicates with the adapter instance through the unified call interface. The adapter instance converts the received call request conforming to the unified call interface into the protocol format required by the corresponding AI service provider's external service interface and forwards it. It also converts the response received from the external service interface back into the unified call interface format and returns it to the business application layer.

[0014] It should be noted that the system provided in this embodiment achieves technical decoupling between business logic and specific AI service providers through the organic coordination of five functional levels, solving technical problems such as high coupling, low scalability, and difficult maintenance in the integration of multiple AI service providers.

[0015] First, the establishment of the service abstraction interface layer brings about the technical effect of interface unification. This layer defines a unified calling interface independent of specific AI service providers, including standardized request and response data structures. This allows the business application layer to program only based on this abstract interface without worrying about the API differences of specific underlying vendors. When it is necessary to adjust request parameters or process response data, only unified modifications need to be made at the abstract interface level, avoiding the tedious operation of scattered modifications in business code and significantly improving code maintainability.

[0016] Secondly, the collaborative work between the configuration management layer and the abstract factory layer enables dynamic service selection. The configuration management layer stores the identifier information of the currently selected AI service provider, and the abstract factory layer dynamically creates the corresponding adapter instance based on this identifier. This mechanism allows switching AI service providers to be done only by modifying configuration data, without altering any business code. For example, when a user switches from service provider A to service provider B, the abstract factory layer automatically creates the corresponding adapter, and the business application layer is completely unaware of this switch, achieving the technical effect of runtime dynamic switching and solving the problem of high switching costs in traditional technologies.

[0017] Third, the dynamic loading module in the adapter implementation layer brings about resource optimization. This module scans a preset directory to identify and register available adapters at system startup, and loads corresponding instances on demand and caches them for reuse at runtime. This lazy loading strategy avoids the memory waste caused by loading all adapters at system startup, while the caching mechanism reduces the performance overhead of frequently creating instances. For systems integrating dozens of AI service providers, this mechanism can significantly reduce memory usage and improve response speed.

[0018] Fourth, the protocol conversion mechanism enables interoperability between heterogeneous systems. Adapter instances are responsible for converting requests from the unified API into the protocol format required by the vendor's external interface, and converting the response back to the unified format. This bidirectional conversion mechanism shields the differences between different vendors in API protocols, data formats, authentication methods, etc., allowing the business application layer to call the AI ​​capabilities of any vendor in a unified way. When integrating a new AI service provider, only the corresponding adapter needs to be developed; there is no need to modify the core business logic, fundamentally improving system scalability.

[0019] Preferably, each adapter in the adapter implementation layer includes a request conversion module and a response conversion module; the request conversion module is used to convert the request data structure of the unified call interface into the request format required by the external service interface of the corresponding AI service provider, and the response conversion module is used to convert the response data returned by the external service interface into the response data structure of the unified call interface.

[0020] It should be noted that in this embodiment, the request conversion module is specifically responsible for converting the internal request format to the format required by the vendor, while the response conversion module is specifically responsible for converting the vendor's response back to the internal format. When a vendor's API upgrade causes changes in request parameters, only the request conversion module needs to be modified; when the vendor's response structure changes, only the response conversion module needs to be modified. The scope of modification is limited to a single module, reducing the complexity of code modification and the risk of introducing errors. For multiple vendors using the same protocol standard, the request conversion module can reuse the same conversion logic, only needing to distinguish the endpoint addresses and authentication information of different vendors through configuration, reducing redundant development work. The independent request conversion module and response conversion module can be unit tested separately, making test cases more focused and test coverage easier to achieve.

[0021] Preferably, the dynamic loading module is further configured to: read the description file corresponding to each adapter, the description file containing at least the AI ​​service providers, protocol types and version information supported by the adapter; and register the information in the description file into the adapter registry for querying by the abstract factory layer.

[0022] It should be noted that in this embodiment, by scanning the description files in a preset directory, the system can automatically identify newly installed adapters and register them in the adapter registry, realizing a pluggable adapter extension mechanism. Users only need to place the adapter file in the specified directory, and the system can automatically recognize and use it without modifying configuration files or restarting the system. After the metadata such as AI service provider, protocol type, and version information contained in the description file is registered in the adapter registry, the abstract factory layer can make intelligent decisions based on this metadata. For example, when a user selects a service provider, the abstract factory layer can query the registry to find the corresponding version of the adapter; when multiple versions of the adapter exist, the latest version or a specified version can be selected. The registration of version information enables the system to manage different versions of adapters from the same service provider simultaneously. When the vendor's API upgrade causes incompatibility with the old version, a smooth transition to the new version can be achieved, avoiding service interruption.

[0023] Preferably, the business application layer includes a desktop floating window module, which is used to create and manage a transparent, borderless, always-on-top floating window. The floating window supports mouse event penetration and provides interactive controls for window dragging, scaling, and transparency adjustment, as well as integrating the application icon into the system tray.

[0024] It's worth noting that the transparent, borderless, and always-on-top floating window design allows the digital human to remain permanently on the desktop, responding to user interactions without interfering with other applications. The transparent window background blends seamlessly with the desktop background, creating a smoother and more natural visual experience. Mouse event passing-through functionality allows mouse events to pass through to the underlying window when the user clicks on areas outside the digital human's image, without affecting other applications. Drag-and-drop positioning, scaling, and transparency adjustments provide users with ample customization options, allowing them to place the digital human anywhere on the screen and adjust its size and transparency to their liking. Users can quickly show / hide the floating window, open the configuration interface, switch AI service providers, and exit the application using the tray icon. The tray icon can also display system status (such as in conversation or waiting to be woken up), providing a visual representation of the status. The system's automatic saving of window position and size ensures that users can restore their preferred settings every time they launch the application, avoiding the hassle of repeated adjustments.

[0025] Furthermore, the system also includes a multimodal interaction module, which is coupled to the business application layer and the adapter implementation layer, and is used to integrate at least one of the following interaction methods: an offline voice wake-up submodule, used to monitor microphone input in real time and activate the system when a preset wake-up word is detected; a voice recognition interaction submodule, used to call the voice recognition service through the adapter to convert the user's voice into text; a voice synthesis interaction submodule, used to call the voice synthesis service through the adapter to convert the digital human's reply text into speech and play it; and a global shortcut key submodule, used to quickly activate the system or switch services through preset shortcut keys.

[0026] It's important to note that offline voice wake-up is suitable for situations where hands are limited (such as cooking or driving); global shortcut keys are suitable for scenarios where voice input is inconvenient, such as office environments; and text input mode is suitable for noisy environments. The coexistence of multiple interaction methods ensures that users can easily interact with the digital human in any scenario. The multimodal interaction module itself does not implement specific speech recognition and synthesis algorithms, but instead calls external services through an adapter architecture. This design avoids redundant development of underlying AI capabilities and allows for flexible selection of the optimal voice service provider. The offline wake-up module runs in an independent thread with extremely low resource consumption and continuously monitors the microphone; it only initiates the speech recognition service, which requires a network connection, upon detecting a wake word. This tiered wake-up mechanism ensures a readily available wake-up experience while avoiding continuous consumption of network and cloud resources, thus reducing usage costs.

[0027] Preferably, the business application layer further includes a digital human rendering and animation driving module, which is coupled to the speech synthesis interaction submodule. When playing speech, the digital human rendering and animation driving module drives the lip movements of the digital human model in real time according to the phoneme timestamp sequence returned by the speech synthesis service to achieve phoneme-level lip movement synchronization; and drives the digital human model to play corresponding facial expression animations according to the sentiment analysis results of the reply text.

[0028] It should be noted that, based on the phoneme timestamp sequence returned by the speech synthesis service, the lip movements of the digital human model are driven in real time, ensuring a high degree of match between the digital human's lip movements and speech. This prevents users from experiencing any sense of audio-visual asynchrony, enhancing the realism and immersion of the interaction. Sentiment analysis identifies the emotional tendency of the reply text and drives the digital human to play corresponding facial animations (smiling, laughing, frowning, etc.), enabling the digital human to not only "speak" but also "express emotions." Furthermore, facial animations and lip movements can be naturally integrated using weighted blending technology, avoiding abrupt changes in expression switching.

[0029] Preferably, the configuration management layer supports hot configuration updates, specifically including: maintaining a list of configuration observers, and notifying the registered observer modules in the list when the configuration data changes; the observer modules include at least one of the abstract factory layer and the business application layer, and after receiving the notification, the observer modules update their own status in real time according to the new configuration without restarting the system.

[0030] It's worth noting that traditional configuration changes often require an application restart to take effect, while this example implements hot configuration updates through the observer pattern. After a user modifies the configuration and clicks the apply button, the system takes effect immediately, without waiting for a restart, improving the smoothness and immediacy of the operation. Especially in scenarios requiring immediate feedback, such as adjusting window transparency or switching AI service providers, the hot update mechanism delivers a superior user experience.

[0031] Preferably, the system further includes a service governance module, which is coupled to the abstract factory layer and the adapter implementation layer to implement a service degradation and fault tolerance mechanism. The service degradation and fault tolerance mechanism includes at least one of the following: a retry mechanism, which performs a limited number of retries according to a preset strategy when a call fails; a circuit breaker mechanism, which temporarily blocks calls to the corresponding AI service provider when the error rate reaches a threshold; and a primary / backup switchover mechanism, which automatically switches to a backup AI service provider when the primary AI service provider is unavailable.

[0032] It should be noted that the system's availability can be improved through the organic combination of three mechanisms: retry, circuit breaker, and primary / standby failover. Specifically: the retry mechanism can handle occasional failures caused by momentary network fluctuations; the circuit breaker mechanism can prevent the continuous calling of faulty services from causing the fault to spread; the primary / standby failover mechanism automatically switches to the standby service when the primary service is completely unavailable, ensuring service continuity. Traditional fault handling often requires manual troubleshooting and manual switching, while the automatic fault tolerance mechanism in this embodiment can automatically complete fault detection and recovery within seconds, reducing the workload of maintenance personnel. Especially during unattended periods such as nighttime or holidays, the automatic fault tolerance mechanism can ensure continuous system availability. When the error rate of an AI service provider reaches a threshold, the circuit breaker temporarily cuts off calls to that service, preventing the entire system from responding slowly or crashing due to a single service failure. After the circuit breaker enters a half-open state, it allows a small number of probe requests to detect whether the service has recovered, realizing automatic fault recovery detection.

[0033] Preferably, the service governance module is also used to monitor the availability and performance metrics of each AI service provider, and automatically trigger the abstract factory layer to switch the currently used adapter instance according to a preset strategy; the preset strategy includes at least one of the following: a preferred strategy based on response time, an optimization strategy based on cost, and a round-robin or weight allocation strategy based on load balancing.

[0034] It should be noted that the response time-based optimization strategy can automatically select the service provider with the fastest response time, ensuring users receive the lowest latency interactive experience; the cost-based optimization strategy can prioritize the lowest-priced service provider while maintaining service quality, reducing operating costs; and the load balancing strategy can distribute requests across multiple service providers, avoiding service interruptions caused by quota limitations of a single service provider. Furthermore, in this embodiment, the system can continuously monitor metrics such as the number of requests, token consumption, and response time for each service provider, and periodically generate usage reports and cost optimization suggestions. For example, for simple problems, it recommends using the lower-cost service provider, while for complex problems, it recommends using the more capable service provider, achieving refined cost control and performance optimization.

[0035] Preferably, the request data structure and response data structure of the unified call interface adopt JSON format. The request data structure includes at least one or more of the following: user message, dialogue history, system prompt, temperature parameter, and maximum generation length. The response data structure includes at least one or more of the following: generated text content, completion status, and error information.

[0036] It's worth noting that JSON, as a lightweight data exchange format, boasts excellent readability and cross-platform compatibility, allowing for easy parsing and generation by various programming languages. This format choice reduces the difficulty of adapter development, enabling third-party developers to easily create new adapters for the system. Fields in the request data structure, such as user messages, conversation history, system prompts, temperature parameters, and maximum generation length, cover the core parameters for large language model calls; fields in the response data structure, such as generated text content, completion status, and error messages, cover the main information of the call result. This standardized design allows the business application layer to call AI capabilities from different vendors using a unified format, eliminating the need to learn different parameter names and formats for each vendor. The unified inclusion of error message fields in the response data structure allows the business application layer to handle errors returned by various vendors in a consistent manner, eliminating the need to write different error handling logic for each vendor, simplifying code implementation, and improving system robustness.

[0037] It should be noted that the above embodiments are merely preferred embodiments of the present invention, and the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention, and the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A desktop floating window digital human system with an open protocol, characterized in that, include: The service abstraction interface layer is used to define a unified calling interface that is independent of specific AI service providers; The business application layer, coupled with the service abstraction interface layer, is used to initiate AI capability call requests through the unified call interface and receive return results. A configuration management layer is used to store and manage configuration data, which includes at least the identification information of the currently selected AI service provider; The abstract factory layer is coupled to the service abstract interface layer and the configuration management layer, respectively, and is used to dynamically determine and create the corresponding adapter instance based on the configuration data. The adapter implementation layer contains multiple adapters, each corresponding to an AI service provider and implementing the unified call interface, and is used for protocol conversion with the external service interface of the corresponding AI service provider; the adapter implementation layer includes a dynamic loading module, which is used to scan a preset adapter directory when the system starts, identify and register available adapters, load the corresponding adapter instance on demand according to the call request of the abstract factory layer when the system runs, and cache the loaded adapter instance for reuse; The business application layer obtains the adapter instance corresponding to the current configuration through the abstract factory layer, and communicates with the adapter instance through the unified call interface; The adapter instance converts the received call request conforming to the unified call interface into the protocol format required by the corresponding AI service provider's external service interface before forwarding it, and converts the response received from the external service interface back into the unified call interface format before returning it to the business application layer.

2. The system according to claim 1, characterized in that, Each adapter in the adapter implementation layer includes a request conversion module and a response conversion module; the request conversion module is used to convert the request data structure of the unified call interface into the request format required by the external service interface of the corresponding AI service provider, and the response conversion module is used to convert the response data returned by the external service interface into the response data structure of the unified call interface.

3. The system according to claim 1, characterized in that, The dynamic loading module is also used to: read the description file corresponding to each adapter, the description file containing at least the AI ​​service providers, protocol types and version information supported by the adapter; and register the information in the description file into the adapter registry for querying by the abstract factory layer.

4. The system according to claim 1, characterized in that, The business application layer includes a desktop floating window module, which is used to create and manage a transparent, borderless, always-on-top floating window. The floating window supports mouse event penetration and provides interactive controls for window dragging, scaling, and transparency adjustment, as well as integrating the application icon into the system tray.

5. The system according to any one of claims 1-4, characterized in that, The system also includes a multimodal interaction module, which is coupled to the business application layer and the adapter implementation layer, and is used to integrate at least one of the following interaction methods: an offline voice wake-up submodule, which is used to monitor microphone input in real time and activate the system when a preset wake-up word is detected; The speech recognition interaction submodule is used to call the speech recognition service through the adapter to convert the user's speech into text; the speech synthesis interaction submodule is used to call the speech synthesis service through the adapter to convert the digital human's reply text into speech and play it. The global shortcut key submodule is used to quickly activate the system or switch services using preset shortcut keys.

6. The system according to claim 5, characterized in that, The business application layer also includes a digital human rendering and animation driving module, which is coupled with the speech synthesis interaction submodule. When playing speech, the digital human rendering and animation driving module drives the lip movements of the digital human model in real time according to the phoneme timestamp sequence returned by the speech synthesis service, so as to achieve phoneme-level lip movement synchronization. And based on the sentiment analysis results of the reply text, drive the digital human model to play the corresponding facial animation.

7. The system according to claim 1, characterized in that, The configuration management layer supports hot configuration updates, specifically including: maintaining a list of configuration observers, and notifying the registered observer modules in the list when the configuration data changes; the observer modules include at least one of the abstract factory layer and the business application layer, and after receiving the notification, the observer modules update their own status in real time according to the new configuration without restarting the system.

8. The system according to claim 1, characterized in that, The system also includes a service governance module, which is coupled to the abstract factory layer and the adapter implementation layer to implement service degradation and fault tolerance mechanisms. The service degradation and fault tolerance mechanisms include at least one of the following: a retry mechanism, which performs a limited number of retries according to a preset strategy when a call fails; a circuit breaker mechanism, which temporarily blocks calls to the corresponding AI service provider when the error rate reaches a threshold; and a primary / backup switchover mechanism, which automatically switches to a backup AI service provider when the primary AI service provider is unavailable.

9. The system according to claim 8, characterized in that, The service governance module is also used to monitor the availability and performance metrics of each AI service provider, and automatically trigger the abstract factory layer to switch the currently used adapter instance according to a preset strategy; the preset strategy includes at least one of the following: a preferred strategy based on response time, an optimization strategy based on cost, and a round-robin or weight allocation strategy based on load balancing.

10. The system according to claim 2, characterized in that, The unified call interface uses JSON format for its request and response data structures. The request data structure includes at least one or more of the following: user message, dialogue history, system prompt, temperature parameter, and maximum generation length. The response data structure includes at least one or more of the following: generated text content, completion status, and error message.

Images