A desktop floating AI digital human terminal and its interaction method

Abstract

This invention discloses a desktop floating AI digital human terminal and its interaction method, comprising: a terminal body, a display screen, an optical prism, and a circuit module; the display screen is set on the terminal body and the image output is controlled by the circuit module; the optical prism is attached to the display screen; the circuit module includes: a main control module, an integrated power supply module, a screen display module, a bus communication and level conversion module, an IO expansion module, an interaction and status feedback module, and a debugging and external expansion interface connected to the main control module; compared with the traditional planar display mode, by utilizing the refraction and reflection optical principles of the optical prism, the digital human image output from the screen is projected to form a floating pseudo-holographic image, effectively improving the three-dimensionality and visual layer of the digital human display; at the same time, the whole adopts a compact desktop shell structure, which is small in size, convenient to place, and suitable for normal desktop display, greatly improving the product display effect and user experience.

Description

Technical Field

[0001] This invention belongs to the field of AI digital human technology, and relates to a desktop floating AI digital human terminal and its interaction method. Background Technology

[0002] Existing AI digital human desktop terminals suffer from numerous technical shortcomings in terms of hardware architecture, imaging effects, and software functionality, making it difficult to meet the demands for high-quality intelligent display and interaction. Firstly, the hardware integration is poor; the pin resources of the terminal's main control board are often heavily occupied by basic peripherals such as the screen and audio, leaving insufficient resources for peripheral expansion. Furthermore, the internal devices often use peripherals from multiple voltage domains, which easily leads to level mismatch issues, resulting in unstable communication and a high failure rate.

[0003] Secondly, the imaging technology has obvious defects. Most of the mainstream desktop AI digital humans currently use traditional flat screen displays, which lack stereoscopic imaging effects and are insufficient in terms of technological sense and display quality. The few devices with levitation imaging function generally have problems such as complex structure, large size, blurry imaging and poor adaptability. They cannot be adapted to the lightweight and compact use scenarios of small desktop ornaments, and the pseudo-holographic levitation imaging effect is poor and the viewing experience is not good.

[0004] Finally, the software functionality is incomplete, the existing terminal interaction mode is simplistic, and it lacks intelligent functions such as personalized character customization, exclusive voiceprint recognition, and customized voice generation. Human-computer interaction is rigid and inflexible, and the system's scalability is weak, making it difficult to flexibly connect to third-party cloud services and AI interfaces, thus limiting its ability to iterate functions and adapt to different scenarios. In summary, there is an urgent need for a desktop floating AI digital human terminal and its supporting control system that features high hardware integration, stable communication, excellent lightweight optical levitation imaging, rich intelligent interaction, and strong scalability. Summary of the Invention

[0005] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0006] A desktop floating AI digital human terminal includes: a terminal body, a display screen, an optical prism, and a circuit module;

[0007] The display screen is mounted on the main body of the terminal and outputs images via a circuit module; while the optical prism is attached to and placed on the display screen.

[0008] The built-in display screen outputs digital human images. By utilizing the light refraction and reflection principle of optical prisms, the screen image is projected onto the imaging visible area, forming a floating pseudo-holographic visual effect. This allows the human eye to observe a three-dimensional floating digital human image, enhancing the three-dimensionality and technological feel of the digital human display.

[0009] The circuit module includes:

[0010] Main control module: Utilizing an ESP32-S3-WROOM-1-N16R8 microcontroller module, serving as the core brain of the AI ​​digital human, responsible for voice wake-up, Wi-Fi / Bluetooth network communication, audio data stream encoding / decoding, and screen UI rendering; the module outputs multiple core signals, including I2S interfaces for audio interaction (I2S_MCLK, I2S_LRCK, etc.), RGB data pins driving the display (such as S3_IO10), and I2C and serial port (UART / USB) pins for peripheral communication.

[0011] Power module: Includes a Type-C power socket, CP2102 chip, and 1.8V LDO regulator, providing power supply for multiple voltage domains such as VCC_3V3, 5V, and VCC_1V8 to meet the power requirements of the main controller and various peripherals.

[0012] Screen display module: It adopts a 40-pin FPC connector for connecting to the display screen; the interface is connected to 3.3V and 5V power supply, and is densely connected to multiple GPIO pins of ESP32-S3 (such as S3_IO11 to S3_IO21, etc.) for parallel RGB data transmission. It also connects to I2C signals (I2C_SCL / SDA) and extended IO signals (EX_IO series) for screen configuration and touch screen control.

[0013] Bus communication and level conversion module: The PCA9306DCUR bidirectional voltage level converter is used to bidirectionally convert the main controller's 3.3V I2C signal to the 1.8V voltage domain, solving the voltage mismatch problem between different sensors and screen peripherals, and ensuring stable communication of high-frequency or low-power I2C peripherals (such as specific audio codecs or sensors).

[0014] IO expansion module: It adopts the TCA9554I2C to parallel port expansion chip, which expands 8 independent IO ports (EX_IO0-EX_IO7) through the main I2C bus control. These ports are allocated to screen backlight, touch reset or other external state control, solving the problem of RGB screen and audio interaction occupying a large number of main control pins.

[0015] Interaction and Status Feedback Module: Includes a physical reset button (SW1) and a boot button (SW2), and reserves an external "wake-up button" interface (CN6, connected to S3_BOOT_IO0), allowing users to directly wake up the sleeping AI digital human by physical pressing; reserves a WS2812 programmable RGB light strip interface (CN1), controlled by a single data line (S3_IO4), used to provide ambient lighting feedback for emotions or status.

[0016] Debugging and external expansion interfaces: Includes a serial / USB interface (CN5), with VCC_3V3, USB+, USB- and GND brought out, utilizing the ESP32-S3's native USB interface for firmware burning, AI model updates and low-level data debugging; reserves a standard I2C interface (CN4) and an expansion IO interface (CN8) to provide space for future upgrades, facilitating the connection of external temperature and humidity sensors, infrared human body sensors or additional microphone array boards.

[0017] A desktop-mounted floating AI digital human terminal interaction method, running on the aforementioned desktop-mounted floating AI digital human terminal, is based on the FreeRTOS code framework. It retains the original basic operating architecture and completes independent development and optimization, realizing anthropomorphic digital human real-time interaction, intelligent dialogue response, real-time screen rendering, and visual human-computer operation functions. Core functions include:

[0018] Customizable agent roles: Supports customizable agent persona parameters, allowing configuration of agent personality, response scripts, interaction logic, and memory rules. The agent's decision-making logic can be modified to adapt to different usage scenarios.

[0019] Voice interaction: It integrates voice acquisition and semantic analysis functions, supports real-time human voice pickup, and completes speech-to-text recognition; it generates response content based on semantic analysis logic, and realizes two-way voice dialogue with the audio playback module.

[0020] Low-level driver: Based on the FreeRTOS framework, the low-level hardware driver is optimized, including screen rendering driver, audio driver, and peripheral interaction driver. It is responsible for hardware initialization, data transmission and instruction scheduling, and ensures stable and coordinated operation of peripherals such as screen, Wi-Fi, Bluetooth, microphone, and speaker.

[0021] Voice Cloning: The built-in voice cloning algorithm module extracts timbre features by sampling a small number of human voice samples and generates simulated timbre adapted to the digital human, thereby achieving customized voice effects for the digital human.

[0022] Voiceprint recognition: Equipped with voiceprint recognition function, it collects and stores the user's voiceprint features to identify the speaker's identity, distinguish interactive users, and realize personalized interaction logic such as permission recognition and exclusive response.

[0023] Button wake-up and interruption: Configure physical button interaction logic to support one-click wake-up of the device to quickly enter interactive standby mode; it also has a voice interruption function, allowing users to interrupt the current response and switch to a new conversation at any time during the interaction.

[0024] Battery detection and low battery protection: Real-time collection of battery voltage data to dynamically monitor the remaining battery power of the device; setting a low battery threshold to trigger low battery reminders, background power saving and frequency reduction, and power failure protection mechanisms.

[0025] PWM backlight adjustment: The screen backlight is controlled by PWM dimming logic. The software outputs control signals with different duty cycles to achieve multi-level brightness adjustment, supporting manual level adjustment and adaptive brightness control.

[0026] MCP Service: Equipped with the MCP communication service module, it has the capabilities of network request encapsulation, data parsing, protocol adaptation and message forwarding. It is used to connect to third-party service interfaces and external APIs to achieve interconnection and interoperability between external cloud data and third-party AI service interfaces.

[0027] The beneficial effects of this invention are:

[0028] 1. Compared to traditional flat display modes, this product utilizes the refraction and reflection optical principles of optical prisms to project the digital human image output from the screen into a floating pseudo-holographic image, effectively enhancing the three-dimensionality and visual depth of the digital human display. At the same time, the overall design adopts a compact desktop shell structure, which is small in size, easy to place, and suitable for normal desktop display, greatly improving the product display effect and user experience.

[0029] 2. This invention uses an ESP32-S3 core main controller paired with dedicated function chips to build a circuit module. By expanding multiple IO resources through the TCA9554 chip, it effectively solves the problem of RGB screens and audio interaction modules occupying a large number of main controller pins and limiting peripheral expansion. At the same time, relying on the PCA9306DCUR chip, it realizes bidirectional conversion of 3.3V and 1.8V dual voltage domain I2C signals, realizes adaptation to peripherals with different voltage specifications, completely solves the defects of incompatibility and unstable signal of cross-voltage domain device communication, and greatly improves the reliability of the overall hardware operation.

[0030] 3. The control system is built based on the FreeRTOS framework, integrating diverse intelligent functions such as customizable intelligent agent persona, voice cloning, voiceprint recognition, two-way voice interaction, and voice interruption wake-up. It can realize personalized human-computer interaction and exclusive permission recognition, breaking the limitations of the traditional digital human interaction mode of being single and lacking customization capabilities, making human-computer interaction more natural and intelligent.

[0031] 4. Equipped with an MCP service module, it can flexibly connect to various third-party cloud services and external API interfaces, and its function expansion is unrestricted; at the same time, the hardware side reserves multiple sets of standard I2C, expansion IO and USB debugging interfaces, which can connect to various sensors and peripheral modules, facilitating subsequent function iteration upgrades and personalized customization, and significantly improving the device's versatility and future development potential. Attached Figure Description

[0032] Figure 1 This is a schematic diagram of the module connection structure of the present invention.

[0033] Figure 2 This is a schematic diagram of the structure of the present invention.

[0034] Figure 3 This is a schematic diagram of the main control module in this invention.

[0035] Figure 4 This is a schematic diagram of the power module circuit in this invention.

[0036] Figure 5 This is another structural schematic diagram of the power module circuit in this invention.

[0037] Figure 6 This is a schematic diagram of the screen display module circuit in this invention.

[0038] Figure 7 This is a schematic diagram of the bus communication and level conversion module circuit in this invention.

[0039] Figure 8 This is a schematic diagram of the IO expansion module circuit in this invention.

[0040] Figure 9 This is a schematic diagram of the interactive and status feedback module circuit in this invention.

[0041] Figure 10 This is a schematic diagram of the debugging and external expansion interface circuit in this invention. Detailed Implementation

[0042] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. It should be understood that this application is not limited to the exemplary embodiments disclosed herein. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0043] In the description of this invention, it should be understood that the terms "length", "width", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0044] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0045] In the embodiments of this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0046] Please refer to Figure 1-10 Example 1: Hardware Terminal Example

[0047] This embodiment includes: a hardware terminal and a circuit module system;

[0048] The hardware terminal includes: the terminal body, the display screen, the optical prism, and the circuit module;

[0049] The terminal body serves as the outer shell structure, with the display screen mounted on it. A circuit module system is housed within the terminal body, enabling image output from the display screen. An optical prism is attached to and placed on the display screen.

[0050] The digital human image is output through the built-in display screen. By using the light refraction and reflection principle of the optical prism, the screen image is projected onto the imaging visible area to form a floating pseudo-holographic visual effect, so that the human eye can observe a three-dimensional floating digital human image, which enhances the three-dimensionality and technological feel of the digital human display.

[0051] The circuit module system includes:

[0052] Main control module: The VDD pin of the ESP32-S3-WROOM-1-N16R8 module is uniformly connected to the VCC_3V3 power supply, all GND pins are connected to the system ground plane, and the EN pin is connected to VCC_3V3 through a 10kΩ pull-up resistor and connected in parallel to GND with a 100nF decoupling capacitor.

[0053] The module brings out the I2S_MCLK, I2S_LRCK, I2S_BCK, I2S_DATA_IN, and I2S_DATA_OUT pins to form an I2S audio interface, with a reserved connection to an external audio codec chip.

[0054] Twelve GPIO pins, from S3_IO10 to S3_IO21, are brought out as an RGB parallel data bus for transmitting screen display signals;

[0055] The S3_IO1 and S3_IO2 pins are brought out as the main I2C bus (I2C_SCL, I2C_SDA), the S3_IO3 and S3_IO4 pins are brought out as general-purpose GPIO, the S3_BOOT_IO0 pin is brought out as the wake-up signal input, and the native USB_D+ and USB_D- pins are brought out for debugging and programming.

[0056] Power module circuit: The VBUS pin of the Type-C power socket outputs 5V voltage, which is connected to the 5V power supply pin of the 40-Pin RGB screen FPC connector; another input is to the VIN pin of the CP2102 chip, which outputs VCC_3V3 voltage to power the ESP32-S3 module, TCA9554 chip, PCA9306DCUR chip, and most peripherals; the VCC_3V3 voltage is also input to the input of the 1.8V LDO regulator, which outputs VCC_1V8 voltage to power I2C peripherals in the 1.8V voltage range; each power output is filtered by a 10µF electrolytic capacitor and a 0.1µF ceramic capacitor connected in parallel.

[0057] Screen display module circuit: The 3.3V power supply pin of the 40-Pin FPC connector is connected to VCC_3V3, and the 5V power supply pin is connected to the 5V voltage output by Type-C; the RGB data pins of the connector are connected to the S3_IO10 to S3_IO21 pins of the ESP32-S3 module; the I2C_SCL and I2C_SDA pins of the connector are connected to the 1.8V output of the PCA9306DCUR chip; the touch control pins and backlight control pins of the connector are connected to the EX_IO0 to EX_IO7 pins extended from the TCA9554 chip, respectively.

[0058] Bus communication and level conversion module circuit: The VREF1 pin of the PCA9306DCUR chip is connected to VCC_3V3, the VREF2 pin is connected to VCC_1V8, and the EN pin is connected to VCC_3V3 to enable the chip; the A1 and A2 pins of the enable chip are connected to the main I2C bus (I2C_SCL, I2C_SDA) of the ESP32-S3 module, respectively, and the B1 and B2 pins are connected to the I2C pin of the 40-Pin FPC connector and the I2C interface of the 1.8V voltage domain peripheral, respectively, to realize bidirectional conversion between 3.3V and 1.8V I2C signals.

[0059] IO expansion module circuit: The VCC pin of the TCA9554 chip is connected to VCC_3V3, and the GND pin is connected to the system ground; the SCL and SDA pins of the chip are connected to the main I2C bus of the ESP32-S3 module, and the A0, A1, and A2 address pins are all grounded, setting the chip's I2C address to 0x20; the P0 to P7 pins of the chip output a total of 8 expansion IO ports, EX_IO0 to EX_IO7, which are respectively allocated to functions such as screen backlight control, touch reset, and status indication.

[0060] Interaction and Status Feedback Module Circuit: One end of the physical reset button (SW1) is connected to the EN pin of the ESP32-S3 module, and the other end is connected to GND; one end of the boot button (SW2) is connected to the S3_BOOT_IO0 pin, and the other end is connected to GND; the two pins of the external wake-up button interface (CN6) are connected to S3_BOOT_IO0 and GND respectively; the VCC pin of the WS2812 LED strip interface (CN1) is connected to VCC_3V3, the GND pin is connected to system ground, and the DATA pin is connected to the S3_IO4 pin of the ESP32-S3 module.

[0061] Debugging and external expansion interface circuit: The VCC pin of the serial port / USB interface (CN5) is connected to VCC_3V3, the USB+ and USB- pins are connected to the native USB_D+ and USB_D- pins of the ESP32-S3 module respectively, and the GND pin is connected to the system ground; the SCL and SDA pins of the standard I2C interface (CN4) are connected to the main I2C bus of the ESP32-S3 module, the VCC pin is connected to VCC_3V3, and the GND pin is connected to the system ground; the expansion IO interface (CN8) brings out 8 general-purpose GPIO pins and power and ground pins for connecting various sensors and peripherals.

[0062] Example 2: Software Control System Example

[0063] This embodiment provides an AI digital human interaction control system running on the aforementioned hardware terminal. The specific operation process is as follows:

[0064] System initialization: After the device is powered on, the FreeRTOS operating system starts and initializes the underlying hardware drivers (screen, audio, Wi-Fi, Bluetooth, buttons, power management, etc.) in sequence, and loads the pre-stored AI digital human model, intelligent agent human parameters and voiceprint database.

[0065] Standby and wake-up: The system enters a low-power standby state and waits for a wake-up signal; when the Boot button is pressed or an external wake-up button signal is detected, the system is immediately woken up and enters interactive mode; at the same time, the WS2812 LED strip displays the corresponding status light (such as a solid green light when woken up).

[0066] Voice interaction process: The microphone collects the user's voice signal and transmits it to the main control module via the I2S interface for speech-to-text processing; the semantic analysis module parses the text content and generates response text by combining the agent's persona and memory rules; if the voice cloning function is enabled, the voice cloning module is called to convert the response text into audio data with a user-customized timbre, which is then output to the speaker via the audio driver; during the interaction, if a new voice input is detected, the voice interruption function is immediately triggered to terminate the current response and process the new dialogue request.

[0067] Personalized interaction: When a user's voice is detected, the voiceprint recognition module extracts voiceprint features and compares them with the database. After identifying the user's identity, it calls the corresponding exclusive response logic and memory data; the WS2812 light strip changes dynamically according to the interaction state (such as flashing blue when listening, breathing rhythm when thinking, and jumping light when speaking).

[0068] Power and display management: The power detection module collects the battery voltage in real time. When the remaining power is lower than the set threshold, it triggers a low power reminder and automatically reduces the CPU frequency and screen brightness. The screen backlight adjusts the brightness level through PWM signal according to the ambient light intensity (reserved sensor interface) or the user's manual setting.

[0069] Third-party service integration: When a user requests external data or services, the MCP service module encapsulates the network request, connects to the cloud server or third-party API interface via Wi-Fi, obtains and parses the data, and generates the corresponding response content to be fed back to the user.

[0070] System upgrades and debugging: The system can be upgraded by connecting to a computer via the native USB interface, which allows for firmware flashing, AI model updates, and underlying data debugging. It can also be upgraded by connecting external sensors or peripherals via the reserved I2C and expansion I / O interfaces to expand system functionality.

[0071] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0072] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A desktop floating AI digital human terminal, characterized in that, include: Terminal body, display screen, optical prism and circuit module; The display screen is mounted on the main body of the terminal and the image output is controlled by the circuit module. The optical prism is attached to the display screen; The circuit module includes: a main control module, which uses a microcontroller module of model ESP32-S3-WROOM-1-N16R8; the circuit module also includes: an integrated power supply module, a screen display module, a bus communication and level conversion module, an IO expansion module, an interaction and status feedback module, and a debugging and external expansion interface connected to the main control module.

2. The desktop floating AI digital human terminal according to claim 1, characterized in that, The power module includes a Type-C power socket, a CP2102 chip, and a 1.8V LDO regulator, providing power supply across multiple voltage domains: 3.3V, 5V, and 1.8V.

3. A desktop floating AI digital human terminal according to claim 1, characterized in that, The screen display module uses a 40-Pin FPC connector, which is connected to 3.3V and 5V power supplies, and connects to the RGB data pins, I2C signals and extended IO signals of the main control module.

4. A desktop floating AI digital human terminal according to claim 1, characterized in that, The bus communication and level conversion module uses the PCA9306DCUR bidirectional voltage level converter to realize bidirectional conversion between the main control 3.3V12C signal and the 1.8V voltage domain signal.

5. A desktop floating AI digital human terminal according to claim 1, characterized in that, The IO expansion module uses the TCA9554I2C to parallel port expansion chip.

6. A desktop floating AI digital human terminal according to claim 1, characterized in that, The interaction and status feedback module includes: a physical reset button, a boot button, a reserved external wake-up button interface, and a WS2812 programmable RGB light strip interface, with the light strip interface controlled by the S3_IO4 pin.

7. A desktop floating AI digital human terminal according to claim 1, characterized in that, The debugging and external expansion interfaces include a serial / USB interface, a standard I2C interface, and an extended IO interface. The native USB interface is used for firmware burning, AI model updates, and low-level data debugging.

8. A desktop floating AI digital human terminal interaction method, operating on the desktop floating AI digital human terminal described in any one of claims 1-7, characterized in that, Built on the FreeRTOS code framework, its core functions include: a custom intelligent agent role module, a voice interaction module, a low-level driver module, a voice cloning module, a voiceprint recognition module, a button wake-up and interruption module, a power detection and low power protection module, a PWM backlight adjustment module, and an MCP service module.

9. The desktop floating AI digital human terminal interaction method according to claim 8, characterized in that, The custom intelligent agent role module supports configuring role personality, response scripts, interaction logic, and memory rules; the voice cloning module extracts timbre features by sampling a small number of human voice samples to generate customized simulated timbres; the voiceprint recognition module is used to identify the speaker's identity and realize permission recognition and exclusive responses.

10. The desktop floating AI digital human terminal interaction method according to claim 8, characterized in that, The MCP service module has the capabilities of network request encapsulation, data parsing, protocol adaptation, and message forwarding, and is used to connect to third-party service interfaces and external APIs.

Images