A headless remote streaming interaction system and method for cross-platform native conversation isolation

Abstract

The application discloses a cross-platform native session isolation headless remote streaming interaction system and method, and belongs to the technical field of computer remote desktop streaming. The application relies on the operating system native remote session service, creates independent interactive sessions isolated from each other through silent authentication, automatically allocates system native virtual display buffer and virtual HID input channel, and does not need third-party virtual graphics card and mouse driver. The session can be separated from the front-end connection background and run headlessly. The application supports H.264 / H.265 / AV1 adaptive coding and AI picture quality optimization, can capture pictures according to unique session identification and complete point-to-point streaming interaction, and realizes multi-user session level isolation. The application can be adapted to Windows, Linux and domestic Kirin systems, solves the problems of traditional scheme driving conflict, picture crosstalk, weak compatibility and inability to run headlessly, and is suitable for high-performance remote interaction scenes such as industrial simulation, three-dimensional design and computing power sharing.

Description

Technical Field

[0001] This invention belongs to the field of computer remote desktop streaming technology, specifically relating to a headless remote streaming interaction system and method with cross-platform native session isolation, which can be applied to high-performance remote interaction scenarios such as industrial 3D design simulation, computing power time-sharing, and secure remote office for government and enterprises. Background Technology

[0002] Remote desktop streaming technology is widely used in current industrial 3D design, simulation rendering, time-sharing computing power, and remote office scenarios in government and enterprises. Existing mainstream technical solutions mainly fall into two categories: The first category is the traditional VDI virtual desktop solution, which relies on physical main screen mapping, third-party virtual graphics cards, and external virtual keyboard and mouse drivers to achieve multi-user distribution. It has drawbacks such as complex driver adaptation, poor version compatibility, easy crosstalk between multi-user screens, and low system stability. At the same time, it generally relies on manual device pairing, fixed authentication methods, and fixed encoding formats, resulting in high maintenance costs and poor scalability.

[0003] The second category consists of remote streaming solutions modified from open-source frameworks. These solutions generally suffer from issues such as single platform, incomplete session isolation, reliance on the main control desktop, session destruction or screen freezing upon disconnection, and inability to truly run headless. They are also unable to adapt to multi-system, high-concurrency, and high-stability computing power sharing and industrial simulation scenarios.

[0004] Existing technologies generally suffer from the following technical defects: 1. Additional third-party virtual display and virtual keyboard and mouse drivers are required, which has a high probability of driver conflicts and low system fault tolerance; 2. Multi-user sessions rely on global main screen rendering, and screen capture is not isolated, which can easily lead to screen flickering, black screens, and keyboard and mouse malfunctions. 3. The certification method is singular and cannot meet the multi-level security certification needs of government, enterprise, and military industries; 4. The encoding format is fixed and cannot be adaptively adjusted according to the terminal and network, resulting in poor compatibility with high-end and low-end devices; 5. The platform has weak compatibility and cannot be simultaneously compatible with Windows, Linux, and the domestic Kylin multi-system architecture; 6. It cannot achieve true headless persistent operation; disconnection will freeze or destroy the session, and it cannot support 24 / 7 computing power time-sharing service.

[0005] Therefore, the industry urgently needs a remote streaming interaction architecture that is independent of third-party virtual hardware, universal across all platforms, completely isolated from sessions, adaptable to multiple authentication methods, adaptive to AI encoding, and truly headless and persistent. Summary of the Invention

[0006] Purpose of the invention: To address the shortcomings of existing technologies, this invention aims to provide a headless remote streaming interaction system and method with cross-platform native session isolation, completely solving the technical pain points of traditional solutions such as complex drivers, screen crosstalk, poor stability, single platform, weak adaptability, and inability to remain headless indefinitely.

[0007] Technical solution To achieve the above objectives, the present invention adopts the following core technical architecture: This invention completely abandons the traditional architecture of external virtual graphics cards, external virtual keyboards and mice, main screen binding, and manual pairing. It relies on the native remote session mechanism of each operating system as the isolation basis, creates independent interactive sessions through silent authentication, reuses the system's native virtual display and virtual input resources, and, together with adaptive AI encoding stream and session lifecycle management, achieves standardized, highly stable, and low-latency headless remote multi-user interaction across the entire platform.

[0008] The core innovative architecture of this invention includes: a silent pluggable authentication layer, a system native session isolation layer, a native virtual device adaptation layer, an AI adaptive encoding streaming layer, a precise session interaction layer, and a lifecycle intelligent management layer.

[0009] Beneficial effects Compared with the prior art, the present invention has the following significant technical advantages: 1. Zero third-party driver dependency: It relies entirely on the system's native virtual display and virtual HID mechanism, removing all external virtual hardware drivers, greatly reducing failure points and compatibility issues, and significantly improving stability; 2. Truly headless background persistence: The session, rendering environment, and virtual resources are continuously retained after the login connection is disconnected, without relying on the physical screen, to achieve industrial-grade headless computing power services; 3. Complete session isolation: Screen capture and input are targeted by session ID, and multiple user screens, keyboards and mice, and resources are completely isolated, eliminating the risk of crosstalk; 4. Universally compatible across platforms: Unified adaptation to Windows, Linux, and the domestic Kylin system, without being limited by system version iterations; 5. Multi-security authentication compatibility: Pluggable multi-authentication system, adaptable to multiple security scenarios including civilian, enterprise, and military applications; 6. Intelligent adaptive streaming: H.264 / H.265 / AV1 dynamic switching + AI image quality optimization, adapting to all levels of terminal devices and network environments; 7. Extremely simple and efficient architecture: The login link and the streaming link are completely decoupled. The streaming does not depend on the remote login protocol, resulting in lower latency and higher throughput; 8. Intelligent resource management: Automatic idle resource recycling, disconnection reconnection, and session management, adaptable to high-concurrency long-term operation scenarios. Detailed Implementation

[0010] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and preferred embodiments, but this does not limit the scope of protection of the present invention.

[0011] Example 1: Implementation on the Windows Platform Combination Figure 1 and Figure 2 As shown, this embodiment implements headless streaming interaction based on the native remote session service of the Windows server series system. The silent authentication login module 1 runs in a background interfaceless mode, initiating authentication and session creation requests to the system's native remote session service.

[0012] Combination Figure 3 As shown, the silent authentication login module 1 has a built-in pluggable authentication unit, including an account password authentication unit 7, a digital certificate authentication unit 8, a biometric authentication unit 9, and a hardware key authentication unit 10. The authentication method can be flexibly switched according to the security level of the scenario, and no manual interaction interface is required throughout the process.

[0013] After identity verification is successful, the system's native session management module 2 creates an independent interactive session for the user at the kernel level. The native virtual hardware and software adaptation module 3 automatically allocates a dedicated virtual display buffer 18 and virtual HID channel 19 to the session. The entire process does not rely on physical display output devices and does not load any third-party virtual graphics cards or virtual keyboard and mouse drivers.

[0014] After session initialization, the front-end authentication and login link can be disconnected, while the session and desktop environment continue to run in the background, achieving true headless operation. Combined with... Figure 5 As shown, the adaptive AI streaming encoding module 4 schedules the H.264 encoding unit 13, H.265 encoding unit 14, and AV1 encoding unit 15 to adaptively select the optimal encoding format based on the server hardware capabilities and client decoding specifications, and completes image noise reduction, edge enhancement, and dynamic bitrate adjustment through the AI ​​image quality optimization unit 16.

[0015] The session-oriented transmission interaction module 5 captures the virtual display buffer screen of the corresponding session based on the unique session identifier, and transmits it to the remote client point-to-point after encoding; at the same time, it accurately delivers the keyboard, mouse and touch commands returned by the client to the virtual HID channel 19 dedicated to the session. The input commands and screen are strictly bound according to the session ID, and there is no cross-session crosstalk.

[0016] Combination Figure 6 As shown, the session lifecycle management module 6 monitors the session's running status throughout the entire process: when the network is disconnected, the session remains active in the background and resumes interaction by directly binding to the original session after reconnection; when the session is idle for a timeout, it automatically reclaims video memory, RAM, and process resources; the administrator can initiate a forced command to cancel a specified session, achieving unified management and control throughout the entire lifecycle.

[0017] Example 2: Implementation on the Linux Platform Combination Figure 4 As shown, this embodiment implements headless streaming interaction with the same architecture based on the native remote session service of the Linux distribution operating system. The system's underlying layer relies on the native remote session service to create independent Xorg interactive sessions, automatically generating dedicated virtual display devices and input channels for each session.

[0018] Silent authentication login module 1 is adapted to the Linux system authentication system, supporting authentication methods such as digital certificates and keys; the upper-layer streaming encoding, AI image quality optimization, session-oriented interaction, and lifecycle management logic are completely reused from Example 1. Combined with... Figure 5 As shown, the encoding and optimization units of the adaptive AI streaming encoding module 4 are uniformly scheduled under the Linux platform, requiring no additional adaptation development.

[0019] This embodiment verifies the cross-platform versatility of the architecture of the present invention: only the adaptation of the underlying system session interface needs to be completed, and the upper-layer core business logic is completely universal and can be quickly migrated to different Linux distributions.

[0020] Example 3: Implementation of the Domestic Kylin System Combination Figure 4 As shown, the domestically developed Kylin system is based on the Linux kernel architecture and is fully compatible with the full-stack technology architecture of this invention. It creates isolated sessions through the system's native remote session service, reusing native virtual display and virtual input resources, while also adapting to domestic security authentication standards and being able to access domestic biometric and digital certificate systems.

[0021] The streaming encoding, AI optimization, and session lifecycle management logic are completely universal with the aforementioned embodiments, enabling high-performance headless remote interaction in a domestically produced hardware and operating system environment, meeting the needs of domestic substitution scenarios such as government and enterprise, and military industries.

[0022] Cross-platform universal operating logic Combination Figure 4 As shown, the underlying system session service interfaces of the three embodiments differ in platform adaptation, but the overall architecture, authentication logic, virtual resource reuse method, stream isolation mechanism, adaptive encoding, and lifecycle management are completely unified. The unified upper-layer architecture covers the entire process of authentication, streaming, session scheduling, and lifecycle management, while the underlying layers respectively connect to the Windows native remote session service and the Linux / Kylin native remote session service, achieving "one technical architecture, coverage of all mainstream systems." It is unaffected by system version iterations and open-source framework updates, possessing strong versatility and foresight. Attached Figure Description

[0023] Figure 1This is a block diagram of the overall architecture of the cross-platform headless session isolation streaming system of the present invention; Figure 2 This is a schematic diagram illustrating the complete process of session creation, headless persistent operation, and directional streaming interaction in this invention. Figure 3 This is a block diagram showing the internal structure of the pluggable multi-authentication module of the present invention. Figure 4 This is a schematic diagram of the cross-platform layered adaptation architecture of the present invention; Figure 5 This is a block diagram showing the internal components of the adaptive AI streaming encoding module of the present invention; Figure 6 This is a schematic diagram of the state transition for session lifecycle management in this invention.

[0024] Explanation of reference numerals in the attached figures 1-Silent Authentication Login Module 2-System Native Session Management Module 3-Native Virtual Hardware and Software Adaptation Module 4-Adaptive AI Stream Encoding Module 5-Session-oriented transmission and interaction module 6-Session Lifecycle Management Module 7-Account Password Authentication Unit 8-Digital Certificate Authentication Unit 9-Biometric Authentication Unit 10-Hardware Key Authentication Unit 13-H.264 coding unit 14-H.265 encoding unit 15-AV1 encoding unit 16-AI Image Quality Optimization Unit 18-Virtual Display Buffer 19-Virtual HID Channel

Claims

1. A headless remote streaming interaction system with native session isolation across platforms, characterized in that, include: Silent authentication login module, system native session management module, native virtual hardware and software adaptation module, adaptive AI streaming encoding module, session-oriented transmission and interaction module, session lifecycle management module; The silent authentication and login module is configured to run in the background without a graphical interface. It supports multiple pluggable authentication methods and silently initiates authentication and connection requests to the native remote session service of the server operating system without any manual intervention. The system's native session management module is configured to receive authentication requests and complete legality verification. For each legitimate user identity, it creates physically isolated independent interactive sessions at the kernel layer, with desktop environments, rendering resources, and input resources being completely independent between different user sessions. The native virtual hardware and software adaptation module is configured to rely on the native mechanism of the operating system to automatically allocate a dedicated virtual display buffer and a dedicated virtual HID input channel for each independent interactive session, without relying on physical display devices and without mounting third-party virtual graphics card drivers and third-party virtual keyboard and mouse drivers. The adaptive AI streaming encoding module is configured to automatically detect the server-side hardware encoding capabilities, client-side hardware decoding specifications, and real-time network status, adaptively switch between H.264, H.265, and AV1 video encoding formats, and incorporate artificial intelligence optimization algorithms to perform noise reduction, edge enhancement, and dynamic bitrate intelligent adjustment processing on the image. The session-oriented transmission interaction module is configured to capture the native virtual display screen of the corresponding session based on the unique session identifier, complete the encoded stream transmission through the point-to-point transmission protocol, and accurately deliver the keyboard, mouse and touch input commands of the remote client to the dedicated virtual HID input channel of the corresponding session to achieve session-level input isolation. The session lifecycle management module is configured to enable persistent disconnected sessions, automatic reconnection after disconnection, recycling of idle resources, and mandatory session management by the administrator. After the front-end login network connection is disconnected, the interactive session, virtual display buffer, and running program continue to run in the background.

2. The system according to claim 1, characterized in that, The silent authentication login module supports the following authentication methods: account password authentication, domain account authentication, digital certificate authentication, biometric authentication, hardware key (UKey) authentication, and dynamic token authentication. These authentication methods can be switched as needed.

3. The system according to claim 1, characterized in that, The system's overall architecture is cross-platform compatible and can be adapted to all Windows server systems, Linux distributions, and the domestic Kylin operating system. All platforms uniformly use the native remote session service to create isolated sessions, and the upper-layer streaming interaction and lifecycle scheduling logic are completely universal.

4. The system according to claim 1, characterized in that, The session-oriented transmission and interaction module is completely independent of the remote login protocol link. After the session is created, the image capture, encoding transmission, and input interaction all run independently without occupying or relying on the remote login channel resources.

5. The system according to claim 1, characterized in that, The adaptive AI streaming encoding module is specifically optimized for 3D design and industrial simulation scenarios. It intelligently balances compression rate and image fidelity in high-precision model rendering, reducing remote interaction latency.

6. A headless remote streaming interaction method with native session isolation across platforms, characterized in that, Includes the following steps: S1. Silent background authentication: The silent authentication login module runs in a background mode without a user interface. It calls the preset authentication credentials to complete the identity verification and initiates a session creation request to the operating system's native remote session service. S2. Create native isolated session: After the system verifies the identity, an independent interactive session is generated at the kernel level, and the system's native virtual display buffer and native virtual input channel are automatically allocated to the session; S3, Headless Persistent Session: After completing session initialization, disconnect the front-end authentication connection, and the interactive session, desktop rendering environment, and virtual display resources continue to persist in the background, achieving headless operation without a physical screen; S4. Session-level targeted screen capture: The self-developed streaming server accurately captures the native virtual display screen data of the corresponding session through a unique session identifier; S5, Adaptive AI Encoding Processing: Automatically matches the optimal encoding format based on the terminal's decoding capabilities and network status, and optimizes image quality and transmission bitrate through AI algorithms; S6, Point-to-point streaming interaction: Transmits the encoded video stream point-to-point to the remote client, which decodes, renders, and displays it; at the same time, it receives remote input commands and sends them back to the dedicated virtual HID channel of the corresponding session to complete remote interaction; S7. Full lifecycle management: The system monitors the online status of sessions in real time, enabling session preservation during disconnection, restoration of interaction upon reconnection, automatic reclamation of idle and timed-out resources, and unified management of administrator sessions.

7. The method according to claim 6, characterized in that, In step S2, each operating system reuses its own native remote session mechanism. The Windows platform relies on the system's remote desktop session service, while the Linux and Kylin platforms rely on the open-source remote session service to uniformly achieve session isolation. The underlying adaptation interfaces are differentiated, while the upper-layer scheduling logic is universalized.

8. The method according to claim 6, characterized in that, In step S4, the host's global main control screen is not captured at all; only the private virtual display buffer of the corresponding independent session is bound, thus completely eliminating the problem of crosstalk between multiple user screens.

9. The method according to claim 6, characterized in that, In step S6, the virtual HID input channels of each session are isolated from each other, and the input commands are strictly bound according to the session ID, so there is no risk of cross-session input cross-control.

10. The method according to claim 6, characterized in that, In step S7, idle resource reclamation includes automatically releasing video memory, RAM, virtual device handles, and process resources to prevent server resource leakage caused by long-term operation by multiple users.

Images