Screen display system

Abstract

The invention discloses a screen display system, which relates to the technical field of computer man-machine interaction, and comprises a display integration module used for acquiring display contents of a plurality of computers and integrating the display contents of the plurality of computers on a single display; the input distribution module is used for intelligently distributing keyboard and mouse input to a target computer; the boundary detection module is used for detecting a computer area corresponding to the mouse position in real time to obtain real-time double-click mouse coordinates and sending the real-time double-click mouse coordinates to the intention recognition module; the intention recognition module is used for acquiring object types and distinguishing operation intentions according to real-time double-click mouse coordinates and the object types; and the display control module is used for realizing switching between windowing and full-screen display.

Description

Technical Field

[0001] This invention relates to the field of computer human-computer interaction technology, specifically a screen display system. Background Technology

[0002] In existing technologies, KVM switches can enable multiple computers to share a single keyboard, monitor, and mouse. However, traditional KVM requires manual switching and cannot achieve seamless mouse movement. Existing solutions cannot intelligently recognize user intent. Summary of the Invention

[0003] To address the shortcomings mentioned in the background section, the present invention aims to provide a screen display system.

[0004] Firstly, the objective of this invention can be achieved through the following technical solution: a screen display system, comprising: The display integration module is used to acquire the display content of multiple computers and integrate the display content of multiple computers onto a single monitor. The input allocation module is used to intelligently allocate keyboard and mouse input to the target computer; The boundary detection module is used to detect the computer area corresponding to the mouse position in real time, obtain the real-time mouse coordinates of double-click, and send the real-time mouse coordinates of double-click to the intent recognition module. The intent recognition module is used to obtain the object type and distinguish the operation intent based on the real-time mouse coordinates of the double-click and the object type. The display control module is used to switch between windowed and full-screen display.

[0005] In conjunction with the first aspect, in some implementations of the first aspect, the system further includes: the process by which the display integration module integrates the display content of multiple computers onto a single display, including: Step 1: Host registration and frame buffer initialization; Step Two: Desktop Canvas Creation and Layout Management; Step 3: Display data capture and processing; Step 4: Image Compositing; Step 5: Output and Synchronization; Through the integration process, the visual output of multiple independent hosts is seamlessly integrated into a unified interactive interface.

[0006] In conjunction with the first aspect, in some implementations of the first aspect, the system further includes: the process by which the input allocation module intelligently allocates keyboard and mouse input to the target computer, including: Step 1: Input capture and normalization; Step Two: Mouse position monitoring and target computer identification; Step 3: Routing and assigning input events; Step 4: State synchronization during switchover; The input allocation module works closely with the intent recognition module: When the input assignment module detects a double-click event, it performs a regular click assignment. Forward the event and its precise local coordinates (X_local, Y_local) to the intent recognition module; Based on these coordinates, the intent recognition module queries the desktop information of the currently active host to determine whether the pixel belongs to a blank desktop or a shortcut icon; Based on the judgment result, the intent recognition module issues the following instruction: If the area is blank: the command display control module will enlarge the current host's display window to full screen; If it is a shortcut: the instruction input allocation module allows the double-click event and sends it to the target host as a regular double-click event, so that its operating system can execute the operation to open the program; The input allocation module executes corresponding actions based on the instructions.

[0007] In conjunction with the first aspect, in some implementations of the first aspect, the system further includes: the process by which the boundary detection module detects the computer area corresponding to the mouse position in real time, including: Mouse position determination and target computer mapping process Step 1: Define the coordinate system The origin (0, 0) is located at the top left corner of the physical display. The X-axis is positive to the right, and the Y-axis is positive downwards. The physical mouse's coordinates in this coordinate system are (X_g, Y_g); Host virtual local coordinate system: The origin (0, 0) is located at the top left corner of the host display window; The maximum coordinate value corresponds to the host's resolution setting, and its local coordinate system ranges from (0,0) to (1920, 1080). Step 2: Establishing and maintaining the layout mapping table; Step 3: Target computer determination algorithm; Step 4: Algorithm for converting global coordinates to local coordinates; Event capture: The system continuously monitors the movement events of the physical mouse; Coordinate acquisition: Get the current position (X_g, Y_g) of the mouse in the global physical coordinate system; Read the pre-established layout mapping table, which records the position and size information of the display area of ​​each host; Region traversal: By looping through all host regions, check whether the current mouse coordinates fall within a certain host display area; Hit detection: Use a boundary comparison algorithm to determine whether the coordinates are within the rectangular area; Coordinate transformation: If a certain area is hit, the global coordinates are converted to the local coordinates of the host using a proportional conversion formula; Event routing: Sends the converted mouse events to the corresponding target host; Error handling: If the mouse is not in any defined area, the system maintains its current state. This process ensures that the mouse can move seamlessly between multiple computer display areas and accurately locate the target computer for control.

[0008] In conjunction with the first aspect, in some implementations of the first aspect, the system further includes: the process by which the intent recognition module distinguishes the operation intent based on the real-time mouse coordinates of the double-click and the object type, including: The click location coordinates are obtained by double-clicking the event, and the click object type is identified based on the click location coordinates. The click object type includes blank areas, icons, and windows. In conjunction with the first aspect, in some implementations of the first aspect, the system further includes: triggering a full-screen switch when the type of the clicked object is a blank area, and executing program startup when the type of the clicked object is a shortcut.

[0009] In conjunction with the first aspect, in some implementations of the first aspect, the system further includes: the display control process of the display control module, comprising: The control process of the display control module. The core function of the display control module The display control module is responsible for receiving the composite image from the display integration module and executing specific display layout strategies, full-screen switching, and visual enhancement. Its control process is a closed-loop system that dynamically responds to external commands and internal state changes. The display control module management process is as follows: Step 1: Loading the layout strategy: The display control module has multiple built-in layout strategies. The system loads the corresponding layout configuration file according to the user's selection or adaptive strategy. Step 2: Display area calculation: Calculate the global coordinates and size of the display window for each computer based on the number of currently connected computers and the resolution of the physical monitors; Step 3: Command Issuance and Rendering The calculated layout information is sent to the compositing engine of the display integration module, which is then instructed to scale, position, and render each host screen according to the new layout strategy.

[0010] In addition to the full-screen switching control process and visual feedback and enhancement.

[0011] In another aspect of the present invention, in order to achieve the above-mentioned objective, a terminal device is disclosed, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor. The memory stores the computer program capable of running on the processor, and when the processor loads and executes the computer program, it employs a screen display system as described above.

[0012] In another aspect of the present invention, in order to achieve the above-mentioned objective, a computer-readable storage medium is disclosed, wherein a computer program is stored in the computer program, and when the computer program is loaded and executed by a processor, a screen display system as described above is employed.

[0013] The beneficial effects of this invention are: This invention enables a truly seamless multi-computer control experience by intelligently recognizing user operation intentions, saving hardware costs and office space, and ultimately improving the efficiency of multi-computer operation. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Figure 1 This is a schematic diagram of the system structure of the present invention; Figure 2 This is a schematic diagram illustrating the operation of the present invention. Detailed Implementation

[0015] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0016] Example 1: like Figure 1 As shown, a screen display system includes: The display integration module is used to acquire the display content of multiple computers and integrate the display content of multiple computers onto a single monitor. The process by which the display integration module integrates the display content of multiple computers onto a single monitor includes: Display the integration process of the integrated module. Display the workflow of the integrated module. The display integration module is one of the core components of the system of this invention. It is responsible for synthesizing display signals from multiple computers (hereinafter referred to as "hosts") into a unified desktop environment and outputting it to a single physical monitor. The integration process of this module mainly includes the following steps: Step 1: Host Registration and Frame Buffer Initialization First, the system establishes a connection with each host via wired (e.g., HDMI, DisplayPort, USB-C) or wireless (e.g., Miracast, WiDi). Each host is assigned a unique logical identifier (e.g., ID_01, ID_02). After the connection is established, the system creates and allocates an independent Virtual Frame Buffer (VFB) in memory for each host to continuously receive and store complete frame display data transmitted from each host.

[0017] Step Two: Desktop Canvas Creation and Layout Management The system creates a total composite desktop canvas on the display output end based on user-preset or automatically calculated layout strategies (such as even distribution, master-slave, picture-in-picture). The resolution of this canvas is equal to or higher than the native resolution of the physical monitor to ensure display clarity. At the same time, the system defines a display window for each host, which is the position and size of that host's screen on the total canvas (such as the coordinates and length and width of a rectangular area).

[0018] Step 3: Display Data Capture and Processing The graphics card output signals of each host are captured by the system in real time. For a software solution, this is achieved through a lightweight agent running on each host, which captures frame changes and transmits them to the integrated module via network sockets. For a hardware solution, a dedicated video capture chip (such as an FPGA) decodes the input video stream and writes the frame data to the corresponding VFB.

[0019] Step 4: Image Compositing Step 5: Output and Synchronization The final composited desktop canvas is sent to the graphics output interface to drive the physical monitor for display. To ensure smooth operation, the system uses Vertical Sync (VSync) technology to synchronize the composite frame rate with the physical monitor's refresh rate, avoiding screen tearing.

[0020] Key technological innovations Software-based graphics card emulation: In software solutions, a virtual graphics card is simulated by an agent program installed on the host machine. The host operating system recognizes this virtual graphics card as an additional physical monitor and outputs extended desktop views to it, thus achieving plug-and-play compatibility without requiring any modifications to the host operating system.

[0021] Intelligent resolution management: The integrated module can automatically adapt to the optimal resolution of different source hosts. For example, when a user turns a display window into full-screen mode, the module will dynamically instruct the host to switch its output resolution to match the native resolution of the physical monitor, thereby obtaining the best full-screen display effect; when exiting full-screen mode, it will automatically restore the original window size and resolution.

[0022] Low-latency transmission optimization: To improve real-time performance, the agent does not transmit the entire frame buffer. Instead, it uses differential frame capture technology to capture and transmit only the screen areas that have changed since the previous frame (dirty rectangles), which greatly reduces the amount of data transmitted and lowers network bandwidth usage and system latency.

[0023] Through the above integration process, the present invention successfully integrates the visual output of multiple independent hosts into a unified interactive interface, laying a solid foundation for subsequent input control and intent recognition.

[0024] The input allocation module is used to intelligently allocate keyboard and mouse input to the target computer; The input allocation module intelligently allocates keyboard and mouse input to the target computer, including: Input allocation module workflow The input allocation module is key to achieving "one keyboard and mouse controlling multiple machines" in this invention. Its core task is to seamlessly route the user's physical input actions to the correct target computer in real time. This process is a dynamic and continuous intelligent decision-making loop, with the following specific steps: Step 1: Input Capture and Normalization The system captures all raw input events (KeyDown / KeyUp, MouseMove, MouseClick, MouseWheel) from the single physical keyboard and mouse at the driver layer. These event data are then packaged and standardized into a unified internal message format, with a high-precision timestamp appended for processing by subsequent modules.

[0025] Step Two: Mouse Position Monitoring and Target Computer Determination This is the core of achieving seamless mouse movement. The system continuously monitors the absolute coordinates (X_global, Y_global) of the physical mouse on the composite desktop canvas.

[0026] The module maintains a layout mapping table that records the precise boundary coordinates of the display window corresponding to each host on the composite canvas (e.g., PC1: (x1, y1) to (x2, y2)).

[0027] By comparing the global mouse coordinates with the layout mapping table in real time, the module can immediately determine which host's display area the mouse is currently in, and thus mark that host as the current active host.

[0028] Step 3: Routing and Distributing Input Events Once the current active host is determined, the module begins input allocation: Mouse events: Convert global coordinates (X_global, Y_global) to local coordinates (X_local, Y_local) relative to the currently active host's display window. Then, send the mouse move, click, or scroll wheel event only to the input stack of the currently active host, simulating input from its local mouse.

[0029] Keyboard events: Send all captured keyboard events only to the currently active host, simulating input from its local keyboard.

[0030] Step 4: State synchronization during handover (to prevent state desynchronization) To prevent keyboard and mouse status confusion during host switching (e.g., one computer thinks the "Shift key" is still pressed, while another does not), the module implements a state synchronization mechanism. When the mouse cursor moves from one host area to another, triggering an active host switch: The module sends a command to the previous active host to "force release" all modifier keys (such as Ctrl, Shift, Alt) and mouse buttons, ensuring that its input state is reset.

[0031] Then, subsequent input events are routed to the new active host.

[0032] Working in conjunction with the double-click intent recognition module The operation intent recognition module also includes: The pixel sampling unit is used to sample and analyze the pixels surrounding the mouse click position to distinguish between desktop blank areas, application icons, and window controls. The behavior mapping unit is used to map the same physical operation to different execution instructions based on the sampling analysis results and the operation type.

[0033] The input allocation module works closely with the operation intent recognition module to achieve the innovative function of "double-clicking a blank area to go full screen": When the input assignment module detects a double-click event, it will suspend the execution of regular click assignment.

[0034] It forwards the event and its precise local coordinates (X_local, Y_local) to the operation intent recognition module.

[0035] Based on this coordinate, the intent recognition module queries the desktop information of the currently active host (through the agent program) to determine whether the pixel belongs to a "blank desktop" or a "shortcut icon".

[0036] Based on the judgment result, the intent recognition module issues the following instruction: If the area is blank: the instruction display control module will enlarge the current host's display window to full screen.

[0037] If it is a shortcut: the instruction input allocation module allows the double-click event to be sent to the target host as a regular double-click event, so that its operating system can execute the operation to open the program.

[0038] The input allocation module executes corresponding actions based on the instructions.

[0039] Key technological innovations Real-time mapping algorithm from absolute coordinates to relative coordinates: This invention implements an efficient and low-latency coordinate transformation algorithm, ensuring that when the mouse moves between host display windows of different resolutions and sizes, the cursor moves linearly and naturally without any jumping sensation.

[0040] Virtual input injection technology: Whether at the software level (simulating virtual HID devices through agent programs on each host) or at the hardware level (simulating USB hosts through microcontrollers), the system can ensure that the input signals sent to the target host are recognized as native hardware inputs by its operating system, thereby achieving compatibility with most operating systems without the need to install specific drivers.

[0041] State machine management: The module maintains a precise input state machine to track the state of each modifier key and mouse button. This is the foundation for achieving seamless and error-free switching, and completely solves the "key jamming" problem that may occur when switching traditional KVM.

[0042] Through the above process, the present invention successfully realizes intelligent, seamless, and intuitive control of multiple computers by a single physical input device, greatly improving the user's work efficiency and experience.

[0043] The boundary detection module is used to detect the computer area corresponding to the mouse position in real time, obtain the real-time mouse coordinates of double-click, and send the real-time mouse coordinates of double-click to the intent recognition module. The process by which the boundary detection module detects the computer area corresponding to the mouse position in real time includes: Mouse position determination and target computer mapping process Mathematical basis and process of mouse position mapping Determining the mouse position and mapping it to the target computer is a real-time, continuous spatial geometric calculation process. Its core lies in establishing a mapping relationship between two coordinate systems: the global physical coordinate system and the virtual local coordinate system of each host.

[0044] Step 1: Define the coordinate system Global Coordinate System: The origin (0, 0) is located at the top left corner of the physical display.

[0045] The X-axis is positive to the right, and the Y-axis is positive downwards.

[0046] The maximum coordinates are the native resolution of the physical display (e.g., (3840, 2160) corresponds to a 4K display).

[0047] The physical mouse's coordinates in this coordinate system are (X_g, Y_g).

[0048] Host Virtual Local Coordinate System: Each host has its own independent local coordinate system.

[0049] The origin (0, 0) is located at the top left corner of the host display window.

[0050] The maximum coordinate value corresponds to the resolution set for the host (for example, a host set to 1920x1080 has a local coordinate system range of (0,0) to (1920, 1080)).

[0051] Step 2: Establishing and Maintaining the Layout Mapping Table The system internally maintains a dynamic layout mapping table, which serves as the basis for position determination. Its data structure can be represented as follows: Host ID Global X-axis start coordinate Global Y-axis start coordinate Global X-axis end coordinate Global Y-axis end coordinate Host resolution X Host resolution Y Step 3: Target Computer Determination Algorithm Step 4: Algorithm for converting global coordinates to local coordinates 1. Event Capture: The system continuously monitors the movement events of the physical mouse.

[0052] 2. Coordinate acquisition: Get the current position (X_g, Y_g) of the mouse in the global physical coordinate system.

[0053] 3. Table Query: Reads a pre-established layout mapping table, which records the location and size information of the display area for each host.

[0054] 4. Region Traversal: By looping through all host regions, check whether the current mouse coordinates fall within a certain host display area.

[0055] 5. Hit Detection: Use a boundary comparison algorithm to determine whether the coordinates are within the rectangular area.

[0056] 6. Coordinate Conversion: If a certain area is hit, the global coordinates are converted to the local coordinates of the host using a proportional conversion formula.

[0057] 7. Event Routing: Sends the converted mouse events to the corresponding target host.

[0058] 8. Exception handling: If the mouse is not in any defined area, the system maintains the current state.

[0059] This process ensures that the mouse can move seamlessly between multiple computer display areas and accurately locate the target computer for control.

[0060] Key technological innovations Scale-based dynamic coordinate transformation: The transformation algorithm used in this invention is dynamic and resolution-independent. Regardless of how large the host display window is scaled, the algorithm can accurately map a global physical pixel to the correct position on the target host virtual desktop by calculating the scale, ensuring the accuracy and linearity of cursor movement.

[0061] Highly efficient region hit testing: By maintaining a lookup table based on rectangular regions and using simple comparison operations, the system can determine the mouse location in real time with extremely low computational overhead (O(n) complexity, where n is the number of hosts), ensuring the system's response speed.

[0062] Handling edge cases: Window overlap: In a preferred embodiment, the layout map may contain z-order (depth information). When windows overlap, mouse events are preferentially assigned to the visible host window at the top layer (with the largest z-order value).

[0063] Cursor style synchronization: The system can also obtain the local cursor style (such as arrow, I-shaped pointer, wait circle) in real time through the agent program on each host, and update it synchronously on the global cursor to provide visual consistency.

[0064] Through the aforementioned precise mathematical mapping and efficient search and judgment process, this invention enables the mouse to move seamlessly, naturally, and accurately across different host screen spaces, creating a unified "one-shot" interactive experience for users.

[0065] The intent recognition module is used to obtain the object type and distinguish the operation intent based on the real-time mouse coordinates of the double-click and the object type. The process by which the intent recognition module distinguishes operation intents based on real-time mouse coordinates of double-clicks and object types includes: The click location coordinates are obtained by double-clicking the event, and the click object type is identified based on the click location coordinates. The click object type includes blank areas, icons, and windows.

[0066] When the type of the clicked object is a blank area, a full-screen switch is triggered; when the type of the clicked object is a shortcut, the program is launched.

[0067] The display control module is used to switch between windowed and full-screen display.

[0068] The display control process of the display control module includes: The control process of the display control module. The core function of the display control module The display control module is the core component in this invention that manages the final visual presentation. It is responsible for receiving the composite image from the display integration module and executing specific display layout strategies, full-screen switching, visual enhancement, and other operations. Its control process is a closed-loop system that dynamically responds to external commands and internal state changes.

[0069] Display layout management process The display control module supports multiple preset and custom layout modes, and its management process is as follows: Step 1: Loading the layout strategy The module has multiple built-in layout strategies (such as even distribution, column layout, master-slave layout, picture-in-picture, etc.), and users can also customize the layout. The system loads the corresponding layout configuration file based on the user's selection or adaptive strategy.

[0070] Step 2: Display Area Calculation Based on the number of currently connected computers (N) and the resolution of the physical monitor (W_total × H_total), the module calculates the global coordinates and size of the display window for each computer.

[0071] Step 3: Command Issuance and Rendering The calculated layout information is sent to the compositing engine of the display integration module, which is then instructed to scale, position, and render each host screen according to the new layout strategy.

[0072] Full-screen switching control process Visual feedback and enhancement To enhance user experience, the display control module also integrates the following visual control processes: Highlighted border indicator: When the mouse enters the display area of ​​a host, the module renders a highlighted border (such as a 2-pixel wide semi-transparent blue band) at the edge of that area.

[0073] The highlight effect automatically hides when the mouse moves out of the frame or when full-screen operation is performed.

[0074] This function is achieved by overriding the rendering layer, without interfering with the host's own display output.

[0075] Smooth transition animation: When switching layouts (especially full-screen / exit full-screen), the module uses a graphics interpolation algorithm to insert intermediate frames between the initial and final states, generating a smooth animation transition effect and avoiding abrupt visual jumps.

[0076] Overlay control interface: The module can render a semi-transparent control bar on the composite screen (which can be accessed via keyboard shortcuts or mouse gestures) for quickly switching layout modes, locking input devices, etc. This control layer is always on the top layer.

[0077] Key technological innovations State saving and restoration mechanism: This invention implements efficient layout state serialization and caching technology. Before entering full-screen mode, the current complex multi-window layout state (including position, size, and stacking order) is quickly saved to memory; when exiting full-screen mode, it can be instantly and accurately restored, providing a smooth user experience.

[0078] Resolution adaptive synchronization technology: The display control module can communicate with the host agent program to collaboratively change the host's output resolution. In full-screen mode, it automatically sets the resolution to the monitor's physical resolution for optimal clarity; upon exiting, it automatically reverts to the host's original working resolution. This process is seamless and requires no manual adjustment from the user.

[0079] Non-intrusive visual enhancement layer: All visual feedback (such as highlighted borders and control bars) is completed in the compositing rendering layer of this system without modifying any content output on the host side, ensuring broad compatibility and host security.

[0080] Through the above process, the display control module endows the static multi-screen display with dynamic, intelligent, and user-friendly interactive capabilities, which is the core value of this invention in improving work efficiency.

[0081] Specifically, the present invention will be further illustrated below through embodiments: This invention supports multiple preset layouts (tiling, master-slave, etc.), dynamically adjusts the size of each computer display area, and remembers and restores full-screen status.

[0082] The hardware layer includes: A cluster of host computers (multiple units), a single high-resolution monitor, and a standard keyboard and mouse; optional features include: dedicated switching controller hardware. The software layer includes: Displays the compositing engine, input routing module, action intent analyzer, and window management controller.

[0083] The system also includes: A cross-system data exchange module is used to establish a shared clipboard and drag-and-drop channel between the multiple computers, enabling data copied on one computer to be pasted into applications on another computer.

[0084] The display control module further includes: The global layout management unit is used to respond to user global hotkeys and simultaneously restore all display windows of connected computers to the preset layout configuration.

[0085] A context-aware multi-computer input processing method includes the following steps: Capture user input actions and their absolute coordinates on the composite display canvas; Determine the target computer to which the coordinates belong and its local coordinates; Analyze the context of the input operation, including the operation type, visual attributes of the clicked target, and the operation gesture; Based on the analysis results, a choice is made between routing the operation command to the target computer for execution or triggering system-level display control functions.

[0086] Specifically, the present invention will be further illustrated below through embodiments: The innovation of this invention is mainly reflected in the following three aspects, far exceeding the simple switching concept of traditional KVM switches: 1. Architectural Innovation: A Paradigm Shift from "Switching" to "Integration" Traditional KVM uses a "select-switching" model, where a user can only interact with one computer at a time. This invention proposes a new paradigm of "parallel access and intelligent routing."

[0087] Technical features: The system does not physically or logically "switch" signal paths, but rather creates a unified virtual interaction plane that is higher than all host operating systems. The display output of all hosts is composited and rendered onto a virtual canvas in real time, while all user input is captured and analyzed by intelligent agents at the virtual canvas level and then distributed in parallel to the corresponding target hosts.

[0088] Technical effect: It enables seamless, pixel-level precision movement of the mouse cursor between different host screens, completely eliminating the black screen delay and physical boundary feeling during traditional KVM switching, and integrating multiple computers into a "supercomputer" with multiple display areas in terms of user experience.

[0089] 2. Algorithm Innovation: Context-Based Intelligent Input Intent Understanding This is the core difference between this invention and all existing technologies. The system possesses the ability to perceive and make decisions regarding the user's operational intentions.

[0090] Technical features: The dual-judgment mechanism not only determines the mouse position (on which computer), but also delves deeper into the intent of the operation at that position (what the user wants to do). This is achieved by combining coordinate information, low-level pixel sampling analysis (such as determining whether the click target is a blank desktop, an icon, a window title bar, or window content), and operation gestures (such as double-click speed and trajectory).

[0091] Dynamic behavior mapping: The same physical operation (such as double-clicking) can trigger drastically different system-level responses (such as full-screen switching) or host-level responses (such as opening a program) depending on the judgment result. This context-dependent interaction model of "one operation with multiple meanings" is unprecedented.

[0092] Technical effect: It provides a highly intelligent and efficient human-computer interaction method. Users do not need to learn additional switching buttons or commands. Intuitive operation can trigger complex system functions, which greatly reduces the cognitive load of multi-machine collaborative operation.

[0093] Functional Innovation: Cross-host global state management and collaboration This invention enables information and operations distributed across different computers to interact with each other.

[0094] Technical features: Global state synchronization engine: The system maintains a unified clipboard and a shared drag-and-drop area for files. Users can copy text or files from host A and paste them directly into applications on host B. Seamless cross-system data flow breaks down data silos between computers.

[0095] Unified display management: Provides system-level hotkeys (such as Ctrl+Super+Left / Right) to jump the cursor between computer windows, or restore the layout of all computer windows to the default state with one click. This management capability transcends the permissions of a single host operating system.

[0096] Technical effect: It realizes true cross-computer collaborative work, not just control, and logically integrates multiple physical computers into a functional whole.

[0097] Based on the same inventive concept, this invention also provides a computer device, comprising: one or more processors, and a memory for storing one or more computer programs; the programs include program instructions, and the processor executes the program instructions stored in the memory. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, used to implement one or more instructions, specifically for loading and executing one or more instructions stored in a computer storage medium to implement the above-described method.

[0098] It should be further explained that, based on the same inventive concept, the present invention also provides a computer storage medium storing a computer program, which, when executed by a processor, performs the above-described method. This storage medium can be any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0099] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0100] The foregoing has shown and described the basic principles, main features, and advantages of this disclosure. Those skilled in the art should understand that this disclosure is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of this disclosure. Various changes and modifications can be made to this disclosure without departing from its spirit and scope, and all such changes and modifications fall within the scope of this disclosure as claimed.

Claims

1. A screen display system, characterized in that, include: The display integration module is used to acquire the display content of multiple computers and integrate the display content of multiple computers onto a single monitor. The input allocation module is used to intelligently allocate keyboard and mouse input to the target computer; The boundary detection module is used to detect the computer area corresponding to the mouse position in real time, obtain the real-time mouse coordinates of double-click, and send the real-time mouse coordinates of double-click to the intent recognition module. The intent recognition module is used to obtain the object type and distinguish the operation intent based on the real-time mouse coordinates of the double-click and the object type. The display control module is used to switch between windowed and full-screen display.

2. The screen display system according to claim 1, characterized in that, The process by which the display integration module integrates the display content of multiple computers onto a single monitor includes: Step 1: Host registration and frame buffer initialization; Step Two: Desktop Canvas Creation and Layout Management; Step 3: Display data capture and processing; Step 4: Image Compositing; Step 5: Output and Synchronization; Through the integration process, the visual output of multiple independent hosts is seamlessly integrated into a unified interactive interface.

3. A screen display system according to claim 1, characterized in that, The input allocation module intelligently allocates keyboard and mouse input to the target computer, including: Step 1: Input capture and normalization; Step Two: Mouse position monitoring and target computer identification; Step 3: Routing and assigning input events; Step 4: State synchronization during switchover; The input allocation module works closely with the intent recognition module: When the input assignment module detects a double-click event, it performs a regular click assignment. Forward the event and its precise local coordinates (X_local, Y_local) to the intent recognition module; Based on these coordinates, the intent recognition module queries the desktop information of the currently active host to determine whether the pixel belongs to a blank desktop or a shortcut icon; Based on the judgment result, the intent recognition module issues the following instruction: If the area is blank: the command display control module will enlarge the current host's display window to full screen; If it is a shortcut: the instruction input allocation module allows the double-click event and sends it to the target host as a regular double-click event, so that its operating system can execute the operation to open the program; The input allocation module executes corresponding actions based on the instructions.

4. A screen display system according to claim 1, characterized in that, The process by which the boundary detection module detects the computer area corresponding to the mouse position in real time includes: Mouse position determination and target computer mapping process Step 1: Define the coordinate system The origin (0, 0) is located at the top left corner of the physical display. The X-axis is positive to the right, and the Y-axis is positive downwards. The physical mouse's coordinates in this coordinate system are (X_g, Y_g); Host virtual local coordinate system: The origin (0, 0) is located at the top left corner of the host display window; The maximum coordinate value corresponds to the resolution host set for this host, and its local coordinate system ranges from (0,0) to (1920,1080). Step 2: Establishing and maintaining the layout mapping table; Step 3: Target computer determination algorithm; Step 4: Algorithm for converting global coordinates to local coordinates; Event capture: The system continuously monitors the movement events of the physical mouse; Coordinate acquisition: Get the current position (X_g, Y_g) of the mouse in the global physical coordinate system; Read the pre-established layout mapping table, which records the position and size information of the display area of ​​each host; Region traversal: By looping through all host regions, check whether the current mouse coordinates fall within a certain host display area; Hit detection: Use a boundary comparison algorithm to determine whether the coordinates are within the rectangular area; Coordinate transformation: If a certain area is hit, the global coordinates are converted to the local coordinates of the host using a proportional conversion formula; Event routing: Sends the converted mouse events to the corresponding target host; Error handling: If the mouse is not in any defined area, the system maintains its current state. This process ensures that the mouse can move seamlessly between multiple computer display areas and accurately locate the target computer for control.

5. A screen display system according to claim 1, characterized in that, The process by which the intent recognition module distinguishes operation intents based on real-time mouse coordinates of double-clicks and object types includes: The click location coordinates are obtained by double-clicking the event, and the click object type is identified based on the click location coordinates. The click object type includes blank areas, icons, and windows.

6. A screen display system according to claim 5, characterized in that, When the type of the clicked object is a blank area, a full-screen switch is triggered; when the type of the clicked object is a shortcut, the program is launched.

7. A screen display system according to claim 1, characterized in that, The display control process of the display control module includes: The control process of the display control module. The core function of the display control module The display control module is responsible for receiving the composite image from the display integration module and executing specific display layout strategies, full-screen switching, and visual enhancement. Its control process is a closed-loop system that dynamically responds to external commands and internal state changes. The display control module management process is as follows: Step 1: Loading the layout strategy: The display control module has multiple built-in layout strategies. The system loads the corresponding layout configuration file according to the user's selection or adaptive strategy. Step 2: Display area calculation: Calculate the global coordinates and size of the display window for each computer based on the number of currently connected computers and the resolution of the physical monitors; Step 3: Command Issuance and Rendering The calculated layout information is sent to the compositing engine of the display integration module, which is then instructed to scale, position, and render each host screen according to the new layout strategy. In addition to the full-screen switching control process and visual feedback and enhancement.

8. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that, The memory stores a computer program that can run on a processor, and when the processor loads and executes the computer program, it employs a screen display system according to any one of claims 1 to 7.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is loaded and executed by the processor, it employs a screen display system according to any one of claims 1 to 7.

Images