Cross-screen input method control method and system for dual-screen remote desktop

Abstract

The application relates to the field of remote desktop and human-computer interaction technology, and provides a cross-screen input method control method and system for a double-screen remote desktop. The method displays a remote desktop session on a first display screen, starts an input agent containing a native editable control on a target display screen different from the first display screen, makes the native editable control obtain a focus and a window token, establishes an input connection request display system input method, opens a remote input channel according to an actual display state of the system input method on the target display screen and a display state callback belonging to a current control request identifier, injects a combined and completed confirmation text and a control operation key event into the remote desktop session after the confirmation text and the control operation key event are respectively verified, and uniformly releases a window and an input connection under an input closing condition. The method has beneficial effects of stable remote picture display, high system input method compatibility, few input method state misjudgments and low abnormal state residual rate.

Description

Technical Field

[0001] This application relates to the field of remote desktop and human-computer interaction technology, and in particular to a cross-screen input method control method and system for dual-screen remote desktop. Background Technology

[0002] When users remotely control a computer using a tablet, the most common inconvenience is that once the on-screen keyboard is brought up, it occupies the area currently displaying the remote computer's screen. The code, spreadsheets, terminal windows, or office software the user is viewing may be obscured, and the remote screen may suddenly shrink, shift upwards, or rearrange. To see the content clearly, the user needs to repeatedly hide the keyboard, adjust the screen, and then reopen the keyboard, frequently interrupting the input process.

[0003] While dual-screen tablets provide an additional screen, typical remote desktop applications cannot simply move the system input method to the second screen. The system input method can only display and function stably if a native input control exists on the target screen that can gain focus and establish an input connection. Simply drawing a keyboard interface or directly issuing a command to display the input method may result in issues such as the keyboard not popping up, popping up on an incorrect screen, or Chinese candidate words or voice input not working properly. Summary of the Invention

[0004] This application provides a cross-screen input method control method and system for dual-screen remote desktop, aiming to solve the problem that although dual-screen tablets provide another screen, ordinary remote desktop applications cannot simply move the system input method to the second screen. The system input method can only be stably displayed and work when there is a native input control on the target screen that can gain focus and establish an input connection.

[0005] In a first aspect, embodiments of this application provide a cross-screen input method control method for a dual-screen remote desktop, applied to a terminal device including a first display screen and a second display screen. The method includes: displaying a remote desktop session on the first display screen, generating a control request identifier in response to an input method activation request, and selecting a target display screen that is different from the first display screen; activating an input agent containing native editable controls on the target display screen, transmitting the control request identifier and remote session information to the input agent, and verifying that the actual display screen identifier of the input agent is consistent with the target display screen identifier. Give the native editable control focus and window token, establish an input connection through the native editable control and request the display of the system input method; listen to the actual display status of the input method on the target screen, and when it is confirmed that the system input method is actually displayed on the target screen and the display status callback belongs to the current control request identifier, open the remote input channel; Upon receiving input events, the system processes the combined confirmation text and control operation key events separately, and injects them into the remote desktop session after verifying the control request identifier and remote session information. Upon responding to the input closing condition, the system stops receiving input events, hides the system input method and terminates the input agent, and releases the relevant windows and input connections.

[0006] In some embodiments, the step of displaying a remote desktop session on the first display screen, generating a control request identifier in response to an input method activation request, and selecting a target display screen different from the first display screen includes: launching a remote desktop client on the first display screen, establishing a remote desktop session with a remote computer, and displaying the remote desktop session in full screen on the first display screen; responding to an input method activation request, generating a control request identifier that is unique within a predetermined range, and binding the control request identifier to the current remote session information; obtaining a list of online display screens, and selecting a second display screen different from the first display screen as the target display screen from the list of online display screens.

[0007] In some embodiments, the step of activating an input agent containing native editable controls on a target display screen and passing a control request identifier and remote session information to the input agent includes: activating an input agent on the target display screen, the input agent containing native editable controls, the native editable controls maintaining focusability and editability; passing a control request identifier, remote session information, and a target display screen identifier to the input agent; and the input agent receiving and storing the control request identifier, remote session information, and target display screen identifier.

[0008] In some embodiments, verifying that the actual display screen identifier of the input agent matches the target display screen identifier includes: after the input agent is started, reading the actual display screen identifier of the display screen where the input agent window is actually located; comparing the actual display screen identifier with the target display screen identifier; when the actual display screen identifier matches the target display screen identifier, confirming that the input agent is running on the target display screen; when the actual display screen identifier does not match the target display screen identifier, ending the input agent instance and restarting it.

[0009] In some embodiments, the step of enabling the native editable control to gain focus and a window token, establishing an input connection through the native editable control, and requesting the display of the system input method includes: setting the input proxy window as the foreground window, enabling the native editable control to gain input focus; the native editable control obtaining a valid window token, establishing an input connection with the system input method service through the native editable control; and sending a display request to the system input method service through the input connection, requesting the system input method to be displayed on the target display screen.

[0010] In some embodiments, the step of monitoring the actual display status of the input method on the target display screen, and confirming that the system input method is actually displayed on the target display screen and that the display status callback belongs to the current control request identifier, and then opening the remote input path, includes: registering a monitoring callback for changes in the input method window area on the target display screen and a monitoring callback for changes in the visibility of the input method; when a display status callback is received, extracting the control request identifier carried by the display status callback, and comparing the control request identifier carried by the display status callback with the current control request identifier; when the control request identifier carried by the display status callback is consistent with the current control request identifier, and the system input method window is visible on the target display screen, confirming that the system input method is actually displayed, and then opening the remote input path.

[0011] In some embodiments, receiving input events involves processing the combined confirmation text and control operation key events separately, including: receiving input events when the remote input channel is open; for input events with a pre-editing stage, extracting the combined and confirmed new text as confirmation text; and generating independent control operation key events for Enter, Backspace, and Tab control operations.

[0012] In some embodiments, the step of injecting the control request identifier and remote session information into the remote desktop session after verification includes: extracting the control request identifier and remote session identifier carried by the input event; comparing the control request identifier carried by the input event with the current control request identifier, and comparing the remote session identifier carried by the input event with the current remote session information; when both the control request identifier and the remote session identifier match, injecting the confirmation text or control operation key event into the current remote desktop session; and discarding the input event when either identifier does not match.

[0013] In some embodiments, responding to the input closing condition, stopping receiving input events, hiding the system input method and ending the input agent, and releasing the relevant window and input connection includes: detecting the input closing condition, which includes a user hide command, a user close command, an open timeout, the end of the remote desktop session, application destruction, the target display being turned off, or the target display being removed; responding to the input closing condition, stopping receiving new input events, sending a hide request to the system input method service through the input connection to hide the system input method; ending the input agent, releasing the input agent window, native editable controls, and input connection, and clearing the binding relationship between the control request identifier and the remote session information.

[0014] Secondly, this application provides a cross-screen input method control system for dual-screen remote desktops. The system includes: a session display unit, used to display a remote desktop session on a first display screen, generate a control request identifier in response to an input method activation request, and select a target display screen that is different from the first display screen; to activate an input agent containing native editable controls on the target display screen, transmit the control request identifier and remote session information to the input agent, and verify that the actual display screen identifier of the input agent is consistent with the target display screen identifier; The channel opening unit is used to enable the native editable control to gain focus and window token, establish an input connection through the native editable control and request the display of the system input method; it monitors the actual display status of the input method on the target display screen, and opens the remote input channel when it confirms that the system input method is actually displayed on the target display screen and the display status callback belongs to the current control request identifier. The connection release unit is used to receive input events, process the combined confirmation text and control operation key events separately, and inject them into the remote desktop session after verifying the control request identifier and remote session information; respond to the input closing condition, stop receiving input events, hide the system input method and end the input agent, and release the relevant window and input connection.

[0015] The method provided in this application has the following beneficial effects: 1. The system input method is carried by a target display screen that is different from the remote screen, and the remote computer screen is continuously displayed on the first display screen, which can reduce the scaling, displacement, layout rearrangement and rendering surface reconstruction of the remote screen when the input method appears, and maintain the stability of the remote screen display area.

[0016] 2. By invoking the system input method through a native editable control with a valid window token, focus, and input connection, the system input method can reuse candidate words, Chinese combination input, and voice input capabilities without modifying the system window management service or input method service, thus improving the compatibility of the system input method.

[0017] 3. Using the actual display status of the input method on the target screen as the basis for input enable can avoid the situation where the input method is successfully called but there is no problem, which can be caused by relying solely on the display call return result, thus reducing misjudgment of the input method status.

[0018] 4. By using control request identifiers to uniformly identify input agents, display status callbacks, and input events, the probability of residual focus, input connections, or agent windows after quick on / off, session end, and screen removal can be reduced, thus reducing abnormal state residue.

[0019] 5. Separating the processing of combined text and control operation button events can reduce the risk of pre-edited text being repeatedly injected into the remote session during Chinese, voice input, or candidate word confirmation, and improve the accuracy of complex text input.

[0020] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a schematic flowchart illustrating the steps of a cross-screen input method control method for a dual-screen remote desktop according to an embodiment of this application; Figure 2 This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application; Figure 3 This is a schematic diagram illustrating the principle of a cross-screen input method control method for a dual-screen remote desktop according to an embodiment of this application; Figure 4 This is a schematic block diagram of a cross-screen input method control system for a dual-screen remote desktop provided in one embodiment of this application; Figure 5 This is a schematic block diagram of the structure of a terminal device provided in an embodiment of this application.

[0023] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Detailed Implementation

[0024] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0025] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the order described. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.

[0026] It should be understood that, in order to clearly describe the technical solutions of the embodiments of the present invention, the terms "first" and "second" are used in the embodiments of the present invention to distinguish identical or similar items with essentially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and the terms "first" and "second" are not necessarily different.

[0027] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0028] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0029] When users remotely control a computer using a tablet, the most common inconvenience is that once the on-screen keyboard is brought up, it occupies the area currently displaying the remote computer's screen. The code, spreadsheets, terminal windows, or office software the user is viewing may be obscured, and the remote screen may suddenly shrink, shift upwards, or rearrange. To see the content clearly, the user needs to repeatedly hide the keyboard, adjust the screen, and then reopen the keyboard, frequently interrupting the input process.

[0030] While dual-screen tablets provide an additional screen, typical remote desktop applications cannot simply move the system input method to the second screen. The system input method can only display and function stably if a native input control exists on the target screen that can gain focus and establish an input connection. Simply drawing a keyboard interface or directly issuing a command to display the input method may result in issues such as the keyboard not popping up, popping up on an incorrect screen, or Chinese candidate words or voice input not working properly.

[0031] In actual use, users may also quickly open, close, and reopen the keyboard, or disconnect from the remote connection or switch screens during input. Since window creation, focus acquisition, and input method display are not completed simultaneously, the application may mistakenly believe that the keyboard has been displayed, or it may leave an invisible input window after closing, causing the keyboard to fail to open the next time, input to be unresponsive, or the interface state to become chaotic.

[0032] Please refer to Figure 1 This application provides a cross-screen input method control method for dual-screen remote desktops, applicable to, for example... Figure 2The diagram shows a terminal device comprising a first display screen and a second display screen. The terminal device can be a foldable dual-screen tablet, a dual-screen laptop, a tablet with an external second display screen, a dual-screen handheld terminal, or any computing device with at least two independent displays. The first and second displays can be two physically independent displays, or a single display screen divided into two display areas by software (such as the left and right display areas in the unfolded state of a foldable screen). The operating system of the terminal device can be Android, HarmonyOS, iOS, Windows, or other operating systems that support multi-display management and system input method frameworks. It should also be noted that all information involved in the method provided in this application was extracted with the authorization of the relevant users and in accordance with relevant regulations, and will not infringe on user privacy.

[0033] Please refer to Figure 2 This application also provides a terminal device, including a first display screen, a second display screen, a processor, a memory, and a communication module. The first display screen is used to display a remote desktop session screen, and the second display screen is used to display an input agent and a system input method. The processor executes computer program instructions stored in the memory to implement the cross-screen input method control method provided in the embodiments of this application. The communication module is used to establish a network connection with a remote computer and transmit remote desktop screen data and input event data.

[0034] Please refer to Figure 3 The cross-screen input method control principle of this application embodiment is as follows: the first display screen displays a remote desktop session; the target display screen (second display screen) starts an input agent containing native editable controls, the native editable controls gain focus and request to display the system input method; the display screen identifier verification ensures that the input agent is running on the target display screen; the actual display status of the input method is monitored, and the remote input path is opened when the display status callback is confirmed to belong to the current control request identifier; the content entered by the user through the system input method generates input events, which are divided into two categories: confirmation text and control operation keys; the input events are injected into the remote desktop session after being verified by the control request identifier and remote session information, thereby realizing cross-screen input.

[0035] The provided method for controlling cross-screen input methods in a dual-screen remote desktop includes steps S101 to S103. Details are as follows: Step S101. Display the remote desktop session on the first display screen, generate a control request identifier in response to the input method activation request, and select a target display screen that is different from the first display screen; start an input agent containing native editable controls on the target display screen, pass the control request identifier and remote session information to the input agent, and verify that the actual display screen identifier of the input agent is consistent with the target display screen identifier.

[0036] Specifically, the core function of step S101 is to establish and display a remote desktop session on the first display screen, generate a unique control request identifier when the user requests to enable the input method, select the second display screen as the target display screen, start an input agent containing native editable controls on the target display screen, and verify that the input agent is indeed running on the target display screen and not incorrectly displayed on the first display screen, thus laying the foundation for the correct pop-up of the system input method on the target display screen in the future.

[0037] Remote Desktop Session Establishment and Display: The user launches the remote desktop client application on their terminal device, enters the remote computer's network address (Internet Protocol address or hostname), port number, username, and authentication credentials, and initiates a remote connection request. The remote desktop client establishes a network connection with the remote computer via a communication module (Wi-Fi, cellular network, or Ethernet), including Transmission Control Protocol connection establishment, transport layer security encryption handshake, user authentication, and remote desktop session negotiation (parameters such as display resolution, color depth, audio redirection, and clipboard redirection). After successful connection establishment, the remote computer continuously sends remote desktop screen data, compressed with video encoding, to the terminal device. The client decodes this data and displays it full-screen on the first display screen. Full-screen display means the remote desktop screen occupies the entire display area of ​​the first screen, hiding the local status bar and navigation bar, maximizing the visible space of the remote screen. The remote desktop client simultaneously generates and records remote session information, including session identifier, remote computer address, login username, session establishment time, and negotiated display parameters, storing this information in the session management module for subsequent input event session verification and routing.

[0038] Input method activation request response and control request identifier generation: The input method activation request is triggered by the user (clicking the keyboard icon button, clicking the input area on the remote screen, pressing a hardware shortcut key, voice command, etc.) or automatically by the application (prompting the user to activate when the currently active window on the remote computer is a text input application). Upon receiving the activation request, the input method control module immediately generates a control request identifier. The control request identifier is a unique identifier that does not repeat within a predetermined range. Generation methods include auto-incrementing serial numbers (maintaining a global counter, atomically incrementing each time), random numbers (generated by a cryptographically secure pseudo-random number generator), universally unique identifiers (128-bit UUID), or a combination of timestamp and random number. After generating the control request identifier, it is bound to the current remote session information, establishing a mapping relationship and storing it in the session binding table. The key in the binding table is the control request identifier, and the values ​​are remote session information, request status, target display screen identifier, input agent reference, etc. The binding relationship ensures that all subsequent operations can be associated with the correct remote session through the control request identifier, while also distinguishing different input method activation requests and avoiding state crosstalk during rapid on / off switching.

[0039] Target Display Selection: The input method control module calls the operating system's multi-display management interface to obtain a list of all currently online displays. Each entry includes the display identifier, name, resolution, refresh rate, status, and connection method. The first display identifier (the display where the remote desktop client window is located) is confirmed and marked as occupied. Then, other displays different from the first display are selected from the list as the target display. Selection strategies include: defaulting to the second built-in screen; prioritizing multiple screens (prioritizing physically adjacent built-in screens, then external monitors, and finally wireless displays); user-specified target displays have the highest priority; and excluding displays that are offline, in mirror mode, or fully occupied by high-priority applications. After selection, the target display identifier is bound to the control request identifier, and a pre-check is performed to ensure that the available area of ​​the target display meets the minimum display requirements of the input method.

[0040] Input Agent Startup: The input method control module creates an input agent window on the target display screen. The input agent is an application window or component running on the target display screen, its core function being to act as a bridge between the system input method and the remote desktop client: it obtains input focus and input connection from the system input method by including a native editable control, triggers the system input method to be displayed on the target display screen, and receives input events generated by the input method and forwards them to the remote desktop client. The input agent window is set to have no title bar, no border, semi-transparent or fully transparent background, is not displayed in the recent tasks list, and its window type is a normal application window to ensure it can obtain window focus and window token. The window size can be set to full screen or small size.

[0041] Native editable controls are standard text input controls provided by the operating system, possessing focusable and editable attributes. The control is set to focusable, touch-mode focusable, editable, and enabled, with the input type set to text (which can be dynamically adjusted according to the remote desktop scenario). To avoid displaying redundant text input boxes on the target screen, the control is made invisible or extremely small (1×1 pixel) or moved out of the visible area, while still maintaining focusability and editability—this "invisible but focusable" design is the core technique of the input agent. Text change listeners and key event listeners are registered for the control to capture input events generated by the input method. Data is exchanged between the input agent and the remote desktop client via in-process method calls, callback interfaces, or inter-process communication mechanisms, with communication latency in the millisecond range.

[0042] Parameter passing and display screen identifier verification: After the input agent starts, the input method control module passes the control request identifier, remote session information (including session identifier), target display screen identifier, configuration parameters, and communication channel identifier to the input agent. The input agent receives these and stores them in member variables; these identifiers are then appended to each subsequent input event for verification.

[0043] After the input agent window is fully displayed, the system calls the operating system interface to read the actual display screen identifier of the screen where the window is located (by querying the window handle or window coordinates), and compares it precisely with the pre-transmitted target display screen identifier. If they match, it confirms that the input agent is running on the target display screen, the verification passes, and the input method control module is notified to update the status. If they do not match, it indicates that the input agent is running on the wrong display screen (possibly due to system window routing policies, user dragging, hot-plugging of the display screen, etc.), and a recovery process is executed: the current input agent instance is terminated (the window is closed, resources are released), the input agent is recreated on the target display screen, parameters are retransmitted, and verification is performed again. A maximum number of retries is set (e.g., 3 times). If there are consecutive failures, it is determined to be a system-level anomaly, and the system reverts to the compatibility mode of the input method on the first display screen and prompts the user. After the verification passes, the input agent can also continuously monitor changes in the display screen where the window is located. If the input is moved to another screen during input, the input path is paused and an attempt is made to resume.

[0044] After step S101 is completed, the first display screen is showing the remote desktop session, the control request identifier has been generated and bound to the remote session information, the target display screen has been selected, the input agent has been successfully started on the target display screen and verified by the display screen identifier, and is in a ready state.

[0045] Step S102. Make the native editable control gain focus and window token, establish an input connection through the native editable control and request the display of the system input method; listen to the actual display status of the input method on the target screen, and when it is confirmed that the system input method is actually displayed on the target screen and the display status callback belongs to the current control request identifier, open the remote input channel.

[0046] Specifically, the core function of step S102 is to enable the native editable control to obtain input focus and window token, establish an input connection and request the display of the system input method, and then listen to the actual display status of the input method (instead of only relying on the return value of the display request). Only when it is confirmed that the input method is actually displayed on the target screen and the callback belongs to the current control request identifier will the remote input channel be opened to avoid input errors caused by "the request is successful but the input method does not actually pop up".

[0047] Window Foregrounding and Focus Acquisition: Set the input agent window as the foreground window (moved to the top of the window stack to receive keyboard and touch input). Since the input agent is located on the target display screen and the remote desktop window is on the first display screen, they can remain active on their respective screens without interfering with each other. After the window is brought to the foreground, call the focus request interface of the native editable control to give it input focus. The control with focus receives input events generated by the user through the soft keyboard, physical keyboard, etc. To ensure stable focus acquisition, the control must meet the following conditions: focusable, visible (even at a very small size), attached to the window view hierarchy, and the window has been created and displayed; if the request fails, a delay and retry can be performed or the focus of other controls can be cleared before requesting again. Register a focus change listener for the control to monitor the focus state in real time, and attempt to restore focus or trigger the shutdown process if focus is unexpectedly lost.

[0048] Window token acquisition and input connection establishment: After a native editable control gains focus, it obtains a valid window token for its own window through the control's window token acquisition interface. A window token is a unique token in the operating system's window management system that identifies the application to which a window belongs. When establishing an input connection, the system input method service needs to verify whether the requesting control belongs to a valid, displayed window; the window token is the verification credential. Only controls holding a valid window token can successfully establish an input connection and request the display of the input method. The window token also implicitly associates the window with the display screen. The input method service determines the display screen based on the token, thus displaying the input method on the corresponding display screen—this is the key mechanism for cross-screen input method display: the input proxy window is located on the target display screen, its token is associated with the target display screen, and subsequent requests for the display of the input method will be displayed on the target display screen.

[0049] After a control gains focus and a valid token, the system automatically triggers the establishment of an input connection: After the input method manager verifies the window token, it establishes an input connection channel between the application process and the input method service process, returning a connection object for requesting to show / hide the input method and receiving input events. The input delegate can override the control's input connection creation callback, returning a custom input connection wrapper class. It can override methods such as text submission, deleting surrounding text, sending key events, executing editor actions, and ending combined text, intercepting and capturing all input events before they reach the control—this is the most reliable way to receive input events. Input connection establishment is an asynchronous process. Successful connection is confirmed by listening to connection-related callbacks or checking the input method's activation status; retrying occurs on failure, and a shutdown process is triggered if the limit is exceeded.

[0050] Input method display request and actual display status monitoring: After a successful input connection is established, a display request is sent to the system input method service through the input connection, requesting that the soft keyboard be displayed on the target screen. The display request is an asynchronous operation. After receiving the request, the input method service needs to load, lay out, and render the input method interface, which typically takes 100ms to 500ms before it is actually displayed. The return value of the display request only indicates whether the request has been accepted, and does not mean that the input method has actually been displayed—therefore, it is necessary to monitor the actual display status.

[0051] The input agent registers callbacks to listen for changes in the input method window area and visibility on the target display screen. Listening methods include: detecting changes in the visible area of ​​the window when the input method pops up via changes in window decoration areas (WindowInsets); listening for changes in the input method window's position and show / hide events via system-level event hooks; and periodically polling the input method manager's activation status interface. When registering callbacks, the current control request identifier is bound to the callback, ensuring that each display state callback carries the corresponding control request identifier.

[0052] Callback control request identifier verification and input method actual display confirmation: When a display status callback is detected, the control request identifier carried in the callback is extracted and compared with the current control request identifier (the activity identifier generated by the most recent enable request). If they match, it means the callback belongs to the current session, and confirmation continues; if they do not match, it means the callback belongs to an old session (old callbacks arrive late during quick toggle) or another session, and the callback is discarded, and the input path is not opened based on it. Control request identifier verification is a key mechanism to prevent old callbacks from mistakenly triggering the input path.

[0053] Based on successful identifier verification, confirm that the system input method window is visible on the target display screen, which must simultaneously meet the following conditions: visibility is true; window size is greater than a preset threshold (excluding zero-size windows that have been created but not rendered); and the display screen identifier of the window is equal to the target display screen identifier. Optionally, the visibility state must last for 100ms to 300ms to exclude brief flickering. Set a timeout timer (2 to 5 seconds) from the time the display request is sent. If no confirmation is received within the timeout period, the display request is retried. If it still fails, the display is considered to have failed, the input agent is terminated, and the user is notified.

[0054] Remote Input Path Activation: After confirming the input method is actually displayed, the input method control module activates the remote input path: setting the path status flag to "activated," notifying the input agent to start receiving and forwarding input events (previously, the input agent was in a display waiting state, and captured events were discarded), notifying the input injection module to prepare to receive events, and displaying a brief "Keyboard is now displayed on the second screen" message on the first display screen (disappearing after 2 to 3 seconds). The remote input path is the input event transmission channel from the target display system input method to the input agent to the communication channel to the remote desktop client to the remote computer. Before the path is activated, user input is not injected into the remote session; it is only formally injected after activation. This gate mechanism ensures that input is only received after the input method is correctly displayed, preventing accidental touches when not displayed from being interpreted as input events.

[0055] After step S102 is completed, the native control has obtained focus and token, the input connection has been established, the system input method has been actually displayed on the target screen and passed the identifier verification, the remote input channel has been opened, and the system has entered the cross-screen input working state.

[0056] Step S103. Receive input events, process the combined confirmation text and control operation key events separately, and inject them into the remote desktop session after verifying the control request identifier and remote session information; respond to the input closing condition, stop receiving input events, hide the system input method and end the input agent, and release the relevant windows and input connections.

[0057] Specifically, the core function of step S103 is to receive input events after the channel is opened, and process them separately into two categories: confirmation text and control operation keys. Each type of event is injected into the remote desktop session only after being double-verified by the control request identifier and remote session information. When the input closing condition is detected, a unified closing process is executed: stop receiving, hide the input method, end the input agent, release all resources, and clear the binding relationship to avoid resource residues that would cause the input method to be unable to be used normally in the future.

[0058] Input event reception and classification: After the remote input channel is enabled, user operations on the system input method of the target display (clicking the soft keyboard, Chinese Pinyin combination input, voice input, handwriting input, physical keyboard input, etc.) generate input events that are passed to the native editable controls through the input connection, triggering text changes or key events. The input agent receives events through channels such as custom input connection interception callbacks (the most reliable layer), text change listeners, key event listeners, and editor action listeners, and deduplicates and merges the events. Each event is appended with a reception timestamp, current control request identifier, current remote session identifier, and event sequence number.

[0059] Input events are handled in two separate categories: Confirmation text processing: For input methods with a pre-editing stage (Chinese, Japanese, Korean, etc.), when a user inputs Pinyin, the Pinyin letters are displayed in the pre-editing area and have not yet been finalized. They should not be injected into the remote desktop (otherwise, the remote computer will receive unconfirmed Pinyin, causing confusion). Only when the user presses the space bar, enter key, or clicks a candidate word, and the pre-edited text is replaced with the final text and submitted, is the newly added text that has been combined and confirmed extracted as the confirmation text. Extraction methods include comparing the control content before and after the text change (if the newly added substring contains non-Latin characters and the pre-editing area has been cleared, it is the confirmation text), listening to the text submission callback of the custom input connection (most accurate), and tracking changes in the state of the pre-editing area. For inputs without a pre-editing stage, such as English, each newly added letter is the confirmation text. After the confirmation text is extracted, encoding conversion (converted to UTF-8 or UTF-16 supported by the remote desktop protocol), length checking (if it exceeds the single injection limit, it is split), invisible character processing (newline characters and tabs are converted to corresponding control key events instead of being injected into the text), and temporary storage in a first-in-first-out event queue.

[0060] Control key processing: For control operations such as Enter (confirm input / execute command / newline), Backspace (delete the character before the cursor), and Tab (switch focus / autocomplete / indent), independent control key events are generated, including key codes, key actions (pressed / released), modifier key states (Shift / Ctrl / Alt, etc., used to identify shortcut keys), key characters, timestamps, and identifiers. It can be extended to handle arrow keys, delete keys, function keys, shortcut keys (copy / paste / undo / select all, etc.), and the Esc key. Control keys and confirmation text are stored in different queues and processed separately: confirmation text is injected in batches through a text injection channel (high efficiency, supports Chinese characters), while control keys are injected key-by-key through a keyboard event channel (simulating real key presses and releases to ensure the application correctly recognizes shortcut keys and consecutive backspaces).

[0061] Double Validation and Remote Injection: After receiving an event forwarded by the input proxy, the input injection module of the remote desktop client performs double validation: extracting the control request identifier carried by the event and comparing it with the current control request identifier (ensuring the event belongs to the current input method session and the time dimension is correct); extracting the remote session identifier carried by the event and comparing it with the session identifier in the current remote session information (ensuring the event target is the current remote session and the spatial dimension is correct). Injection only occurs if both match; otherwise, the event is discarded and logged (for scenarios such as delayed arrival of old session events, multi-session crosstalk, abnormal restart of the input proxy, and communication errors).

[0062] After successful verification, injection is performed according to event type: Confirmation text injection—select character sequence injection (short text, character-by-character encoding and sent via keyboard channel) or clipboard injection (long text, written to the remote clipboard and sent with a paste shortcut key) based on length, sent to the remote computer via remote desktop protocol, received and inserted into the currently active window; Control operation key injection—map local key codes to scan codes or virtual key codes of the remote desktop protocol, set modifier key states, and sequentially send key press events, short delays (10ms to 50ms simulating real key press duration), and key release events; send a sequence of key combinations: modifier key press → primary key press and release → modifier key release, simulating the real operation sequence. After injection, the result is recorded (success / failure, time, content summary). Limited retries are allowed on failure; if the remote connection is interrupted, a shutdown process is triggered.

[0063] Input Closure Condition Detection and Unified Closure Release: The input method control module detects input closure conditions in multiple ways, including: user hiding commands (clicking the hide button, collapsing the input method, gesture hiding); user closing commands (disabling cross-screen input method in settings, switching back to single-screen mode); opening timeout (input method not actually displayed within a preset time after display request); remote desktop session termination (user actively disconnects, remote computer shuts down, network interruption timeout); application destruction (user closes application, system performs low memory reclamation); target display screen shutdown (folded screen turns off, external monitor power off, screen off); target display screen removal (external monitor hot-plugged disconnect, folded screen removed by the system after folding, wireless display screen disconnected); focus loss timeout (control loses focus and does not recover within a preset time); user switches to other applications. When multiple conditions are triggered simultaneously, they are handled according to the following priority: user-initiated > exception failure > session termination > display change.

[0064] Upon detecting the shutdown condition, execute the following unified process: First, stop receiving new input events—set the pathway status to closed, notify the input agent to stop processing new events, clear the event queue to be injected, and cancel all timers; Second, hide the system input method—send a hide request to the input method service via the input connection, listen for the hide callback to confirm the input method is hidden, and forcibly disconnect the input connection if no confirmation is received within a timeout; Third, end the input agent—clear control focus, destroy the input agent window (release view resources, window token expires), and mark the instance as destroyed; Fourth, release all related resources—unregister all control listeners and system-level listeners (display changes...). (To avoid memory leaks, check network status, etc.), disconnect the input connection (end combined text, notify the input method service to release connection resources), close the communication channel, cancel unfinished asynchronous tasks, and release temporary data structures; fifth step, clear the control request identifier and remote session information binding—remove the current identifier entry from the session binding table, set the current control request identifier to an invalid value, mark the remote session as having no active input method, and clear the target display screen identifier; sixth step, close the completion confirmation—check that resources have been released item by item, reset the input method control module status to the initial waiting state, update the interface (make the open button clickable again), and record the close log (reason, time, release result, and time consumed).

[0065] The shutdown process is designed to be idempotent (multiple executions do not report errors) and includes null value checks and exception handling at each step. Single-step exceptions do not interrupt subsequent resource release, ensuring maximum resource release. If the shutdown reason is recoverable (user actively hides it), the remote desktop session remains displayed, and the user can reopen it (generate a new identifier and re-execute the entire process); if it is unrecoverable (session ends, application is destroyed), the client terminates accordingly.

[0066] After step S103 is completed, the input events have been correctly classified, validated and injected, the closing condition has been responded to, the input method has been hidden, the input agent has ended, all resources have been released, the binding has been cleared, and the system has been restored to the initial state of waiting for the next opening request, with no resource residue.

[0067] In some embodiments, the step of displaying a remote desktop session on the first display screen, generating a control request identifier in response to an input method activation request, and selecting a target display screen different from the first display screen includes: launching a remote desktop client on the first display screen, establishing a remote desktop session with a remote computer, and displaying the remote desktop session in full screen on the first display screen; responding to an input method activation request, generating a control request identifier that is unique within a predetermined range, and binding the control request identifier to the current remote session information; obtaining a list of online display screens, and selecting a second display screen different from the first display screen as the target display screen from the list of online display screens.

[0068] Launch the remote desktop client on the first display screen to establish a remote desktop session with the remote computer. The remote desktop session is displayed in full screen on the first display screen.

[0069] The user launches the remote desktop client from the terminal device's application list. The application process initializes the network module, video decoding module, audio playback module, input injection module, and user interface. The user selects a saved connection configuration or manually enters the remote computer's network address (Internet Protocol address or hostname), port number, authentication information (username, password or token, biometrics, single sign-on credentials), and session parameters (display resolution, color depth, audio / clipboard / drive redirection switch). After clicking connect, a request is initiated.

[0070] The network connection establishment process involves the following steps: domain name resolution (hostname to IP address), Transmission Control Protocol (TCP) three-way handshake to establish a reliable transmission connection, Transport Layer Security (TLS) handshake to establish an encrypted channel, user authentication, session parameter negotiation, and remote computer creating a session and returning a success notification. The entire process typically takes 1 to 10 seconds, depending on network latency and server performance.

[0071] After the session is established, the remote computer's screen capture module captures the changing areas of the screen (inter-frame differential reduces data volume), compresses the data using a video encoder (H.264, H.265 / High Efficiency Video Coding, VP9, ​​AV1, etc., dynamically adjusting the bitrate and frame rate according to bandwidth), and sends it to the client through a virtual channel. Upon receiving the data, the client first decodes it using a hardware decoder, then renders it to the remote desktop window on the first display screen using a graphics processor, employing double or triple buffering to avoid screen tearing. The screen refresh rate is typically 15fps to 60fps, and can be as low as 5fps to 10fps for static images to save bandwidth.

[0072] The remote desktop client window is set to full-screen mode: its size is set to the full resolution of the first display screen, with no title bar and no border. The status bar and navigation bar are hidden through immersive mode. On foldable / dual-screen devices, the window is explicitly designated to run on the first display screen. Full-screen display ensures that the remote screen occupies the entire visible area of ​​the first display screen—which is precisely why the input method needs to be displayed on the second screen: every inch of the first screen is used to display the remote screen and cannot be obscured by the input method.

[0073] The session management module generates and maintains remote session information, including: session identifier (a unique identifier assigned by the protocol or generated locally), remote computer identifier (address, name, operating system type and version), user information (username, domain, permissions), connection information (establishment time, connection type: direct / relay / VPN, network protocol, encryption protocol, bandwidth estimate, round-trip time), session parameters (resolution, color depth, dots per inch, list of enabled redirection functions), and session status (connecting / connected / disconnected / reconnecting, last active time, idle duration). Session information is continuously updated during the session and is used for input event validation, connection quality monitoring, and reconnection recovery.

[0074] In response to the input method activation request, a unique control request identifier within a predetermined range is generated, and the control request identifier is bound to the current remote session information.

[0075] Input method activation requests can be generated in several ways: user-initiated (clicking the keyboard icon button floating at the edge of the remote screen, double-clicking or long-pressing the input area on the remote screen, custom gestures, physical keyboard shortcuts, voice assistant commands); and application-initiated (receiving information about the currently active window on the remote computer via the remote desktop protocol virtual channel, popping up a prompt when the focused control is a text editing type, and generating the request after user confirmation; or prompting after detecting cursor blinking / input box outline through remote screen image recognition). The activation request includes fields such as request identifier, type (user-initiated / application-initiated), timestamp, trigger source, current remote session identifier, and user preference settings.

[0076] The control request identifier generation methods include: an auto-incrementing sequence number (maintaining a 32-bit unsigned integer global counter, atomically incrementing, ranging from 0 to 4294967295, which will not be exhausted within a single runtime cycle); a universally unique identifier (generating a 128-bit UUID by calling a system interface, with extremely strong global uniqueness but a relatively long string); and a timestamp plus a random number (combining the lower 32 bits of a millisecond-level timestamp with a 16-bit random number to form a 48-bit identifier, balancing time order and avoiding conflicts within the same millisecond). Generation must be an atomic operation to ensure no duplication in a multi-threaded environment.

[0077] The binding data structure is a session binding table (hash table). The key is the control request identifier, and the value is a binding information object, which includes: control request identifier, remote session information reference or copy, request timestamp, request status enumeration (pending startup / input agent started / display verification passed / input method display confirmed / input channel open / closing / closed), target display identifier, input agent instance reference, input method display status, creation time, and last activity time. After generating the identifier, a binding object is created and inserted into the table. During insertion, it checks whether the key already exists (if there is a conflict, it is regenerated). The purpose of binding is to allow all subsequent operations to look up the corresponding remote session and status through the identifier, ensuring that each session's input method is managed independently and does not interfere with each other in multi-session scenarios. The binding is created when the identifier is generated and removed from the table after the process is closed and resources are released. Callbacks / events arriving later with the old identifier are discarded because the binding cannot be found.

[0078] Retrieve a list of online displays, and select a second display, distinct from the first display, as the target display.

[0079] The system calls the operating system's multi-display management interface to obtain a list of all online displays. For each display, the following information is extracted: display identifier (a unique integer assigned by the system), name, physical size, resolution, refresh rate, orientation (landscape / portrait / rotation angle), status (online / offline / sleep), whether it is the default display, whether it is an internal display, and connection method (internal / HD multimedia interface / video graphics array / Universal Serial Bus Type-C / wireless display). A display change listener is registered to update the list in real time, and a forced refresh is performed when a target display is selected to ensure the latest status is used.

[0080] The primary display screen identifier is determined by querying the display screen where the remote desktop client window handle is located, recorded, and marked as occupied, thus excluding it during target selection. In dual-screen folding devices, if the user manually drags the remote window to another screen, the identifier is dynamically queried based on the actual screen the window is located on, without hard-coding.

[0081] Target display selection strategy: Basic filtering (excluding the primary display, offline / screen-off display, mirrored display, and display occupied by high-priority applications); direct selection of a single candidate (for typical dual-screen devices, after excluding the primary display, only the second built-in display remains, which is selected directly); priority ranking of multiple candidates (priority 1: built-in display physically adjacent to the primary display; priority 2: external display of similar size; priority 3: user-specified screen (highest priority); priority 4: screen with the highest resolution and refresh rate; priority 5: screen closest to the primary display), selecting the highest-ranked screen; optional user confirmation dialog box (when there are multiple candidates, available screens are listed for the user to choose from, and the result is remembered as the default for the next time). After selection, the target display identifier is recorded in the bound object. Pre-check that the available display area is greater than the minimum display requirement of the input method (e.g., width ≥ 400 pixels, height ≥ 200 pixels), if not, the next candidate is selected.

[0082] In some embodiments, the step of activating an input agent containing native editable controls on a target display screen and passing a control request identifier and remote session information to the input agent includes: activating an input agent on the target display screen, the input agent containing native editable controls, the native editable controls maintaining focusability and editability; passing a control request identifier, remote session information, and a target display screen identifier to the input agent; and the input agent receiving and storing the control request identifier, remote session information, and target display screen identifier.

[0083] Launch an input agent on the target display screen. The input agent contains native editable controls that retain focusability and editability.

[0084] The architecture of the input proxy needs to resolve a core contradiction: it must have a real input target on the target screen that can gain focus from the input method to trigger its pop-up, yet it cannot display an actual text input box on the target screen (the text should be injected into the remote desktop, not displayed locally), and it must not interfere with the remote desktop session on the first screen or preempt the first screen's focus. Architectural implementation methods include: in-application window approach (the input proxy acts as an independent window within the remote desktop client application; this approach has low latency due to intra-process communication and is simple to implement, and is preferred); independent component / service approach (an independent thread or message loop within the same process); and independent application approach (an independent application communicates via inter-process communication; this is completely independent but involves communication latency and is complex to deploy).

[0085] When creating the input proxy window, the target display screen must be explicitly specified (either the initial display screen is specified in the creation parameters, or the window is moved to the target display screen's coordinate range after creation). Window properties are carefully designed: no title bar, no border, no system buttons; background is completely transparent or semi-transparent (almost invisible but still focusable); size is full-screen (covering the entire target screen area, facilitating focus on controls at any position and ensuring token validity) or small (a tiny window at the bottom only holds controls); it is not displayed in the recent tasks list / application switcher; it does not acquire control of the system navigation bar; the window type is a normal application window, not a system window (ensuring it can gain window focus and tokens). After window creation, it waits for a creation completion callback to confirm success; if it fails, it retryes up to 3 times with a delay; if it still fails, it notifies the client of a display error.

[0086] The native editable control is the standard text input control in the operating system. Initialization includes: creating an instance and adding it to the window's root view (ensuring it's attached to the view tree, a prerequisite for obtaining tokens and focus), requesting layout and redraw; setting properties—focusable to true, touch-mode focusable to true, enabled to true, editable to true, clickable to true, long-press clickable to true, setting the input type to text (supporting mixed Chinese, English, numbers, and symbols, dynamically adjusting to numbers / telephone numbers, etc., depending on the remote scenario), setting the input method options to not display the extracted text area, not use full-screen input mode, empty prompt text, empty initial text, and hidden cursor (to avoid the blinking cursor being seen in transparent windows); setting the size and position—using the minimized size (1×1 pixel placed in the upper left corner of the window). Transparent (background text and cursor are fully transparent, normal size but not visible), or moved out of the visible area (position set outside the window's visible area, still able to gain focus in the view hierarchy). Minimizing the size and placing it in a corner is recommended for simplicity and good compatibility. Register listeners—text change listener (captures three callbacks for text changes), key event listener (captures physical / soft keyboard function keys), focus change listener (captures gaining / losing focus), and editor action listener (captures Enter / Complete / Next). Verify that it is invisible but can be focused—call the focus request interface to check if it returns true, the focused property is true, the window token is not empty, and the visibility is visible (it must be visible even at a very small size; setting it to invisible / removed means it cannot be focused). Once the verification passes, it is ready.

[0087] Input agents run on a separate message loop thread (user interface thread) to ensure timely processing of window messages. Within the same application, they share the main thread; independent components run on their own threads. Input event handling involving time-consuming operations (encoding conversion, network transmission) is processed asynchronously on worker threads to avoid blocking the user interface thread and causing the input method to become unresponsive. Thread-safe data structures or inter-thread communication mechanisms are used to ensure multi-threaded consistency when communicating with remote desktop clients.

[0088] The control request identifier, remote session information, and target display identifier are passed to the input agent.

[0089] The parameters transmitted include: control request identifier (a unique identifier for this start request); remote session information (session identifier, remote computer address / name, login username, connection type, encryption protocol, etc., with the session identifier being the core for subsequent event verification); target display screen identifier (an integer identifier of the target screen on which the input agent should run, used for self-verification); input agent configuration parameters (native control input type, visibility, window transparency, size policy, display timeout, forwarding mode, etc., are optional); and communication channel configuration (event bus topic name, service name, local socket port, shared memory name, and other endpoint information).

[0090] Passing methods: In-process constructor parameter passing (passed through the constructor when creating an instance, stored in member variables; simple, direct, and thread-safe; preferred); In-process setter method parameter passing (passed through the setter method when reusing an instance; thread safety is ensured by using synchronization keywords or atomic references); Inter-process message parameter passing (initialization messages are sent via inter-process communication when the process / service is independent; the serialized format includes all parameters; after receiving, it is deserialized and stored; after sending, it waits for confirmation and resends after timeout); Persistent temporary storage parameter passing (optional; written to a temporary file or shared preferences; read after startup; has latency and is not preferred).

[0091] The input proxy defines member variables to store parameters: the current control request identifier (the `volatile` keyword ensures multi-threaded visibility), the remote session information object, the target display identifier, the configuration object, and a communication channel reference, protected by `volatile` or synchronization. It provides a parameter receiving method (or constructor to receive parameters), internally performing parameter validity checks (identifier must be non-zero / non-empty, session information must be non-empty and the session identifier must be non-empty, and the target display identifier must be valid), assigning values ​​to member variables, recording the receiving timestamp, sending a receiving confirmation via the communication channel, and returning an error code and logging if validation fails. Parameters remain unchanged throughout the input proxy's lifetime (unless the identifier is updated upon restarting), and are set to null upon shutdown to aid garbage collection. When handling input events, the identifier is read and attached to the event via a getter method; the getter method reads a `volatile` variable or reads within a synchronized block to ensure the latest value.

[0092] In some embodiments, verifying that the actual display screen identifier of the input agent matches the target display screen identifier includes: after the input agent is started, reading the actual display screen identifier of the display screen where the input agent window is actually located; comparing the actual display screen identifier with the target display screen identifier; when the actual display screen identifier matches the target display screen identifier, confirming that the input agent is running on the target display screen; when the actual display screen identifier does not match the target display screen identifier, ending the input agent instance and restarting it.

[0093] After the input agent is started, it reads the actual display screen identifier of the actual display screen where the input agent window is located.

[0094] Timing of data retrieval: Window creation is an asynchronous process. From calling the creation interface to actual display and layout completion, it takes 50ms to 500ms. Immediate retrieval may result in a default screen identifier or an invalid identifier, leading to misjudgments. Therefore, the following methods are chosen: Retrieve data in the window creation completion callback (after confirming the window has been attached to the window manager and completed the initial layout in callbacks such as window attachment and focus changes); Delayed retrieval (retrieve data after a 200ms to 500ms delay via the message loop, giving the window sufficient display time); Retrieve data in the layout completion listener (register a global layout listener or pre-draw listener, and retrieve data in the first layout completion callback, at which point the window position and size are determined). It is recommended to retrieve data in the callback with a fallback: attempt to retrieve data in the creation callback, and if an invalid identifier is encountered, retry after a 300ms delay.

[0095] Reading methods: Querying via window handle (obtaining the input proxy window handle, calling the system interface to query the current display screen, and obtaining the identifier; this is the most direct and accurate method); or querying via window coordinates (obtaining the screen coordinates of the top-left corner of the window, traversing all display screens to check which screen coordinate range the coordinates fall within, thus determining the current screen). After reading, a validity check is performed: the identifier is non-zero / non-empty, the identifier exists in the current online display screen list, and the identifier is not the identifier of the first display screen (further confirmation is needed if the identifier of the first display screen is read). If invalid, a delay and retry are performed.

[0096] Compare the actual display screen label with the target display screen label.

[0097] The actual display screen identifier read is precisely matched against the target display screen identifier pre-passed and stored in member variables (integers or string equality). The display screen identifier is a unique identifier assigned by the system; only a complete match indicates that the correct target screen is being used. Context information is recorded during the comparison for log debugging: actual identifier, target identifier, comparison result, read timestamp, current window position and size (for analyzing the cause of inconsistencies), and a snapshot of the online display screen list.

[0098] If the actual display screen identifier matches the target display screen identifier, confirm that the input agent is running on the target display screen.

[0099] When the input agent's internal status flag is set to "Display verification passed," it sends a verification pass notification (carrying a control request identifier, actual identifier, and pass timestamp) to the input method control module via the communication channel. The control module updates the status of this identifier in the binding table to "Display verification passed" and triggers the subsequent focus acquisition and input method display request process (step S102). The pass event is logged. After verification, the input agent is in a verified ready state, waiting to acquire focus, establish a connection, and request to display the input method. Verification is a prerequisite for all subsequent operations. Only by confirming that the input agent is on the correct screen can it be ensured that the input method that pops up later is also on the target screen and does not obstruct the first screen of remote display.

[0100] If the actual display screen identifier does not match the target display screen identifier, terminate the input agent instance and restart it.

[0101] Possible reasons for inconsistency: The target screen parameter specified when the window was created was not effective (some system versions have incomplete support for specified screen startup, and the window is placed on the default screen); the user manually dragged the window during the creation process; in a multi-screen environment, the system window routing policy routes the window to an unexpected screen (after hot-plugging an external monitor, the system reassigns the display, and the target identifier has become invalid or been remapped); in display mirroring mode, the two screens share content, and the window may logically be considered to be on the default screen; the system window manager automatically places the new window on the screen where the mouse cursor is located instead of the specified screen.

[0102] In case of inconsistency, execute the recovery process (at this time, the control has not yet gained focus or established a connection, so closing is relatively simple): send a notification to the client that the display verification failed and is in recovery (carrying the identifier, actual / target identifier, and reason for failure); cancel unfinished delayed tasks and timers; clear the focus if the control has gained focus; destroy the input agent window to release view resources; mark the instance as destroyed; and log the close.

[0103] Restart: Reconfirm the target display screen identifier (re-obtain the online list to confirm the target screen is still online and available; if offline, select the next available screen or prompt the user); recreate the input agent window on the target screen (try different creation parameters or specifying methods, such as directly specifying the screen using window parameters this time if the last activity failed to start, or immediately move it to the target screen's coordinate range using the window movement interface after creation); retransmit the control request identifier, remote session information, and target identifier; perform verification again after the new instance starts (read the actual identifier, compare, confirm, or restore).

[0104] Retry Limit and Degradation: A maximum of 3 retries are set, with an additional delay between each retry (immediate for the first, 200ms for the second, and 500ms for the third) to allow the window management service time to stabilize. If three consecutive retries fail to resolve the issue, the system is deemed unable to launch on the target screen (possibly due to system-level anomalies: target screen driver error, window management service crash, or multi-screen routing policy restrictions). Degradation is then implemented: the cross-screen solution is abandoned, and a fallback to compatibility mode is implemented (the input method is displayed on the first screen, which may partially obscure the remote screen, but the input function remains functional). An error message and operation suggestions are displayed to the user ("Unable to display the keyboard on the second screen; the system has switched to the first screen. Please check the second screen connection status or try restarting the application"), including a retry button and a confirmation switch button. Detailed error logs are recorded (actual / target identifier, online list, creation parameters, and system version for each retry). If the user selects to retry, the counter is reset and the process is repeated. If the user confirms using the first screen, the input agent is launched on the first screen (the target identifier is the first screen identifier; verification usually passes).

[0105] Continuous monitoring (optional enhancement): After successful verification, register a display change listener and a window position change listener. If the window is moved to another screen during input, immediately pause the input path, hide the input method, and attempt to move it back to the target screen. After moving it back, re-verify. If successful, redisplay the input method and open the path. If it cannot be moved back or verification fails, execute the shutdown process and prompt the user to avoid the input method suddenly popping up to the error screen or the state becoming chaotic after input is interrupted.

[0106] In some embodiments, the step of enabling the native editable control to gain focus and a window token, establishing an input connection through the native editable control, and requesting the display of the system input method includes: setting the input proxy window as the foreground window, enabling the native editable control to gain input focus; the native editable control obtaining a valid window token, establishing an input connection with the system input method service through the native editable control; and sending a display request to the system input method service through the input connection, requesting the system input method to be displayed on the target display screen.

[0107] Set the input agent window as the foreground window to give the native editable control input focus.

[0108] The foreground window is the currently active window interacting with the user, has the highest window stacking order, can receive keyboard touch input, and its contained controls can gain input focus. The input proxy window must become a foreground window for native controls to gain focus. Calling the window-foregrounding interface moves the window to the top of the stacking order, requests window focus, or sets a specific window flag to ensure visibility. Since the input proxy is located on the target screen and the remote desktop window is on the first screen, they can maintain foreground / active states on their respective displays without interference—the input proxy gains focus on the second screen to trigger the input method, while the remote desktop window continues to display the remote view on the first screen (screen rendering does not depend on the foreground state; rendering continues as long as the window is visible). If there can only be one foreground window globally, foregrounding the input proxy may cause the remote desktop window to lose its foreground state, but this does not affect screen display; focus can be switched back to the remote desktop window after input is complete. If window-foregrounding fails (due to system permission restrictions or abnormal window state), it will retry up to 3 times after a delay of 100ms to 300ms. If it continues to fail, it will attempt to minimize and then restore the window state to force refresh. If this still fails, the error will be logged and an attempt will be made to directly request control focus (some system controls requesting focus may indirectly cause the window to become foreground).

[0109] Once the window becomes the foreground window, it immediately calls the native control focus request interface to gain input focus. The returned boolean value indicates success or failure. The focus direction can also be specified, and other controls can be cleared of focus before the request. The control with focus receives input events from the user via input devices such as the soft keyboard, physical keyboard, or stylus. Its focus state changes to "focused," triggering a focus change callback. The input agent executes subsequent input connection establishment within the callback. Stable focus acquisition requires the following conditions: "Focusable" is true, "Focusable" is true in touch mode, the control is visible (even at very small sizes, it must be visible in the view hierarchy; setting it to invisible / removed prevents focus), it is attached to the window's view hierarchy, and the window has been created and displayed. If failure occurs, the cause is investigated and a retry is delayed. A focus change listener is registered for the control to monitor in real time. If focus is unexpectedly lost (the focus acquisition parameter in the callback is false and not actively closed), a re-request for focus is delayed. If multiple attempts fail to recover focus, an input closing process (focus loss timeout) is triggered to ensure stable input method connection and prevent input method hiding or input interruption due to focus loss during input.

[0110] The native editable control obtains a valid window token and establishes an input connection to the system input method service through the native editable control.

[0111] A window token is a unique token in the operating system's window management system that identifies the application / activity to which a window belongs. Its core function is to verify whether the system input method service's input method requests (show / hide, establish connection) originate from a valid, already displayed window—by checking if the window token carried in the request is valid (corresponding to the currently existing window). Only controls holding valid tokens have their input method requests accepted; requests from controls without valid tokens are rejected, and the input method will not pop up. Therefore, obtaining a valid token is a prerequisite for establishing an input connection. Tokens are assigned and associated with windows by the system when a window is created and attached to the window manager. Controls obtain tokens through the token acquisition interface after being attached to the window's view hierarchy. A non-empty token indicates validity (further verification of the token's existence is possible); invalid tokens wait for the window attachment callback to be re-acquired or are retried after a delay. The token not only identifies the application to which the window belongs but also implicitly associates the associated display screen—the input method service determines the window's display screen based on the token, thus displaying the input method on the corresponding screen. The input agent window is located on the target screen, and its token is associated with the target screen. Subsequent requests for the input method will then be displayed on the target screen; this is the key mechanism for cross-screen display. If the input agent is on the first screen, the input method will also be displayed on the first screen, obscuring the remote screen. This is why step S101, display verification, ensures the input agent is on the target screen. The input method service display routing logic in a multi-screen system may vary depending on the version: some systems strictly route according to the screen where the window is located, while others display the input method on the default screen / main display. In this case, the target screen needs to be explicitly specified through the system's multi-screen input method interface. Before sending a display request, the input agent checks whether the system supports displaying on the specified screen. If it does, it specifies the screen; otherwise, it relies on the implicit routing of the screen where the window is located (the default behavior in most systems).

[0112] After a control gains focus and a valid token, the system automatically triggers the establishment of an input connection: After verifying the token, the input method manager establishes an input connection channel between the application process and the input method service process, and returns a connection object. The application uses this object to request to show / hide the input method, pass editing context (selected text, cursor position), and receive text and key events submitted by the input method. The establishment is an asynchronous process (involving inter-process communication). After the connection object is created, it still needs to wait for the input method service to receive confirmation. Success is confirmed by listening to connection-related callbacks (full-screen mode reports and cursor updates indicate that the service has interacted through the connection) or by checking the input method activation status interface (returning true indicates that the connection has been established and the service has been associated with the control). A timeout timer is set from 500ms to 1000ms. After the timeout, the activation status is checked. If it fails (activation is still false after the timeout), the cause is investigated (invalid token, control not truly gaining focus, input method service abnormality, system input method disabled), and the focus and connection are re-requested. The maximum number of retries is 3. If it still fails, the shutdown process is triggered to prompt the user.

[0113] One of the core technologies of input proxy is to override the input connection creation callback of the control, returning a custom input connection wrapper class. The following methods are overridden to intercept input events generated by the input method: Text Submission Method (intercepts the confirmation text submitted by the input method, extracts the text content and forwards it to the event handling module, then calls the parent class method to submit the text normally to the control—since the control is invisible, the text is not displayed; not calling the parent class may cause abnormal internal state of the input method, so it is recommended to call it); Delete Surrounding Text Method (intercepts backspace / delete operations and converts them into control key events); Send Key Event Method (intercepts physical / soft keyboard function key events, extracts the key codes and actions and forwards them as control key events); Execute Editor Action Method (intercepts Enter / Complete / Next and converts them into corresponding control keys); End Combined Text Method (intercepts the combined text completion event to mark the end of the pre-editing stage). Through custom connections, the input proxy can intercept and capture all input events before the input method events reach the control, which is the most reliable way to receive them (it is more low-level than text change listeners and can capture key events and control operations).

[0114] When establishing a connection, configuration parameters can be passed to the input method service via control properties: input type (text-like / number-like / telephone-like / date / time-like, etc., affecting keyboard layout and candidate word strategy), input method options (disabling full-screen input mode, not displaying the extracted text area, displaying the next button, etc.), and private input method options (voice input switch, handwriting mode switch, and other private options supported by specific input methods). The input agent can dynamically adjust the input type according to the current scenario of the remote desktop: set it to text-like and disable autocorrection when opening the code editor; set it to number-like when opening the calculator; set it to text-like and disable suggestions when opening the terminal command line, improving the adaptability of cross-screen input experience.

[0115] Send a display request to the system input method service via the input connection, requesting the system input method to be displayed on the target screen.

[0116] After a successful input connection is established, a display request is sent to the input method service via the connection. The parameters are a display flag (implicit display / forced display / explicit display, etc.) and an input connection identifier. The display request is an asynchronous operation. After receiving it, the input method service needs to load, lay out, and render the input method interface, which typically takes 100ms to 500ms to actually display. The return value only indicates whether the request has been accepted, not whether it has actually been displayed—this is the key problem that this application aims to solve: a successful return of the display request cannot be used as the basis for judging whether it has been displayed; the actual display status must be monitored (following step S102). The display request may not actually be displayed due to reasons such as input method application loading / crash, insufficient system resources, token expiration, input method service display policy restrictions, or routing errors in multi-screen environments. Therefore, it is necessary to confirm the display status by monitoring it. When the request fails (returns false), check if the control holds a valid token and focus (if not, re-request), if the system input method is enabled (if not, prompt the user), and if the input agent window is on the target screen (if not, correct the position first). Retry 200ms up to 3 times. If it continues to fail, record the error and trigger the shutdown process with the message "Keyboard display failed, please check input method settings".

[0117] In some embodiments, the step of monitoring the actual display status of the input method on the target display screen, and confirming that the system input method is actually displayed on the target display screen and that the display status callback belongs to the current control request identifier, and then opening the remote input path, includes: registering a monitoring callback for changes in the input method window area on the target display screen and a monitoring callback for changes in the visibility of the input method; when a display status callback is received, extracting the control request identifier carried by the display status callback, and comparing the control request identifier carried by the display status callback with the current control request identifier; when the control request identifier carried by the display status callback is consistent with the current control request identifier, and the system input method window is visible on the target display screen, confirming that the system input method is actually displayed, and then opening the remote input path.

[0118] Register callbacks for listening to changes in the input method window area on the target display screen and for listening to changes in the input method visibility.

[0119] As mentioned earlier, a successful return of the display request does not mean that the input method has actually been displayed. It is necessary to register a listener callback to monitor the actual state of the input method window (position, size, visibility) in real time. Only when the window is actually present on the target screen (area is non-zero, visibility is visible) can it be confirmed that it has been displayed. The listener callback is based on the actual display state as the technical condition for opening the input path.

[0120] Input method window area change monitoring: Detection is achieved through a window decoration area (WindowInsets) change listener—when the input method pops up from the bottom, the visible height of the input proxy window decreases; when hidden, it reappears. The decoration area object is obtained in the decoration area change callback, and its visibility is determined through the input method area visibility judgment interface. The window area size is obtained through the input method area acquisition interface. This method is accurate and reliable. Alternatively, object position change and show / hide events can be registered through system-level event hooks. The callback checks whether the event window is the input method window (determined by the window class name) and obtains the position and size. Or, the input method window can be checked for visibility and position every 50ms to 100ms using a polling method.

[0121] Input method visibility change monitoring: In addition to determining the visibility of the decoration area, it can periodically poll the input method manager's activation status and text acceptance status interface (called every 100ms; activation returning true and text acceptance returning true usually indicate that it has been displayed and is accepting input); or listen for input method show / hide system notifications (sent by some systems); other systems can poll through the traditional input method interface's open status query or text service framework activity flag query.

[0122] Listener callbacks are associated with the target screen: During registration, ensure that the callback only focuses on the input method window state on the target screen. This is achieved by comparing the display identifier of the input method window with the target identifier in the callback, and only handling events on the target screen while ignoring other screens. The decoration area listener targets the input proxy window, and since the input proxy is located on the target screen, its input method decoration area naturally corresponds to the input method on the target screen (when it pops up at the bottom of the target screen, it affects the visible area of ​​the input proxy window on the target screen). System-level event hooks can restrict listening to specific process events through process parameters or filter the screen where the window is located in the callback.

[0123] Control Request Identifier and Callback Binding: Each display status callback must carry a corresponding control request identifier to distinguish callbacks from different start requests. Binding method: When the input agent registers a callback, it uses the current control request identifier as callback context data (held by a member variable of a custom listener class, or passed by a pointer to a callback data structure); when the callback is triggered, the identifier is read from the context and appended to the callback event and passed to the client; if the callback is a system-level global callback and cannot be directly bound, it is read from the input agent's currently active control request identifier member variable when triggered (the input agent only processes one identifier at a time, so reading the current value is sufficient).

[0124] When a display status callback is received, the control request identifier carried in the display status callback is extracted and compared with the current control request identifier.

[0125] When the listener callback is triggered, the input agent (or the client input method control module) receives the display status callback event, which includes: callback type (region change / visibility change / show / hide), input method window region information (position coordinates, width, height, only region change callbacks have this), input method window visibility (true / false, only visibility change callbacks have this), callback occurrence timestamp (milliseconds), control request identifier (read from the callback context or the current value of the input agent), and the display screen identifier where the input method window is located (optional).

[0126] Extract the control request identifier field from the callback event. If the callback does not explicitly carry it (system-level global callback), read it from the current activity identifier member variable of the input agent (the identifier is stored when the input agent starts and remains unchanged throughout its lifecycle unless it is restarted and updated).

[0127] The input method control module maintains the current control request identifier, which represents the identifier generated by the most recent enable request and is in an active (not closed) state. When enabling, a new identifier is generated and the current value is updated. When closing, it is set to an invalid value (0 or empty) and read from the current value.

[0128] The callback's carried identifier is precisely matched against the current identifier (integers or UUIDs): If they match, the callback belongs to the currently active input method session, is valid, and input method visibility confirmation continues; if they don't match, the callback belongs to a previous session (old session callback arrives late) or another session, is expired / invalid, the callback is discarded, and the input path is not opened based on it. A log is recorded (including the callback's carried identifier, the current identifier, the callback type, and a timestamp) for debugging. Identifier verification is a key mechanism to prevent old callbacks from mistakenly triggering the input path during rapid on / off switching: A typical scenario is that the user clicks to open (generating identifier 6), the input method starts loading but is not displayed, the user immediately clicks to hide (identifier 6 is cleared, the current identifier is set to 0), and then immediately clicks to open again (generating identifier 7). At this time, the old session's (identifier 6) display callback arrives late, carrying identifier 6, which does not match the current identifier 7, is discarded, and the path is not opened incorrectly; without identifier verification, when the old callback arrives, it is mistakenly considered to be displayed and the path is opened, but at this time the user has already closed the old input method and opened a new input method (the new input method may not yet be displayed), causing the path to open before the input method is actually displayed, and the user's touch is misinterpreted as an input event, causing confusion.

[0129] When the control request identifier carried by the status callback is consistent with the current control request identifier, and the system input method window is visible on the target display screen, confirm that the system input method is actually displayed and open the remote input channel.

[0130] Based on the successful identifier verification, confirming that the input method window is visible on the target display screen requires the following conditions to be met simultaneously: visibility is true (the callback visibility field is true, or the input method visibility judgment in the decoration area returns true, or the input method window visibility query returns true); the area is not zero (the window width is greater than the preset threshold, such as 50 pixels, and the height is greater than 50 pixels, excluding zero-size windows that have been created but not rendered; for decoration area methods, the bottom decoration is greater than 0, i.e., the input method height); the display screen is the target display screen (the display screen identifier of the window is equal to the target identifier, the screen is queried by the window position, or the screen where the input proxy window corresponding to the decoration area is located is the target screen); optional continuous visibility confirmation (the visible state is required to last for 100ms to 300ms, the timer is started when visibility is first detected, and if it is still visible after the expiration, it is confirmed; if it is hidden, it is considered as flickering and ignored to avoid misjudgment of brief flickering).

[0131] Once all confirmation conditions are met, the input method control module executes the following: setting the input method display status flag to "displayed"; recording the display confirmation timestamp; notifying the input agent that the input method has been confirmed and displayed via the communication channel, and the input agent enters the input forwarding ready state; opening the remote input channel; displaying a brief visual prompt ("The keyboard is displayed on the second screen" text or a small keyboard icon, which disappears automatically after 2 to 3 seconds) in the corner of the remote screen on the first display screen to inform the user that they can start inputting; and recording the confirmation event log (including control request identifier, confirmation time, input method window size, and target identifier).

[0132] Detailed Implementation of Remote Input Path Enabling: The remote input path is the channel for transmitting and injecting input events from the target screen system input method to the input agent (custom connection / listener capturing events), to the communication channel, to the remote desktop client input injection module, to the remote desktop protocol, and finally to the remote computer. Before the path is enabled, events captured by the input agent are discarded or temporarily stored without forwarding. After enabling, events are forwarded and injected into the remote session. Enabling Operation: The input method control module sets the path status flag to enabled and sends an enable input forwarding command (carrying a control request identifier) ​​to the input agent through the communication channel. After receiving the command, the input agent sets its internal forwarding flag to true and begins forwarding the captured confirmation text and control keys to the client through the communication channel. The client input injection module prepares to receive events (initializing the injection queue and resetting the injection status). Path enabling is a clear state switch. It is only executed when both conditions are met: "identifier verification passed + input method actual display confirmation." If either condition is not met, it will not be enabled. This gate mechanism ensures that input reception will not begin before the input method is actually displayed (avoiding misinterpretation of touch when it is not displayed), will not be enabled incorrectly when an old callback arrives, and will not be enabled when the input method pops up an error screen (avoiding injection in erroneous scenarios). Once the channel is opened, the events captured by the input agent immediately enter the confirmation text or control key processing flow, and after double verification, they are injected into the remote session, so that user input is applied to the remote computer in real time.

[0133] In some embodiments, receiving input events involves processing the combined confirmation text and control operation key events separately, including: receiving input events when the remote input channel is open; for input events with a pre-editing stage, extracting the combined and confirmed new text as confirmation text; and generating independent control operation key events for Enter, Backspace, and Tab control operations.

[0134] Receive input events when the remote input channel is open.

[0135] Input event reception is only performed when the input channel is open. Before the channel is open, the input agent may have captured events (such as accidental touches by the user during input method display), but these events are discarded or temporarily stored without processing or forwarding. After the channel is open, the input forwarding state is entered to formally receive and process the events. Event reception control based on the channel state ensures that events are only processed after the input method is correctly displayed and the user confirms that input can begin, avoiding state confusion.

[0136] Input events originate from user operations on the target screen system's input method and potentially connected physical keyboard key operations. They are divided into two main categories: confirmation text events (the final text content entered by the user through the input method, including Chinese, English, numbers, symbols, emoticons, etc.) and control operation key events (control function keys triggered by the user through the input method or physical keyboard, including Enter, Backspace, Tab, and expandable arrow keys, delete keys, function keys, combination keys, cancel keys, etc.). These events are received, processed, and injected into the input agent to ensure the correctness of text input and control operations.

[0137] The receiving channels include: custom input connection interception callbacks (the lowest level and most reliable, rewriting methods such as text submission, deleting surrounding text, sending key events, executing editor actions, and ending combined text, intercepting and capturing all events before the input method event reaches the control); native control text change listeners (capturing control text changes to indirectly obtain confirmation text, but cannot capture control keys that do not change the text, such as arrow keys); key event listeners (capturing physical keyboard or soft keyboard function key events, including control keys); and editor action listeners (capturing editor actions such as Enter, Finish, Next, etc.). Multiple channels can be used simultaneously, and events are deduplicated and merged (e.g., custom connection event text change listeners will not process them again), ensuring that each event is processed only once.

[0138] Each event is appended with: event reception timestamp (milliseconds), current control request identifier (read from member variable), current remote session identifier (read from member variable), event sequence number (an auto-incrementing integer used for sorting and deduplication), and event source channel (identifying which receiving channel it came from, used for debugging deduplication), which are passed to the client along with the event for subsequent verification, sorting, and deduplication.

[0139] For input events in input methods with a pre-editing stage, extract the newly added text that has been combined and confirmed as the confirmation text.

[0140] Detailed Explanation of the Pre-editing Stage: Input methods for pictographic or syllabic characters such as Chinese, Japanese, and Korean typically have a pre-editing stage. When a user inputs pinyin, the pinyin letters are first displayed in the input method's pre-editing area (or the control's combined text area). At this time, the text is not yet finalized, and the user can select candidate words, modify pinyin, or cancel input. When the user presses the space bar, enter key, or clicks a candidate word, the pre-edited text is replaced with the final text and submitted to the control. This final submitted text is called the combined and confirmed new text, i.e., the confirmation text. The text in the control during the pre-editing stage is temporarily modifiable and should not be injected into the remote desktop (otherwise, the remote computer will receive unconfirmed pinyin letters, causing input chaos). Only the combined and confirmed text should be injected—it is crucial to distinguish between pre-edited text and confirmation text, and only inject the confirmation text into the remote session. This is a key challenge in cross-screen input method processing.

[0141] Confirmation of text extraction method: Compare the content of the control before and after the text change (in each text change callback, compare the full text beforeText and afterText before the change, calculate the newly added substring (the appended part at the end), and check whether the newly added substring contains non-Latin characters (Chinese / Japanese / Korean, etc.) - if it contains them, it means that one combination input has been completed (pinyin is replaced with text), and the newly added substring is the confirmation text; if it only contains Latin letters (pinyin), it means that it is still in the pre-editing stage and will not be processed yet; at the same time, check whether the pre-editing area has been cleared (judged by the input connection combination text state or the control selected text state), if it is not empty, the combination is not completed and will not be confirmed yet); listen for the submission event (some input methods trigger a specific submission event when submitting the confirmation text). This means that when the text submission method of the custom input connection is called, the input agent directly obtains the submitted text content from the callback parameter, which is the most accurate extraction method; tracking the state of the pre-edited area (the input agent maintains the state of the text content in the pre-edited area. When the text in the pre-edited area changes from non-empty to empty and the corresponding content is added to the control text, it indicates that a combined input is completed. The added text is the confirmation text. The state of the pre-edited area can be obtained through the input connection combined text range query interface); for inputs without a pre-editing stage, such as English (when the user clicks the letter key on the soft keyboard, the letter is directly submitted to the control without a pre-editing stage. Each time a new letter is added, it is the confirmation text. The text change listener or the text submission method of the custom connection is called each time the submitted text is directly extracted as the confirmation text).

[0142] Confirmation text content processing: Encoding conversion (converting the text to a character encoding supported by the remote desktop protocol, such as UTF-8 or UTF-16, ensuring correct transmission of Chinese characters, emoticons, and special symbols; paying attention to handling multi-byte characters in proxy pairs); Length check (checking the length of the confirmation text; if it exceeds the maximum single input length limit of the remote desktop protocol, such as 256 characters, splitting it into multiple injections, each not exceeding the limit; when splitting, avoid splitting in the middle of multi-byte characters); Invisible character processing (invisible characters such as newlines and tabs are converted into corresponding control operation key events based on semantics (newline → Enter key event, tab → Tab key event) instead of text injection, because some applications on the remote computer may have special processing for control characters (such as executing commands instead of inserting text using newlines in terminal windows). After conversion, invisible characters in the original confirmation text are removed, and the remaining visible text is injected as confirmation text. Control key events corresponding to invisible characters are generated and injected separately); Text temporary storage (temporarily storing the confirmation text in the input event queue, waiting for verification before injection; the queue uses a first-in-first-out order to ensure injection according to the user's input order to avoid disorder).

[0143] For Enter, Backspace, and Tab control operations, generate independent control operation key events.

[0144] Control operation keys are keys that users input via input methods or physical keyboards, which do not produce displayable text but perform specific control functions. This application focuses on three core types of control keys: Enter key (commonly used in remote computer applications to confirm input, execute commands, and add newlines; the key code varies across different systems but the semantics are consistent); Backspace key (deletes the character before the cursor; unlike the Delete key—Backspace deletes the character before the cursor, Delete deletes the character after the cursor); Tab key (used to switch focus between input controls, insert tabs, autocomplete, etc.; commonly used for autocomplete or indentation in remote desktop terminal windows or code editors). Extended processing includes: Arrow keys (up, down, left, right arrows, to move the cursor or select text); Delete key (deletes the character after the cursor); Function keys (F1 to F12, triggering application-specific functions); Combination keys (control keys plus letter keys to perform copy / paste / undo / select all editing operations, Shift key plus letter keys to input uppercase or symbols, Alternate key plus letter keys to execute menu shortcuts, etc.); Cancel key (cancels or closes dialog boxes); Escape key (cancels the current operation or closes the menu). The expanded control button processing enhances the completeness of remote desktop operations, allowing users to not only input text but also perform common editing operations via a second-screen input method or physical keyboard.

[0145] Control operation key event generation: When the input agent detects a control key in the key event listener callback or the key event sending method of the custom input connection, it generates an independent event, which includes: key code (the key identifier defined by the operating system, such as Enter / Backspace / Tab key code, which is a system-level identifier and may be different in different systems. It is mapped to the remote desktop protocol key code during subsequent injection); key action (press or release, usually only pressing or pressing and releasing are processed to simulate real key presses. The real key operation sequence is to press first and then release. The remote desktop protocol usually needs to send pressing and releasing separately to correctly simulate it); modifier key status (whether the Shift key, Control key, Alternate key, and Function key (Windows key / Command key) are pressed at the same time, used to identify combination keys, and represented in bitmask form, each bit corresponds to a modifier key press state); key character (key corresponds to Unicode character, if present, such as Enter corresponding to newline character, Tab corresponding to tab character, some control keys have no printable characters, so the character field is empty); event timestamp, control request identifier, remote session identifier, and event sequence number. Control button events and confirmation text events are stored in different queues (or in a unified queue, distinguished by an event type field) for easy processing.

[0146] The significance of separating control operation keys and confirmation text: Different injection methods (confirmation text is typically injected in batches via the remote desktop protocol text injection channel, which is efficient and can correctly handle multi-byte characters such as Chinese characters; control keys are typically injected one key at a time via the remote desktop protocol keyboard event channel, simulating real key presses and releases to ensure the application correctly recognizes key actions such as shortcuts and consecutive backspaces); different processing logic (confirmation text requires handling encoding, length splitting, and invisible character conversion; control keys require handling key code mapping, modifier key combinations, and key repetition rate); error isolation (text injection failure does not affect key injection, and vice versa; separate processing improves system robustness); and consistent user experience (separate processing ensures that both text input and control operations work correctly on the remote computer, providing a consistent keyboard experience with the local computer).

[0147] In some embodiments, the step of injecting the control request identifier and remote session information into the remote desktop session after verification includes: extracting the control request identifier and remote session identifier carried by the input event; comparing the control request identifier carried by the input event with the current control request identifier, and comparing the remote session identifier carried by the input event with the current remote session information; when both the control request identifier and the remote session identifier match, injecting the confirmation text or control operation key event into the current remote desktop session; and discarding the input event when either identifier does not match.

[0148] Extract the control request identifier and remote session identifier carried by the input event.

[0149] After receiving the input event forwarded by the input agent through the communication channel, the remote desktop client input injection module first parses the event and extracts each field: event type (confirmation text or control key), event content (the text content of the confirmation text, or the key code / action / modifier key state / character of the control key), control request identifier, remote session identifier, event timestamp, and event sequence number. The parsed data is stored as a structured object for easy subsequent validation and injection. The control request identifier is appended when the input agent receives the event (read from the current identifier in the member variable), indicating which start request the event belongs to; the remote session identifier is appended when the input agent receives the event (read from the session identifier in the current remote session information in the member variable), indicating which remote desktop session the event target is.

[0150] Compare the control request identifier carried by the input event with the current control request identifier, and compare the remote session identifier carried by the input event with the current remote session information.

[0151] The current control request identifier is read from the input method control module (the activity identifier generated by the most recent start request; updated when starting, and set to an invalid value of 0 or empty when closing). The event, carrying the control request identifier, is precisely matched against the current control request identifier (integers or UUIDs): Match – the event belongs to the current input method session, and the control request identifier verification passes (time dimension correct); No match – the event belongs to an old session or another session, and verification fails.

[0152] The current remote session information is read from the session management module (the remote session information currently displayed on the first screen and active), and the remote session identifier is extracted from it. The event carrying the remote session identifier is precisely matched and compared with the current remote session identifier (string equality or number equality): Match - the event target is the current remote session, and the remote session identifier verification passes (spatial dimension is correct); No match - the event may come from another remote session input agent, and the verification fails.

[0153] Double validation is a logical AND relationship; both validations must pass before injection can continue: the control request identifier validation ensures that the event belongs to the current input method session (time dimension correctness), and the remote session identifier validation ensures that the event target is the current remote desktop session (spatial dimension correctness). The two dimensions together ensure the correctness and security of input events, avoiding event crosstalk in multi-session and multi-round opening scenarios.

[0154] When both the control request identifier and the remote session identifier match, an acknowledgment text or control operation key event will be injected into the current remote desktop session.

[0155] After both verifications pass, the input injection module distributes the event to the corresponding injection process based on the event type: confirmation text events are distributed to the text injection process, and control key events are distributed to the key injection process. The two processes are executed independently without interfering with each other.

[0156] Confirmation text injection detailed process: Encoding conversion (converting the confirmation text to a character encoding supported by the remote desktop protocol; remote desktop protocols typically support Unicode character injection using UTF-16, converting the text from local encoding such as UTF-8 to UTF-16 to ensure correct transmission of Chinese characters, emoticons, and special symbols; pay attention to handling multi-byte characters of proxy pairs); Injection method selection (selecting based on text length and protocol support: short texts such as a single Chinese character or a few letters use character sequence injection, converting the text character by character to Unicode code points and sending it through the remote desktop protocol keyboard event channel; good compatibility but lower efficiency; long texts such as a sentence or a paragraph use clipboard injection, writing the text to the remote clipboard (sent through the remote desktop protocol clipboard virtual channel)). Send the text via the paste shortcut (control key + V). The remote computer's currently active window will paste the text, which is highly efficient and can correctly handle special characters, but requires clipboard redirection support. Some remote desktop protocols provide dedicated text injection channels to directly send text content and automatically insert it on the remote end, which is the most efficient. Injection execution (the text is sent to the remote computer through the corresponding channel of the remote desktop protocol according to the selected method. The remote session identifier is carried when sending to ensure that the injection is performed in the correct session). Injection result confirmation (the remote desktop protocol usually returns the injection result as successful / failed, or indirectly verifies it through changes in the remote screen (such as the input text appearing on the remote screen). The input injection module records the injection result (success / failure, injection time, text content summary) for log debugging.

[0157] Detailed process for injecting control operation keys: Key code mapping (converting local operating system key codes to scan codes or virtual key codes defined by the remote desktop protocol. Different systems have different key code definitions (e.g., Android key codes differ from Windows virtual key codes), requiring conversion via a mapping table. The mapping table contains the correspondence between local key codes and remote protocol key codes, covering all commonly used keys (letters, numbers, symbols, function keys, control keys, arrow keys, etc.). The mapping table has explicit entries for the three core control keys: Enter, Backspace, and Tab.); Modifier key state setting (setting the corresponding flag bit in the remote desktop protocol keyboard event based on the modifier key state (whether the up / control / alternate / function key is pressed). Protocol keyboard events typically include a modifier key state field, represented by a bitmask, where each bit corresponds to one modifier key); Key press event sending (sending the KeyDown key press event via the remote desktop protocol keyboard event channel, including the mapped key code, modifier key state, timestamp, and remote session identifier); Short delay (simulating the actual key press duration, inserting a short delay of 10ms to 50ms between the press and release events). The duration should not be too long (to avoid repeated key presses) nor too short (to avoid the remote computer not recognizing the key press as valid). Physical keyboard key press events can use the original press duration, while soft keyboard key presses use the default delay. Key release event sending (sends a KeyUp event after the delay, including the mapped key code, modifier key state (the modifier key state may have changed at release, so the latest state is used), and timestamp); Combination key injection sequence (for combination keys such as Control key + C for copy, the injection sequence is: first send the modifier key (Control key) press event → short delay → send the primary key (C key) press event). Event → Brief delay → Send primary key (C key) release event → Brief delay → Send modifier key (control key) release event, simulating the user's actual key combination operation sequence to ensure that the remote computer application correctly recognizes the key combination and executes the corresponding function, such as copy; multiple modifier key combinations, such as control key plus shift key plus A, send all modifier key presses in sequence, then send primary key press and release, and finally send all modifier key releases in sequence; injection result confirmation (similar to text injection, record key injection results (success / failure, time, key code, modifier key status) for log debugging, and limit retries or exception handling if it fails).

[0158] Remote computer response after injection: After injection, the remote computer's operating system receives keyboard events or text insertion operations and distributes them to the currently active window (the application with input focus). The application then performs the corresponding operation: confirming that the text has been inserted at the current cursor position (displaying the input text in word processing software, displaying the input command in a terminal window, and displaying the input code in a code editor); and controlling the keys to execute corresponding functions (pressing Enter to confirm input or execute a command, backspace to delete the character before the cursor, using a tab to switch focus or autocomplete, and using shortcut keys to perform copy, paste, or undo, etc.). The user sees the input effect on the first screen of the remote desktop, achieving a cross-screen input experience where input is done on the second screen and viewed on the first.

[0159] The input event is discarded if any identifier does not match.

[0160] When the control request identifier or the remote session identifier does not match (either), the input injection module discards the event and does not perform injection. It logs the event (including event type, content summary, control request identifier, current control request identifier, remote session identifier, current remote session identifier, reason for discard, and timestamp) for debugging and troubleshooting. The discarded event does not enter the injection queue and has no effect on the remote desktop session.

[0161] Typical scenarios for event discarding include: delayed arrival of old session events (when a user quickly closes and reopens the input method, the old session's input event arrives late, carrying an old identifier that doesn't match the current identifier, and is therefore discarded); multi-session crosstalk (a user connects to multiple remote computers simultaneously, each session has an independent input agent, and an input event from one session might be sent to another session's input injection module due to a communication channel error, carrying a remote session identifier that doesn't match the current session's identifier, and is therefore discarded); abnormal restart of the input agent (after an abnormal restart of the input agent, the control request identifier is updated, and old events that were present before the restart arrive, carrying the old identifier, are discarded); and communication channel errors (communication channel data errors cause the event identifier field to be tampered with, resulting in an inconsistency with the current identifier, and the event is discarded to avoid incorrect injection). The discarding mechanism ensures that only events belonging to the current input method session and targeting the current remote session are injected, guaranteeing input correctness and security.

[0162] State retention after discarding: After a discarded event, the input injection module's state remains unchanged (the injection queue is not updated, and the current and session identifiers are not changed). It continues to receive subsequent events. The discarding operation is idempotent; discarding the same event multiple times does not produce side effects. If multiple events are discarded consecutively (e.g., control request identifiers continue to mismatch), the input injection module can trigger an alarm (notifying the input method control module to check if the current state is abnormal, such as whether the input proxy is closed but the communication channel is still sending events), facilitating timely detection and recovery from abnormal states.

[0163] In some embodiments, responding to the input closing condition, stopping receiving input events, hiding the system input method and ending the input agent, and releasing the relevant window and input connection includes: detecting the input closing condition, which includes a user hide command, a user close command, an open timeout, the end of the remote desktop session, application destruction, the target display being turned off, or the target display being removed; responding to the input closing condition, stopping receiving new input events, sending a hide request to the system input method service through the input connection to hide the system input method; ending the input agent, releasing the input agent window, native editable controls, and input connection, and clearing the binding relationship between the control request identifier and the remote session information.

[0164] Detect input closing conditions, including user-hidden commands, user-closed commands, open timeout, remote desktop session termination, application destruction, target display screen being turned off, or target display screen being removed.

[0165] Input method closure conditions are events or states that trigger the end of a cross-screen input method session, requiring the input method to be closed and related resources released. The closure conditions defined in this application are divided into several categories: user-initiated closure (user-hidden command, user-close command) – the user actively requests closure; abnormal failure (open timeout) – the input method fails to display within the expected time, triggering closure; session end (remote desktop session end, application destruction) – the remote session or the application itself ends, rendering the input method meaningless, triggering closure; and display change (target display off, target display removed) – the target display becomes unusable, preventing the input method from continuing to display on the target screen, triggering closure. These conditions cover all scenarios that may lead to the input method needing to be closed, including user-initiated operations, abnormal failures, session lifecycle changes, and hardware state changes.

[0166] Detailed Explanation and Detection Methods for Each Closure Condition: User Hide Command – The user actively clicks the hide keyboard button on the remote desktop client interface, clicks the hide / collapse button on the input method interface, or triggers hiding via gestures (such as swiping down the input method area). Detection methods include listening to the hide button click event, listening to the input method hide notification (system-sent hide broadcast or callback), and listening to the input method interface collapse button click (via touch event in the input agent window or input method hide callback). User Closure Command – The user actively disables the cross-screen input method switch in settings, switches back to single-screen mode, or exits cross-screen input mode. Detection methods include listening to changes in the cross-screen input method switch status in settings and listening to display mode switching events (from dual-screen to single-screen). Switching to single-screen mode), listening for user actions that disable the input method function; Opening timeout—If the input method is not actually displayed within a preset timeout period (configurable from 2 to 5 seconds) starting from the timeout period after sending the input method display request (no callback with the same identifier and actual display is received), it is considered a display failure and triggers closure. The detection method is to start a timeout timer when sending the display request, and check whether it has actually been displayed when it expires. If it is not displayed, the opening timeout closure condition is triggered; Remote desktop session termination—The remote desktop connection is disconnected, including user-initiated disconnection, remote computer shutdown / restart, network interruption causing connection timeout, remote session being logged out by the remote computer, etc. The detection method is to listen for remote desktop client connection status change events (connection termination, etc.). (Enable callback), monitor network status changes (remote connection usually drops when network is disconnected), periodically check remote connection activity (if heartbeat packet times out and no response is received, it is determined to be disconnected); application destruction—the remote desktop client application is closed by the user (removed from the recent tasks list), reclaimed by the system due to low memory (process is killed), or the application lifecycle ends by calling the destruction interface. The detection method is through application lifecycle callbacks (destruction callback, stop callback) and through process survival detection (the system calls back relevant lifecycle methods before the process is killed); target display screen shutdown—the target display screen (second screen) is turned off by the user, such as the second screen turning off after a foldable screen is folded, or the external monitor being turned off by the user. Power supply and target display entering screen-off / sleep state are detected by registering a display status change listener (screen-on / screen-off notification) which is triggered when the target screen changes to screen-off / sleep state, and periodically querying the target screen status (online and screen-on status via the display management interface); Target display removal—the target display is removed from the online display list, such as when an external monitor is hot-plugged and disconnected (connection cable unplugged), when the second screen of a foldable screen is marked as removed by the system after folding (no longer exists as an independent display), or when a wireless display disconnects. This is detected by registering a display connection / disconnection listener which is triggered when the target screen disappears from the online list, and periodically retrieving the online display list and checking whether the target screen is still in the list.

[0167] Closing condition priority and conflict handling: When multiple closing conditions are triggered simultaneously (such as a user hide command and target screen closing occurring at the same time), they are handled according to priority: user-initiated closing (user hide / close command) has the highest priority (explicit user intent); exception failure (open timeout) is next; session termination (remote session termination / application destruction) is next; and display change (target screen closing / removal) has the lowest priority. The closing process is executed after a high-priority condition is triggered. Low-priority conditions may have already been processed in the closing process (resources released) and will not be triggered again. Conflict handling ensures that the closing process is not repeatedly executed when multiple conditions occur simultaneously, preventing exceptions.

[0168] In response to the input closing condition, stop receiving new input events, send a hide request to the system input method service through the input connection, and hide the system input method.

[0169] Upon detecting the shutdown condition, immediately execute the first step: Stop receiving new input events—set the remote input channel status to closed; notify the input agent to stop receiving new events (the input agent enters a closed state, no longer processing new text changes or key event callbacks, or processing them and then discarding them without forwarding); clear the input event queue (discard any pending events that have not yet been injected; for received but not injected confirmation text, injection is determined based on the shutdown reason: if the user actively hides it, it is not injected (the user may not want to retain incomplete input); if the remote session ends, it cannot be injected (the connection has been broken); if the opening timeout occurs, the input method may not display any valid events in the queue, and by default, the queue is cleared and residual events are not injected); cancel all incomplete timers (display timeout timers, focus loss timeout timers, persistent visibility confirmation timers, etc.) to prevent timers from triggering during or after shutdown, causing abnormal states. Stopping the reception of input events is the first step in the shutdown process, ensuring that no new events are generated or injected during shutdown to avoid state chaos.

[0170] Step 2: Hide the system input method – After stopping receiving events, the input agent sends a hide request to the input method service via the input connection, requesting to hide the system input method. The parameters include a hide flag (e.g., implicit hide only, not always hide, etc., controlling the hiding behavior) and an input connection identifier to ensure the hide request is associated with the correct connection. Upon receiving the request, the input method service executes the input method interface hiding animation (e.g., swipe down, fade out) and destroys it (releasing input method interface resources). The hide request return value only indicates whether the request was accepted, not whether it has actually been hidden. Confirmation of hiding is required via a display status listener callback (receiving a hide callback and the input method window being invisible, or the input method visibility check in the decorative area returning false). If no confirmation to hide is received within a preset time (1 to 2 seconds), a forced close is executed: the input connection is forcibly disconnected (the input connection disconnection interface is called to notify the input method service to release all resources related to the connection, including the input method interface); if the input method is still not hidden after disconnection (a very rare exception), the current input method is forcibly hidden through the system-level input method management interface (e.g., calling the input method manager's hide interface and passing in the current control); a forced close event log is recorded (including the reason for closing, the time the hide request was sent, the confirmation timeout, and the forced close method). The hide confirmation and forced close ensure that the input method interface is completely hidden and does not remain on the target display screen. After the system input method is hidden, the target display screen regains its full visible area (no longer occupied by the soft keyboard), allowing the user to perform other operations on the target screen. Simultaneously, the input method service releases input method interface resources (memory, graphics buffer, etc.) to reduce system resource consumption.

[0171] End the input agent, release the input agent window, native editable controls and input connection, and clear the binding relationship between the control request identifier and remote session information.

[0172] Step 3: End the input proxy instance – End the input proxy after the input method is hidden: Clear the focus of native editable controls (call the control clear focus interface to make the control no longer hold input focus, avoiding focus residue that prevents other controls from gaining focus); Destroy the input proxy window (call the window destruction interface to release all view resources in the window view hierarchy, including native editable controls, all listeners, and all subviews; after the window is destroyed, it is removed from the window manager and no longer displayed, and the window token becomes invalid); Mark the input proxy instance as destroyed (set the internal state flag to destroyed, rejecting any subsequent method calls to avoid operations on destroyed instances that may cause exceptions); If the input proxy is an independent activity or service, call the end interface to end its lifecycle (remove it from the activity stack or service manager).

[0173] Step 4: Release all related resources – Native editable control resources: Controls are destroyed along with the window, but all listeners must be explicitly unregistered (text change listeners, key event listeners, focus change listeners, editor action listeners, etc.; remove listeners from the control to prevent listeners from holding control references, thus preventing the control from being garbage collected), clear the control's text content (set it to null to release text buffer resources), clear the control's custom input connection (if a custom connection wrapper class is used, disconnect it from the input method service and release the connection object), and set the control reference to null (to help the garbage collector identify recyclables); Input connection release: Input connections should be explicitly released after the window is destroyed – end the current combined text (if there is unfinished pre-edited text, call the input connection end combined text interface to notify the input method service to end the combined state to prevent the service from maintaining the combined state and causing resource leaks), disconnect the input connection (call the input connection disconnect interface to notify the input method service to release all resources related to the connection, including the input method session state, candidate word cache, editing context, etc.; after disconnection, the connection object is no longer valid, and any requests through this connection are rejected). 1) Release input connection objects (set references to null to aid garbage collection). Input connection release may be automatically completed when the window is destroyed in some systems, but explicit release ensures timely release of resources and avoids the input method service remaining in an invalid connection state. Other related resources: Close communication channels (close the communication channel between the input agent and the remote desktop client, such as closing local sockets, unregistering event bus subscriptions, unbinding service connections, closing shared memory, etc. to avoid communication channel resource leaks); Cancel unfinished asynchronous tasks (cancel all unfinished asynchronous tasks such as display request retries, injection retries, display verification retries, etc., to avoid tasks continuing to execute after component destruction, leading to null pointer exceptions or state exceptions. Use a task cancellation flag when canceling to ensure that the task can respond to the cancellation request); Release temporary data structures (clear the input event queue, clear cached data, release temporary buffers, etc.); Unregister system-level listeners (if system-level listeners such as display change listeners, network status listeners, and system broadcast receivers are registered, they must be explicitly unregistered to avoid the system maintaining references to destroyed components, leading to memory leaks—this is one of the most common sources of memory leaks).

[0174] Step 5: Clear the binding relationship between the control request identifier and remote session information—After resource release, the input method control module clears the binding relationship between the current control request identifier and remote session information: Remove the entry corresponding to the current control request identifier from the session binding table (delete the key-value pair in the hash table to release the memory occupied by the binding information object); Set the current control request identifier to an invalid value (such as 0 or empty, indicating that there is currently no active input method session); Mark the input association status of the current remote session information as no active input method (update the input method status field in the remote session information to indicate that the remote session currently has no associated cross-screen input method session); If the control request identifier is an auto-incrementing sequence number, retain the counter value and do not reset it (ensure that the subsequently generated identifier is not duplicated to avoid the identifier wrapping around and conflicting with the old identifier); Clear the current target display identifier (set to an invalid value to indicate that no target display is currently selected). After clearing the binding relationship, the old control request identifier is no longer associated with any remote session, and subsequent callbacks / events carrying the old identifier will be discarded by the verification mechanism because the binding cannot be found, and will not interfere with the new input method session—this is a key step to prevent residual input connections after closing from causing the next inability to open, by thoroughly clearing the binding and releasing resources to ensure that each time the input method is opened, it is in a completely new and clean state.

[0175] Step 6: Closure Completion Confirmation and State Reset – After all closure and release operations are completed, the input method control module performs a closure completion confirmation: It checks each resource to ensure it has been released (window destroyed, input connection disconnected, listener logged off, communication channel closed, binding cleared, current identifier invalidated) to ensure nothing is missed; it resets the internal state of the input method control module to its initial state (waiting for input method activation request, all status flags restored to default values); it notifies the remote desktop client to update the interface (e.g., hiding the keyboard activation visual cue, restoring the keyboard activation button to clickable status, clearing the cross-screen input mode interface identifier); it records the closure event log (closure reason, closure time, resource release result (all successful or partial failure and failure items), closure process time, etc., for debugging and performance analysis). If the closure reason is recoverable (user actively hides rather than remote session termination / application destruction), the remote desktop session remains displayed and the user can request activation again (a new identifier is generated, and steps S101 to S103 are re-executed); if it is not recoverable (remote session termination, application destruction), the remote desktop client terminates or enters a disconnected state accordingly.

[0176] The shutdown process is designed to be idempotent (multiple shutdown operations will not cause errors or resource leaks). Null value checks and status checks are added to each step (e.g., skipping the disconnection operation if the input connection is broken, skipping the destruction operation if the window is destroyed, skipping the unregistering operation if the listener has been unregistered, and skipping the clearing operation if the binding has been cleared). This avoids null pointer exceptions or illegal state exceptions when repeatedly closing or partially closing and then closing again. Furthermore, critical steps in the shutdown process are wrapped in exception handling. Even if an exception is thrown at a certain step (e.g., an input method service failure causing a hidden request to fail, or a system exception thrown when the window is destroyed), the entire shutdown process will not be interrupted. Subsequent resource release steps continue to execute, ensuring that as many resources as possible are released to avoid overall resource leaks due to exceptions in a single step. Idempotency and exception protection ensure the robustness of the shutdown process, maximizing resource release under various abnormal scenarios.

[0177] After step S103 and Example 8 are completed, the input events have been correctly classified, verified and injected into the remote desktop session, the input closing condition has been detected and responded to, the input event reception has stopped, the system input method has been hidden, the input agent has ended, resources such as windows, input connections, listeners, and communication channels have been released, the control request identifier and the binding of remote session information have been cleared, the system has returned to the initial state of waiting for the next input method opening request, the entire cross-screen input method session cycle has ended completely without resource residue, and the next session can be started at any time.

[0178] In some embodiments, this embodiment provides a complete end-to-end comprehensive description of all technical content using a typical use case—a user writing a document on a foldable dual-screen tablet computer via remote desktop control of an office computer—covering the entire process from establishing a remote desktop connection to enabling, inputting, and disabling the cross-screen input method.

[0179] Scenario 1: Establishing a remote desktop connection and displaying the first screen: The user holds a foldable dual-screen tablet (with the left and right screens serving as the first and second displays, respectively) and opens the remote desktop client application. After the application launches, the user selects "Office Computer" from the saved connection list and clicks connect. The remote desktop client establishes a network connection with the office computer via the home Wi-Fi: the connection is successfully established after steps including domain name resolution, transmission control protocol connection establishment, transport layer security encryption handshake, user authentication, and remote desktop session negotiation. The office computer's desktop screen is transmitted to the tablet after video encoding and compression. The remote desktop client decodes the video and displays it full-screen on the first display (left screen), allowing the user to see the complete desktop of the office computer, including any open word processing software windows. The session management module records remote session information (session identifier "office-pc-session-123", remote computer name, login username, connection establishment time, negotiated display resolution, etc.).

[0180] Scenario 2: Input method activation request and control request identifier generation: The user needs to input a piece of Chinese text in the word processing software on their office computer and click the floating keyboard icon button on the remote desktop client interface to trigger an input method activation request. Upon receiving the request, the input method control module immediately generates a control request identifier (an auto-incrementing serial number; currently, it's the 5th activation, so the identifier is 5). This identifier is then bound to the current remote session information (session identifier "office-pc-session-123") and stored in the session binding table. The binding entry records identifier 5, session identifier, request status "pending activation," and request timestamp.

[0181] Scenario 3: Target Display Screen Selection and Input Agent Startup: The input method control module calls the system's multi-display management interface to obtain the list of online displays: display identifier 0 is the left screen (the first display is showing the remote desktop), and identifier 1 is the right screen (the second display is currently showing the tablet's main screen). Confirming that the first display, identifier 0, is already occupied, the second display (identified as 1), distinct from the first display, is selected as the target display. After selection, the input agent is started on the target display (right screen): a full-screen window (normal application window type) with no title bar, no border, and a semi-transparent background is created. A native editable control is created within the window (focusable, touch-sensitive, editable, enabled, text input type, 1×1 pixel size, placed in the upper left corner of the window—invisible but focusable). Text change listeners, key event listeners, focus change listeners, and editor action listeners are registered for the control. The control input connection is overridden to create a callback to return a custom input connection wrapper class for intercepting input events. After the input agent starts, the input method control module passes control request identifier 5, remote session information (session identifier "office-pc-session-123"), and target display identifier 1 to the input agent, which stores these in member variables.

[0182] Scenario 4: Display Screen Label Verification: After the input agent window is created and displayed, wait 300 milliseconds, call the system interface to read the actual display screen identifier of the window, and query through the window handle to find that the actual identifier is 1. Compare the actual identifier 1 with the target identifier 1. If the result is consistent, it is confirmed that the input agent is running on the target display screen. Then, send a verification pass notification (carrying identifier 5) to the input method control module through the communication channel. The control module updates the status of identifier 5 in the binding table to display screen verification pass.

[0183] Scenario 5: Focus acquisition, window token and input connection establishment: After successful verification, the input agent sets the window as the foreground window, calls the native control focus request interface, and the control successfully gains input focus, triggering the focus change listener. Once the control gains focus, it obtains a valid window token (not empty) through the token acquisition interface. The system input method manager detects that the control has gained focus and holds a valid token, automatically triggering the input connection establishment: a custom input connection object is created through the control input connection creation callback, establishing a connection channel between the application process and the input method service process. The input agent confirms successful connection establishment by listening to connection-related callbacks.

[0184] Scenario 6: Confirmation of Input Method Display Request and Actual Display: After the input connection is successfully established, the input agent sends a display request (explicit display flag) to the input method service through the connection, requesting the soft keyboard to be displayed on the target screen. The display request returns true (accepted), but the input method interface is not actually displayed yet. The input agent has registered a listener callback for changes in the target screen's input method window area and visibility, and binds the current control request flag 5 to the callback. After sending the display request, a display timeout timer is started (timeout 3 seconds). After approximately 250 milliseconds (input method application loading and interface rendering time), the system's Chinese Pinyin input method soft keyboard interface pops up at the bottom of the target screen (right side screen), occupying the lower half of the screen. The input method window area change listener callback is triggered, carrying flag 5, input method window area information (width equal to screen width, height approximately 45% of screen height), and callback type "area change". The input agent extracts the flag 5 carried in the callback and compares it with the current flag 5. At the same time, it checks that the input method window visibility is true, the display screen flag is 1 (target screen), and the size is greater than the threshold, confirming that the system input method is actually displayed on the target screen. The input agent stops the timeout timer, notifies the input method control module that the input method has been confirmed for display, and the control module opens the remote input channel: the channel status is set to open, the input agent is notified to start receiving and forwarding events, and the text prompt "The keyboard is now displayed on the right side screen" is displayed in the upper right corner of the remote screen on the first display screen (disappears after 2 seconds).

[0185] Scenario 7: Chinese Text Input and Confirmation Text Processing After the remote input path is enabled, the user sequentially taps the five letter keys n, i, h, a, and o on the Chinese Pinyin input method soft keyboard on the right screen. Each time a letter is tapped, the custom input connection text submission method is invoked to submit a single Latin letter (Pinyin is displayed in the candidate word area during the pre-editing stage). The input agent detects that the submitted content contains only Latin letters and the pre-editing area is not empty, determines that it is in the pre-editing stage, and does not process or forward it for the time being. The user taps the space bar, the input method replaces the pre-edited Pinyin "nihao" with the Chinese candidate word "nihao (你好)" and submits it to the control, the custom input connection text submission method is invoked to submit the text "你好", and the pre-editing area is cleared at the same time (combination completed). The input agent detects that the submitted text contains Chinese characters and the pre-editing area has been cleared, determines that it is newly added text confirmed upon combination completion, and extracts "你好" as the confirmed text. The input agent attaches the receiving timestamp, control request identifier 5, remote session identifier "office-pc-session-123", and event sequence number 1 to the confirmed text event, and forwards it to the remote desktop client input injection module through the communication channel. After receiving it, the input injection module extracts the identifier 5 and compares it with the current identifier 5 for a match, extracts the remote session identifier "office-pc-session-123" and compares it with the current remote session identifier for a match, and the double verification is passed. The confirmed text injection is performed: "你好" is converted into the UTF-16 encoding supported by the remote desktop protocol, and sent to the office computer through the text injection channel. The currently active window of the word processing software on the office computer receives the text, inserts the two characters "你好" at the cursor position, and the user can see "你好" appearing in the word processing software through the remote screen of the first display screen.

[0186] Scenario 8: Control operation key input and key event processing: After the user finishes inputting "你好", they press the Enter key on the soft keyboard to start a new line. The custom input connection sends the key event method is invoked, with the key code being Enter and the action being press. The input agent detects that the key code is Enter, generates an independent control operation key event (key code: Enter, action: press, modifier key status: none, event timestamp, identifier 5, session identifier, event sequence number 2) and forwards it to the client. After the double verification is passed, the input injection module performs control key injection: maps the local Enter key code to the Enter scan code of the remote desktop protocol, sets the modifier key status to none, sends a key press event, delays for 20 milliseconds, and sends a key release event. The word processing software on the office computer receives the Enter event and executes a line break, and the cursor moves to the next line. The user continues to input "This is text input through a dual-screen cross-screen input method", the process is similar to Scenario 7, the confirmed text is injected into the remote session after verification, and the text appears in the word processing software character by character.

[0187] Scenario 9: Combination key operation (extended function): The user needs to select all the text they just typed for copying by pressing the Control key plus the A key combination on the physical keyboard (or the Bluetooth keyboard connected to the tablet). The custom input connection's key event sending method is called sequentially: Control key pressed, A key pressed, A key released, Control key released. When the input agent detects a Control key press, it records the modifier key state (Control key bit is 1). When it detects an A key press, it recognizes it as a Control key plus A key combination based on the modifier key state, generating a control key event sequence. This event sequence is forwarded through the communication channel. After double verification, the input injection module executes the key combination injection sequence: send Control key press event → delay → send A key press event → delay → send A key release event → delay → send Control key release event. The office computer's word processing software receives the Control key plus A key combination and performs a select all operation, selecting all text in the document.

[0188] Scenario 10: Input Closure and Resource Release After the user completes text input, clicking the "Hide Keyboard" button on the remote desktop client interface triggers the user's hide command (input closing condition). The input method control module detects this and immediately executes a unified shutdown process: Step 1: Stop receiving new events – Set the path status to closed, notify the input agent to stop processing new events, clear the event queue (no residual events), and cancel the timer; Step 2: Hide the system input method – Send a hide request to the input method service via the input connection, the input method service executes the soft keyboard hide animation (fade out by sliding down), and approximately 150 milliseconds later, the input method window area change listener callback is triggered (area change to zero area, carrying identifier 5) to confirm it is hidden; Step 3: End the input agent – ​​Clear control focus, destroy the input agent window (window disappears from the right side of the screen, token expires), and mark the instance as destroyed; Step 4: Release all resources – Log off. All control listeners and system-level display change listeners are disconnected from the input connection (ending combined text, notifying the input method service to release connection resources), the communication channel is closed, unfinished asynchronous tasks are canceled, and temporary data structures are released. The fifth step clears the binding relationship—removing the 5th entry from the session binding table, setting the current control request flag to 0 (invalid value), marking the remote session as having no active input method, and clearing the target display flag. The sixth step confirms the completion of the shutdown—checking that resources have been released item by item, resetting the input method control module state to the initial waiting state, updating the interface (making the "on" button clickable again), and recording the shutdown log (reason: user-hidden command; time: all resources successfully released; time: approximately 400 milliseconds). After shutdown, the right screen resumes displaying the tablet's main screen, while the left screen continues displaying the office computer's remote desktop. The system returns to its initial state, waiting for the next startup request. Users can continue to view the remote screen or click the keyboard button again to start a new cross-screen input method session (generating a new control request flag 6).

[0189] Scenario 11: Abnormal Scenario – Validation of the Quick Input Method Switch Identifier: To illustrate the role of control request identifier verification, consider an abnormal scenario: A user clicks a keyboard button to activate the input method (generating identifier 6). The input method begins loading but is not yet displayed. The user immediately clicks the hide button (triggers closure, identifier 6 is cleared, and the current identifier is set to 0). Then, the user immediately clicks the keyboard button to reopen the input method (generating identifier 7). At this time, the input method display callback from the old session (identifier 6) arrives delayed (because the input method takes time to load, the old session's display request may only be completed after it has been closed), and the callback carries identifier 6. The input injection module extracts the callback carrying identifier 6 and compares it with the current identifier 7. If the result is inconsistent, it is determined to be an expired callback and discarded, and the remote input path is not opened based on it. Without identifier verification, when the old callback arrives, it will be mistakenly assumed that the input method has been displayed and the path will be opened. However, at this time, the user has already closed the old input method and opened a new input method (the new input method may not yet be displayed), causing the path to open before the input method is actually displayed. The user's touch is misinterpreted as an input event, causing input chaos. Through identifier verification, only callbacks belonging to the current session (identifier 7) are recognized, ensuring the correctness of the state.

[0190] Scenario 12: Abnormal Scenario – Continuous Monitoring of Hot-Swapping of the Target Display Screen: During cross-screen input (with the input method already displayed on the right screen and the input path open), the user folds the dual-screen tablet (the right screen is folded and turned off, and the system marks it as removed). The input agent's registered display change listener is triggered, detecting that the target display (identifier 1) has been removed from the online display list (target display removal is a condition for input shutdown). Upon detection, the input method control module immediately executes a unified shutdown process: stops receiving events, hides the input method (the input method interface disappears as the right screen turns off), terminates the input agent, releases all resources, and clears the control request identifier binding. After shutdown, the system returns to its initial state. When the user unfolds the tablet, the right screen lights up again, and the system adds it back to the online display list (the identifier may change to 1 or a new identifier). When the user clicks the keyboard button again, the input method control module retrieves the online display list again, reselects the target display (the current right screen), and re-executes the complete startup process (generating a new control request identifier, starting the input agent, verifying the display, gaining focus, displaying the input method, and opening the input path). Continuous monitoring and shutdown processes ensure that the system responds correctly when the target display is hot-plugged, leaving no input method or input agent residue on the removed display, and preventing the input method from failing to open subsequently.

[0191] Through the complete end-to-end description of the above twelve scenarios, all the technical contents of the dual-screen remote desktop cross-screen input method control method of this application—including remote session display, control request identifier generation and binding, target display screen selection, input agent startup and native editable control design, display screen identifier verification and inconsistency recovery, focus acquisition and window token, input connection establishment and custom connection interception, input method display request and actual display status monitoring, control request identifier callback verification, remote input path gate opening, separate processing of confirmation text and control operation keys, dual verification and remote injection, multi-type input closing condition detection, unified closing and full resource release, binding clearing, idempotency and exception protection—are all fully and practically described.

[0192] In some embodiments, an intelligent input method parameter adaptive optimization method based on user input behavior learning is provided. By collecting and analyzing user behavior data during cross-screen input, machine learning algorithms are used to dynamically optimize input method display parameters, input type prediction, and candidate word ranking, thereby improving the personalized experience and input efficiency of cross-screen input.

[0193] Specifically, after the remote input channel is activated, the input agent collects user input behavior data while forwarding input events, including: input speed (number of characters entered per minute), backspace frequency (number of backspaces per minute reflects the input error rate), candidate word selection position (the position of the word selected by the user in the candidate word list reflects the candidate word ranking matching degree), input type switching frequency (frequency of switching between Chinese / English / numbers), frequently used vocabulary list (frequently entered words by the user), input time period distribution, and the ratio of physical keyboard to soft keyboard usage. This behavior data, along with control request identifiers and remote session identifiers, is stored together to form a user input behavior profile.

[0194] The input method control module incorporates a lightweight machine learning model (decision tree, gradient boosting tree, or shallow neural network) to provide optimized input method parameter suggestions based on user input behavior profiles. These suggestions include: input method display height suggestions (based on user hand gestures and input speed, recommending the proportion of screen height occupied by the input method; for fast-inputting users, a higher keyboard is recommended to increase key area and reduce accidental touches, while for slow-inputting users, a lower keyboard is recommended to retain more screen space for viewing remote screens); and input type prediction (based on the user's current remote desktop activity window type (determined through analysis of remote screen content or window information returned by the remote computer) and historical input type switching patterns, predicting the user's most likely next input method). The system automatically switches the keyboard layout based on the input type—it automatically switches to an English layout with symbols when a terminal window is opened on a remote desktop, and automatically switches to a Chinese Pinyin layout when a word processing software is opened, reducing the number of times users need to manually switch; it also features personalized candidate word sorting (based on the user's frequently used vocabulary list and candidate word selection history, it reorders the candidate word list returned by the input method, prioritizing frequently selected words and reducing the time users spend flipping through pages to find candidate words, thus improving the efficiency of Chinese input); and it predicts the key magnification area (based on the user's accidental touch pattern (key areas with high backspace frequency), it dynamically adjusts the size of the soft keyboard touch area, enlarging the touch response area for keys that are prone to accidental touches to reduce the accidental touch rate).

[0195] The machine learning model is trained online: the initial model is pre-trained on general user data, and after deployment to the terminal device, it is continuously and incrementally updated based on the current user's actual input behavior data (the model parameters are updated with the behavior data of each cross-screen input session). The model gradually adapts to the user's personal input habits, achieving personalized parameter adaptive optimization. Model updates are performed on the local device (federated learning or local incremental learning), and user input behavior data is not uploaded to the cloud to protect user privacy.

[0196] By learning user input behavior and intelligent parameter adaptive optimization, the cross-screen input method can not only be displayed correctly on the second screen, but also dynamically optimize the input experience according to the user's personal habits, significantly improving input efficiency and comfort. This solves the problem that fixed parameter configurations cannot adapt to the personalized needs of different users. At the same time, all learning and optimization are performed locally to protect user input privacy.

[0197] Please see Figure 4 As shown, Figure 4This is a schematic diagram of the cross-screen input method control system 200 for dual-screen remote desktop provided in this application embodiment. The cross-screen input method control system 200 for dual-screen remote desktop is used to execute the steps of the cross-screen input method control method for dual-screen remote desktop shown in the above embodiments. The cross-screen input method control system 200 for dual-screen remote desktop can be a single server or a server cluster, or it can be a terminal, such as a handheld terminal, laptop, wearable device, or robot.

[0198] like Figure 4 As shown, the cross-screen input method control system 200 for dual-screen remote desktop includes: a session display unit 201, used to display the remote desktop session on the first display screen, generate a control request identifier in response to the input method activation request, select a target display screen that is different from the first display screen; start an input agent containing native editable controls on the target display screen, transmit the control request identifier and remote session information to the input agent, and verify that the actual display screen identifier of the input agent is consistent with the target display screen identifier; The channel opening unit 202 is used to enable the native editable control to obtain focus and window token, establish an input connection through the native editable control and request the display of the system input method; listen to the actual display status of the input method on the target display screen, and when it is confirmed that the system input method is actually displayed on the target display screen and the display status callback belongs to the current control request identifier, the remote input channel is opened; The connection release unit 203 is used to receive input events, process the combined confirmation text and control operation key events separately, and inject them into the remote desktop session after verifying the control request identifier and remote session information; respond to the input closing condition, stop receiving input events, hide the system input method and end the input agent, and release the relevant window and input connection.

[0199] It should be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the cross-screen input method control system and each module of the dual-screen remote desktop described above can be referred to the corresponding content in the various embodiments of the cross-screen input method control method of the dual-screen remote desktop, and will not be repeated here.

[0200] The aforementioned cross-screen input method control method for dual-screen remote desktop can be implemented as a computer program, which can be used in various ways, such as... Figure 4 It runs on the system shown.

[0201] Please see Figure 5 , Figure 5 This is a schematic block diagram of the structure of a terminal device provided in an embodiment of this application. The terminal device includes a processor, a memory, and a network interface connected via a device bus, wherein the memory may include a storage medium and internal memory.

[0202] The storage medium can store operating devices and computer programs. These computer programs include program instructions that, when executed, cause the processor to perform any cross-screen input method control method for a dual-screen remote desktop.

[0203] The processor provides computing and control capabilities to support the operation of the entire terminal device.

[0204] Internal memory provides an environment for the execution of computer programs in non-volatile storage media. When the computer program is executed by the processor, it enables the processor to execute any cross-screen input method control method for dual-screen remote desktops.

[0205] This network interface is used for network communication, such as sending assigned tasks. Those skilled in the art will understand that... Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the solution of this application and does not constitute a limitation on the terminal to which the solution of this application is applied. Specific terminal devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0206] It should be understood that the processor can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among these, a general-purpose processor can be a microprocessor or any conventional processor.

[0207] In one embodiment, the processor is configured to run a computer program stored in a memory to perform the following steps: displaying a remote desktop session on a first display screen, generating a control request identifier in response to an input method activation request, and selecting a target display screen that is different from the first display screen; activating an input agent containing native editable controls on the target display screen, transmitting the control request identifier and remote session information to the input agent, and verifying that the actual display screen identifier of the input agent is consistent with the target display screen identifier; Give the native editable control focus and window token, establish an input connection through the native editable control and request the display of the system input method; listen to the actual display status of the input method on the target screen, and when it is confirmed that the system input method is actually displayed on the target screen and the display status callback belongs to the current control request identifier, open the remote input channel; Upon receiving input events, the system processes the combined confirmation text and control operation key events separately, and injects them into the remote desktop session after verifying the control request identifier and remote session information. Upon responding to the input closing condition, the system stops receiving input events, hides the system input method and terminates the input agent, and releases the relevant windows and input connections.

[0208] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to implement the steps of the cross-screen input method control method for dual-screen remote desktop as provided in any embodiment of this application.

[0209] The computer-readable storage medium can be an internal storage unit of the terminal device described in the foregoing embodiments, such as the hard disk or memory of the terminal device. Alternatively, the computer-readable storage medium can be an external storage device of the terminal device, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., provided on the terminal device.

[0210] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A cross-screen input method control method for a dual-screen remote desktop, applied to a terminal device including a first display screen and a second display screen, characterized in that, include: Display the remote desktop session on the first display screen, generate a control request identifier in response to the input method enable request, and select a target display screen that is different from the first display screen; Initiate an input agent containing native editable controls on the target display screen, pass the control request identifier and remote session information to the input agent, and verify that the actual display screen identifier of the input agent matches the target display screen identifier; Give the native editable control focus and window token, establish an input connection through the native editable control, and request the display of the system input method; Monitor the actual display status of the input method on the target display screen, and when it is confirmed that the system input method is actually displayed on the target display screen and the display status callback belongs to the current control request identifier, open the remote input channel; Upon receiving input events, the system processes the combined confirmation text and control operation key events separately, and injects them into the remote desktop session after verifying the control request identifier and remote session information. Upon responding to the input closing condition, the system stops receiving input events, hides the system input method and terminates the input agent, and releases the relevant windows and input connections.

2. The method according to claim 1, characterized in that, The steps of displaying the remote desktop session on the first display screen, generating a control request identifier in response to the input method activation request, and selecting a target display screen different from the first display screen include: launching the remote desktop client on the first display screen, establishing a remote desktop session with the remote computer, and displaying the remote desktop session in full screen on the first display screen; In response to the input method activation request, generate a unique control request identifier within a predetermined range and bind the control request identifier to the current remote session information; Retrieve a list of online displays, and select a second display, distinct from the first display, as the target display.

3. The method according to claim 1, characterized in that, The step of activating an input agent containing native editable controls on the target display screen and transmitting control request identifiers and remote session information to the input agent includes: activating an input agent on the target display screen, wherein the input agent contains native editable controls, and the native editable controls maintain focusability and editability; Pass the control request identifier, remote session information, and target display identifier to the input agent; The input agent receives and stores the control request identifier, remote session information, and target display identifier.

4. The method according to claim 3, characterized in that, The verification that the actual display screen identifier of the input agent is consistent with the target display screen identifier includes: after the input agent is started, reading the actual display screen identifier of the actual display screen where the input agent window is located; Compare the actual display screen label with the target display screen label; If the actual display screen identifier matches the target display screen identifier, confirm that the input agent is running on the target display screen; if the actual display screen identifier does not match the target display screen identifier, terminate the input agent instance and restart it.

5. The method according to claim 1, characterized in that, The step of enabling the native editable control to gain focus and window token, establishing an input connection through the native editable control, and requesting the display of the system input method includes: setting the input proxy window as the foreground window, and enabling the native editable control to gain input focus; The native editable control obtains a valid window token and establishes an input connection to the system input method service through the native editable control; Send a display request to the system input method service via the input connection, requesting the system input method to be displayed on the target screen.

6. The method according to claim 1, characterized in that, The process of monitoring the actual display status of the input method on the target display screen, and confirming that the system input method is actually displayed on the target display screen and that the display status callback belongs to the current control request identifier, opens the remote input channel, including: registering the input method window area change monitoring callback and the input method visibility change monitoring callback on the target display screen. When a display status callback is received, the control request identifier carried in the display status callback is extracted and compared with the current control request identifier. When the control request identifier carried by the status callback is consistent with the current control request identifier, and the system input method window is visible on the target display screen, confirm that the system input method is actually displayed and open the remote input channel.

7. The method according to claim 1, characterized in that, The input event reception process combines the confirmed text and control operation button events, including: receiving input events when the remote input channel is open; For input events of input methods with a pre-editing stage, extract the newly added text that has been combined and confirmed as the confirmation text; For Enter, Backspace, and Tab control operations, generate independent control operation key events.

8. The method according to claim 7, characterized in that, The process of injecting the control request identifier and remote session information into the remote desktop session after verification includes: extracting the control request identifier and remote session identifier carried by the input event; Compare the control request identifier carried by the input event with the current control request identifier, and compare the remote session identifier carried by the input event with the current remote session information; If both the control request identifier and the remote session identifier match, an acknowledgment text or control operation key event will be injected into the current remote desktop session; if neither identifier matches, the input event will be discarded.

9. The method according to claim 1, characterized in that, The response to the input closing condition, stopping the reception of input events, hiding the system input method and ending the input agent, and releasing the relevant window and input connection, includes: detecting the input closing condition, which includes user hiding command, user closing command, opening timeout, remote desktop session end, application destruction, target display screen closing or target display screen removal; In response to the input closing condition, stop receiving new input events, and send a hide request to the system input method service through the input connection to hide the system input method; End the input agent, release the input agent window, native editable controls and input connection, and clear the binding relationship between the control request identifier and remote session information.

10. A cross-screen input method control system for a dual-screen remote desktop, applied to a terminal device including a first display screen and a second display screen, for implementing the method as described in any one of claims 1-9, characterized in that, include: The session display unit is used to display the remote desktop session on the first display screen, generate a control request identifier in response to input method activation requests, and select a target display screen that is different from the first display screen. Initiate an input agent containing native editable controls on the target display screen, pass the control request identifier and remote session information to the input agent, and verify that the actual display screen identifier of the input agent matches the target display screen identifier; The pathway opening unit is used to enable the native editable control to gain focus and window token, establish an input connection through the native editable control, and request the display of the system input method; Monitor the actual display status of the input method on the target display screen, and when it is confirmed that the system input method is actually displayed on the target display screen and the display status callback belongs to the current control request identifier, open the remote input channel; The connection release unit is used to receive input events, process the combined confirmation text and control operation key events separately, and inject them into the remote desktop session after verifying the control request identifier and remote session information; respond to the input closing condition, stop receiving input events, hide the system input method and end the input agent, and release the relevant window and input connection.

Images