Remote application input method control method and device, product, equipment and medium

Abstract

The invention discloses a remote application input method control method and device, a product, equipment and a medium, and the method comprises the steps: obtaining a virtual key message which is a preset message sent by a remote application client under the condition that the remote application client detects that the remote application client meets a preset condition, and enabling the virtual key message to be a preset key message sent by the remote application client when the remote application client meets the preset condition; if the preset condition is met, representing that the local input method of the remote application client is about to display a candidate window; responding to the virtual key message, and obtaining the latest position of a keyboard input cursor; and returning the latest position of the keyboard input cursor to the remote application client. In this way, the difference between the experience and the experience of a local input method applied locally can be reduced, and the user experience is improved.

Description

Technical Field

[0001] This application relates to the field of remote control technology, and in particular to remote application input method control methods, devices, products, equipment and media. Background Technology

[0002] With the development of Internet technology, mobile office through RAPP (RemoteApp) technology has become a basic requirement for more and more office workers. RAPP technology has many advantages that traditional CS (Client-Server) deployment methods do not have, such as more secure data transmission, less network bandwidth consumption, and simpler deployment and maintenance.

[0003] Currently, there are local input method mapping schemes based on remote applications to leverage the advantages of client-side input methods, such as complete interface elements, word selection memory data, and user-preferred input methods, in order to achieve the advantages of Chinese input in remote applications. However, there are still differences between the local input method experience and the local application experience, which affects the user experience. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide a remote application input method control method, device, product, equipment, and medium that can reduce the difference between the remote application input method experience and the local application input method experience, thereby improving the user experience. The specific solution is as follows:

[0005] Firstly, this application provides a remote application input method control method, applied to a remote application server, comprising:

[0006] Obtain virtual key messages, wherein the virtual key messages are preset messages sent by the remote application client when it is detected that the remote application client meets preset conditions, and meeting the preset conditions indicates that the local input method of the remote application client is about to display a candidate window;

[0007] In response to the virtual key message, obtain the latest position of the keyboard input cursor;

[0008] The latest position of the keyboard input cursor is sent back to the remote application client.

[0009] Optional, also includes:

[0010] A preset detection module is used to detect whether the input method is disabled in the currently focused window of a remote application; wherein, the preset detection module is a module used to detect whether the input method is disabled in the focused window outside the target process, wherein, the target process is the process currently interacting with the user in the remote application session;

[0011] If the preset detection module does not detect a disabled input method, it sends a preset detection message to the currently focused window to trigger the preset hook module to detect whether the input method is disabled in the focused window within the target process, and obtain the detection result.

[0012] The remote application client is notified based on the detection results so that it can make the appropriate settings.

[0013] Optionally, the step of using a preset detection module to detect whether the input method is disabled in the currently focused window includes:

[0014] The system utilizes a preset detection module and detects whether the input method is disabled in the currently focused window based on a preset period.

[0015] Optionally, the step of using a preset detection module to detect whether the input method is disabled in the currently focused window includes:

[0016] The system utilizes a preset detection module and triggers a detection based on preset mouse and keyboard messages to determine whether the input method is disabled in the currently focused window.

[0017] Optional, also includes:

[0018] The preset hook module responds to the focus acquisition message and detects whether the input method is disabled in the focus window within the target process, and obtains the detection result.

[0019] Optional, also includes:

[0020] Maintain the data message window identifier corresponding to the process activating the input method proxy module;

[0021] When input data is received from the remote application client, the input data is sent to the corresponding data message window based on the data message window identifier of the process that is currently activating the input method module.

[0022] Secondly, this application provides a remote application input method control device, applied to a remote application server, comprising:

[0023] The virtual key message acquisition module is used to acquire virtual key messages, wherein the virtual key messages are preset messages sent by the remote application client when it is detected that the remote application client meets preset conditions, and meeting the preset conditions indicates that the local input method of the remote application client is about to display a candidate window.

[0024] The latest cursor position acquisition module is used to acquire the latest position of the keyboard input cursor in response to the virtual key message;

[0025] The latest cursor position feedback module is used to send the latest position of the keyboard input cursor back to the remote application client.

[0026] Thirdly, this application provides a computer program product, including a computer program / instruction that, when executed by a processor, implements the aforementioned remote application input method control method.

[0027] Fourthly, this application provides an electronic device, including a memory and a processor, wherein:

[0028] The memory is used to store computer programs;

[0029] The processor is used to execute the computer program to implement the aforementioned remote application input method control method.

[0030] Fifthly, this application provides a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the aforementioned remote application input method control method.

[0031] As can be seen from the above scheme, the present invention provides a remote application input method control method, applied to a remote application server, comprising: acquiring virtual key messages, wherein the virtual key messages are preset messages sent by the remote application client when it detects that the remote application client meets preset conditions, and meeting the preset conditions indicates that the local input method of the remote application client is about to display a candidate window; responding to the virtual key messages, acquiring the latest position of the keyboard input cursor; and transmitting the latest position of the keyboard input cursor back to the remote application client.

[0032] As can be seen, the beneficial effects of the present invention are as follows: by obtaining the virtual key message sent by the remote application client when the remote application client meets the preset conditions, the timing for obtaining the latest position of the keyboard input cursor is determined. In response to the virtual key message, the latest position of the keyboard input cursor is obtained and sent back to the remote application client. In this way, at the critical moment, that is, when the candidate window of the local input method of the remote application client is about to be displayed, the virtual key message is obtained, and then the latest position of the keyboard input cursor is obtained and sent back, ensuring that the experience of the input method candidate window position is consistent with that of the local application's local input method, reducing the difference between the experience of the local application's local input method and improving the user experience.

[0033] Correspondingly, the remote application input method control device, product, equipment, and readable storage medium provided by the present invention also have the above-mentioned technical effects. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0035] Figure 1 A flowchart of a remote application input method control method provided in this application embodiment;

[0036] Figure 2 A schematic diagram of remote application input method control provided in an embodiment of this application;

[0037] Figure 3 This is another schematic diagram of remote application input method control provided in an embodiment of this application;

[0038] Figure 4 This is another schematic diagram of remote application input method control provided in the embodiments of this application;

[0039] Figure 5 A schematic diagram of a remote application input method control device provided in an embodiment of this application;

[0040] Figure 6 This is a structural diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0041] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0042] First, the terms used in this application will be explained:

[0043] TSF: Text Services Framework, is a next-generation input method framework adopted by Microsoft Windows operating systems in Windows XP and later. TSF is a powerful framework suitable for applications requiring complex text processing and multilingual support. With TSF, developers can create smarter and more flexible text input experiences to meet the needs of modern users.

[0044] IME: Input Method Editor, generally referring to the input method component of an operating system. In Windows systems, it specifically refers to the older IMM-IME input method framework and its corresponding IME input method messages. Although newer operating systems generally use the latest TSF framework input method, the various IME input method messages received by applications are still retained by the TSF input method framework for compatibility and adaptation.

[0045] IMM: Input Method Manager. In the old IMM-IME input method framework, the IMM is located between the application and the input method, passing messages between the application and the input method and managing the IME module.

[0046] With the development of internet technology, mobile office work via RAPP (Remote App) technology has become a basic requirement for more and more office workers. RAPP technology has many advantages that traditional CS (Client-Server) deployment methods do not have, such as more secure data transmission, less network bandwidth consumption, and simpler maintenance. However, RAPP cannot yet achieve the same ease of use as local programs. A typical and commonly used example is the research on the ease of use of input methods.

[0047] To leverage the advantages of client-side input methods in remote applications, including complete interface elements, word selection memory data, and user-preferred input methods, patent CN102968190A – "Local Input Method Mapping Device and Method Based on Remote Applications" – was developed. Correspondingly, the aDesk desktop cloud product version implements a Windows remote application client-side local input method system. In this patented solution, the input method proxy module, within the Windows remote application system, may load a copy of the Tsf input method module for each application process. The input method control module, implemented as the xShell communication control module in the Windows remote application system, starts a process instance module for each remote application user session. However, this patented solution has some shortcomings: the input method proxy module in the system obtains the keyboard cursor input position and whether the input method is switched to English input mode, corresponding to the Windows Tsf input method module obtaining the keyboard cursor input position and the Tsf input method's own GUID_COMPARTMENT_KEYBOARD_OPENCLOSE switch state. This switch state is analogous to the user actively pressing the shift key to switch between Chinese and English input modes using the local input method. This input method state does not correspond to common scenarios such as disabling IME input controls and web password input boxes. Therefore, it cannot achieve the simulated local effect of the client input method automatically switching to English input mode when operating remote applications to process controls such as password boxes. In the initial adaptation implementation of the Windows remote application system, the candidate window position remains after the last input when a new client input method is initiated. In the initial adaptation implementation of the Windows remote application system, the client input method system experiences data loss for word selection in complex multi-process applications. This application provides effective solutions to these problems:

[0048] (1) Detecting the problem of disabling IME control input: Input method interface state control: GUID_COMPARTMENT_KEYBOARD_OPENCLOSE is used to identify the open or closed state of the input method keyboard interface (such as soft keyboard, candidate word window, etc.). Its value is a boolean type: TRUE (1): indicates that the input method keyboard interface is active and the user can perform input interaction. FALSE (0): indicates that the input method keyboard interface is closed and the input method enters background silent mode. The corresponding Tsf input method usage scenario for this function point is that when the user selects a Chinese input method to edit text, if the shift key is pressed, it will become like the US keyboard and the input method candidate window will not pop up. This technical point cannot be used for the case where the input method is disabled by the editing control itself, such as the password box control on the webpage or the password box control of the Windows native application. When the input method is disabled by the editing control itself, the corresponding Tsf input method module cannot be loaded at all, so the input method interface state control GUID_COMPARTMENT_KEYBOARD_OPENCLOSE mentioned above cannot be connected and recognized. Therefore, identifying whether the input method is currently disabled within the editing control itself is a new feature supported by this application. Experimental analysis revealed that in scenarios where Windows program controls are set to disable the input method, the TSF input method control cannot be invoked at all, thus making it impossible to collect Chinese and English input status information for IME tracking as designed in the original patent. To address this, this application employs an additional resident control module (i.e., a preset detection module) to continuously detect and identify input method disabling in the focused window of the remote application user session, and sends the results back to the xShell input method control module, which in turn sends them back to the client. By returning the information to the client regarding whether the input method is currently disabled within the editing control itself, the client can switch its own input method disabling settings to simulate a situation where the input method is disabled as if it were a local editing control.

[0049] (2) The problem of delayed acquisition of keyboard input cursor position by the input method proxy module: Experimental analysis revealed that the Windows system Tsf input method module is designed as a keyboard message event-driven model. If the keyboard input cursor position in the remote application document is moved by clicking with the mouse, the Tsf input method module will have no opportunity to determine the new cursor input position before the user initiates a new keyboard input. In contrast, when the local application uses the local input method, when the user starts pressing a key to trigger the Tsf input method keyboard response, the keyboard cursor position recorded by the input method itself may still be the historical position after the last input. However, after the Tsf input method obtains the keyboard response call opportunity, it will open the candidate window and immediately obtain the latest keyboard cursor input position. Therefore, when using the Tsf input method locally in Windows, interface elements such as the candidate window can often be displayed in the correct latest position in a timely manner. For the remote application client input method system, when the user presses a key on the client side to trigger the appearance of the client input method candidate window, the corresponding key has already been swallowed by the client side. It is not until the client completes a word selection input and the system sends the word selection data to the remote application server that the input method proxy module actually loaded by the remote application is triggered to obtain the keyboard cursor position after this text input. This input method candidate window positioning experience is very inconsistent with that of the local input method in the local application. To address this, this application predicts the timing when the local input method will pop up the candidate window from the system client side, sends a predefined special virtual key message, and enables the input method proxy module actually loaded by the remote application to detect the latest keyboard cursor input position at this critical moment, and promptly sends the obtained coordinate information back to the client, notifying the client to calibrate the candidate window display position.

[0050] (3) Input problem in complex multi-process applications: The conventional approach to input data routing for input methods is that the xShell module obtains the input data, calls the Windows system API to obtain the focus window, and then calls the system API to convert the window handle to its associated process ID, sending the input data to the SangforTsf module loaded by that process. However, due to the relatively complex structure of Windows applications, Tsf input method modules that are not in the process of the focus window may process the current input. To address this, this application utilizes the timing of the input focus setting interface call in the Tsf input method framework. The input method proxy module obtains the process ID of the current module and reports it to xShell, ensuring that the xShell input method control module always records the process ID of the latest activated input method proxy module, maintaining correct routing and tracking of input data.

[0051] Thus, the improved client-side local input method system achieves better implementation results on the Windows remote application server platform, primarily by implementing the following four key performance indicators:

[0052] ① The stability of the local input method system on the client side when connecting to remote applications. The Windows input method system is designed with a series of complex interactions between its applications and the input method module. Any processing error may cause the remote application to be unable to continue inputting text, or the input method function window to be fixed in a fixed position and unable to be moved.

[0053] ② The accuracy of the client's local input method system in connecting with the remote application, that is, whether the position of the input method function window is updated in a timely, accurate and appropriate manner after a series of text editing and modification inputs are performed for the remote application and then the input method function window is called up again;

[0054] ③ The compatibility of the local input method system on the client side with remote applications, that is, the need to widely connect and adapt to the text input interaction of various remote applications;

[0055] ④ The local input method system on the client side ensures local consistency with remote applications, which means supporting the same IME disable control and password box control as the local system, providing an ultimate input experience with automatic English keyboard input.

[0056] See Figure 1 As shown in the figure, this application discloses a remote application input method control method, applied to a remote application server, including:

[0057] Step S11: Obtain virtual key messages, wherein the virtual key messages are preset messages sent by the remote application client when it detects that the remote application client meets preset conditions, and meeting the preset conditions indicates that the local input method of the remote application client is about to display a candidate window.

[0058] The virtual key message is a preset message, specifically a message used to trigger the acquisition of the latest position of the keyboard input cursor. This message is sent by the remote application client to the remote application server when the candidate window is about to be displayed on the remote application client's local input method. Furthermore, this application embodiment can provide corresponding methods for determining whether the preset conditions are met for various platforms. When the candidate window is not displayed, if the first valid letter key is detected being pressed, the preset conditions are considered met; this method is universal. Additionally, for Windows platforms, the determination can be based on a specified input method message. In optional implementations, the determination can be based on the input method messages WM_IME_STARTCOMPOSITION and WM_IME_NOTIFY (IMN_OPENCANDIDATE). For Linux platforms, the determination is made through an input method context interface created using x11.

[0059] Step S12: In response to the virtual key message, obtain the latest position of the keyboard input cursor.

[0060] In an optional implementation, the latest keyboard input broadcast position can be obtained through the currently active input method proxy module, which is a TSF input method module running on the remote application server for input method proxying. The remote application server also runs an input method control module, thereby realizing local input method mapping based on the remote application. The input method proxy module is used to send the obtained remote application input method related information to the input method control module; the input method control module is used to send input method control commands to the client according to the input method related information, control the client to operate the local input method, obtain the client's character information, and forward the character information to the input method proxy module; the input method proxy module is also used to input characters to the remote application according to the character information forwarded by the input method control module.

[0061] Step S13: Send the latest position of the keyboard input cursor back to the remote application client.

[0062] In this embodiment, the input method control module can transmit the latest position of the keyboard input cursor back to the remote application client, so that the client can set the position of the input method prompt box based on the latest position of the keyboard input cursor.

[0063] See Figure 2 As shown, Figure 2This is a schematic diagram of remote application input method control provided in an embodiment of this application. DUMMYWindow for Remote APPN is the remote application interaction window, mainly used to display the screen of the remote application. It can invoke the input method candidate window and receive input method word selection data through input method message specifications, but it does not actually input corresponding content from the control on the client program. SBCSession refers to the remote application client process, IME Message is the Windows-specific IME message, Local IME refers to the client's local input method, SDX-NET is a remote desktop network communication protocol, DVC_CHANNEL_IME is a dedicated channel for transmitting input method-related data, xIMEMon is the control itself that disables external monitoring processes for the input method, i.e., the preset detection module, xShell refers to the remote application user session message gateway, i.e., the input method control module, xAppHook is the application / process unified injection module, i.e., the preset hook module, Remote AppN is a specific remote application, and RPC is remote call. In this embodiment of the application, the input method function module, i.e., the input method proxy module, is designed and customized for the client's local input method system. The TSF Interface is the interaction interface between the Windows-specific TSF input method and the system TSF framework. The input method proxy module is a self-developed TSF input method component. As a TSF input method component, it can register to receive keyboard input events, thus gaining additional timely key press response calls. At this point, the input method proxy module uses methods provided by the TSF framework to obtain the latest input cursor position, achieving timely feedback of the cursor input position to the client (the client has an actual cursor input position, which directly affects the accuracy of the pop-up candidate window position, conforming to the user's local application usage habits). The following process has been added:

[0064] ① Determine when the client's local input method is about to display the candidate window as the initiation point for the process, and send a specified pre-emptive virtual key message. An example of its implementation is as follows:

[0065] General platform method: When the candidate window is not popped up, the first valid letter key press will activate the input method candidate window. When the candidate window is popped up, clicking the mouse to select and send the candidate word, pressing the ESC key, the space bar, the Enter key, or pressing a number key will cancel or confirm sending the candidate word and close the candidate window. This general platform method refers to a method that can be used across various platforms for client-side key recognition. After the window is activated, the first letter key press will almost always trigger the pop-up of the input method candidate window (if no pre-sent virtual key message flag has been set, it will be set and sent). Pressing the ESC key, space bar, Enter key, or number key will cancel or confirm sending the candidate word and close the candidate window (effectively clearing the pre-sent virtual key message flag).

[0066] The response method for a specified platform and timing can be mentioned in the examples.

[0067] Windows platform method: Determine based on received input method messages WM_IME_STARTCOMPOSITION and WM_IME_NOTIFY(IMN_OPENCANDIDATE).

[0068] Linux platform method: Creating an input method context interface via x11:

[0069] XCreateIC(xim, XNClientWindow, DefaultRootWindow(display),XNPreeditAttributes, &preedit_attr, XNCallback, preedit_callback, NULL);

[0070] Register a callback to display the pre-edited area. The pre-edited area is similar to the candidate window interface on the Windows platform. The callback to display the pre-edited area is a callback interface that the system can call before this interface pops up.

[0071] ② Respond to the preceding virtual key message, notifying the input method agent module or the designated xInput independent process module to inject the corresponding virtual key operation into the current user session. That is, a custom message can be sent to the input method agent module for a subsequent response, or the API can be directly called to inject key events into the system session. For example, on the Windows platform, calling the SendInput() function can inject keyboard input. Specifying an xInput independent process module means designating a separate process module named xInput.

[0072] ③ By utilizing the TSF keyboard event notification interface to obtain the timing of the call and the latest position of the cursor, the cursor input position can be promptly reported back to the client.

[0073] In this embodiment, in response to the lead virtual key message, the input method agent module or the designated xInput independent process module is notified to inject the corresponding virtual key operation into the current user session. The input method agent module and the designated xInput independent process module are parallel execution paths that can be adopted depending on the compatibility of the remote virtual software.

[0074] It should be noted that the TSF input method in Microsoft Windows systems is designed as a keyboard key-triggered functional module. For example, if the keyboard cursor position is changed by using the mouse, the TSF input method module will not immediately detect and respond. Therefore, it can be analyzed that existing technical solutions, which only obtain the keyboard input focus position change after a valid IME input occurs, will result in a lag in cursor input position determination. In order to achieve a smooth experience similar to the local input method in the SBC remote application client input method function, this embodiment adds a virtual key to predict when the input method candidate window will pop up based on key presses and input method messages on the client side. This virtual key sends a message as a pre-key keyboard cursor positioning request to activate the input method prompt candidate window. In this way, even if the virtual key sent to the remote application server is slightly delayed compared to the local input method, the input method candidate window and other interfaces themselves support position correction after the interface is opened, thus achieving a more local input method experience.

[0075] Furthermore, in an optional implementation, this application embodiment can also utilize a preset detection module to detect whether the input method is disabled in the currently focused window; wherein, the preset detection module is a module used to detect whether the input method is disabled in the focused window outside the target process, wherein the target process is the process currently interacting with the user in the remote application session; if the preset detection module does not detect a disabled input method, a preset detection message is sent to the currently focused window to trigger a preset hook module to detect whether the input method is disabled in the focused window within the target process, and obtain the detection result. Based on the detection result, the remote application client is notified so that the remote application client can make corresponding settings.

[0076] In an optional implementation, this application embodiment notifies the remote application client when the detection result indicates a change in state, i.e., from disabled to not disabled or from not disabled to disabled.

[0077] In other words, in this embodiment, an external probe of the target process can be performed first. If no disabled input method is detected, an internal probe of the target process is triggered. The hook function in the preset hook module is injected into the process of each remote application. If a disabled input method is detected, there is no need to issue an internal probe of the target process.

[0078] Furthermore, this embodiment of the application can utilize a preset detection module and call a first preset method to detect whether the input method is disabled in the currently focused window; wherein, the preset detection module is a module used to detect whether the input method is disabled in the focused window outside the target process, wherein the target process is the process currently interacting with the user in the remote application session; if the preset detection module does not detect a disabled input method, a preset detection message is sent to the currently focused window to trigger a preset hook module to receive a response call within the target process, and to call a second preset method to detect whether the input method is disabled in the focused window, thereby obtaining the detection result. The first preset method can be a method that calls the GetGUIThreadInfo() function or the ImmGetDefaultIMEWnd() function. The second preset method can be to determine the mode of the process in which the current focus window is located. In this embodiment, the mode of the process can be determined based on the mode configuration information. When it is a specified mode, it is determined that the input method is disabled. The mode configuration information can include the correspondence between the process name and the mode. That is, some processes can be configured to be in the mode of disabling the input method by default, or the GUID_COMPARTMENT_KEYBOARD_DISABLED and GUID_COMPARTMENT_EMPTYCONTEXT information can be queried from the TSF framework interface for the currently focused editing item.

[0079] In an optional implementation, the step of using a preset detection module to detect whether the input method is disabled in the currently focused window may include: using the preset detection module to detect whether the input method is disabled in the currently focused window based on a preset period; correspondingly, if the preset detection module does not detect a disabled input method, a preset detection message is sent to the currently focused window to trigger a preset hook module to detect whether the input method is disabled in the focused window within the target process, and the detection result is obtained. Based on the detection result, the remote application client is notified so that the remote application client can make corresponding settings.

[0080] That is, the embodiments of this application can periodically detect whether the input method is disabled.

[0081] In an optional implementation, the step of using a preset detection module to detect whether the input method is disabled in the currently focused window may include: using a preset detection module and triggering the detection of whether the input method is disabled in the currently focused window based on preset mouse messages and preset keyboard message responses.

[0082] Among them, the preset mouse message can be the left mouse button release message, and the preset keyboard message can be the keyboard TAB key release message.

[0083] Furthermore, in response to the focus acquisition message, the preset hook module detects whether the input method is disabled in the focus window within the target process, and obtains the detection result.

[0084] Among them, the focus acquisition message is the WM_SETFOCUS message. The preset hook module responds to the WM_SETFOCUS message of each application window to detect the disabled input method of the corresponding window and sends the result to the preset detection module. The window pointed to by the detection result is identified as the current focus window, which is a valid detection result. The valid detection result is sent to the remote application client.

[0085] Further, see Figure 3 As shown, Figure 3 This is another schematic diagram of remote application input method control provided in this application embodiment. This application embodiment adds a dedicated process module (i.e., a preset detection module) on the remote application server side to assist in initiating detection messages to the currently focused window. xIMEMon is an external monitoring process for disabling the input method in the control itself, i.e., the preset detection module. The xAppHook injection module (i.e., the preset hook module) actually initiates identification and judgment of the focused window within the process and transmits the results to xShell (i.e., the input method control module). xShell only notifies the client to set the corresponding input method state when it detects a state reversal. If it detects that the current editing control itself has disabled the input method, it may send a message to xShell specifying the disabled input method state as 1. If it detects that the control does not have a disabled input method, it sends a message to xShell specifying the disabled input method state as 0. xShell only sends a message to the client to provide feedback when its own state changes; if the state remains the same, there is no feedback.

[0086] In an optional implementation, a fixed-period timer can be set for the input method disabling detection program to perform out-of-process detection and send specified detection messages to the focused window. The xAppHook injection module then detects whether the input method is disabled in the focused window within the target process and sends the result to the input method disabling detection program. In other words, specified detection messages are periodically sent to the focused window, and the xAppHook injection module then detects whether the input method is disabled in the focused window within the target process and sends the result to the input method disabling detection program.

[0087] In another alternative implementation, detection efficiency can be optimized by replacing the fixed-period Timer scheme with a detection method triggered by some feature messages. This can include the following steps:

[0088] ① The input method disable detection program obtains the call timing when the left mouse button is released and the keyboard TAB key is released by registering low-level mouse and keyboard message hooks. It performs out-of-process detection. If no input method is detected, it sends a specified detection message to the focused window. Then, the xAppHook injection module detects whether the input method is disabled in the focused window within the target process and sends the result to the input method disable detection program.

[0089] ② The xAppHook injection module responds to the WM_SETFOCUS message of each application window, performs input method disabling detection for the corresponding window, and sends the result to the input method disabling detection program. The window pointed to by the detection result is identified as the currently focused window, which is a valid detection result.

[0090] This application embodiment calls the system API to obtain the focused window and sends a specified message to the focused window to trigger it. The focused window belongs to the currently active process, which is the process that the remote application session is actually activating user interaction in. Among them, ② corresponds to how to detect the situation where the user directly clicks to open the remote application without any additional keyboard and mouse operations. ① corresponds to the situation where the user may have initiated an operation to switch the remote application process and window, covering the possibility that the target editing control may have changed, and supplementing the detection process. The WM_SETFOCUS message is a fixed message of the Windows system, indicating that the application window has gained focus. xAppHook is injected into each remote application process and can respond to the WM_SETFOCUS message when each remote application window gains focus. The detection control disables input method detection, and the detection result and the current window handle value are returned to xShell. xShell calls the system API to obtain the currently focused window, and the window specified in the verification feedback result is still the currently focused window (indicating that the program is in a stable state with focus), which is a valid detection result. If it is an unoptimized timed detection, such strict verification is not necessary, because the next detection will come soon, and the result can be detected quickly when the focus is stable.

[0091] In this application embodiment, multiple methods for determining whether an input method is disabled can be invoked. The SBC remote application may have a native input method disabling control, or it may call the password input control built into the Windows operating system, or it may be a web control that disables the input method in a browser. Therefore, multiple input method disabling determination methods need to be invoked sequentially on the server side.

[0092] ①If the GetGUIThreadInfo() function call fails, it is assumed that the focused window cannot be detected, and the client can disable the input method;

[0093] ②If the ImmGetDefaultIMEWnd() function returns an empty window handle, it means that the input method has not been loaded properly, and the client can disable the input method;

[0094] ③ The injection module determines the process of the currently focused window and sets it to pass input mode; the client should disable the input method.

[0095] ④ If the injection module uses the TSF framework interface to query the GUID_COMPARTMENT_KEYBOARD_DISABLED and GUID_COMPARTMENT_EMPTYCONTEXT information of the currently focused edit item and either one is not empty, the client should disable the input method;

[0096] ⑤ If none of the above tests indicate that the input method is disabled, the client should enable the input method.

[0097] Example of setting the input method to disable status in the client response:

[0098] ① Windows clients enable or disable input methods by calling the ImmAssociateContext(Ex) system function;

[0099] ② The Linux client enables or disables input method settings by calling the x11 related interface functions XSetICFocus() / XUnsetICFocus().

[0100] In this embodiment, if the remote application is not editing the disabled input method control, the subsequent input by the client program will not trigger the input method process. If it is not being edited to disable the input method control, the subsequent input by the client program will trigger the input method process.

[0101] You can mention in the examples that ① is a response method that can be used by Windows clients and ② is a response method that can be used by Linux clients.

[0102] Among these features, the use of timer polling for detection makes the system structure simpler and clearer; the adoption of detection efficiency optimization methods improves operating efficiency and makes it closer to the actual system implementation.

[0103] Furthermore, in an optional implementation, this application embodiment can also maintain the data message window identifier corresponding to the process activating the input method proxy module; when input method input data is received from the remote application client, the input method input data is sent to the corresponding data message window based on the data message window identifier corresponding to the process currently activating the input method module, using the input method control module.

[0104] In other words, the input method control module is used to send input data from the input method to the corresponding data message window. The data message window can be understood as the data receiving window of the remote application input method proxy module. In this embodiment, after the input method proxy module is loaded by the application process, a data message window corresponding to the process identifier is created. The data message window identifier can be the process identifier or other unique identifiers that distinguish the window. Furthermore, when the input method proxy module is activated, the process identifier and data message window identifier of the currently activated input method proxy module are obtained and reported to the input method control module. If the data message window identifier is the process identifier, only the process identifier can be uploaded as the data message window identifier. The input method control module maintains and tracks the data message window identifier of the activated input method proxy module, and when it receives input data from the client's input method, it sends it to the corresponding data message window. This solves the problem of multi-process input data routing in complex software. See also... Figure 4 As shown, Figure 4 This is a schematic diagram of another remote application input method control provided in the embodiments of this application.

[0105] ① The input method proxy module is loaded and used by the application process. When the dynamic library is loaded, a data message window with a name related to the process PID is created to receive input data from the client's input method.

[0106] For example, the window name can be TSF_IMEInput_$PID.

[0107] ②Use the TSF framework to subscribe to the input focus activation callback interface call, obtain the PID of the process where the currently active input method module is located, and report it to xShell:

[0108] The TSF input method framework provides examples of its active input focus interface:

[0109] HRESULT ITfKeyEventSink::OnSetFocus(BOOL fForeground);

[0110] HRESULT iTfThreadMgrEventSink::OnSetFocus(ITfDocumentMgr *pdimFocus,ITfDocumentMgr *pdimPrevFocus);

[0111] ③xShell maintains and tracks the PID of the active input method process. When it receives input data from the client input method, it sends a data message to the specified process PID-related name in the data message window.

[0112] The input method proxy module loads a DLL independently for each process, registering a window with a different name to receive data. Existing technology retrieves the currently active window, finds its corresponding process, and then sends data to the input method proxy module loaded by that process. However, in practice, this method sometimes fails to accurately determine the actual input process. Therefore, a TSF input method activation callback interface is adopted. At the moment of input focus activation in the corresponding input method proxy module, a mechanism is used to autonomously retrieve the current process and report it, ensuring that xShell always tracks and records the process and window information to which input method data is to be sent.

[0113] As can be seen, this embodiment of the application obtains virtual key messages sent by the remote application client when it detects that the remote application client meets preset conditions, thereby determining the timing for obtaining the latest position of the keyboard input cursor. In response to the virtual key messages, the latest position of the keyboard input cursor is obtained and sent back to the remote application client. Thus, at a critical moment—when the remote application client's local input method is about to display the candidate window—the virtual key messages are obtained, and the latest position of the keyboard input cursor is obtained and sent back, ensuring that the input method candidate window position experience is consistent with the local application's local input method. This reduces the difference between the experience and the local application's local input method, improving the user experience. Furthermore, it can detect whether the input method is disabled and solve the problem of input data routing in complex software with multiple processes.

[0114] This enables automatic identification of remote application operations that disable input method controls, automatic English keyboard input, and ensures accurate and lag-free positioning of the input method candidate window on the client side, including accurate cursor positioning. It also resolves the issue of multi-process input data routing in complex software.

[0115] See Figure 5 As shown, this application embodiment provides a remote application input method control device, applied to a remote application server, including:

[0116] The virtual key message acquisition module 11 is used to acquire virtual key messages, wherein the virtual key messages are preset messages sent by the remote application client when it is detected that the remote application client meets preset conditions, and meeting the preset conditions indicates that the local input method of the remote application client is about to display a candidate window.

[0117] The latest cursor position acquisition module 12 is used to acquire the latest position of the keyboard input cursor in response to the virtual key message;

[0118] The cursor latest position feedback module 13 is used to send the latest position of the keyboard input cursor back to the remote application client.

[0119] In an optional implementation, the device further includes: a preset detection module, a preset hook module, and a notification module; wherein the preset detection module is used to detect whether the input method is disabled in the currently focused window of the remote application; wherein the preset detection module is a module used to detect whether the input method is disabled in the focused window outside the target process, wherein the target process is the process currently interacting with the user in the remote application session; if the preset detection module does not detect that the input method is disabled, a preset detection message is sent to the currently focused window to trigger the preset hook module to detect whether the input method is disabled in the focused window within the target process, and obtain the detection result; the notification module is used to notify the remote application client based on the detection result, so that the remote application client can make corresponding settings.

[0120] In an optional implementation, a preset detection module is used to detect whether the input method is disabled in the currently focused window based on a preset period.

[0121] In an optional implementation, a preset detection module is used to detect whether the input method is disabled in the currently focused window based on preset mouse messages and preset keyboard message responses.

[0122] In an optional implementation, a pre-defined hook module is provided to detect whether the input method is disabled in the focus window within the target process in response to a focus acquisition message, and to obtain the detection result.

[0123] Furthermore, the device also includes an identifier maintenance module, used to: maintain the identifier of the data message window corresponding to the process activating the input method proxy module;

[0124] The input data sending module is used to send the input data to the corresponding data message window based on the data message window identifier of the process that is currently activating the input method module when it receives input data from the input method of the remote application client.

[0125] As can be seen, this embodiment of the application obtains the virtual key message sent by the remote application client when it detects that the remote application client meets preset conditions, thus determining the timing for obtaining the latest position of the keyboard input cursor. In response to the virtual key message, the latest position of the keyboard input cursor is obtained and sent back to the remote application client. In this way, at a critical moment—when the remote application client's local input method is about to display the candidate window—the virtual key message is obtained, and the latest position of the keyboard input cursor is obtained and sent back. This ensures that the input method candidate window position experience is consistent with that of the local application's local input method, reducing the difference between the two and improving the user experience. Furthermore, it can detect whether the input method is disabled and solve the problem of input data routing in complex software with multiple processes.

[0126] See Figure 6 As shown in the figure, this application discloses an electronic device 20, including a processor 21 and a memory 22; wherein, the memory 22 is used to store a computer program; the processor 21 is used to execute the computer program, the remote application input method control method disclosed in the foregoing embodiment.

[0127] For details regarding the specific process of the remote application input method control method described above, please refer to the relevant content disclosed in the foregoing embodiments, which will not be repeated here.

[0128] Furthermore, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk, or optical disk, and the storage method can be temporary storage or permanent storage.

[0129] In addition, the electronic device 20 also includes a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26; wherein, the power supply 23 is used to provide operating voltage for the various hardware devices on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0130] Furthermore, embodiments of this application also disclose a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the remote application input method control method disclosed in the foregoing embodiments.

[0131] For details regarding the specific process of the remote application input method control method described above, please refer to the relevant content disclosed in the foregoing embodiments, which will not be repeated here.

[0132] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0133] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0134] The remote application input method control method, device, product, equipment, and medium provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A remote application input method control method, characterized in that, Applied to remote application servers, including: Obtain virtual key messages, wherein the virtual key messages are preset messages sent by the remote application client when it is detected that the remote application client meets preset conditions, and meeting the preset conditions indicates that the local input method of the remote application client is about to display a candidate window; In response to the virtual key message, obtain the latest position of the keyboard input cursor; The latest position of the keyboard input cursor is sent back to the remote application client.

2. The remote application input method control method according to claim 1, characterized in that, Also includes: A preset detection module is used to detect whether the input method is disabled in the currently focused window of a remote application; wherein, the preset detection module is a module used to detect whether the input method is disabled in the focused window outside the target process, wherein, the target process is the process currently interacting with the user in the remote application session; If the preset detection module does not detect a disabled input method, a preset detection message is sent to the current focus window to trigger the preset hook module to detect whether the focus window has a disabled input method in the target process and obtain the detection result. The remote application client is notified based on the detection results so that it can make the appropriate settings.

3. The remote application input method control method according to claim 2, characterized in that, The method of using a preset detection module to detect whether the input method is disabled in the currently focused window includes: The system utilizes a preset detection module and detects whether the input method is disabled in the currently focused window based on a preset period.

4. The remote application input method control method according to claim 2, characterized in that, The method of using a preset detection module to detect whether the input method is disabled in the currently focused window includes: The system utilizes a preset detection module and triggers a detection based on preset mouse and keyboard messages to determine whether the input method is disabled in the currently focused window.

5. The remote application input method control method according to claim 4, characterized in that, Also includes: The preset hook module responds to the focus acquisition message and detects whether the input method is disabled in the focus window within the target process, and obtains the detection result.

6. The remote application input method control method according to any one of claims 1 to 5, characterized in that, Also includes: Maintain the data message window identifier corresponding to the process activating the input method proxy module; When input data is received from the remote application client, the input data is sent to the corresponding data message window based on the data message window identifier of the process that is currently activating the input method module.

7. A remote application input method control device, characterized in that, Applied to remote application servers, including: The virtual key message acquisition module is used to acquire virtual key messages, wherein the virtual key messages are preset messages sent by the remote application client when it is detected that the remote application client meets preset conditions, and meeting the preset conditions indicates that the local input method of the remote application client is about to display a candidate window. The latest cursor position acquisition module is used to acquire the latest position of the keyboard input cursor in response to the virtual key message; The latest cursor position feedback module is used to send the latest position of the keyboard input cursor back to the remote application client.

8. A computer program product, characterized in that, Includes a computer program / instruction that, when executed by a processor, implements the remote application input method control method as described in any one of claims 1 to 6.

9. An electronic device, characterized in that, Includes memory and processor, wherein: The memory is used to store computer programs; The processor is configured to execute the computer program to implement the remote application input method control method as described in any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that, Used to store a computer program, wherein the computer program, when executed by a processor, implements the remote application input method control method as described in any one of claims 1 to 6.

Images