Device control method, electronic device, and computer-readable storage medium

Abstract

The present application relates to the technical field of user equipment, and discloses a device control method, an electronic device, and a computer-readable storage medium. The method is applied to a client. The client is used to control cross-device flow of a target application between a first device and a second device.

Description

Equipment control methods, electronic devices and computer-readable storage media

[0001] Related applications

[0002] This application claims priority to Chinese patent application No. 202411619205.8, filed on November 13, 2024, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application relates to the field of terminal equipment technology, and in particular to equipment control methods, electronic devices, and computer-readable storage media. Background Technology

[0004] With the development of information technology, smart terminal devices have become an indispensable part of people's daily lives and work. These devices are becoming increasingly diverse, including laptops, personal computers, mobile phones, tablets, surveillance cameras, smart TVs, smart speakers, in-vehicle infotainment systems in smart cars, and smart conference terminals. Users often need to use different devices to meet diverse work and entertainment needs. Some related technologies enable collaborative work between smart terminals, such as screen sharing and projection. However, these technologies can only achieve unidirectional data flow transmission or view projection, and cannot provide users with the ability to bidirectionally run applications between two devices or two operating systems. This limits the convenience of using different devices and reduces device efficiency. Summary of the Invention

[0005] The main objective of this application is to provide a device control method, electronic device, and computer-readable storage medium, which aims to solve the technical problem in the related art that can only realize unidirectional data flow transmission or view projection, and cannot support bidirectional flow of application programs between two devices or two operating systems.

[0006] To achieve the above objectives, this application provides a device control method applied to a client, wherein the client is used for cross-device application flow between a first device and a second device, the method comprising:

[0007] When the activation of the application flow function for the first application is detected, the first application data running the first application is obtained from the first device and sent to the second device to be displayed in the first window of the display interface of the second device; wherein, when a first operation instruction is received, the first window displays the updated first application data generated by the first application in response to the first operation instruction, and the first operation instruction is an operation instruction for the first application triggered on the second device.

[0008] When the activation of the application flow function for the second application is detected, the second application data running the second application is obtained from the second device and sent to the first device to be displayed in the second window of the display interface of the first device; wherein, when a second operation instruction is received, the second window displays the updated second application data generated by the second application in response to the second operation instruction, and the second operation instruction is an operation instruction for the second application triggered on the first device.

[0009] In addition, to achieve the above objectives, this application also provides an electronic device, which includes: a memory, a processor, and a device control program stored in the memory and executable on the processor. When the device control program is executed by the processor, it implements the steps of the device control method described above.

[0010] In addition, to achieve the above objectives, this application also provides a computer-readable storage medium storing a device control program, which, when executed by a processor, implements the steps of the device control method described above.

[0011] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a device control program that, when executed by a processor, implements the steps of the device control method described above. Attached Figure Description

[0012] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0013] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0014] Figure 1 is a flowchart illustrating the cross-device transfer method used in this application;

[0015] Figure 2 is a schematic diagram of cross-device application flow provided in a specific embodiment of this application;

[0016] Figure 3 is a schematic diagram of the Windows device initialization process provided in a specific embodiment of this application;

[0017] Figure 4 is a schematic diagram of the Android device initialization process provided in a specific embodiment of this application;

[0018] Figure 5 is a schematic diagram of the hardware operating environment involved in the cross-device transfer method in the embodiments of this application.

[0019] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0020] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0021] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0022] Among related technologies, remote desktop and screen sharing technologies have facilitated remote work and collaboration. These technologies were initially designed as one-way control methods between devices, enabling only unidirectional data flow or view projection between two devices. They cannot support bidirectional applications between two devices.

[0023] Bidirectional data transfer between two devices involves efficiently exchanging data and control information between different operating systems, ensuring application consistency on both operating systems, and translating API calls from one operating system into those of the other to ensure functional consistency. These issues require significant R&D investment and time to resolve.

[0024] Therefore, related technologies lack bidirectional switching capabilities between the two systems: they can only achieve unidirectional application or screen sharing, failing to meet users' needs for bidirectional application switching between Windows and Android systems. This unidirectionality limits users' flexibility in switching between different devices and the continuity of their workflow.

[0025] The main solution of this application embodiment is a device control method applied to a client. The client is used for cross-device application flow between a first device and a second device. The method includes: upon detecting a trigger to enable application flow function for a first application, obtaining first application data running the first application from the first device and sending the first application data to the second device to display in a first window of the second device's display interface; wherein, upon receiving a first operation instruction, the first window displays updated first application data generated by the first application in response to the first operation instruction, the first operation instruction being an operation instruction triggered on the second device for the first application; upon detecting a trigger to enable application flow function for a second application, obtaining second application data running the second application from the second device and sending the second application data to the first device to display in a second window of the first device's display interface; wherein, upon receiving a second operation instruction, the second window displays updated second application data generated by the second application in response to the second operation instruction, the second operation instruction being an operation instruction triggered on the first device for the second application.

[0026] This application embodiment runs the user-selected application in the background of the source device (i.e., the device where the application is actually installed), while transferring the application's interface display and user control from the source system (i.e., the source device's operating system) to the target system (i.e., the target device's operating system). This allows the user to interact with the application on the target device (i.e., the device where the application is actually displayed and can be interacted with after it has been transferred from the source device). Thus, from the user's perspective, the application has been transferred from the source device to the target device, but in reality, the application is still running on the source device; only the interface display and user control have been transferred to the target device. Through the above mechanism, users can freely transfer applications from the source device to the target device. This allows the client to interact with the source and target devices, exchanging display data and control information. Display data from the source device is sent to the target device, where the application's interface is displayed. User control information on the target device is then converted and sent back to the source device, enabling the application running on the source device to understand and execute the corresponding operations, generating new display data. This solves the technical challenge of converting API calls from one operating system to another to ensure functional consistency. The client then sends the new display data to the target device, displaying the changes caused by user actions. This achieves efficient exchange of data and control information between different operating systems, ensuring application consistency across both systems. This solves the technical problem of related technologies that only allow unidirectional data flow or view projection, unable to support bidirectional application flow between two devices or operating systems.

[0027] This application's embodiments overcome the limitations of related technologies, achieving for the first time bidirectional application flow between Windows and Android systems, filling the gap in cross-platform bidirectional application flow technology. Through innovative system architecture and module design, it solves key technical challenges in cross-platform application flow. In today's digital age, users often own multiple devices simultaneously, such as Windows laptops, Android smartphones, and tablets. These devices each have their own advantages in hardware performance, operating systems, and application ecosystems. Achieving seamless collaboration and resource sharing between different devices can greatly improve user productivity and user experience.

[0028] Furthermore, the device control method in this application embodiment runs the user-selected application in the background of the source device, while transferring the application's interface display and user operation from the source system to the target system. This allows the user to interact with the application on the target device. From the user's perspective, the application has moved from the source device to the target device, but in reality, the application still runs on the source device; only the interface display and user operation have been transferred. This provides an application-level workflow solution, breaking down the isolation between devices. Users can freely switch between multiple operating environments within an application. This is not only suitable for collaboration between personal devices but also for remote work, development, and debugging workflows in enterprise-level scenarios. The device control method in this application embodiment opens up many new use cases. For example, during the design process, a user can create a design on a PC (Personal Computer) for a target application that needs testing or debugging, then transfer it to an Android mobile device to test touch interaction or mobile UI (User Interface) layout, and then drag it back to the PC for modification.

[0029] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0030] Please refer to Figure 1, which is a flowchart illustrating the cross-device transfer method used in this application.

[0031] In this embodiment, the device control method is applied to the client, and the client is used for cross-device application flow between the first device and the second device. The method includes steps S100 to S200.

[0032] In step S100, when it is detected that the application flow function for the first application is triggered, the first application data running the first application is obtained from the first device and the first application data is sent to the second device to be displayed in the first window of the display interface of the second device; wherein, when a first operation instruction is received, the first window displays the updated first application data generated by the first application in response to the first operation instruction, and the first operation instruction is an operation instruction for the first application triggered on the second device.

[0033] In step S200, when the activation of the application flow function for the second application is detected, the second application data running the second application is obtained from the second device and sent to the first device to be displayed in the second window of the display interface of the first device; wherein, when a second operation instruction is received, the second window displays the updated second application data generated by the second application in response to the second operation instruction, and the second operation instruction is an operation instruction for the second application triggered on the first device.

[0034] In this embodiment, the first device is distinct from the second device. The first and second devices can be electronic devices running different operating systems; for example, the first device is a desktop computer running Windows, and the second device is a smartphone running Android, or a desktop computer running Linux. Alternatively, the first and second devices can be electronic devices running the same operating system; for example, the first device is a smartphone running Android, and the second device is a tablet computer running Android, or another smartphone running Android.

[0035] In this embodiment, the client is the core component for enabling cross-device application flow between the first device and the second device. The client can be a central control device that communicates or is electrically connected to both the first and second devices simultaneously. This central control device can include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs (televisions), desktop computers, or any electronic device capable of performing the aforementioned functions. The client can also be a virtual device or functional module integrated into the first and / or second devices to implement the device control method provided in this embodiment, such as applications, functional services, chip systems, modems, etc., installed on the first and / or second devices. The client supports cross-platform communication, application data acquisition and transmission, operation command processing, user interface adaptation, status synchronization, and user interaction functions to ensure efficient and reliable cross-device application flow between the first and second devices.

[0036] Furthermore, the application flow function in this embodiment refers to the ability for an application to flow freely between different devices. When the application flow function of an application is enabled, the application can flow from the source device to the target device. When the application flow function of an application is disabled, the application cannot flow from the source device to the target device, and the application that has flowed to the target device will be flowed back to the source device.

[0037] To facilitate understanding of the concepts of source device and target device mentioned above, in one example, application A is actually installed on device 1, and after being transferred across devices, it is displayed on device 2 and can interact with application A on device 2. In this case, device 1 is regarded as the source device of application A, and device 2 is regarded as the target device of application A.

[0038] As those skilled in the art will recognize, application data refers to all visual output data and related status information generated by an application during its operation, typically including visual and audio content. Visual content primarily includes user interface elements of the application, such as windows, icons, text, images, animations, and video frames, while audio content primarily includes sounds generated by the application, such as background music, voice prompts, and video audio.

[0039] In this embodiment, the first application is the application installed on the first device, the second application is the application installed on the second device, the first application data refers to the application data of the first application, and the second application data refers to the application data of the second application.

[0040] In this embodiment, separate windows can be created on the display interfaces of the first and second devices to display application data. The first window is created from the display interface of the second device and is specifically for displaying the first application data, while the second window is created from the display interface of the first device and is specifically for displaying the second application data.

[0041] In this embodiment, at least one window specifically for displaying application data can be pre-opened on the display interface of the second device, and one of them can be selected as the first window. Alternatively, after detecting that the application flow function of the first application is enabled or receiving the first application data, a first window specifically for displaying the first application data can be opened on the display interface of the second device, thereby displaying the first application data in the first window for the user to interact with the first application on the second device. Correspondingly, in this embodiment, at least one window specifically for displaying application data can also be pre-opened on the display interface of the first device, and one of them can be selected as the second window. Alternatively, after detecting that the application flow function of the second application is enabled or receiving the second application data, a second window specifically for displaying the second application data can be opened on the display interface of the first device, thereby displaying the second application data in the second window for the user to interact with the second application on the first device.

[0042] In this embodiment, the first operation instruction can be an operation instruction generated based on keyboard and mouse events, such as mouse movement, clicking, scrolling, and keyboard key input. Of course, the first operation instruction can also be an operation instruction generated based on touch screen operation or voice input; this embodiment does not specifically limit it in this way.

[0043] For example, in one feasible implementation, the first operation instruction includes an operation instruction generated by at least one of cursor movement, key input, voice input, click event, swipe event, box selection event, and drag event.

[0044] As those skilled in the art will understand, cursor movement refers to an event in which the position of the cursor on the screen changes using a device such as a mouse, touchpad, or keyboard. Key input refers to an event in which characters, symbols, or shortcut key operations are entered using a keyboard, physical keys, or virtual keys. Voice input refers to an event in which voice commands are input using a microphone. A click event refers to an event in which a button, link, or other interactive element on the screen is activated by clicking with a mouse or tapping on a touchscreen. A swipe event refers to an event in which an object on the screen is moved or screen content is scrolled using a swipe gesture on a touchscreen. A selection event refers to an event in which an area or element on the screen is selected by dragging with a mouse or multi-touch on a touchscreen. A drag event refers to an event in which an object on the screen is moved or screen content is scrolled using a drag-and-drop mechanism.

[0045] In this embodiment, the first operation instruction can be a combination of various types of user interaction behaviors, including but not limited to the aforementioned cursor movement, key input, voice input, click events, swipe events, box selection events, and drag events. These operation instructions are triggered on the second device and sent to the first device through appropriate conversion and transmission mechanisms to control the behavior of the application flowing from the first device in the second device.

[0046] The second operation instruction is similar to the first operation instruction. It can also be an operation instruction generated based on keyboard and mouse events, or an operation instruction generated based on touch screen operation or voice input, or an operation instruction generated by at least one of the following: cursor movement, key input, voice input, click event, swipe event, box selection event, and drag event.

[0047] In this embodiment, when the activation of the application flow function for the first application is detected, the first application data running the first application is obtained from the first device and sent to the second device so that the first application data is displayed in the first window of the display interface of the second device. Upon receiving a first operation instruction, the first application running on the first device is triggered to execute the first operation instruction, and the updated first application data generated by the first application in response to the first operation instruction is sent to the second device so that the first window displays the updated first application data. Thus, the first application runs on the first device, while the interface display and interaction are transferred to the second device, realizing cross-device application flow from the first device to the second device. Simultaneously, upon detecting the activation of the application flow function for the second application, this embodiment obtains the second application data running on the second device and sends the second application data to the first device. The second application data is then displayed in the second window of the first device's display interface. Upon receiving a second operation command, the second application running on the second device is triggered to execute the second operation command, and the updated second application data generated by the second application in response to the second operation command is sent to the first device. This causes the second window to display the updated second application data, thereby running the second application on the second device while transferring the interface display and interaction to the first device, achieving cross-device application flow from the second device to the first device. Through the above steps, this embodiment overcomes the limitations of related technologies, realizing bidirectional cross-device application flow from the first device to the second device and from the second device to the first device. This allows users to seamlessly flow and run applications between two systems, filling the gap in bidirectional cross-device application flow technology and solving the technical problem that related technologies can only achieve unidirectional data flow transmission or view projection, and cannot support bidirectional application flow between two devices or two operating systems.

[0048] This embodiment enhances the user experience, allowing users to seamlessly switch applications between different devices without restarting them on each device, saving time and effort. The device control method provided in this embodiment allows users to run applications on one device while displaying and interacting on another, offering greater flexibility. For example, a user can run complex office software on a desktop computer while viewing and editing documents on a smartphone, and vice versa.

[0049] This embodiment improves work efficiency by allowing users to handle different tasks simultaneously on multiple devices. For example, a user can run multiple applications on a desktop computer while viewing and editing data from one of the applications on a tablet, or run an application on a smartphone while viewing and editing data from that application on a desktop computer. Through the device control method provided in this embodiment, team members can collaborate on different devices, sharing and editing data from the same application in real time, thus enhancing the efficiency of team collaboration.

[0050] This embodiment optimizes resource utilization by concentrating computationally intensive tasks on more powerful devices (such as desktop computers) while transferring display and interaction tasks to portable devices (such as smartphones and tablets). Through the device control method provided in this embodiment, the battery life of portable devices (such as smartphones and tablets) can be effectively extended because most computational tasks can be handled by more powerful devices, while the portable devices only need to provide display and interaction functions.

[0051] This embodiment supports devices with different operating systems (such as HarmonyOS, Windows, Android, and Linux), enabling cross-platform application flow and expanding the applicability of the application. At the same time, by opening a dedicated window for displaying application data, it solves the problem of user interface adaptation and ensures that the display effect of the user interface is consistent on different devices.

[0052] This embodiment, when the application streaming function for a first application is triggered, obtains first application data generated by running the first application from the first device and sends the first application data to the second device for display in a first window on the second device's display interface. Upon receiving a first operation command, the first window displays updated first application data generated by the first application in response to the first operation command, which is an operation command triggered on the second device for the first application. Similarly, when the application streaming function for a second application is detected to be triggered, the embodiment obtains second application data generated by running the second application from the second device and sends the second application data to the first device for display in a second window on the first device's display interface. Upon receiving a second operation command, the second window displays updated second application data generated by the second application in response to the second operation command, which is an operation command triggered on the first device for the second application. This ensures that the user can normally control the streaming application on both systems. Through the above mechanism, bidirectional flow between the two systems is achieved. Compared to related technologies that can only achieve application flow in one direction, such as only supporting one-way flow from Windows to Android or from Android to Windows, this cannot meet the user's need to freely switch applications between different devices and operating systems. This limitation forces users to repeatedly launch and operate the same applications when operating across platforms, reducing work efficiency and user experience. The embodiments of this application realize bidirectional application flow between Windows and Android systems, allowing users to seamlessly switch between the two systems and run applications, improving the convenience of cross-platform operation. Users do not need to repeatedly launch and operate the same applications on different devices, significantly improving work efficiency and user experience, thus solving the technical problem of related technologies that can only achieve unidirectional data flow transmission or view projection, and cannot support bidirectional flow of applications between two devices or two operating systems.

[0053] This application's embodiments overcome the limitations of related technologies, achieving for the first time bidirectional application flow between Windows and Android systems, filling the gap in cross-platform bidirectional application flow technology. Through innovative system architecture and module design, it solves key technical challenges in cross-platform application flow. In today's digital age, users often own multiple devices simultaneously, such as Windows laptops, Android smartphones, and tablets. These devices each have their own advantages in hardware performance, operating systems, and application ecosystems. Achieving seamless collaboration and resource sharing between different devices can greatly improve user productivity and user experience.

[0054] Furthermore, the device control method in this application embodiment runs the user-selected application in the background of the source device (i.e., the device where the application is actually installed), while transferring the application's interface display and user operation from the source system (i.e., the source device's operating system) to the target system (i.e., the target device's operating system). This allows the user to interact with the application on the target device (i.e., the device where the application is actually displayed and can be interacted with after it has been transferred from the source device). Thus, from the user's perspective, the application has been transferred from the source device to the target device, but in reality, the application is still running on the source device; only the interface display and user operation have been transferred to the target device. This provides an application-level flow solution that breaks down the isolation between devices, allowing users to freely switch between multiple operating environments within an application. This is not only suitable for collaboration between personal devices but can also be applied to remote work, development, debugging, and other workflows in enterprise-level scenarios. The device control method of this application opens up many new use cases. For example, during the design process, a user can create a design on a PC (Personal Computer) for a target application that needs to be tested or debugged, then transfer it to an Android mobile device to test touch interaction or mobile UI (User Interface) layout, and then drag it back to the PC for modification.

[0055] In this embodiment, application data acquisition and display are performed in real time, ensuring a consistent user experience on the target device compared to the source device and reducing perceived latency. This embodiment allows users to view and control applications on one device while operating on another, increasing operational flexibility and enabling users to operate the same application across different devices, switching seamlessly and breaking down device barriers, thus greatly expanding the application's usage scenarios. Furthermore, the dedicated window design makes application display more focused, allowing users to concentrate on operating a single application without being distracted by other applications.

[0056] To further enhance user experience and ease of operation, this embodiment allows users to choose whether to provide a toolbar menu when displaying the first application data in the first window of the current display interface of the first device. When deciding to provide a toolbar menu, users can choose which functions to provide, such as floating window function, full-screen window function, hiding function, screenshot function, screen recording function, refresh function, return function, and close function.

[0057] For example, in one feasible implementation, the device control method further includes step A10.

[0058] Step A10: Obtain the preset toolbar menu information and send the toolbar menu information to the first device or the second device so as to render the toolbar menu information on the display interface of the first device or the second device.

[0059] The toolbar menu information includes at least one of the following function options: floating window function, full-screen window function, hide function, and close function. The floating window function sets the first or second window to be displayed as a floating window; the full-screen window function sets the first or second window to be displayed in full-screen mode; the hide function hides the display of the first or second window while keeping the application flow function enabled; and the close function closes the display of the first or second window and disables the application flow function.

[0060] As those skilled in the art will recognize, a toolbar menu is a user interface component that typically contains a series of commonly used function buttons or menu items, providing a series of convenient operations for the application. Toolbar menu information refers to relevant information about the tool menu bar, which is used to render the tool menu bar on the display interface.

[0061] In this embodiment, users can preset toolbar menu information, including the required function options, in the client for different applications according to their own usage habits. Then, after the application flows to the target device, the toolbar menu information preset for the application is automatically obtained and sent to the target device so that the toolbar menu corresponding to the toolbar menu information is rendered in the display interface of the target device, thereby providing the function options included in the toolbar menu information in the target device.

[0062] For example, upon detecting a trigger to enable application flow functionality for the first application, or after sending data from the first application to the second device, pre-set tool menu bar information for the first application can be obtained and sent to the second device. This allows the second device to render a tool menu bar corresponding to the first application within its display interface, thus providing the required functional options. Similarly, upon detecting a trigger to enable application flow functionality for the second application, or after sending data from the second application to the first device, pre-set tool menu bar information for the second application can be obtained and sent to the first device. This allows the first device to render a tool menu bar corresponding to the second application within its display interface, thus providing the required functional options.

[0063] When the toolbar menu is displayed on the second device's screen, the floating window function sets the first window to be displayed as a floating window, the full-screen window function sets the first window to be displayed in full-screen mode, the hide function hides the first window, and the application flow function of the first application remains enabled, while the close function closes the display of the first window and disables the application flow function of the first application. Conversely, when the toolbar menu is displayed on the first device's screen, the floating window function sets the second window to be displayed as a floating window, the full-screen window function sets the second window to be displayed in full-screen mode, the hide function hides the second window, and the application flow function of the second application remains enabled, while the close function closes the display of the second window and disables the application flow function of the second application.

[0064] In this embodiment, when rendering the toolbar menu, the toolbar menu can be rendered at the top or bottom of the display interface, it can be rendered as a sidebar on the side of the display interface, it can be rendered directly into the viewport (e.g., the first viewport, the second viewport), or it can be rendered as a floating window connected to the viewport. This embodiment does not impose too many restrictions on this, and users can flexibly set it according to their own preferences.

[0065] In this embodiment, when the first window is displayed as a floating window on the display interface of the second device using the floating window function, the area size of the first window is smaller than the area size of the display interface of the second device. When the first window is displayed in full screen on the display interface of the second device using the full-screen window function, the area size of the first window is equal to the area size of the display interface of the second device. When the display of the first window is hidden using the hide function, it can be either masked or hidden in the sidebar, top, or bottom. Correspondingly, when the second window is displayed as a floating window on the display interface of the first device using the floating window function, the area size of the second window is smaller than the area size of the display interface of the first device. When the second window is displayed in full screen on the display interface of the first device using the full-screen window function, the area size of the second window is equal to the area size of the display interface of the first device. When the display of the second window is hidden using the hide function, it can be either masked or hidden in the sidebar, top, or bottom.

[0066] When a window is hidden using the hide function, it can be unhidden again using the same hide function, or it can be unhidden using the floating window function or the full-screen window function.

[0067] This implementation sends preset toolbar menu information to the corresponding device, renders the toolbar menu information, and provides users with various function options. This allows users to perform common operations more conveniently and efficiently through the various functions provided by the toolbar menu when transferring the first application to the second device or vice versa. This effectively enhances the practicality of cross-device application transfer, making the cross-device application transfer method more complete and applicable to more practical application scenarios.

[0068] In addition to pre-setting toolbar menu information for the application, when the application is transferred across devices to the target device, the user can be prompted and given corresponding options to choose in real time whether a toolbar menu is needed based on the current needs, and what functions the toolbar menu should include. The user can also cancel the toolbar menu at any time during the application's cross-device transfer process, and redefine the functions included in the toolbar menu.

[0069] In one feasible implementation, the equipment control method further includes steps S300 to S400.

[0070] Step S300: Upon receiving a first operation instruction or a second operation instruction, based on the system function mapping relationship between the first operating system and the second operating system, the first operation instruction or the second operation instruction is converted into an operation instruction in a processing format adapted to the operating system to be transferred. Here, the first operating system is the operating system of the first device, the second operating system is the operating system of the second device, the operating system to be transferred is the operating system of the device to be transferred, and the device to be transferred is the first device or the second device.

[0071] In step S400, the operation instructions adapted to the processing format of the operating system to be transferred are sent as target operation information to the device to be transferred. The target operation information is used to trigger the first application running on the first device to execute the first operation instruction, or to trigger the second application running on the second device to execute the second operation instruction.

[0072] In this embodiment, the system function mapping relationship refers to the mapping relationship between the operation instructions corresponding to the same operation in different operating systems, and the target operation information is the operation information that can be directly understood and executed by the device to be transferred.

[0073] In this embodiment, when the received operation instruction is a first operation instruction, the device to be transferred is a first device, and the target operation information is used to trigger a first application running on the first device to execute the first operation instruction. When the received operation instruction is a second operation instruction, the device to be transferred is a second device, and the target operation information is used to trigger a second application running on the second device to execute the second operation instruction.

[0074] In this embodiment, when the first operating system and the second operating system are different operating systems, and the first operation instruction cannot be directly understood and executed by the first device, it is necessary to use the system function mapping relationship between the first and second operating systems to convert the first operation instruction into an operation instruction adapted to the processing format of the first operating system. That is, the target operation information corresponding to the first operation instruction that can be directly understood and executed by the first device is used. Then, the target operation information is sent to the first device, triggering the first application running in the background of the first device to execute the first operation instruction, ensuring that the first application can accurately execute the first operation instruction. Correspondingly, when the first operating system and the second operating system are different operating systems, and the second operation instruction cannot be directly understood and executed by the second device, it is also necessary to use the system function mapping relationship between the first and second operating systems to convert the second operation instruction into an operation instruction adapted to the processing format of the second operating system. That is, the target operation information corresponding to the second operation instruction that can be directly understood and executed by the second device is used. Then, the target operation information is sent to the second device, triggering the second application running in the background of the second device to execute the second operation instruction, ensuring that the second application can accurately execute the second operation instruction.

[0075] This implementation method ensures that the source device can directly execute the received operation information through system function mapping without further processing. This achieves a seamless connection between the operation action on the target device and the actual execution on the source device. As a result, the operation triggered by the user on the target device for the streaming application (i.e., the application that is streamed from the source device to the target device) can be immediately reflected in the application that is actually running on the source device, triggering it to respond to the user's operation, generate new application data, and display it synchronously on the display interface of the target device.

[0076] This implementation ensures that different operation instructions for the same operation on different operating systems can be accurately understood and executed, effectively solving the compatibility problem of operation instructions between different operating systems when applications are transferred across devices. This allows users to seamlessly operate the same application on different devices, achieving a consistent operating experience even on different operating systems. It provides a new solution for multi-device collaboration, especially in scenarios requiring cross-platform operation.

[0077] In one feasible implementation, the device control method further includes at least one of the following steps.

[0078] If a touch operation is detected that drags the application window of the first application to the first preset flow area of ​​the first device, it is determined that the application flow function for the first application is triggered, wherein the first preset flow area is a preset area on the display interface of the first device.

[0079] If a touch operation is detected that drags the application window of the second application to the second preset flow area of ​​the second device, it is determined that the application flow function for the second application is triggered, wherein the second preset flow area is a preset area on the display interface of the second device.

[0080] As those skilled in the art will know, an application window refers to the window that an application displays on the screen after it is launched.

[0081] In this embodiment, the first preset flow area is used to trigger the application flow function of the application installed on the first device, while the second preset flow area is used to trigger the application flow function of the application installed on the second device.

[0082] In this embodiment, when the application window of the application is dragged to the first preset flow area or the second preset flow area, the application flow function of the application is triggered.

[0083] This implementation allows users to intuitively trigger the application flow function by simply dragging and dropping. This method is simple, easy to understand, convenient, and efficient, reducing the cognitive burden on users. Furthermore, users can immediately see the system's response as they drag the application window to the preset flow area, enhancing the immediacy and feedback of the operation. In addition, this method has a clear intent; the system can accurately identify the user's intention, avoiding misoperations and achieving precise cross-device application flow.

[0084] This implementation provides a simple and intuitive way to trigger the application migration function. Users simply drag and drop the application window to the preset migration area to activate the application migration function, transferring it from the source device to the target device. This method not only improves the convenience and immediate feedback of the operation but also enhances the user's sense of control and operational consistency. Whether in scenarios such as document editing, game control, or online meetings, users can achieve cross-device migration of applications through simple drag-and-drop operations, thereby achieving higher work efficiency and a better user experience.

[0085] In one feasible implementation, the device control method includes at least one of the following steps.

[0086] If the activation of the application flow function for the first application is detected, the data of the first application will be hidden from display on the display interface of the first device.

[0087] If the activation of the application flow function for the second application is detected, the data of the second application will be hidden from display on the display interface of the second device.

[0088] In this embodiment, when the application flow function for the first application is triggered, the first application flows from the first device to the second device. At this time, on the display interface of the first device, the data of the first application can be hidden, that is, the application window of the first application is not displayed on the display interface of the first device, thereby freeing up the display resources of the first device, reducing the graphics processing burden of the first device, improving the overall performance of the system on the first device, and effectively utilizing the screen space of the first device, avoiding unnecessary visual interference, so that the user can focus on other tasks or applications on the first device, while the display and interaction of the first application are carried out on the second device, realizing multi-device collaborative work.

[0089] Correspondingly, when the application flow function for the second application is triggered, the second application flows from the second device to the first device. At this time, on the display interface of the second device, it is also possible to choose to block the display of the second application data, that is, not to display the application window of the second application on the display interface of the second device, thereby freeing up the display resources of the second device, reducing the graphics processing burden of the second device, improving the overall performance of the system on the second device, and effectively utilizing the screen space of the second device, avoiding unnecessary visual interference, so that users can focus on other tasks or applications on the second device, while the display and interaction of the second application are carried out on the first device, realizing multi-device collaborative work.

[0090] This implementation, when the application flow function for an application is triggered, masks the application data of that application in the display interface of the application's source device. This method not only frees up display resources on the source device and improves system performance, but also reduces visual interference on the source device, enhancing the clarity and focus of user operations. Furthermore, masking the display strengthens system security on the source device and protects user privacy. Whether in document editing, game control, or online meetings, users can effectively manage application data and seamlessly switch between cross-device operations by masking the display, thereby achieving higher work efficiency and a better user experience.

[0091] In addition to masking the application data of the application on the display interface of the source device, this embodiment can also display the application data of the application on both the display screen of the source device and the display screen of the target device.

[0092] In one feasible implementation, the device control method includes at least one of the following steps.

[0093] If the system detects that the application flow function for the first application has been triggered to be closed, the system will remove the blocking display of the first application's data on the first device's display interface and switch the first application from background to foreground operation.

[0094] If the system detects that the application flow function for the second application has been triggered to be closed, the system will remove the blocking display of the second application data on the second device's display interface and switch the second application from background to foreground operation.

[0095] This implementation, upon detecting a trigger to disable the application flow function for an application, removes the masking display of the application's data on the source device's screen and switches the application from running in the background to running in the foreground. This allows the user to continue operating the application on the source device after the flow returns. This method improves the intuitiveness and immediate feedback of the operation, enhancing its continuity and flexibility. Whether in document editing, game control, or online meetings, users can control the application flow status through simple preset touch operations, achieving effective application data management and seamless switching between devices, resulting in higher work efficiency and a better user experience.

[0096] In one feasible implementation, the device control method further includes at least one of the following steps.

[0097] If it is detected that the second device drags the first window to the second preset flow area in the form of a floating window based on the first touch operation, it is determined to trigger the shutdown of the application flow function for the first application, wherein the second preset flow area is a preset area on the display interface of the second device.

[0098] If it is detected that the first device drags the second window to the first preset flow area in the form of a floating window based on the second touch operation, it is determined to trigger the shutdown of the application flow function for the second application, wherein the first preset flow area is a preset area on the display interface of the first device.

[0099] As those skilled in the art will know, touch operation refers to an interaction method performed by directly contacting the surface of a touchscreen. It typically involves using a finger (or stylus) to perform actions such as clicking, sliding, pinching, and rotating on the display screen to trigger corresponding functions or interface changes on the device. In this embodiment, the first touch operation is used in the second device to drag a first window as a floating window to a second preset transition area, while the second touch operation is used in the first device to drag a second window as a floating window to the first preset transition area.

[0100] In this embodiment, the first preset flow area is used to trigger the application flow function of closing the application flow to the first device, while the second preset flow area is used to trigger the application flow function of closing the application flow to the second device.

[0101] In this embodiment, when the window displaying the application data of an application is dragged to the preset flow area, the application flow function of the application will be turned off, thereby flowing it back to the source device. That is, the cross-device flow of the application between the source device and the target device is stopped, and the data transmission and interaction of the application between the two devices are terminated.

[0102] This implementation allows users to easily manage cross-device applications through intuitive drag-and-drop actions. A simple drag-and-drop action can close the application flow, simplifying the operation process of cross-device application flow and enhancing the intuitiveness and convenience of operation.

[0103] In addition to dragging the window to the preset flow area as a floating window via touch screen operation, this embodiment also allows dragging the window to the preset flow area as a floating window via mouse, thereby disabling the application's flow function. Furthermore, the application's flow function can also be directly disabled via voice input, key input, or other methods.

[0104] This implementation provides a simple and intuitive way to trigger the application migration function to close an application. Users simply drag and drop the application's data window, displayed as a floating window, to the preset migration area to trigger the application migration function, transferring the application from the target device back to the source device. This method not only improves the convenience and immediate feedback of the operation but also enhances the user's sense of control and operational consistency. Whether in scenarios such as document editing, game control, or online meetings, users can achieve cross-device migration of applications through simple drag-and-drop operations, thereby achieving higher work efficiency and a better user experience.

[0105] In one feasible implementation, the device control method includes at least one of the following steps.

[0106] If the system detects that the application flow function for the first application has been triggered to be closed, the display of the first window on the display interface of the second device will be closed.

[0107] If the system detects that the application flow function for the second application has been triggered to be closed, the display of the second window on the display interface of the first device will be closed.

[0108] In this embodiment, when the application's application flow function is turned off, the window displaying the application data of the application on the target device's display interface will also be canceled, thereby freeing up the space occupied by the application on the target device's display interface, so that the user can focus on other tasks or applications on the target device when there is no need to flow the application to the target device.

[0109] In one feasible implementation, the device control method includes at least one of the following steps.

[0110] If the touch position corresponding to the touch operation command on the second device is detected to be in the first window, the touch operation command on the second device is determined to be a first operation command for the first application.

[0111] If the touch position corresponding to the touch operation command on the first device is detected to be in the second window, the touch operation command on the first device is determined to be a second operation command for the second application.

[0112] In this embodiment, touch operation instructions refer to operation instructions generated based on touch operations, and touch position refers to the position on the screen of the electronic device where the user's touch operation occurs.

[0113] In this embodiment, after the target device detects a touch operation command generated by the user's touch operation, it records the touch position corresponding to the touch operation command and checks whether it is located within a window displaying application data for a specific application, in order to determine whether the touch operation command is an operation command for a specific application. If the touch position is located within a window specifically used to display application data for a specific application, then the touch operation command is determined to be an operation command for that application.

[0114] It is easy to understand that, in addition to the methods mentioned above, one can also identify whether the operation commands corresponding to mouse click events, key inputs, and voice inputs are operation commands for a specific application by identifying whether the object of the operation command is a window specifically used to display application data, or by identifying whether the area where the operation command is applied belongs to a window specifically used to display application data.

[0115] To facilitate understanding of the technical concept or principle of the above embodiments of the cross-device transfer method of this application, a specific embodiment is provided.

[0116] In this specific embodiment, to achieve bidirectional application flow between devices, a stable, efficient, and secure network environment needs to be established to ensure the reliability and real-time performance of data transmission. Specifically, this can be achieved through USB (Universal Serial Bus) connection, WIFI connection, local area network connection, etc.

[0117] As shown in Figure 2, this specific embodiment takes the cross-device application flow between an electronic device running a Windows operating system (hereinafter referred to as a Windows device, also known as the first device) and an electronic device running an Android operating system (hereinafter referred to as an Android device, also known as the second device) as an example.

[0118] In this specific embodiment, the software module running on the Windows device includes a flow management module and a flow agent program, and the Android device also runs a software module including a flow management module and a flow agent program. That is, the client can also be a virtual device or functional module integrated into the first device and / or the second device to implement the device control method, such as an application, function service, chip system, modem, etc. installed on the first device and / or the second device.

[0119] The application flow management module is responsible for managing the creation and location settings of virtual displays, handling the capture and transmission of keyboard and mouse events, and managing application flow, including application detection, window capture, data encoding, and network communication. Its components include the following modules.

[0120] Application Management Submodule: Responsible for monitoring application startup, shutdown, and window state changes.

[0121] Window Management Submodule: Responsible for capturing the content of application windows and using the GPU (Graphics Processing Unit) for video encoding.

[0122] Keyboard and Mouse Sharing Submodule: Responsible for capturing user keyboard and mouse input events.

[0123] The network communication submodule is responsible for establishing a communication connection and transmitting data with the flow management module of the Android device in the Windows device, and vice versa in the Android device.

[0124] The streaming proxy program on the Windows device is responsible for receiving window data (i.e., second application data) and input events (i.e., input events corresponding to the first operation command) from the Android device. When the Android application (i.e., the second application) is streamed to the Windows device, it acts as a proxy program to display the Android application's interface, that is, displaying the second application data in the second window of the first device's display interface. On the Android device, it also receives window data (i.e., first application data) and input events (i.e., input events corresponding to the second operation command) from the Windows device. When the Windows application (i.e., the first application) is streamed to the Windows device, it acts as a proxy program to display the Windows application's interface, that is, displaying the first application data in the first window of the second device's display interface. This streaming proxy program is also responsible for providing the user interface, supporting window control (minimize, maximize, close, i.e., hide function, full-screen window function, and close function) and input processing. Its components are as follows:

[0125] Interface display module: In Windows devices, it is responsible for rendering the application interface from Android devices, that is, displaying the second application data on the display interface of the first device in the second window. In Android devices, it is responsible for rendering the application interface from Windows devices, that is, displaying the first application data on the display interface of the second device in the first window.

[0126] Input processing module: On the Windows device, it receives user input events (i.e., input events corresponding to the second operation command) within the agent program window and transmits them back to the Android device. Specifically, upon detecting a second operation command for the second application, it sends the target operation information corresponding to the second operation command to the second device. On the Android device, it receives user input events (i.e., input events corresponding to the first operation command) within the agent program window and transmits them back to the Windows device. Specifically, upon detecting a first operation command for the first application, it sends the target operation information corresponding to the first operation command to the first device.

[0127] In Figure 2, the Windows home screen on a Windows device is the display interface of the first device, the Windows virtual screen is the first preset transition area within the display interface of the first device, the Windows application window is the application window of the first application, and the transition agent window is the second window. Similarly, the Android home screen on an Android device is the display interface of the second device, the Android virtual screen is the second preset transition area within the display interface of the second device, the Android application window is the application window of the second application, and the transition agent window is the first window.

[0128] As shown in Figure 2, when a Windows application window is dragged from the Windows home screen to the Windows virtual screen, the flow management module in the Windows device interacts with the flow management module in the Android device. The window data of the Windows application window captured on the Windows device is passed to the Android device, where the flow proxy program uses the Windows application as the flow application and displays its interface in the flow proxy program window on the Android home screen. Similarly, when an Android application window is dragged from the Android home screen to the Android virtual screen, the application flow steps are executed, transferring the Android application to the Windows device for display.

[0129] In this embodiment, cross-device application flow between Windows devices and Android devices requires the initialization of relevant software modules (i.e., clients integrated into the first and second devices).

[0130] As shown in Figure 3, the initialization process in a Windows device includes the following steps.

[0131] Step S11: Start the workflow management module.

[0132] The first step is to start the flow management module in the Windows device.

[0133] Step S12: Create a Windows virtual screen.

[0134] The second step is to create a Windows virtual screen, which represents the Android home screen on the Android device. The resolution of the Windows virtual screen can be set to be the same as that of the Android home screen on the Android device.

[0135] Step S13: Set the positional relationship between the Windows virtual screen and the Windows main screen.

[0136] The third step is to set the positional relationship between the Windows virtual screen and the Windows main screen, where the position of the Windows virtual screen corresponds to the Android main screen on the Android device.

[0137] Step S14: Start the transfer agent program.

[0138] Fourth step, start the streaming agent program on the Windows device.

[0139] Step S15: Obtain the local IP address and present it as a QR code.

[0140] Step 5: Obtain the local IP address of the Windows device and present it as a QR code.

[0141] Step S16: Start the communication service and wait for the client to connect.

[0142] Step 6: Start the communication service on the Windows device and wait for the client to connect. The client refers to the Android device.

[0143] As shown in Figure 4, the initialization process in an Android device includes the following steps.

[0144] Step S21: Start the workflow management module.

[0145] The first step is to launch the streaming management module on the Android device.

[0146] Step S22: Create an Android virtual screen.

[0147] The second step is to create an Android virtual screen, which represents the Windows home screen on the Windows device. The resolution of the Android virtual screen can be set to be the same as that of the Windows home screen on the Windows device.

[0148] Step S23: Set the positional relationship between the Android virtual screen and the Android main screen.

[0149] The third step is to set the positional relationship between the Android virtual screen and the Android home screen, where the position of the Android virtual screen corresponds to the Windows home screen on the Windows device.

[0150] Step S24: Start the transfer agent program.

[0151] The fourth step is to start the streaming agent program on the Android device.

[0152] Step S25: Scan the QR code on the Windows device to obtain the server IP address.

[0153] Step 5: Use an Android device to scan the QR code displayed on the Windows device to obtain the server's IP address, where "server" refers to the Windows device.

[0154] Step S26: Start the communication service and connect to the server.

[0155] Step 6: Start the communication service on the Android device and connect to the server.

[0156] In this specific embodiment, after successful initialization, the process of transferring the application from the Windows device to the Android device includes the following steps.

[0157] 1. On a Windows device, drag the application window of application A from the Windows home screen to the Windows virtual screen. That is, drag the application window of the first application to the first preset flow area of ​​the first device, or drag the second window as a floating window to the first preset flow area.

[0158] 2. The flow management module on the Windows device listens for dragging and dropping of application window A and captures when application window A is dragged to the Windows virtual screen.

[0159] 3. The streaming management module on the Windows device analyzes the window data (i.e., application data) of application A to determine whether application A is a local application (i.e., an application installed locally) or a streaming application (i.e., an application streamed from an Android device). The processing flow includes the following steps.

[0160] 3.1 If application A is a streaming application (i.e., application A is a second application), it means that the drag-and-drop operation ends the cross-device streaming of application A, specifically including the following steps.

[0161] The Windows device's streaming management module first notifies the Android device's streaming management module to move application A from the background to the foreground, i.e., to display application A on the Android device's home screen; then, it clears application A's information records in the Windows device's streaming management module. In other words, it removes the blocking of the second application's data from the second device's display interface, switching the second application from background to foreground operation.

[0162] 3.2 If application A is a local application (i.e., application A is the first application), it means that the drag operation is to start the cross-device flow of application A, which includes the following steps.

[0163] 3.2.1 The window management module on the Windows device captures the window data of application A (i.e., the first application data) in real time and sends it to the Android device. In other words, it obtains the first application data of the first application running from the first device and sends the first application data to the second device.

[0164] 3.2.2 The streaming agent program on the Android device receives the window data of application A, and adds the Android application's toolbar menu (i.e., the toolbar menu pre-set for the first application) to the user. In other words, the toolbar menu information is rendered on the display interface of the second device, and the data of the first application is displayed in the first window of the display interface of the second device.

[0165] 3.2.3 The window management module on the Windows device obtains the application flow information of application A (such as application name, ID, window ID, or position coordinates) and sends it to the window management module on the Android device for recording.

[0166] 3.2.4 If multiple Windows applications are transferred to an Android device, the window management module on the Windows device captures the window data of each application in real time and sends it to the transfer agent program on the Android device.

[0167] 3.3 After application A is transferred from a Windows device to an Android device, the processing flow of the keyboard and mouse sharing module includes the following steps.

[0168] 3.3.1 All keyboard and mouse operations (including keyboard and mouse operations) on the Android device are intercepted by the keyboard and mouse sharing module on the Android device.

[0169] 3.3.2 On an Android device, when the mouse moves to the streaming agent window, the keyboard and mouse sharing module on the Android device synchronously sends mouse operation information to the keyboard and mouse sharing module on the Windows device. The keyboard and mouse sharing module on the Windows device displays the mouse position information in real time on the Windows device through Windows system event injection methods, such as reading and writing to / dev / uinput.

[0170] 3.3.3 The mouse input event handling process for mouse operations includes the following steps.

[0171] 3.3.3.1 The keyboard and mouse sharing module on the Android device determines whether a mouse input event is a response from a streaming application or a local application, based on the streaming application information recorded by the window management module of the Android device. It also determines whether the mouse click area belongs to the streaming proxy window of the streaming application (i.e., the first window) or the Android application window of the local application (i.e., the application window of the second application). In other words, if the touch position corresponding to the touch operation command on the second device is detected to be within the first window, the touch operation command on the second device is determined to be a first operation command targeting the first application.

[0172] 3.3.3.2 If the mouse input event belongs to a streaming application response, that is, the mouse click area belongs to the streaming agent window of the streaming application, the mouse input event is passed through to the Windows device, and will eventually be distributed to the corresponding streaming application in the Windows background for processing. In other words, upon receiving the first operation instruction, the target operation information corresponding to the first operation instruction is sent to the first device to trigger the first application running in the background of the first device to execute the first operation instruction.

[0173] 3.3.3.3 If the mouse input event is a response of a local application, that is, the mouse click area belongs to the Android application window of the local application, the mouse input event is dispatched to the corresponding Android application in the Android device for processing.

[0174] 3.3.4 The keyboard input event handling process includes the following steps.

[0175] 3.3.4.1 The processing logic for all keyboard input events, following mouse input events.

[0176] 3.3.4.2 If the keyboard input event belongs to a streaming application response, the keyboard input event is passed through to the Windows device, and will eventually be distributed to the corresponding streaming application in the Windows background for processing. That is, upon receiving the first operation instruction, the target operation information corresponding to the first operation instruction is sent to the first device to trigger the first application running in the background of the first device to execute the first operation instruction.

[0177] 3.3.4.3 If the keyboard input event is a local application response, the keyboard input event is dispatched to the corresponding Android application in the Android device for processing.

[0178] 4. The streaming agent on the Android device receives the real-time window data stream (i.e., the first application data) sent by the window management module of the Windows device, overlays it with the Android application's toolbar menu, and presents it to the user, thus realizing the transfer of the Windows application to the Android device. In other words, it obtains the first application data running the first application from the first device, sends the first application data to the second device to be displayed in the first window of the second device's display interface, and renders the toolbar menu information on the second device's display interface.

[0179] Correspondingly, in this specific embodiment, after successful initialization, the process of the application flowing from the Android device to the Windows device includes the following steps.

[0180] 1. On an Android device, enter the multitasking management interface and drag task B to the Android virtual screen. That is, drag the application window of the second application to the second preset flow area of ​​the second device, or drag the first window as a floating window to the second preset flow area.

[0181] 2. The flow management module on the Android device listens for dragging and dropping the application window of task B and captures the application window of task B being dragged to the Android virtual screen.

[0182] 3. The flow management module on the Android device analyzes the window data of task B to determine whether task B is a local application or a flow application. The processing flow includes the following steps.

[0183] 3.1 If task B is a streaming application (i.e., task B is the first application), it means that the drag-and-drop operation ends the cross-device streaming of task B, specifically including the following steps.

[0184] The workflow management module on the Android device first notifies the workflow management module on the Windows device to move task B from the background to the foreground, that is, to move task B to the Windows home screen of the Windows device for display. Then, it clears the information record of task B in the workflow management module of the Android device. In other words, it removes the blocking display of the first application's data on the first device's display interface and switches the first application from background running to foreground running.

[0185] 3.2 If task B is a local application (i.e., task B is a second application), it means that the drag-and-drop operation is to start the cross-device flow of task B, which includes the following steps.

[0186] 3.2.1 The window management module on the Android device captures the window data (i.e., the second application data) of task B in real time and sends it to the Windows device. That is, it obtains the second application data of the second application running from the second device and sends the second application data to the first device.

[0187] 3.2.2 The streaming agent on the Windows device receives the window data of Task B, and overlays it with the title bar menu of the Windows application (i.e., the toolbar menu pre-set for the second application), presenting it to the user. In other words, the toolbar menu information is rendered on the display interface of the first device, and the data of the second application is displayed in the second window of the display interface of the first device.

[0188] 3.2.3 The window management module on the Android device obtains the flow application information of task B and sends it to the window management module on the Windows device for recording.

[0189] 3.2.4 If multiple Android applications are transferred to a Windows device, the window management module on the Android device will capture the window data of each application in real time and send it to the transfer agent program on the Windows device.

[0190] 3.3 After Task B is transferred from the Android device to the Windows device, the processing flow of the keyboard and mouse sharing module includes the following steps.

[0191] 3.3.1 All keyboard and mouse operations (including keyboard and mouse operations) on Windows devices are intercepted by the keyboard and mouse sharing module on Android devices.

[0192] 3.3.2 On Windows devices, when the mouse moves to the streaming agent window, the keyboard and mouse sharing module on the Windows device synchronously sends mouse operation information to the keyboard and mouse sharing module on the Android device. The keyboard and mouse sharing module on the Android device displays the mouse position information in real time on the Android device through Android system event injection methods, such as reading and writing to / dev / uinput.

[0193] 3.3.3 The mouse input event handling process for mouse operations includes the following steps.

[0194] 3.3.3.1 The keyboard and mouse sharing module on the Windows device determines whether a mouse input event belongs to a streaming application response or a local application response based on the streaming application information recorded by the window management module of the Windows device. Specifically, it determines whether the mouse click area belongs to the streaming proxy window of the streaming application (i.e., the second window) or to the Windows application window of the local application (i.e., the application window of the second application). In other words, if the touch position corresponding to the touch operation command on the first device is detected to be in the second window, the touch operation command on the first device is determined to be a second operation command targeting the second application.

[0195] 3.3.3.2 If the mouse input event belongs to the response of the streaming application, that is, the mouse click area belongs to the streaming agent window of the streaming application, the mouse input event is passed through to the Android device, and will eventually be distributed to the corresponding streaming application in the Android background for processing. In other words, upon receiving the second operation instruction, the target operation information corresponding to the second operation instruction is sent to the second device to trigger the second application running in the background of the second device to execute the second operation instruction.

[0196] 3.3.3.3 If the mouse input event is a local application response, that is, the mouse click area belongs to the Windows application window of the local application, the mouse input event is dispatched to the corresponding Windows application in the Windows device for processing.

[0197] 3.3.4 The keyboard input event handling process includes the following steps.

[0198] 3.3.4.1 The processing logic for all keyboard input events, following mouse input events.

[0199] 3.3.4.2 If the keyboard input event belongs to a streaming application response, the keyboard input event is passed through to the Android device, and will eventually be distributed to the corresponding streaming application running in the Android background for processing. That is, upon receiving the second operation instruction, the target operation information corresponding to the second operation instruction is sent to the second device to trigger the second application running in the background of the second device to execute the second operation instruction.

[0200] 3.3.4.3 If the keyboard input event is a local application response, the keyboard input event is dispatched to the corresponding Windows application in the Windows device for processing.

[0201] 4. The streaming agent on the Windows device receives the real-time window data stream (i.e., the second application data) sent by the window management module of the Android device, overlays it with the title bar menu of the Windows application, and presents it to the user, thus realizing the streaming of the Android application to the Windows device. In other words, it obtains the second application data running the second application from the second device and sends the second application data to the first device to be displayed in the second window of the first device's display interface, and renders the toolbar menu information on the first device's display interface.

[0202] This specific embodiment achieves bidirectional application flow between Windows and Android devices through a simple and intuitive user interface design. A unified drag-and-drop operation and instant visual feedback enable users to quickly get started, significantly improving the cross-device application experience. Seamless application switching and multitasking support meet users' collaborative work needs in multi-device environments.

[0203] This specific embodiment achieves true bidirectional application flow, breaking through the unidirectional limitation of related technologies: current technical solutions mainly support unidirectional application interaction from Android devices to Windows systems, and cannot achieve application flow from Windows systems to Android devices. This specific embodiment innovatively achieves bidirectional application flow between Windows and Android systems. Users can transfer Windows applications to Android devices for continued use, and vice versa, truly achieving seamless cross-platform collaboration. Through this specific embodiment, users no longer need to repeatedly open and operate applications on different devices, resulting in a more seamless workflow and more convenient operation. This specific embodiment supports multi-device collaborative work, meeting users' needs in multiple scenarios and with multiple tasks.

[0204] This specific embodiment provides a unified and intuitive interaction method, reducing the learning cost for users: Existing application cross-device switching typically requires complex user settings or relies on specific hardware and software environments, making the operation cumbersome. However, this specific embodiment creates a virtual display in the system, allowing users to switch applications simply by dragging and dropping, resulting in a unified and intuitive operation. With this specific embodiment, users do not need to learn new operating methods, lowering the barrier to entry. This specific embodiment uses the same operating method on both Windows and Android devices, improving the consistency of cross-platform use.

[0205] This specific embodiment enables bidirectional application flow and management across devices, improving multitasking capabilities: related technologies typically only support unidirectional application flow, failing to meet users' bidirectional task requirements. This specific embodiment, however, supports bidirectional application flow. Through this embodiment, users can simultaneously handle multiple tasks on different devices, conveniently switching and managing applications. This specific embodiment improves work efficiency, enhances system flexibility, and meets user needs in complex scenarios.

[0206] This specific embodiment has made comprehensive innovations in multiple technical fields such as the creation of virtual displays, efficient capture and encoding of window content, cross-system input event synchronization, and secure transmission. The various technical modules work closely together to form an organic whole, producing a synergistic effect that surpasses the sum of the effects of each individual technology.

[0207] This specific embodiment achieves cross-platform, bidirectional, and real-time application flow through comprehensive technological innovation, significantly improving the user experience, which is something that simple combinations of technologies cannot achieve.

[0208] This specific embodiment solves a long-standing technical problem: issues such as bidirectional application flow, real-time interaction, and security, which cannot be effectively addressed by related technologies, are comprehensively resolved in this specific embodiment.

[0209] The above examples are only used to help understand this application and do not constitute a limitation on the cross-device transfer method of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0210] In addition, please refer to Figure 5, which is a schematic diagram of the device structure of the hardware operating environment involved in the cross-device transfer method in the embodiments of this application.

[0211] This application also provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the steps of the application cross-device transfer method described above.

[0212] Referring to Figure 5 below, a schematic diagram of an electronic device suitable for implementing embodiments of this application is shown. The electronic device in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs (Televisions), desktop computers, or any electronic device capable of performing the above functions. The electronic device shown in Figure 5 is merely an example and should not impose any limitations on the functions and scope of use of the embodiments of this application.

[0213] As shown in Figure 5, the electronic device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the electronic device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. While electronic devices with various systems are shown in the figures, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.

[0214] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0215] The electronic device provided in this application, employing the cross-device application transfer method described in the above embodiments, solves the technical problem in related technologies that can only achieve unidirectional data stream transmission or view projection, and cannot support bidirectional transfer of application execution between two devices or two operating systems. Compared with related technologies, the beneficial effects of the electronic device provided in this application are the same as those of the cross-device application transfer method provided in the above embodiments, and other technical features of this electronic device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0216] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0217] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of protection of the above claims.

[0218] In addition, this application also provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to perform the steps of the application cross-device transfer method in the above embodiments.

[0219] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0220] The aforementioned computer-readable storage medium may be included in an electronic device or may exist independently without being assembled into an electronic device.

[0221] The aforementioned computer-readable storage medium carries one or more programs. When the one or more programs are executed by an electronic device, the electronic device performs the following steps: upon detecting a trigger to enable an application flow function for a first application, it acquires first application data running the first application from the first device and sends the first application data to a second device to display it in a first window of the display interface of the second device; wherein, upon receiving a first operation instruction, the first window displays updated first application data generated by the first application in response to the first operation instruction, the first operation instruction being an operation instruction for the first application triggered on the second device; upon detecting a trigger to enable an application flow function for a second application, it acquires second application data running the second application from the second device and sends the second application data to the first device to display it in a second window of the display interface of the first device; wherein, upon receiving a second operation instruction, the second window displays updated second application data generated by the second application in response to the second operation instruction, the second operation instruction being an operation instruction for the second application triggered on the first device.

[0222] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0223] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0224] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0225] The computer-readable storage medium provided in this application stores computer-readable program instructions (i.e., a computer program) for performing the steps of the above-described application cross-device transfer method. This solves the technical problem in related technologies that only allow unidirectional data stream transmission or view projection, and cannot support bidirectional transfer of application programs between two devices or two operating systems. Compared with related technologies, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the application cross-device transfer method provided in the above embodiments, and will not be repeated here.

[0226] Furthermore, embodiments of this application also provide a computer program product, including a computer program that, when executed by a processor, implements the steps of the application cross-device transfer method as described in the above embodiments.

[0227] The computer program product provided in this application solves the technical problem in related technologies that can only achieve unidirectional data stream transmission or view projection, and cannot support bidirectional flow of application programs between two devices or two operating systems. Compared with related technologies, the beneficial effects of the computer program product provided in this application are the same as the beneficial effects of the cross-device application flow method provided in the above embodiments, and will not be repeated here.

[0228] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A device control method applied to a client, the client being used for cross-device application flow between a first device and a second device, the method comprising: When the activation of the application flow function for the first application is detected, the first application data running the first application is obtained from the first device and sent to the second device to be displayed in the first window of the display interface of the second device; wherein, when a first operation instruction is received, the first window displays the updated first application data generated by the first application in response to the first operation instruction, and the first operation instruction is an operation instruction for the first application triggered on the second device. When the activation of the application flow function for the second application is detected, the second application data running the second application is obtained from the second device and sent to the first device to be displayed in the second window of the display interface of the first device; wherein, when a second operation instruction is received, the second window displays the updated second application data generated by the second application in response to the second operation instruction, and the second operation instruction is an operation instruction for the second application triggered on the first device.

2. The device control method of claim 1, wherein, The method includes: Obtain preset toolbar menu information and send the toolbar menu information to the first device or the second device to render the toolbar menu information on the display interface of the first device or the second device; The toolbar menu information includes at least one of the following function options: floating window function, full-screen window function, hide function, and close function. The floating window function sets the first window or the second window to be displayed as a floating window; the full-screen window function sets the first window or the second window to be displayed in full-screen mode; the hide function hides the display of the first window or the second window while keeping the application flow function enabled; and the close function closes the display of the first window or the second window and disables the application flow function.

3. The device control method of claim 1, wherein, The method further includes: Upon receiving the first operation instruction or the second operation instruction, based on the system function mapping relationship between the first operating system and the second operating system, the first operation instruction or the second operation instruction is converted into an operation instruction adapted to the processing format of the operating system to be transferred, wherein the first operating system is the operating system of the first device, the second operating system is the operating system of the second device, the operating system to be transferred is the operating system of the device to be transferred, and the device to be transferred is the first device or the second device; The operation instructions adapted to the processing format of the operating system to be transferred are sent as target operation information to the device to be transferred. The target operation information is used to trigger the first application running on the first device to execute the first operation instructions, or to trigger the second application running on the second device to execute the second operation instructions.

4. The device control method of claim 1, wherein, The method includes at least one of the following: If a touch operation is detected that drags the application window of the first application to the first preset flow area of ​​the first device, it is determined that the application flow function for the first application is triggered, wherein the first preset flow area is a preset area on the display interface of the first device. If a touch operation is detected that drags the application window of the second application to the second preset flow area of ​​the second device, it is determined that the application flow function for the second application is triggered, wherein the second preset flow area is a preset area on the display interface of the second device.

5. The device control method of claim 1, wherein, The method includes at least one of the following: If the activation of the application flow function for the first application is detected, the data of the first application will be hidden from display on the display interface of the first device. If the activation of the application flow function for the second application is detected, the data of the second application will be hidden from display on the display interface of the second device.

6. The device control method of claim 5, wherein, The method includes at least one of the following: If the system detects that the application flow function for the first application has been triggered to be closed, the data of the first application will be unblocked on the display interface of the first device, and the first application will be switched from running in the background to running in the foreground. If the system detects that the application flow function for the second application has been triggered to be closed, the data of the second application will be unblocked on the display interface of the second device, and the second application will be switched from running in the background to running in the foreground.

7. The device control method of claim 6, wherein, The method further includes at least one of the following: If it is detected that the second device drags the first window to the second preset flow area in the form of a floating window based on the first touch operation, it is determined to trigger the shutdown of the application flow function for the first application, wherein the second preset flow area is a preset area on the display interface of the second device; If the system detects that the first device drags the second window to the first preset flow area as a floating window based on the second touch operation, it determines that the application flow function for the second application will be turned off. The first preset flow area is a preset area on the display interface of the first device.

8. The device control method of claim 1, wherein, The method includes at least one of the following: If the system detects that the application flow function for the first application has been triggered to be closed, the display of the first window on the display interface of the second device will be closed. If the system detects that the application flow function for the second application has been triggered to be closed, the display of the second window on the display interface of the first device will be closed.

9. The device control method of claim 1, wherein, The method further includes at least one of the following: If the touch position corresponding to the touch operation command on the second device is detected to be in the first window, the touch operation command on the second device is determined to be a first operation command for the first application; If the touch position corresponding to the touch operation command on the first device is detected to be in the second window, the touch operation command on the first device is determined to be a second operation command for the second application.

10. The device control method according to any one of claims 1 to 9, wherein, The first operation instruction includes an operation instruction generated by at least one of cursor movement, key input, voice input, click event, swipe event, box selection event, and drag event.

11. An electronic device, comprising: The electronic device includes: a memory, a processor, and a device control program stored in the memory and executable on the processor, wherein the device control program, when executed by the processor, implements the steps of the device control method as described in any one of claims 1 to 10.

12. A computer readable storage medium, wherein, The computer-readable storage medium stores a device control program, which, when executed by a processor, implements the steps of the device control method as described in any one of claims 1 to 10.

13. A computer program product, wherein, The computer program product includes a device control program, which, when executed by a processor, implements the steps of the device control method as described in any one of claims 1 to 10.

Images