Simulator-based data processing method and device, equipment, storage medium and program product

Abstract

The embodiment of the invention provides a simulator-based data processing method and device, equipment, a storage medium and a program product, and the method comprises the steps: displaying an operation interface of a simulator on a current interface of electronic equipment; the simulator comprises a host end and a client end, and in response to running operation which is executed on an operation interface and aims at a plurality of terminal application programs, a display identifier is distributed to each terminal application program through the client end; the to-be-rendered data of each terminal application program and the corresponding display identifier are mapped and then transmitted to the host end; creating an application window for each terminal application program based on the display identifier through the host end, and displaying the application window of each terminal application program on a current interface of the electronic equipment; and in the application window of each terminal application program, performing window texture rendering based on the to-be-rendered data. Through the application, the resource consumption can be obviously reduced, and the system operation efficiency is improved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the Internet field, and relate to but are not limited to a data processing method and device based on a simulator, equipment, a storage medium and a program product. BACKGROUND

[0002] With the continuous expansion of the mobile application ecosystem, the simulator, as an important tool for running application programs on non-native devices, is widely used in software development, game testing, and multi-task office scenarios. The simulator usually simulates the execution of a target operating system by running a client environment on a host device, thereby allowing users to run various application programs in a non-native environment.

[0003] The Android simulator in the related art generally adopts a "single window carrying" mode, that is, the entire Android system environment is run in a simulator window, and the multi-opening function is realized through resource replication, supporting the simultaneous start of multiple independent Android instances, each instance running an application, and each instance running independently in a virtual terminal. Although this method achieves multi-task processing, it is essentially independent running and does not realize true multi-window integrated display, resulting in low system resource utilization and limited user experience.

[0004] Therefore, the method in the related art has the problems of high resource occupation and large response delay, and it is difficult to realize efficient collaborative display and interaction of multiple terminal application programs in a unified interface, which limits user experience and performance. SUMMARY

[0005] Embodiments of the present application provide a data processing method and device based on a simulator, equipment, a storage medium and a program product, which allocate an independent display identifier to each terminal application program, and transmit the display identifier and the to-be-rendered data to the host end after mapping, so that the host end can create an independent application window for each terminal application program based on the display identifier, thereby realizing independent windowed display of multiple applications, and multiple terminal application programs are run through the host end, realizing that all terminal application programs share the same set of system resources, thereby significantly reducing resource consumption and improving overall running efficiency.

[0006] The technical solution of the embodiments of the present application is as follows: An embodiment of the present application provides a data processing method based on a simulator, comprising: displaying an operation interface of a simulator on a current interface of an electronic device; the simulator comprises a host end and a client end, the host end is simulator software running on the electronic device, and the client end is a virtual terminal environment created on the electronic device; in response to a running operation performed on the operation interface for a plurality of terminal application programs, assigning a display identifier to each terminal application program through the client end; and transmitting, to the host end, to-be-rendered data of each terminal application program after mapping the to-be-rendered data and the corresponding display identifier; creating, by the host end, an application window for each terminal application program based on the display identifier, and displaying the application window of each terminal application program on the current interface of the electronic device; and performing window texture rendering in the application window of each terminal application program based on the to-be-rendered data.

[0007] An embodiment of the present application provides a data processing device based on a simulator, comprising: a display module configured to display an operation interface of a simulator on a current interface of an electronic device; the simulator comprises a host end and a client end, the host end is simulator software running on the electronic device, and the client end is a virtual terminal environment created on the electronic device; a data processing module configured to, in response to a running operation performed on the operation interface for a plurality of terminal application programs, assign a display identifier to each terminal application program through the client end; and transmit, to the host end, to-be-rendered data of each terminal application program after mapping the to-be-rendered data and the corresponding display identifier; a window creation module configured to create, by the host end, an application window for each terminal application program based on the display identifier, and display the application window of each terminal application program on the current interface of the electronic device; and a rendering module configured to perform window texture rendering in the application window of each terminal application program based on the to-be-rendered data.

[0008] In the above scheme, the data processing module is further configured to: based on the display identifier of each terminal application program, construct a graphics buffer queue for each terminal application program in a preset buffer area; store the to-be-rendered data of each terminal application program and the corresponding display identifier in the respective graphics buffer queue after mapping; and transmit the to-be-rendered data of each terminal application program to the host end by using the graphics buffer queue.

[0009] In the above scheme, the data processing module is further configured to: for each terminal application program, compress the to-be-rendered data of the terminal application program to obtain compressed to-be-rendered data; obtain meta information of the to-be-rendered data; the meta information at least comprises the display identifier of the terminal application program; and package and store the compressed to-be-rendered data and the meta information in the graphics buffer queue.

[0010] In the above scheme, the window creation module is further configured to: in response to receiving the first data packet in the graphics buffer queue of any terminal application through the host, parse the first data packet to obtain the first data to be rendered and metadata; and create an application window for the terminal application based on the metadata.

[0011] In the above scheme, the device further includes: a setting module, configured to, after creating an application window for the terminal application based on the metadata, obtain the program name and program icon of the terminal application from the client; set the window title of the application window to the program name of the terminal application; and set the window icon of the application window to the program icon of the terminal application.

[0012] In the above scheme, the data processing module is further configured to: create a communication channel for each terminal application; the communication channel is used to send data through a cross-system communication interface; the channel identifier of the communication channel is the display identifier of the terminal application; and the data to be rendered in the graphics buffer queue of the terminal application is transmitted to the host through the communication channel.

[0013] In the above scheme, the device further includes: a program exit module, used to, after rendering the window texture based on the data to be rendered, in response to a program exit operation for any terminal application, send a program exit notification message for the any terminal application to the host via the client; and in response to the program exit notification message, close the communication channel of the any terminal application and stop displaying the application window of the any terminal application.

[0014] In the above scheme, the device further includes: a window interaction module, used to, after rendering window textures based on the data to be rendered, respond to a window interaction operation for any terminal application via the host terminal to perform an operation response on the application window of any terminal application, and display the operation response result for the application window on the current interface of the electronic device; wherein, the window interaction operation includes at least one of the following: window maximization operation, window minimization operation, and window closing operation; the window interaction operation is performed through an input device connected to the electronic device; in response to the completion of the operation response for the window interaction operation, the operation response result is synchronized to the client terminal.

[0015] In the above scheme, the window interaction module is further configured to: after rendering the window texture based on the data to be rendered, in response to a window interaction operation for any terminal application, encapsulate the window interaction operation into a client-side input operation through the host terminal, and send the client-side input operation to the client terminal; respond to the client-side input operation through the client terminal, and display the operation response result for the application window on the current interface of the electronic device; wherein, the window interaction operation includes at least one of the following: window maximization operation, window minimization operation, and window closing operation; the window interaction operation is executed through an input device connected to the electronic device.

[0016] In the above scheme, the device further includes: a virtual display device processing module, configured to, after assigning a display identifier to each terminal application through the client, create a virtual display device corresponding to the display identifier of the terminal application through the client; generate at least one virtual layer based on the data to be rendered of the terminal application; composite the at least one virtual layer to obtain a virtual composite image of the terminal application; and output the virtual composite image of the terminal application to the virtual display device; the data processing module is further configured to: map the virtual composite image of each terminal application to the corresponding display identifier and transmit it to the host.

[0017] In the above scheme, the virtual display device processing module is further configured to: create a virtual display adapter; the virtual display adapter is used to manage virtual display devices; and call the virtual display adapter to construct a virtual display device corresponding to the display identifier.

[0018] In the above scheme, the virtual display device includes a virtual built-in display device; the virtual display device processing module is further configured to: check the display change event of the hardware synthesizer; and in response to the display change event being a built-in display creation event, call the virtual display adapter to construct a virtual built-in display device corresponding to the display identifier.

[0019] In the above scheme, the device further includes: an external display processing module, used to call an external display creation function to create an external display corresponding to the display identifier in response to the display change event being an external display creation event; the external display is used to display virtual composite images of different terminal applications, and the external display uses a hardware compositor to composite the at least one virtual layer, and uses the hardware compositor to display the virtual composite image on the external display.

[0020] In the above scheme, the virtual display device processing module is further configured to: construct a buffer queue for the terminal application in a preset frame buffer based on the display identifier of the terminal application; the buffer queue is connected to the virtual display device corresponding to the display identifier; store the virtual composite image of the terminal application in the buffer queue; and use the buffer queue to transmit the virtual composite image of the terminal application to the virtual display device.

[0021] In the above scheme, the data processing module is further configured to: in response to the data format of the virtual display device being a multimedia codec format, encode the virtual composite image into a first format corresponding to the multimedia codec format, and map the virtual composite image of the first format with the corresponding display identifier to the host terminal; in response to the data format of the virtual display device being a surface texture format, call the graphics rendering program interface to map the virtual composite image with the corresponding display identifier to the host terminal.

[0022] This application provides an electronic device, including: a memory for storing computer-executable instructions or computer programs; and a processor for executing the computer-executable instructions or computer programs stored in the memory to implement the above-described simulator-based data processing method.

[0023] This application provides a computer program product that stores computer-executable instructions or a computer program, which are used to cause a processor to execute the computer-executable instructions or the computer program to implement the above-described simulator-based data processing method.

[0024] This application provides a computer-readable storage medium storing computer-executable instructions or computer programs, which are used to cause a processor to execute the computer-executable instructions or computer programs to implement the above-described simulator-based data processing method.

[0025] The above solution has the following beneficial effects: In this embodiment, the simulator's operation interface is first displayed on the current interface of the electronic device. The simulator consists of a host running on the electronic device and a client in a virtual terminal environment. Then, when the user executes multiple terminal applications, the client assigns a unique display identifier to each terminal application, maps the data to be rendered of the terminal application to the display identifier, and transmits it to the host. The host then creates and displays an independent application window for each terminal application. Finally, the host completes texture rendering in its respective application window. In this way, on the one hand, by assigning an independent display identifier to each terminal application on the client and mapping the display identifier to the data to be rendered before transmitting it to the host, the host can create an independent native window for each terminal application based on the display identifier, thereby achieving independent windowed display of multiple applications and avoiding the limitations of traditional single-window systems. On the other hand, since multiple terminal applications run through the host, all terminal applications share the same set of system resources, significantly reducing resource consumption and improving overall operating efficiency. Attached Figure Description

[0026] Figure 1 This is an optional architecture diagram of the simulator-based data processing system provided in the embodiments of this application; Figure 2 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application; Figure 3 This is an optional flowchart illustrating a simulator-based data processing method provided in an embodiment of this application. Figure 4 This is a schematic diagram of the implementation process of mapping data to be rendered with display identifiers and then transmitting it to the host, as provided in an embodiment of this application. Figure 5 This is another optional flowchart illustrating the simulator-based data processing method provided in the embodiments of this application; Figure 6 This is a schematic diagram of a response flow for a program exit operation provided in an embodiment of this application; Figure 7 This is a schematic diagram of a response flow for window interaction operations provided in an embodiment of this application; Figure 8 This is a schematic diagram of another response flow for window interaction operations provided in an embodiment of this application; Figure 9 This is a schematic diagram of a PC desktop provided in an embodiment of this application; Figure 10 This is another PC desktop illustration provided in an embodiment of this application; Figure 11This is a schematic diagram of the overall technology chain provided in the embodiments of this application; Figure 12 This is a schematic diagram illustrating the implementation process of the installation terminal application provided in the embodiments of this application; Figure 13 This is the Guest core component and class diagram provided in the embodiments of this application; Figure 14 This is a schematic diagram illustrating the synchronous implementation of a multi-window process between the Host and Guest sides according to an embodiment of this application. Figure 15 This is a schematic diagram of the Host-side transparent transmission of user input events provided in an embodiment of this application; Figure 16 This is a flowchart illustrating the HDMI display optimization scheme provided in the embodiments of this application. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0028] In the following description, references to "some embodiments" refer to a subset of all possible embodiments. However, it is understood that "some embodiments" may be the same or different subsets of all possible embodiments and may be combined with each other without conflict. Unless otherwise defined, all technical and scientific terms used in the embodiments of this application have the same meaning as commonly understood by one of ordinary skill in the art to which the embodiments of this application pertain. The terminology used in the embodiments of this application is for the purpose of describing the embodiments of this application only and is not intended to limit the application.

[0029] In the embodiments of this application, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0030] Before describing the simulator-based data processing method provided in the embodiments of this application, the technical terms involved in the embodiments of this application will be explained first.

[0031] 1) Responding to: used to indicate the conditions or states on which the operation is performed depends. When the conditions or states on which it depends are met, one or more operations can be performed in real time or with a set delay. Unless otherwise specified, there is no restriction on the order in which the multiple operations are performed.

[0032] 2) Emulator: Also known as a mobile emulator application, it is a software tool (i.e., application) used to mimic another computer system or device. The main purpose of an emulator is to run software or games that originally could only run on a specific platform on different hardware or operating systems. An emulator creates a virtual environment by copying the target system's instruction set, hardware architecture, and peripheral interfaces, allowing source code or binary files to be executed on the host system while maintaining compatibility with the original environment. Mobile emulator applications are used to simulate the operating environment of a mobile device on an electronic device. They allow users to experience mobile applications without actually owning a mobile device. They are mainly divided into application market programs, mobile emulator host clients, and mobile emulator guest clients. In this embodiment, the scenario is that a user runs an Android system on a Windows computer, downloads an Android application, installs it, and opens it for normal use.

[0033] 3) Application Market Program: This refers to the software program protected in the embodiments of this application. Users can search for and download software, and manage the running status of the mobile phone emulator Host and Guest in this application market program.

[0034] 4) Emulator User Interface: This is the main interface seen by the user when running emulator software on an electronic device (such as a personal computer (PC)). This interface is used to launch, manage, and control applications in the virtual terminal environment. In this embodiment, the user interface serves not only as the emulator's entry point but also as the basic platform for multi-window display.

[0035] 5) Host Side: In this embodiment, it can also be referred to as the mobile phone emulator host side, which refers to the emulator software itself running on the electronic device, as well as its associated hardware abstraction layer, operating system, and user interface components. The host side is responsible for creating and managing the virtual terminal environment (i.e., the client side), and handling tasks such as data transmission, graphics rendering, and window creation from the client side. The host side provides a series of tools and interfaces so that users can interact with the virtual terminal. In this embodiment, the host side also undertakes the task of receiving application rendering data from the client side and generating independent windows on the host system based on this data. The host side described in this embodiment can be a PC software implemented on the Windows platform, and the product form can be any PC-based software management tool, which can be simply referred to as the host side in this embodiment.

[0036] 6) Client (Guest): In this embodiment, it can also be called a mobile phone emulator Guest, referring to a virtual terminal environment created on an electronic device. For example, this client can be implemented based on the Android system. This virtual terminal environment is used to run multiple terminal applications (i.e., multiple apps), and each terminal application can communicate with the host through the client. The client is responsible for assigning a unique display identifier to each terminal application and mapping the rendering data of the terminal application to the display identifier before transmitting it to the host. In other words, the client is used to run multiple terminal applications and outputs the UI rendering data of these applications to the host through a virtual display mechanism so that the terminal applications can be displayed as independent windows on the host. It should be noted that in a virtual terminal environment, users can install and run mobile applications and perform various operations and experiences, just like using a real mobile phone. The Guest terminal described in this application embodiment can be an Android system implemented based on a VirtualBox virtualization scheme. Its form is a Virtual Disk Image (VDI) file, and it conforms to the VDI file format specification. The Host terminal loads this VDI file to load and run it. In this application embodiment, it can be simply referred to as the Guest terminal.

[0037] 7) Display Identifier: This is an identifier used to identify a terminal application and its corresponding data to be rendered. It is a unique identifier assigned to each terminal application and is used to establish a data mapping relationship between the host and client. The display identifier acts as a bridge between the client and the host, enabling the host to accurately identify and process the data to be rendered from different terminal applications, thereby creating corresponding independent windows for the terminal applications.

[0038] 8) Data to be rendered: This refers to the graphics data generated by the terminal application during its operation, including image frames, texture information, etc. This data needs to be transmitted to the host and used for rendering in a separate window created on the host.

[0039] 9) Graphics Buffer Queue: This is a data structure built on the client side for each terminal application to temporarily store the application's data to be rendered. The graphics buffer queue supports sequential reading and writing, ensuring data integrity and synchronization during transmission. The purpose of the graphics buffer queue is to improve the efficiency and reliability of data transmission, ensuring that the data to be rendered can be transmitted to the host side in a timely and orderly manner. In this embodiment, each terminal application has a graphics buffer queue corresponding to its display identifier.

[0040] 10) Virtual Display Device: This is a software-simulated display device provided by the Android system. The virtual display device is neither connected to nor dependent on a physical screen. Instead, it outputs the image content of the application or system interface to a memory stream for use by other modules. In this embodiment, the client side achieves independent management and transmission of the data to be rendered for each application by creating a virtual display device corresponding to the display identifier of the terminal application. In other words, the virtual display device "virtually" outputs the image content of the terminal application or system interface to an image stream (Surface) in memory. Imagine a user opening an app on their phone. Normally, the app's screen would be displayed on the phone screen, but using Virtual Display, the system "secretly copies" this screen and outputs it to an "invisible screen"—this screen is actually a memory data stream. This data stream can be used for screen recording, uploaded to a computer for display, or displayed as an independent window in an emulator.

[0041] 11) External Display: For example, in this embodiment, it can be implemented as an HDMI Display, which is a physical external display recognized and enabled by the system when an Android device is connected to an external display device (such as a TV, projector, or monitor) via an HDMI interface. When a mobile phone or set-top box is connected to a TV with an HDMI cable, the TV lights up and displays the mobile phone's image; at this time, the TV is an HDMI Display. The Android system automatically detects the connection or disconnection of this cable and "mirrors" or "extends" the image to the TV. In this embodiment, the Windows window on the host side is equivalent to the external display device.

[0042] 12) Cross-system communication interface: This is a technical means used to establish a communication channel between the host and client. This interface supports efficient, low-latency data transmission, ensuring that data to be rendered from multiple terminal applications can be quickly processed and displayed on the host. Cross-system communication interfaces can be implemented based on shared memory, sockets, or other high-performance communication protocols.

[0043] 13) Window texture rendering: This refers to the process of drawing graphics in the native Windows window based on the data to be rendered received from the terminal application on the host side. This process includes texture mapping, color adjustment, scaling, and other operations to ultimately achieve a high-quality visual presentation. Window texture rendering is typically implemented using graphics APIs such as DirectX or OpenGL to ensure rendering efficiency and compatibility.

[0044] 14) Communication channel: This is the data transmission path between the host and the client. Its channel identifier is usually consistent with the display identifier of the terminal application to ensure the correct routing and processing of data.

[0045] 15) Virtual composite image: This is an image result created by compositing one or more virtual layers generated by the client based on the rendering data from the terminal application. This image is used to output to a virtual display device and is ultimately transmitted to the host for window display.

[0046] 16) Frame buffer: This is a pre-defined memory area on the host or client side used to cache rendering data or image frames for subsequent processing and display. The design of the frame buffer helps improve the smoothness and stability of graphics rendering.

[0047] In related technologies, Android emulators generally use a single-window hosting mode to run the Android system. Even if some products offer multi-instance functionality, they can only achieve the multi-window effect by launching multiple complete Android instances. This method not only consumes a lot of resources but also limits the user experience, making it difficult to meet the demands of modern PC environments for multitasking and desktop-level operation. Therefore, the methods in these technologies prevent users from displaying multiple apps in parallel in independent windows like PC applications. All apps run in the same simulated mobile interface, and users cannot freely drag, scale, or operate across application windows like native PC applications, hindering multitasking efficiency. Furthermore, achieving multi-window functionality through multiple instances means launching a complete Android system instance for each app, resulting in excessive resource consumption, decreased system performance, and difficulty supporting large-scale concurrent application operation. It also fails to fully utilize the multi-window and multi-monitor operation features of PCs, impacting operational efficiency and immersion.

[0048] To address the aforementioned problems in related technologies, this application provides a high-efficiency, low-overhead, and highly compatible technical solution to resolve the issue that Android emulators cannot display multiple applications in independent windows on a PC environment. This solution allows multiple Android applications to run on the same Guest system and be presented as independent Windows native windows on the Host side, improving user experience and resource utilization. Specifically, this application provides a data processing method based on an emulator. On the client side, a display identifier is assigned to each terminal application, and the data to be rendered is mapped to the display identifier and transmitted to the host side. The host side creates an application window for each terminal application based on the display identifier and displays the application window of each terminal application on the current interface of the electronic device. Finally, window texture rendering is performed in the application window of each terminal application based on the data to be rendered. This application effectively solves the problems of low efficiency, high resource consumption, and poor user experience in multi-window display in traditional emulators, achieving the goal of displaying multiple Android applications as independent windows on the Host side within a single Guest system, significantly improving resource utilization and user experience.

[0049] Here, we first describe an exemplary application of the simulator-based data processing device according to the embodiments of this application. This simulator-based data processing device is an electronic device used to implement a simulator-based data processing method. The simulator-based data processing device (i.e., electronic device) provided in the embodiments of this application can be implemented as any terminal capable of running simulator applications, such as a laptop, tablet, desktop computer, or intelligent robot. In another implementation, the electronic device provided in the embodiments of this application can also be implemented as a server, wherein the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. The terminal and the server can be directly or indirectly connected through wired or wireless communication, which is not limited in the embodiments of this application. Next, we will describe an exemplary application when the electronic device is implemented as a personal computer (PC).

[0050] See Figure 1 , Figure 1This is an optional architecture diagram of the simulator-based data processing system provided in this application embodiment. The simulator-based data processing system 10 in this application embodiment includes at least a PC 100, a network 200, and a server 300. An simulator application is installed on the PC 100. The simulator includes a host side and a client side. The host side is the simulator software running on the PC 100, and the client side is a virtual terminal environment created on the PC 100. The server 300 can be a backend server for the simulator application. The PC 100 can constitute the simulator-based data processing device of this application embodiment, that is, the simulator-based data processing method of this application embodiment is implemented through the PC 100. The PC 100 is connected to the server 300 through the network 200, which can be a wide area network (WAN), a local area network (LAN), or a combination of both.

[0051] See Figure 1 When running the emulator application, PC 100 can interact with server 300 via network 200, sending program execution commands to server 300 to obtain emulator execution data, and displaying the emulator's operation interface on PC 100's current screen based on this execution data. Then, in response to the user's execution operations for multiple terminal applications, PC 100 assigns a display identifier to each terminal application via the client side; and maps the data to be rendered for each terminal application to the corresponding display identifier and transmits it to the host side. Next, the host side creates an application window for each terminal application based on the display identifier and displays the application window of each terminal application on PC 100's current screen. Finally, window texture rendering is performed in the application window of each terminal application based on the data to be rendered.

[0052] Figure 2 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Figure 2 The illustrated electronic device can be the aforementioned simulator-based data processing device. For example, the electronic device can be implemented as a PC, wherein the electronic device includes: at least one processor 210, memory 250, at least one network interface 220, and user interface 230. The various components in the electronic device are coupled together via a bus system 240. It is understood that the bus system 240 is used to implement communication between these components. In addition to a data bus, the bus system 240 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 2 The general labeled all buses as Bus System 240.

[0053] Processor 210 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0054] User interface 230 includes one or more output devices 231 that enable the presentation of media content, and one or more input devices 232.

[0055] Memory 250 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, etc. Memory 250 may optionally include one or more storage devices physically located remote from processor 210. Memory 250 may include volatile memory or non-volatile memory, or both. Non-volatile memory may be read-only memory (ROM), and volatile memory may be random access memory (RAM). The memory 250 described in this application embodiment is intended to include any suitable type of memory. In some embodiments, memory 250 is capable of storing data to support various operations, examples of which include programs, modules, and data structures, or subsets or supersets thereof, as illustrated below.

[0056] Operating system 251 includes system programs for handling various basic system services and performing hardware-related tasks, such as framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks; network communication module 252 is used to reach other computing devices via one or more (wired or wireless) network interfaces 220, for example, network interfaces 220 include: Bluetooth, WiFi, and Universal Serial Bus (USB); input processing module 253 is used to detect and translate one or more user inputs or interactions from one or more input devices 232.

[0057] In some embodiments, the apparatus provided in this application may be implemented in software. Figure 2A simulator-based data processing device 254 stored in memory 250 is shown. The simulator-based data processing device 254 can be a simulator-based data processing device in an electronic device, and can be software in the form of programs and plug-ins, including the following software modules: display module 2541, data processing module 2542, window creation module 2543, and rendering module 2544. These modules are logically connected and can therefore be arbitrarily combined or further divided according to the functions they implement. The functions of each module will be described below.

[0058] In some embodiments, the apparatus provided in this application can be implemented in hardware. As an example, the apparatus provided in this application can be a processor in the form of a hardware decoding processor, which is programmed to execute the simulator-based data processing method provided in this application. For example, the processor in the form of a hardware decoding processor can be one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0059] The simulator-based data processing methods provided in the embodiments of this application can be executed by an electronic device, wherein the electronic device can be implemented as a PC, that is, the simulator-based data processing methods in the embodiments of this application can be executed by a PC.

[0060] The data processing method based on a simulator provided in the embodiments of this application will be described below. Figure 3 This is an optional flowchart illustrating a simulator-based data processing method provided in an embodiment of this application. The following will be combined with... Figure 3 The steps shown are explained as follows: Figure 3 As shown, taking an electronic device as the execution subject of the simulator-based data processing method as an example, the method includes the following steps S101 to S104: S101, Display the simulator's operation interface on the current screen of the electronic device; The simulator includes a host and a client. The host is the simulator software running on the electronic device, and the client is a virtual terminal environment created on the electronic device.

[0061] An emulator is a software tool installed on an electronic device to simulate the running environment of a target operating system (such as Android or iOS). This allows the emulator to run on the electronic device, simulating the target operating system's environment. Taking a mobile application (e.g., a phone application) that runs on a PC as an example, the emulator can be a mobile emulator. A mobile emulator can simulate the running environment of a mobile operating system on a PC, allowing users to experience applications on a mobile device without actually owning it. An exemplary scenario in this application embodiment is: on a Windows PC, running an Android system via a mobile emulator, downloading an application developed based on Android specifications, installing it, and using it normally. The emulator can include an application market program, an emulator host (Host) and an emulator guest (Guest). For example, when the emulator is a mobile emulator, it can include an application market program, a mobile emulator Host, and a mobile emulator Guest. The application market program refers to the program in the emulator used to perform functions such as application search and application download; the application market program can also be used to manage the running status of the emulator's Host and Guest sides.

[0062] In this embodiment, the host side refers to the emulator software itself and its supporting components running on the electronic device. It is responsible for managing the entire virtual terminal environment and providing a graphical interface for user interaction. For example, a PC-based software management tool can serve as the host side, providing a windowed interface for displaying and managing multiple Android applications. In practice, after launching the emulator application on an electronic device (e.g., a PC), the user first sees the emulator's main interface, which is the entry point for user interaction. For example, when running a PC-based software management tool under a Windows system, the user will see a window similar to a mobile phone screen on the desktop; this window is the emulator's initial interface. On the emulator's initial interface, the user can click to install or open Android applications, and also manage multiple virtual terminal instances. The host side refers to the emulator software itself and its required framework components. The host side is responsible for managing the entire emulator environment and communicating with the client side.

[0063] The client-side is a virtual terminal environment created within the host-side on an electronic device. The client-side can be an Android system instance implemented using VBox virtualization. It is used to run various Android applications. The client-side is represented as a VDI file. It can be created on top of the host-side using virtualization technology (such as VBox). The client-side has complete system functionality, capable of independently running terminal applications and generating graphics rendering data. The client-side allows users to install and run various Android applications within it, and the performance and behavior of these applications should be as close as possible to the experience on a real mobile device.

[0064] S102, in response to the operation of running multiple terminal applications on the operation interface, assigns a display identifier to each terminal application through the client; and transmits the data to be rendered of each terminal application to the host after mapping the data to be rendered with the corresponding display identifier.

[0065] When a user runs multiple Android applications in the emulator, the client assigns a unique display identifier to each application. A display identifier is an identifier used to uniquely identify each terminal application and its corresponding data to be rendered; it is a logical ID and also uniquely identifies the display channel of each application. The display identifier acts as a bridge between the client and host, enabling the host to accurately identify and process rendering data from different terminal applications. For example, when a user launches multiple apps on the client, each app is assigned a unique display identifier, such as DisplayID-001, DisplayID-002, etc. These display identifiers are used not only to distinguish different application windows but also to establish a mapping relationship with the graphics buffer queue. Each app's UI rendering data (i.e., the data to be rendered, including textures, colors, resolution, etc.) is bound to the display identifier and transmitted to the host via a communication channel. Binding UI rendering data to display identifiers and transmitting it to the host via a communication channel ensures that the rendered content of each app can be processed and displayed independently on the host, thus supporting multi-window operation.

[0066] In the specific implementation process, after the UI rendering is completed on the client side, the client side will extract the image frames of each terminal application through the SurfaceFlinger module, and encapsulate the image frames into a rendering package with a DisplayID. This package is then sent to the host side through the communication interface between the host and the client side. Based on the application content identified by the display identifier, the host side can perform subsequent window creation and rendering work.

[0067] In some embodiments, seeFigure 4 , Figure 4 The step S102, which maps the data to be rendered for each terminal application to its corresponding display identifier and transmits it to the host, can be achieved through the following steps S1021 to S1023: S1021, based on the display identifier of each terminal application, constructs a graphics buffer queue for each terminal application in a preset buffer.

[0068] A graphics buffer queue is a set of memory structures allocated to each terminal application to store the data to be rendered for that application, used for temporarily caching graphics frame data. The SurfaceFlinger module in the Android system is responsible for managing these graphics buffer queues. The graphics buffer queues are used to temporarily store the rendering results of each frame of the data to be rendered for each terminal application, so that these rendering results can be transmitted to the host later.

[0069] Each graphics buffer queue corresponds to an independent display channel, ensuring that UI rendering data from different terminal applications does not interfere with each other. The graphics buffer queue supports producer and consumer modes, where the rendering thread of the terminal application acts as the producer, submitting data to be rendered to the graphics buffer queue, while the host end acts as the consumer, retrieving and processing the data. The graphics buffer queue avoids frequent memory copies, improves data transfer efficiency, and guarantees frame order and timestamp consistency.

[0070] A display identifier is a unique identifier assigned by the client to each terminal application to distinguish different display channels. The display identifier is typically a string or integer ID. It is bound to a virtual display device or HDMI monitor to differentiate the rendering streams of different terminal applications, ensuring that each terminal application's application window can be independently created and managed on the host side. In this embodiment, whenever a new terminal application starts, the client creates an independent virtual display for that new application and assigns a unique display identifier. The display identifier is used throughout the transmission to identify and match the rendering data stream of the terminal application, thereby ensuring that the host side can correctly render the terminal application's window into the correct native window.

[0071] It needs to be explained that virtual display is an abstract function provided by the operating system's graphics system. It creates a virtual display channel that does not depend on a physical display device. This channel has an independent display identifier, configuration, and timing. However, the content of the frame buffer of this channel does not directly drive the physical display device. Instead, it is output to an off-screen rendering target (usually a Surface) allocated and controlled by the terminal application. The workflow of virtual display includes: (1) Request creation: A terminal application with permissions (such as screen recording software) sends a request to the DisplayManagerService (DMS) to specify the parameters of the virtual display (width, height, density, etc.) and provide a Surface for receiving image data. (2) Instantiate virtual device: DMS creates a virtual display device instance through the virtual display adapter and registers it in the client's display device list. At this time, the client will think that there is an additional "physical display". (3) Integration into the compositing channel: The surface compositor begins to regard the virtual display as a legitimate output target. In each vertical synchronization signal cycle, the surface compositor performs an independent compositing operation for this virtual display. (4) Data Output and Consumption: The composited image is not sent to the display controller, but is written to the Surface provided by the terminal application through a buffer queue mechanism. The terminal application then obtains the image data from the Surface for subsequent processing (such as encoding into video, streaming over a network, etc.). The virtual display in this embodiment has the same graphics processing channels as the physical display, including optimizations such as layer compositing, color management, and hardware overlay, to ensure the quality and performance of the output image. The opening, closing, and content rendering of the virtual display are independent of the state of the physical display; the physical display can enter a sleep state, while the virtual display can still function normally. Creating a virtual display usually requires higher system privileges because the virtual display may capture all screen content, involving user privacy. Virtual display is essentially a high-level abstraction provided by the operating system's graphics stack. Virtual display decouples graphics rendering logic from physical display hardware, allowing rendering results to be redirected to any software endpoint for processing and consumption, thus enabling powerful capabilities for screen content acquisition, redirection, and redistribution.

[0072] In this embodiment, by creating an independent graphics buffer queue for each terminal application, the design of the graphics buffer queue ensures that the rendering data of different terminal applications does not interfere with each other, thereby improving transmission efficiency and stability. The graphics buffer queue design also supports multi-threaded access and asynchronous processing, which helps reduce latency and improve overall performance. By constructing a graphics buffer queue and binding it to a display identifier, the rendering data of multiple terminal applications can be independently separated within the same client instance and efficiently reused on the host side. This method of constructing a graphics buffer queue and binding it to a display identifier avoids the resource waste caused by repeatedly launching multiple client instances in traditional multi-instance emulators, thus significantly reducing CPU and memory usage.

[0073] S1022, after mapping the data to be rendered for each terminal application to the corresponding display identifier, store it in its respective graphics buffer queue.

[0074] The data to be rendered refers to the graphics data currently generated by the terminal application but not yet composited and output. This data may include pixel information, textures, animation states, etc., of the UI interface. In this embodiment, the data to be rendered is composited in the SurfaceFlinger on the client side and mapped to the corresponding graphics buffer queue according to its display identifier. Mapping the data to be rendered for each terminal application to its display identifier means that when storing data, the data to be rendered for each terminal application is bound to its display identifier, facilitating quick location and retrieval of the corresponding data on the host side based on the display identifier. This mapping mechanism not only improves the orderliness of data organization but also effectively avoids data confusion between different applications, thereby ensuring the accuracy and consistency of the rendered output. This compositing and mapping process ensures that the graphics data of each terminal application is not mixed with the graphics data of other terminal applications, maintaining the independence of the graphics data of each terminal application.

[0075] Mapping refers to the process of establishing a logical association between the data to be rendered and a specific display identifier. In step S1022, the surface compositor packages the composited graphics data according to the display identifier of each terminal application and adds corresponding metadata (such as resolution, rotation angle, timestamp, etc.). Then, the surface compositor writes the entire data packet into the corresponding graphics buffer queue. The mapping process ensures the orderliness of the data to be rendered and lays the foundation for subsequent cross-system transmission.

[0076] In practice, there is a close relationship between the graphics buffer queue and the display identifier. The display identifier determines which graphics buffer queue the data to be rendered should be written to, while the graphics buffer queue is responsible for correctly transmitting the data to be rendered with a specific display identifier to the host. The collaborative work of the graphics buffer queue and the display identifier enables precise management and efficient transmission of rendering data for multiple applications. By mapping the data to be rendered to the display identifier and storing it in the graphics buffer queue, the client can achieve fine-grained control over the content of multiple windows and provide a clear data structure to the host. This approach not only improves rendering efficiency but also enhances the system's support for complex window operations.

[0077] In some embodiments, the process of mapping the data to be rendered for each terminal application to its corresponding display identifier and storing it in its respective graphics buffer queue in step S1022 can be implemented as follows: For each terminal application, firstly, the data to be rendered for that terminal application is compressed to obtain compressed data to be rendered; then, the metadata of the data to be rendered is obtained; the metadata includes at least the display identifier of the terminal application; finally, the compressed data to be rendered and the metadata are packaged and stored in the graphics buffer queue.

[0078] The data to be rendered is image frame data generated by a terminal application (such as an Android application) during client-side runtime, typically including UI elements, textures, color information, etc. This data is synthesized by a surface compositor and prepared for output to a display device or virtual display channel. Due to the large volume of image frame data, this application compresses the data to be rendered for each terminal application to reduce bandwidth consumption during transmission between the client and host sides and to improve overall rendering efficiency. Compression is the process of encoding the data to be rendered to reduce its size while retaining key visual information. The compression of the data to be rendered for each terminal application can be implemented using image compression algorithms (such as JPEG, PNG, or H.264 intra-frame), or a lightweight encoding method can be selected based on the actual scenario. By using these compression techniques, the data size of the data to be rendered can be reduced while maintaining image quality. In this embodiment, a suitable compression algorithm can be selected based on the application scenario; for example, JPEG compression can be used under low bandwidth conditions, while lightweight H.264 intra-frame compression can be used when real-time performance is required. The compression process can be accelerated on the GPU to ensure high throughput and low latency. In this embodiment, the compression process occurs after the surface compositor synthesizes the UI frames of each terminal application, but before sending the UI frames synthesized by the surface compositor to the graphics buffer queue. By performing the operation of compressing the data to be rendered, this embodiment can significantly reduce the transmission overhead of graphics data, which can improve the efficiency of cross-system communication. At the same time, this operation can also reduce the computational burden on the host side during decoding and rendering. Ultimately, these optimizations achieve a smoother multi-window display experience.

[0079] Before storing the compressed data to be rendered into the graphics buffer queue, it is necessary to extract and package the metadata related to the compressed data. This metadata includes, but is not limited to, the following: timestamp (a synchronization time reference used by the operating system (e.g., Windows) during playback and rendering, ensuring consistency of the image across different windows); resolution (the width and height of the current rendered frame, used to indicate to the host that the window size of the terminal application is correctly set); rotation status (indicating whether the current app interface has been rotated to adapt to different device orientations); channel ID (used to uniquely identify the virtual display channel of the app, distinguishing windows of different terminal applications); display identifier (the identity identifier of the terminal application to which the display identifier belongs in the client system, usually assigned by the WindowManagerService, used to identify the source when creating the corresponding window on the host). This metadata is used on the host for critical operations such as window initialization, layout adjustment, and event binding of the terminal application, ensuring that each terminal application window can independently and accurately present the user interface content of the terminal application. Therefore, metadata is a data structure describing the attributes and context of graphics data, used to assist in the management and scheduling of the rendering process. Metadata is crucial for ensuring the correct parsing and synchronization of graphical data. By extracting and transmitting metadata, the host computer can perform precise window creation, layout adjustments, and input event reverse mapping, while also supporting dynamic display configuration updates and resource optimization.

[0080] A graphics buffer queue is a first-in-first-out (FIFO) buffering mechanism used to temporarily store graphics data (i.e., UI frames) for on-demand reading and rendering on the host side. Each app's data to be rendered and its metadata are encapsulated into a data packet and stored sequentially in the corresponding graphics buffer queue. Graphics buffer queues are typically implemented using shared memory or high-performance communication interfaces (such as Socket or Hyper-V channels) to support low-latency and high-concurrency data transmission. The design of the graphics buffer queue allows graphics data from multiple terminal applications to be generated concurrently on the client side and wait for transmission sequentially, avoiding screen stuttering or frame drops caused by graphics data transmission latency. Furthermore, the graphics buffer queue supports dynamic adjustment of its size to adapt to the performance requirements of the entire data processing system under different operating conditions.

[0081] Packaging is the process of combining compressed graphics data and metadata into a unified format, typically involving serialization to maintain data integrity and consistency during transmission. The packaged data packets can be transmitted between the client and host via standardized protocols, and the receiving system parses and renders the packaged packets on the host. By packaging compressed data and metadata and storing them in an orderly manner in the graphics buffer queue, efficient data transmission and synchronized rendering can be achieved, avoiding data loss and out-of-order issues, thus ensuring the stability and smoothness of multi-window displays.

[0082] In this embodiment, by compressing the data to be rendered, extracting metadata, and packaging and storing it in the graphics buffer queue, the transmission bandwidth and resource consumption can be effectively reduced. This improves the transmission efficiency of graphics data and further enables a smoother and more stable multi-window display effect.

[0083] S1023 uses a graphics buffer queue to transfer the data to be rendered from each terminal application to the host.

[0084] The graphics buffer queue acts as a data source, sending the data to be rendered from each terminal application to the host via cross-system communication channels (such as shared memory, sockets, and Hyper-V channels). After reading the data from the graphics buffer queue, the host can parse it into a format suitable for native Windows window rendering. The data transmission process via the graphics buffer queue typically includes data packaging, transmission control, and flow scheduling to ensure stable transmission of the data to be rendered from each terminal application with low latency and high throughput. This data transmission method using the graphics buffer queue fully utilizes the client system's GPU resources while reducing CPU involvement, thus optimizing overall system performance.

[0085] In this embodiment, by constructing a graphics buffer queue for each terminal application, mapping the rendering data of each terminal application to the display identifier and storing it in the corresponding graphics buffer queue, and then using the graphics buffer queue to transmit it to the host, the rendering data of multiple applications can be efficiently separated and independently transmitted. This avoids the screen conflict or rendering chaos caused by sharing the display channel in traditional solutions, and thus significantly improves the stability and user experience of multi-window display Apps in simulator scenarios.

[0086] In the specific implementation process, step S1023 uses a graphics buffer queue to transfer the data to be rendered from each terminal application to the host. This can be achieved in the following way: a communication channel is created for each terminal application; this communication channel is used to send data through a cross-system communication interface; the channel identifier of the communication channel is the display identifier of the terminal application; then, the data to be rendered in the graphics buffer queue of the terminal application is transferred to the host through the communication channel.

[0087] A communication channel is a data transmission channel established between the host and client sides, carrying the rendering data in the graphics buffer queue of each terminal application. The communication channel allows the client to transmit graphics data from its graphics buffer to the host side in real time for independent windowed display. Communication channels can be created using mechanisms such as shared memory, Socket communication, and Hyper-V channels to ensure low-latency and high-throughput data transmission. The establishment of the communication channel enables the UI rendering stream of each terminal application to be independently transmitted to the host side, generating the corresponding native window on the host.

[0088] Each communication channel has a unique identifier, called the channel identifier, used to distinguish different terminal applications. In this embodiment, the channel identifier is set as the display identifier of the terminal application, that is, the VirtualDisplay ID or Surface ID corresponding to the terminal application. By setting the channel identifier as the display identifier of the terminal application, it can be ensured that the graphics data of each terminal application can be accurately mapped to the corresponding native window of the host, avoiding the confusion of graphics data.

[0089] The communication channel supports cross-system communication interfaces. For example, the client can use the Binder mechanism to transfer data to the host, or it can achieve efficient inter-process communication between the client and host through a custom inter-process communication (IPC) protocol. This support for cross-system communication interfaces helps improve response speed and operational stability in multi-window scenarios.

[0090] In practice, when a terminal application starts, the client allocates a unique communication channel to it and binds this channel to the application's graphics buffer queue. Subsequently, the client's surface compositor packages the composited graphics data and sends it to the host via the communication channel. This approach not only reduces system resource consumption but also improves rendering efficiency, making it particularly suitable for large-scale concurrent applications running in multi-instance scenarios.

[0091] A graphics buffer queue is a set of buffers that store graphics data, typically managed by the GPU driver. Each terminal application's graphics buffer queue is bound to its corresponding communication channel, ensuring that the graphics data within the queue can be independently transmitted to the host. The content of the graphics buffer queue can be raw RGB / YUV pixel data or lightweight encoded (such as H.264 intra-frame or JPEG compression) video frames to reduce bandwidth consumption and improve transmission efficiency. There is a close relationship between the communication channel and the graphics buffer queue for graphics data transmission. The graphics buffer queue is responsible for collecting and caching the graphics data to be rendered from the terminal application, while the communication channel is responsible for efficiently transmitting this graphics data to the host. This separate design ensures that the rendering process of each terminal application does not interfere with each other, and it also improves the overall system stability and responsiveness. In this step, the client sends the graphics data from the graphics buffer queue to the host via the aforementioned communication channel. Since each communication channel is associated with a specific terminal application, the host can quickly identify and process the corresponding graphics data based on the channel identifier, and then draw it onto the corresponding native Windows window. In practical applications, the client continuously submits the synthesized frame data to the graphics buffer queue and pushes it to the host via the communication channel. After receiving the graphics data, the host locates the corresponding window instance based on the channel identifier and updates the image content within the window using the DirectX or OpenGL API, thereby achieving a smooth multi-window display experience.

[0092] By transmitting graphics data from the graphics buffer queue to the host via a communication channel, efficient transmission of graphics data and windowed rendering are achieved, avoiding the performance bottleneck caused by relying on full-screen rendering in traditional emulators. At the same time, it ensures that multiple Android applications can run in the same virtual Android system as independent windows, thereby significantly improving user experience and system resource utilization.

[0093] In this embodiment, a communication channel is created for each terminal application, and the data to be rendered in the graphics buffer queue is transmitted to the host through the communication channel. This method enables precise routing and efficient transmission of graphics data, ensuring that the UI content of each terminal application is independently displayed as a native window on the host. This technical solution provides an operating experience closer to native PC applications and improves multitasking capabilities and user satisfaction.

[0094] In some embodiments, see Figure 5 , Figure 5The method may further include the following steps S201 to S204 after assigning a display identifier to each terminal application via the client: (S102' is shown in step S102). S201, for each terminal application, creates a virtual display device on the client side that corresponds to the display identifier of the terminal application.

[0095] A virtual display device (VHD) is a software-simulated display interface. It doesn't connect to an actual physical screen but instead outputs graphical data to an image stream (Surface) in memory for use by other modules. In the Android system, VHDs are implemented through virtual displays. Each terminal application is assigned a unique display channel ID and an independent VHD to handle the rendering of its UI, ensuring isolation between the application's UI and other applications. The design of VHDs avoids conflicts and performance bottlenecks caused by all applications sharing the same display channel. Furthermore, VHDs provide fundamental support for subsequent data extraction and windowed display.

[0096] Here, creating a virtual display device corresponding to the display identifier of the terminal application on the client side can be achieved in the following way: First, create a virtual display adapter; this virtual display adapter is used to manage the virtual display device; then, call the virtual display adapter to construct the virtual display device corresponding to the display identifier.

[0097] A virtual display adapter is a software component used to manage and control virtual display devices. It is responsible for allocating resources, configuring parameters, and coordinating the interaction between multiple virtual display channels. As the core of the virtual display mechanism, the virtual display adapter supports dynamic settings for attributes such as resolution, refresh rate, and pixel format for each independent window, ensuring that the displayed content of different application windows does not interfere with each other.

[0098] In practice, virtual display adapters can be built based on the DisplayManagerService (DMS) interface provided by the Android system. For example, in a PC-based software management tool, when a user opens an Android app, the host sends a request to the client. The client then calls DMS to create a new virtual display adapter instance and assigns a unique display identifier to it. The virtual display adapter is then responsible for initializing the virtual display channels in the surface compositor and binding the Android app's rendering output to these channels. By introducing a unified virtual display channel management system, multiple virtual display channels can be centrally managed, allowing resource configuration based on the needs of different applications, thereby improving system compatibility and stability.

[0099] Calling a virtual display adapter is used to construct a virtual display device corresponding to a specific display identifier (such as an app window). The process of calling a virtual display adapter typically involves specifying parameters such as the size (width and height), orientation, and DPI of the display area, and registering these parameters with the surface compositer so that the operations registered in the surface compositer can enable subsequent graphics compositing and data transmission.

[0100] In real-world scenarios, such as when a user runs two apps in a software management tool on a PC, the host machine will call the virtual display adapter to create independent virtual display device objects for each app. Each app's virtual display device will serve as its rendering target, ensuring that the screens of the two apps are correctly composited and displayed in their respective Windows windows on the host machine. By employing a split-screen or multi-window management system, independent display and rendering of each app can be achieved, avoiding screen conflicts or resource contention, thereby improving multitasking efficiency and user experience.

[0101] In this embodiment, by creating a virtual display adapter and constructing a virtual display device corresponding to the display identifier, independent windowed display of multiple apps can be achieved. This technical solution reduces resource consumption and improves system stability. This improved system stability supports more efficient application windowing operations. This efficient application windowing operation better meets the needs of users using Android emulators in a PC environment.

[0102] In this embodiment of the application, the virtual display device may include a virtual built-in display device; the virtual display adapter is invoked to construct the virtual display device corresponding to the display identifier, which may involve first checking the display change event of the hardware synthesizer; then, in response to the display change event being a built-in display creation event, the virtual display adapter is invoked to construct the virtual built-in display device corresponding to the display identifier.

[0103] The hardware compositor is a module in the Android system responsible for compositing graphics layers into a final image for output to the physical display. Display change events are system events triggered when the status of a connected external display changes (such as plugging / unplugging an HDMI cable, adjusting resolution, etc.). By detecting display change events, the client can respond promptly and update its display configuration. For example, when a user plugs in an external display, the surface compositor detects the display change event and notifies the virtual display device to take appropriate action. This ensures that the client can perceive changes in the host's display environment in real time and make dynamic adjustments.

[0104] The "Create Built-in Display" event refers to the process by which the surface synthesizer actively creates a virtual built-in display device representing the host's built-in display when the system initializes or detects that specific conditions are met. The virtual display adapter is an interface class used to manage virtual built-in display devices, responsible for coordinating the creation and rendering data output of these devices. The display identifier is a unique identifier used to distinguish different display channels. When the "Create Built-in Display" event is detected, the client, based on its current display configuration information (such as resolution, refresh rate, DPI, etc.), invokes the virtual display adapter to generate a virtual built-in display device that matches the display identifier. This process of invoking the virtual display adapter to generate a virtual built-in display device matching the display identifier allows the client to simulate the behavior of a real physical display, thereby achieving the effect of independent multi-window display.

[0105] There is a close data interaction between display change events and the virtual display adapter. When the hardware synthesizer detects a display change event, it passes relevant parameters to the virtual display adapter so that the virtual display adapter can correctly construct a virtual built-in display device that matches the current display environment. For example, upon detecting a new HDMI display connection, the virtual display adapter can dynamically create a virtual display device with the corresponding resolution and refresh rate based on the display's Extended Display Identification Data (EDID). This mechanism allows the client to flexibly adapt to various display configurations, improving system compatibility and stability.

[0106] In this embodiment, by checking the display change events of the hardware synthesizer and calling the virtual display adapter according to the event type to construct a virtual built-in display device, this embodiment can realize real-time perception and dynamic response to the display environment. This embodiment can support the function of independent display of multiple windows. This embodiment can improve user experience and system flexibility.

[0107] In some embodiments, if the display change event is an external display creation event, then in response to the display change event being an external display creation event, an external display creation function can be called to create an external display corresponding to the display identifier; the external display is used to display virtual composite images of different terminal applications, and the external display uses a hardware compositor to composite at least one virtual layer, and uses a hardware compositor to display the virtual composite image on the external display.

[0108] The external display creation function refers to a set of predefined interfaces or APIs in the Android system on the client side. When a display change event indicating the creation of an external display is detected, DMS or other related services call the external display creation function to dynamically generate a new display device instance. The external display creation function receives parameters including display resolution, refresh rate, and DPI, and initializes a new virtual display device or HDMI display device based on these parameters. The calling process of the external display creation function ensures that the client side can flexibly adapt to different display needs, thereby supporting multi-window or multi-monitor application scenarios.

[0109] Virtual composite images are image streams output to a specific Surface after being composited by a surface compositer, based on the graphical data rendered by multiple terminal applications running on the client side. These virtual composite images are not directly displayed on the physical screen, but rather serve as image data streams in memory for further processing and rendering on the host side. The UI content of each terminal application is independently composited into a virtual display channel, forming an independent virtual composite image corresponding to that terminal application. This virtual composite image is ultimately transmitted to the host side and displayed in the form of a native Windows window.

[0110] In practice, the external display creation function not only creates display channels but also ensures that the newly created external display meets the needs of different application scenarios by passing configuration parameters (such as resolution and refresh rate). For example, in a mobile office scenario, a user may need to run multiple Android apps simultaneously and want each app to be displayed as an independent window on the PC desktop. In this case, the external display creation function can dynamically generate multiple display channels according to the user's request, each corresponding to a different app interface. Furthermore, when a user requests the creation of a new external display on the host side, the client-side DMS receives a corresponding event notification and calls the external display creation function to generate the corresponding virtual display. For example, in a software management tool, when a user clicks the "New Window" button, the external display creation function is automatically called to allocate an independent display channel for the new app to be run and initiate the corresponding graphics rendering process. Subsequently, all UI content of the new app will be rendered independently and output to the host-side Windows window through a hardware compositor. The user can then see the new app's independent interface on the desktop and perform operations such as dragging and scaling on the new app.

[0111] S202, Generate at least one virtual layer based on the data to be rendered from the terminal application.

[0112] Virtual layers are graphical layers used in Android's surface compositor to host UI elements (such as buttons and text boxes) of a terminal application. Virtual layers are one or more graphical layers generated for a specific terminal application during the compositor's compositing phase. These layers can be composited by the GPU and ultimately output to the buffer queue of the virtual display device. The process of generating virtual layers is completed in the surface compositor, specifically including identifying the rendering target Surface of the terminal application, adding it to the virtual layer list, and preparing for compositing according to the rendering strategy. The UI rendering result of each terminal application is decomposed into multiple layers for independent compositing and management. Virtual layers can contain different resolutions, scaling ratios, formats, and update frequencies to adapt to the needs of different application scenarios. For example, a video player may contain a main video layer and a subtitle layer, which are rendered independently and then composited. By generating independent virtual layers for each terminal application, the integrity and independence of the UI content of each terminal application can be guaranteed, while improving the efficiency and flexibility of graphics compositing. Generating independent virtual layers for each terminal application helps to achieve high-quality windowed display effects, enhancing user immersion and operating experience.

[0113] S203, composite at least one virtual layer to obtain a virtual composite image for the terminal application.

[0114] Virtual composite images refer to the final image result obtained by merging multiple virtual layers according to a certain order and rules. The virtual layer compositing process is completed by a surface compositor, which is responsible for blending the contents of each layer and outputting it to a specified display target. In the embodiments of this application, the display target is the created virtual display device. Through the layer compositing method completed by the surface compositor, the UI rendering results of each terminal application are integrated into an independent image, thereby facilitating subsequent transmission and display. In addition, lightweight compression (such as H.264 intra-frame encoding) can be performed during the compositing process of the surface compositor to reduce the bandwidth consumption of data transmission and improve overall efficiency.

[0115] S204 outputs the virtual composite image of the terminal application to the virtual display device.

[0116] Outputting virtual composite images from the terminal application to a virtual display device enables the export and submission of virtual composite images from the GPU buffer to the virtual display device. The virtual display device, acting as the receiving end, continuously monitors the virtual composite images from the surface compositer and caches them in the corresponding buffer queue. Since the virtual display device itself is not connected to a physical screen, the graphics data is not directly displayed but is transmitted to the host as intermediate data. The key to this step is ensuring the continuity and stability of the virtual composite image output, thus providing a reliable foundation for subsequent cross-system transmission.

[0117] In this embodiment, outputting the virtual composite image of the terminal application to the virtual display device can be achieved by first constructing a buffer queue for the terminal application in a preset frame buffer based on the display identifier of the terminal application; the buffer queue is connected to the virtual display device corresponding to the display identifier; then, the virtual composite image of the terminal application is stored in the buffer queue; finally, the buffer queue is used to transmit the virtual composite image of the terminal application to the virtual display device.

[0118] A frame buffer is a memory area used to temporarily store graphics data for rendering and display operations. In this embodiment, multiple frame buffers can be preset to support multiple terminal applications running simultaneously and outputting images independently. A buffer queue is a set of graphics buffers arranged in a first-in, first-out (FIFO) order, serving as an intermediate caching mechanism between producers and consumers. Producers (such as surface compositors) are responsible for writing graphics data into the buffer queue, while consumers (such as the host) are responsible for reading and displaying images from the buffer queue. The buffer queue's role is to achieve efficient image transmission and reduce data loss and latency.

[0119] Accordingly, the process of mapping the data to be rendered for each terminal application to its corresponding display identifier and transmitting it to the host in step S102 can be achieved through the following step S205: S205 maps the virtual composite image of each terminal application to its corresponding display identifier and transmits it to the host.

[0120] In this step, a clear mapping relationship is established between the virtual composite image and the display identifier. This mapping relationship ensures that the host can correctly bind the graphics data to the corresponding Windows native window. This mapping relationship allows the UI content of each terminal application to be presented on the host as an independent Windows native window, allowing users to freely drag, scale, and switch between application windows as if operating local PC programs.

[0121] In this embodiment, a virtual display device is created for each terminal application, and a virtual layer is generated. The image is then composited, and the composited graphic data is output to the virtual display device. A mapping relationship is established between this graphic data and the corresponding display identifier, and the mapped data is transmitted to the host. Through this method, multiple terminal applications can be displayed in independent windows, thereby reducing system resource consumption, improving user experience, and promoting better adaptation and integration of Android applications into the PC operating system ecosystem.

[0122] In this embodiment, in response to the data format of the virtual display device being a multimedia codec format, the virtual composite image can be encoded into a first format corresponding to the multimedia codec format, and the virtual composite image in the first format can be mapped to the corresponding display identifier and then transmitted to the host; or, in response to the data format of the virtual display device being a surface texture format, the graphics rendering program interface can be called to map the virtual composite image to the corresponding display identifier and then transmit it to the host.

[0123] Multimedia codec formats refer to standard formats used for compressing and decompressing video or audio data, such as H.264, H.265 (HEVC), VP8, and VP9. Multimedia codec formats use specific algorithms to efficiently compress images, thereby reducing the need for transmission bandwidth and storage space. In this embodiment, when the client detects that the virtual display device uses a multimedia codec format, it converts the virtual composite image into the multimedia codec format for transmission. Converting the virtual composite image into a multimedia codec format for transmission can effectively reduce transmission latency and improve rendering efficiency.

[0124] Surface texture format refers to a texture object format provided by graphics APIs such as OpenGL ES or Vulkan, which can be used in graphics rendering engines. Surface texture format does not compress images but retains the original pixel data, making it suitable for high-precision rendering requirements. In this embodiment, when the client detects that the virtual display device uses a surface texture format, the graphics processing system controlled by the client will not compress the virtual composite image, but will directly transmit the virtual composite image to the host through the SurfaceTexture interface.

[0125] A graphics rendering interface (GFA) refers to a set of function libraries used to control graphics hardware for rendering, such as OpenGL ES, Vulkan, and DirectX. In this embodiment, the client end calls the GFA to encapsulate a virtual composite image into a texture object and transmits it to the host end via shared memory or other high-performance communication methods. The host end then uses the GFA to render the virtual composite image and generate the corresponding native window.

[0126] S103 creates an application window for each terminal application based on a display identifier on the host side and displays the application window of each terminal application on the current interface of the electronic device.

[0127] Upon receiving a display identifier and its corresponding data to be rendered, the host computer creates an independent native window for each display identifier. This native window is the application window of the terminal application, similar to a window handle (HWND) window in Windows. For example, when receiving data to be rendered for DisplayID-001, the host computer creates a window named XX1; when receiving data to be rendered for DisplayID-002, the host computer creates a window named XX2.

[0128] Each application window created on the host side has an independent title bar, icon, minimum size limit, and size control functionality. Once created, the application windows are displayed in the user's desktop environment. Users can freely drag, resize, minimize, or maximize the various application windows created on the host desktop. For example, a user can open multiple different apps simultaneously, each running in an independent application window, with no interference between the windows. This design achieves a windowed integration experience for Android applications on the PC desktop, allowing users to view the interfaces of multiple Android applications at the same time without being limited by the traditional single-window mode, significantly improving the user's multitasking efficiency and ease of operation.

[0129] In some embodiments, when creating an application window for each terminal application based on a display identifier on the host side, it can be implemented in the following way: in response to receiving the first data packet in the graphics buffer queue of any terminal application on the host side, the first data packet is parsed to obtain the first data to be rendered and metadata; then, an application window is created for the terminal application based on the metadata.

[0130] The graphics buffer queue is a queue structure on the client side used to store graphics data. Each terminal application requests an independent graphics buffer from the surface compositor upon startup. This graphics buffer stores the graphics data rendered after the terminal application's UI. Once the first piece of graphics data is ready, it is placed in the graphics buffer queue and transmitted to the host side via the cross-system communication interface. The management of the graphics buffer queue ensures the orderliness and efficiency of the rendered frames. The first data packet is the first data unit in the graphics buffer queue transmitted to the host side. This first data packet contains the graphics data of the first frame and related metadata. This information is crucial for the host side to correctly parse and process the graphics data. For example, the timestamp in the metadata can be used to synchronize the playback time of multiple application windows, avoiding screen stuttering or delays; the channel ID in the metadata is used to determine which terminal application's application window the first data packet in the graphics buffer queue belongs to, thus correctly assigning the first data packet in the graphics buffer queue to the corresponding application window. The reception of the first data packet marks the official start of the terminal application's UI rendering process. The host side can trigger subsequent window creation and rendering processes based on the reception of the first data packet.

[0131] The parsing operation refers to extracting the image data and metadata from the first data packet in the graphics buffer queue and organizing it according to a predefined data structure for subsequent processing. The parsing process includes decoding the image format, verifying data integrity, and extracting key parameters. The parsing process is executed by the host, which decodes the content of the first data packet to extract the first image content and metadata (such as resolution, timestamp, channel ID, etc.). This parsing process ensures that the host can accurately identify the basic attributes and display requirements of the terminal application, thereby creating a suitable native window for the client. In this embodiment, by receiving and parsing the first data packet on the host, the initial display parameters of the terminal application can be accurately obtained, providing basic data support for subsequent window creation and rendering. This enables an efficient multi-window initialization process, thereby improving the overall response speed and operating efficiency of the data processing system, and ultimately enhancing the user experience.

[0132] Creating an application window based on metadata refers to the host generating the corresponding application window based on the metadata parsed from the first data packet. The metadata parsed from the first data packet determines the window's appearance, size, position, and other basic attributes. An application window is a native window created on the host (such as a Windows operating system) to host the UI content of a specific application within a terminal application (Android system). Each application window has an independent window handle (HWND), title bar, icon, size, position, and other attributes, and supports common window operations such as minimizing, maximizing, dragging, and closing. In this embodiment, the application window is dynamically generated based on key parameters in the metadata (such as App name, resolution, channel ID, etc.). The process of creating an application window includes: calling the Windows API to create an HWND window handle; setting the window title to the application name, loading and setting the window icon; and initializing the graphics context (such as DirectX or OpenGL) for subsequent image rendering. Simultaneously, the host establishes a communication channel with the client to continuously receive subsequent graphics data and update the image content in the application window. In this embodiment, by creating application windows based on metadata, it can be ensured that each terminal application can be presented as an independent native window on the host. This presentation method can maintain the consistency and compatibility of application windows, enabling these application windows to better integrate into the PC desktop environment, and ultimately improve the user's operating experience and interaction efficiency.

[0133] In this embodiment, by responding to the reception and parsing of the first data packet and combining it with metadata to create an independent native window, the terminal application can be efficiently displayed on the host side. This reduces system resource consumption, improves multitasking capabilities, further enhances the flexibility and practicality of the Android emulator on the PC platform, and ultimately promotes the development of the cross-platform application ecosystem. In summary, this embodiment can efficiently complete the window initialization and rendering process of the terminal application on the host side. Specifically, in actual implementation, firstly, the host side detects the first data packet in the graphics buffer queue from the client side, indicating that a terminal application is ready for display. Subsequently, the host side parses the data packet, extracting key display parameters such as resolution, timestamp, and channel ID. Next, the host side creates the corresponding native window based on this metadata and initializes the corresponding graphics context to prepare for subsequent image rendering. Finally, the host side continuously receives subsequent graphics data through the communication mechanism with the client side and updates the window content in real time, achieving a smooth display effect. The entire process achieves seamless integration and efficient operation of the terminal application on the PC platform.

[0134] In some embodiments, after creating an application window for a terminal application based on metadata, the host can also obtain the application name and application icon of the terminal application from the client; then, the window title of the application window is set to the application name of the terminal application, and the window icon of the application window is set to the application icon of the terminal application.

[0135] The host program is the emulator's main control program running on the electronic device. This program is responsible for managing the startup, configuration, and status monitoring of the virtual phone environment (client). Through the communication interface with the virtual phone environment, the host program can obtain real-time information about the terminal application, including but not limited to the application name and icon. The application name is a unique identifier for the terminal application, usually defined by the developer, used to indicate the application's function or purpose to the user. The application name can be written into the manifest file of the Android installation package (APK) to uniquely identify the terminal application. The application icon is a small graphic identifier displayed on the desktop or launcher. The icon is designed by the developer and embedded in the APK file to visually help users quickly identify the terminal application.

[0136] The host and client establish a connection via shared memory, Socket communication, or a Hyper-V channel, ensuring the host can obtain metadata about the terminal applications on the client in real time. When the connection is established via shared memory, Socket communication, or a Hyper-V channel, the personalized identifiers of each terminal application can be automatically identified and extracted without additional configuration, resulting in a display effect closer to native PC operating habits. Furthermore, in practical implementation, to ensure that program names and icons are also obtained promptly and accurately, a stable communication mechanism can be configured between the host and client. This stable communication mechanism can be implemented through Remote Procedure Call (RPC), message queues, etc., which guarantee the reliability and efficiency of data transmission.

[0137] An application window is a native Windows window (HWND) created by the client for each independently running terminal application. The application window title is a text label displayed on the desktop, typically used to help users quickly identify the window's content. The application window icon is a small graphic icon in the upper left corner of the application window, also serving an identification function. In this embodiment, the application window title and icon are dynamically set to the corresponding terminal application's program name and icon, respectively. This configuration aims to ensure visual consistency between each application window and its corresponding terminal application, enhancing user recognition and operational experience. For example, when a user opens App1 in a software management tool on a PC, the application window title is App1, and the icon is the standard App1 icon, a display effect completely consistent with running App1 on an Android device. In actual implementation, the application window title and icon settings also support dynamic updates to address potential changes in the program name or icon during terminal application operation, ensuring the user interface remains up-to-date.

[0138] In summary, by obtaining the application name and icon of the terminal application from the client and applying the application name and icon to the corresponding independent window, the user's intuitive understanding of the window content can be improved, the user experience in multi-window scenarios can be enhanced, and an operating experience closer to that of native PC applications can be achieved.

[0139] S104, in each terminal application's application window, perform window texture rendering based on the data to be rendered.

[0140] Window texture rendering refers to the process where the host machine receives the rendering data from the terminal application and then performs graphics drawing on the native Windows window based on this data. The process includes texture mapping, color adjustment, scaling, and other operations, ultimately achieving high-quality visual presentation. For example, when an app on the client side generates a frame of image data, this frame is encapsulated as texture information and transmitted to the host machine via shared memory or sockets. The host machine uses graphics rendering interface technologies (such as DirectX or OpenGL) to draw this texture information. For instance, for the window of the terminal application XX1, the host machine uploads the image frame data obtained from the client to the GPU, generates texture maps, and updates the window's display content. Due to the efficient rendering mechanism, each window can achieve smooth screen refresh and supports high resolution and multi-monitor output. The host machine uses DirectX or OpenGL technologies to achieve smooth and clear window rendering effects. Furthermore, during the rendering process, personalized configurations such as window transparency adjustment and top-mount settings can be applied to meet the operating habits of different users.

[0141] In practical implementation, through steps S101 to S104, multiple apps within a single Android system instance can be displayed as independent windows on the host machine, eliminating the need to launch a separate Android virtual machine instance for each app. This not only reduces system resource consumption but also improves overall operating efficiency and stability. Simultaneously, each application window possesses complete native window attributes, allowing users to switch, adjust, and layout them as if operating a local PC program, greatly enhancing operational flexibility and multitasking capabilities. In other words, by assigning an independent display identifier to each terminal application running on the client machine and mapping its rendering data to the native window on the host machine, the effect of displaying multiple Android applications as independent windows on the host machine is achieved. Compared to traditional multi-instance methods, this solution only requires running one client instance, avoiding the problem of repeatedly loading system resources, thus significantly reducing CPU, memory, and storage consumption and improving overall performance and user experience. Furthermore, since each application window is a native window, it can seamlessly integrate with other applications on the PC, supporting advanced functions such as cross-application copy and paste and file drag and drop, truly achieving deep integration of Android applications and the PC ecosystem.

[0142] The emulator-based data processing method provided in this application assigns a display identifier to each terminal application on the client side and maps the data to be rendered to the display identifier before transmitting it to the host side. The host side creates an application window for each terminal application based on the display identifier and displays it on the current interface of the electronic device. Finally, the host side performs window texture rendering based on the data to be rendered. This method effectively solves the problems of low efficiency, high resource consumption, and poor user experience in multi-window display in traditional emulators, and achieves the goal of displaying multiple Android applications in independent windows on the host side in a single client system, significantly improving resource utilization and user experience.

[0143] In some embodiments, see Figure 6 , Figure 6 The method may further include the following steps S301 to S302 after rendering the window texture based on the data to be rendered: S301, in response to a program exit operation for any terminal application, sends a program exit notification message for any terminal application to the host via the client.

[0144] An application exit operation refers to the act of a user or system actively terminating a terminal application. Application exit can be triggered by the user clicking the close button, selecting the exit menu item, using a shortcut key (such as Alt+F4), or automatically by the system (such as forced shutdown due to insufficient resources). Application exit operations are usually accompanied by the cleanup of application state and the release of resources.

[0145] In this embodiment, the client-side is responsible for communicating with the host-side and forwarding program exit notification messages. When a program exits, the client-side generates the program exit notification message and sends it to the host-side. The program exit notification message is a structured data packet sent by the client-side to the host-side after detecting a program exit operation, informing the host-side which application has been closed. The program exit notification message typically contains metadata such as the target application's unique identifier (e.g., process ID, display ID) and the exit reason (normal or abnormal exit). Alternatively, the data structure of the program exit notification message may include the following fields: application identifier (App ID) (used to uniquely identify the closed terminal application), session identifier (Session ID) (used to identify the client-side instance where the terminal application is running), and timestamp (used to record the time of the exit event).

[0146] The host is responsible for creating and maintaining the communication connection with the client, receiving messages from the client, and executing corresponding control logic based on the message content, such as the emulator management core component closing the communication channel and the emulator management core component destroying the window.

[0147] In actual implementation, when a user closes an application window, the client detects this action and immediately generates a program exit notification message. The client then transmits this notification message to the host through the established communication channel to ensure the system can respond promptly to the user's action of closing an application window.

[0148] S302, in response to the program exit notification message on the host side, closes the communication channel of any terminal application and stops displaying the application window of any terminal application.

[0149] A communication channel is a data transmission link established between the host and client sides, used for real-time exchange of rendering frame data, window states, input events, and other information. Communication channels can be implemented based on shared memory, sockets, or other high-performance communication protocols to ensure low latency and high throughput. An application window is a native Windows window created by the host for each independently running Android app. When an application window is closed, the corresponding terminal application's communication channel also needs to be closed synchronously to prevent invalid data transmission and resource leaks.

[0150] Upon receiving a program exit notification message, the host first verifies the validity of the message source and content. After confirming the identity of the target application, the host calls the underlying interface to close the communication channel associated with the corresponding terminal application. Subsequently, the host destroys the application window instance of the terminal application and releases related resources (such as texture buffers, drawing contexts, etc.), thus completing the entire shutdown process. In a specific implementation, to improve the user experience, the host can perform an animation transition (such as a fade-out effect) before closing the application window, and then officially destroy the application window. In addition, the host can also record the lifecycle state of the application window for subsequent log analysis or performance testing.

[0151] The method in this embodiment responds to the program exit operation and closes the communication channel and application window corresponding to the program exit operation. This method can avoid invalid data transmission and resource occupation, thereby effectively releasing system resources, improving overall operating efficiency and enhancing system stability.

[0152] In some embodiments, see Figure 7 , Figure 7 The method may further include the following steps S401 to S402 after rendering the window texture based on the data to be rendered: S401, in response to window interaction operations for any terminal application, performs operation responses on the application window of any terminal application through the host, and displays the operation response results for the application window on the current interface of the electronic device.

[0153] Window interaction operations include at least one of the following: window maximization, window minimization, and window closing; these operations are performed through an input device connected to an electronic device. Window interaction operations refer to a series of actions performed by a user on a specific application window through an input device connected to an electronic device. For example, a user clicks the maximize button to display the window content in full screen, clicks the minimize button to hide the window in the taskbar, and clicks the close button to end the window's operation. Window interaction operations are typically driven by user intent; users adjust the window state through these operations, thereby managing screen space and multitasking more flexibly. An input device refers to a hardware device used to input data and instructions into a computer system, including but not limited to a mouse, keyboard, touchscreen, stylus, and game controller. In this embodiment, the input device acts as a bridge between the user and the data processing system, responsible for receiving the user's physical operations and converting them into recognizable digital signals, which are then processed by the operating system.

[0154] Electronic devices can establish a communication connection with input devices. Window interaction operations can be triggered by the input device, which can then control the application window. Input devices can include, but are not limited to, any of the following: a mouse, a touchpad, and a handheld control component. A mouse is an external input device for electronic devices, primarily used to control the position of the cursor on the screen, manipulating screen elements through its buttons and scroll wheel. A touchpad is a widely used input device on electronic devices, controlling cursor movement by sensing finger movement; it can be considered a substitute for a mouse. A handheld control component is a handheld input device used to control the position of the cursor on the screen; for example, a handheld control component can include a gamepad. Furthermore, this application does not limit the method of establishing a communication connection between the input device and the electronic device; the input device can be externally connected to the electronic device, or it can be built into the electronic device. External input devices refer to devices that establish a communication connection with electronic devices. These connections can be made via wired or wireless communication; for example, a wired mouse can communicate with an electronic device via wired communication, while a wireless mouse can communicate with an electronic device via wireless communication. Internal input devices, on the other hand, are devices that are integrated into the electronic device itself, such as a touchpad.

[0155] In this embodiment, window interaction operations not only support common mouse click events but also shortcut key combinations (such as Alt+Enter for window switching and Alt+F4 for closing windows) to improve user efficiency. The input device and the host establish a connection via a standard communication protocol (such as Windows' message mechanism) to ensure that operation commands are transmitted in real time and correctly parsed by the standard communication protocol. Operation response refers to the host executing corresponding window management actions based on the operation type after receiving the window interaction operation, specifically including adjusting the window size, position, and state. Operation response result refers to the status feedback information generated by the host after completing the window operation. The operation response result is usually displayed to the user in a graphical interface, indicating that the application window has successfully performed the specified operation. For example, when the user clicks the maximize button, the screen will display the visual effect that the application window is now full-screen; when the user clicks the close button, the application window will disappear, possibly accompanied by an animation indicating that the operation is complete. The above operation response behavior is ultimately reflected in the current interface of the electronic device, allowing the user to intuitively see the result of the window changes. This application embodiment supports multiple window interaction operations, which can improve the user's flexibility in controlling multiple Android application windows, thereby better adapting to the needs of multitasking and improving user experience and work efficiency.

[0156] S402, in response to the completion of the operation response for the window interaction operation, synchronizes the operation response result to the client.

[0157] Synchronization mechanisms can be implemented through shared memory, Socket communication, or remote desktop protocols to ensure low latency and high reliability of data transmission. In practical scenarios, when a user maximizes an Android app window on a PC, the host computer adjusts the window to full-screen mode and synchronizes this window size change to the client's Android system, thus changing the corresponding window state on the client's Android system. This embodiment of the application, by synchronizing the operation response result to the client, ensures that the window states on the host and client sides remain consistent, thereby avoiding display anomalies caused by state asynchrony and further improving overall system stability and collaborative efficiency.

[0158] In this embodiment, the system responds to user window interaction operations on the application window, executes the corresponding operation response on the host side, and synchronously transmits the operation response result to the client side. Through this method, cross-platform window management functionality can be achieved, thereby enhancing the user's control over multiple Android applications in the emulator environment and further providing a more efficient multitasking capability that closely resembles the native PC operating experience.

[0159] In summary, the steps in the above embodiments are closely collaborative: First, the user initiates a window interaction operation through an input device; the host receives and executes the corresponding operation response; then, the operation response result is fed back to the electronic device interface for display; finally, the operation response result is transmitted to the client through a synchronization mechanism to maintain consistency between the window states on the host and client sides. This process design effectively improves window management efficiency and user experience in a cross-platform environment.

[0160] In some embodiments, see Figure 8 , Figure 8 The method may further include the following steps S501 to S502 after rendering the window texture based on the data to be rendered: S501, in response to a window interaction operation for any terminal application, encapsulates the window interaction operation into a client-side input operation on the host side and sends the client-side input operation to the client side.

[0161] S502 responds to input operations from the client and displays the response results for the application window on the current interface of the electronic device.

[0162] Window interaction operations include at least one of the following: window maximization operation, window minimization operation, and window closing operation; window interaction operations are performed through an input device connected to an electronic device.

[0163] Client-side input operations refer to the process by which the host machine converts received window interaction operations into a standard input event format that the emulator can recognize, so that the client (i.e., the Android virtual environment) can correctly parse and respond. Mapping native PC window operation commands to the corresponding InputEvent types (such as ACTION_DOWN, ACTION_UP) in client-side input operations, and attaching necessary coordinate information and timestamps, achieves cross-platform compatibility and consistency.

[0164] An operation response refers to a series of processing logics executed by the client based on the received input from the client, including but not limited to updating the application window state, redrawing the interface content, and calling relevant callback functions. After the operation response is completed, the client will send the new window state back to the host to ensure that the host can update the display effect in a timely manner.

[0165] The current interface of an electronic device refers to the PC desktop environment running the emulator. Users can view all open, independent application windows on the current interface. Each application window can be dragged, resized, stacked, or hidden, just like a local Windows program. These features greatly enhance the continuity of the user experience and operational efficiency.

[0166] In practical implementation, there is a clear mapping relationship between window interaction operations and client-side input operations. After receiving the original window interaction signal from the input device, the host end converts the original window interaction signal into a client-side input operation conforming to the Android system standard to ensure that the client end can correctly recognize and execute the corresponding logical processing. The process of the host end converting the original window interaction signal into a client-side input operation involves not only the conversion of event types, but may also include coordinate system adaptation and timestamp synchronization to ensure the accuracy and real-time performance of the operation. In practical implementation, the result of the operation response ultimately needs to be fed back to the display layer of the host end, so that the user can intuitively see the interface changes caused by the user operation on the current interface of the electronic device. The above-mentioned two-way communication mechanism is a key link in realizing efficient human-computer interaction in the embodiments of this application. This communication mechanism ensures that the operation initiated by the user on the host end can receive an immediate response from the client end and be visually presented on the host end, thereby forming a complete operation loop.

[0167] In summary, in this embodiment, by introducing a reverse mapping mechanism for window interaction operations after window texture rendering, accurate responses to native PC window operations can be achieved on the Android client. This reverse mapping mechanism allows users to freely control the state of client application windows on the host side, thereby improving the flexibility and intuitiveness of multi-window operations and significantly optimizing the multitasking experience in emulator scenarios.

[0168] The following will describe an exemplary application of the embodiments of this application in a real-world application scenario.

[0169] With the rapid development of the mobile application ecosystem, Android emulators have become an important tool for developers to test, gamers and enterprise office users to run Android applications on personal computer platforms. Currently, most mainstream Android emulator products use a single-window hosting mode to run the Android system. Some emulators provide multi-instance functionality, allowing users to launch multiple independent Android instances (i.e., multiple Guest systems) at the same time, with each instance running one application. However, this multi-instance method essentially achieves multitasking through resource copying, rather than true multi-window integration. It can be seen that the solutions in the relevant technologies still have at least one of the following problems: (1) Single-window limitation: Users cannot display multiple terminal applications in parallel in independent windows like PC applications. All terminal applications run in the same simulated mobile phone interface. Users cannot freely drag, scale, and operate across application windows like using native PC applications. This limitation restricts the efficiency of multitasking. (2) High resource consumption: Users achieve multiple application windows through multi-instance. This operation means that every time a new App is opened, the system needs to launch a complete Android system instance, which leads to a large resource consumption and a decline in the overall system performance, making it difficult to support large-scale concurrent application scenarios. (3) The systems in the related technologies cannot make full use of the multi-window, multi-monitor operation features of personal computers, resulting in a fragmented user experience during use, which affects operating efficiency and immersion.

[0170] To address the aforementioned issues, this application proposes a novel Android emulator architecture capable of running multiple apps in a multi-window manner, significantly improving user experience and system resource utilization. Specifically, the technical solution of this application includes the following core elements: In implementing this application, the WindowManagerService and SurfaceFlinger modules within the Android Open Source Project (Android AOSP) framework are customized, overcoming the traditional Android system's design limitation of uniformly compositing all application windows into a single display layer. Specifically, during Activity startup, the emulator system identifies the application's ownership and allocates an independent Virtual Display Channel (VLC) to each application. Each VLC is bound to a unique SurfaceID, ensuring that the UI rendering stream of each VLC with its unique SurfaceID is isolated from other terminal applications. The emulator system generates a corresponding independent PC window (i.e., the application window) on the host side. Furthermore, to improve resource utilization and reduce system overhead, compared to related technologies that rely on "multiple" Android instances to achieve multi-window operation, this application embodiment only needs to start a complete Android system instance (Guest OS). All applications share the same set of system services (such as Zygote, System Server, PackageManager, Battery Service, etc.), avoiding the huge resource consumption caused by repeatedly loading the kernel and running multiple system service processes (System Server). Moreover, to enhance user experience and make the emulator's usage scenarios closer to the operating habits of native PC applications, this application embodiment fundamentally reconstructs the interaction paradigm of the Android emulator, upgrading the original operation mode limited to "simulating a mobile phone screen" to a desktop-level experience with multi-window, multi-tasking, and freely layoutable elements that conforms to PC users' habits.

[0171] Here, we will first describe the product-side implementation of the simulator-based data processing method provided in the embodiments of this application.

[0172] The simulator-based data processing method provided in this application can be applied to software management tools, which support multi-window parallel processing, eliminating the inefficient mode of switching within windows. Figure 9As shown in the PC desktop 900, when a user runs multiple Android applications (i.e., terminal applications) in the software management tool, each Android application automatically generates an independent Windows window (i.e., the application window mentioned above), such as... Figure 9 The example shows Windows window 901 for Android application XX1 and Windows window 902 for Android application XX2. Users can switch between Android applications at any time, just like operating a local PC program.

[0173] Furthermore, it enables cross-platform collaboration, allowing Android applications to better integrate into the Windows ecosystem and improving cross-screen collaboration and multitasking experiences. For example, it allows opening Android application XX1 and Android application XX2 simultaneously, and running them concurrently with PC software. Each Android application window possesses complete native window attributes, such as support for always-on-top windows, adjustable transparency (e.g., a semi-transparent floating chat window), and customization of the window title bar, icon, minimum size, etc. Figure 10 The PC desktop 1001 shown is a Windows window 1002 of the Android application XX1 that is displayed on top.

[0174] The technical implementation process of the simulator-based data processing method provided in the embodiments of this application will be described below.

[0175] The core of this application's embodiments lies in modifying the WindowManager and related rendering logic in Android AOSP, enabling each App's UI rendering data (i.e., data to be rendered) to be independently extracted in the Guest system and transmitted to the Host, where it is drawn into an independent native Windows window in the PC environment. For example... Figure 11 As shown, the entire technology chain includes the following steps ( Figure 11 (The diagram illustrates the process of launching two terminal applications, App1 and App2). In step S1101, the Guest terminal detects the launch events of all Activities and assigns an independent display identifier to each terminal application.

[0176] Here, when the Activity starts, the modified WindowManagerService assigns a unique display identifier (DisplayID) to each terminal application and binds it to a dedicated surface controller (SurfaceControl).

[0177] An Activity is one of the basic building blocks of a terminal application running in the Android system. It represents a single screen of a user interface. Terminal applications running in the Android system are usually composed of multiple Activities, each of which provides different functions or interfaces. At the same time, Activities are also responsible for managing the interaction between the user and the terminal application.

[0178] In step S1102, the Guest terminal calls the surface synthesizer to separate the rendering data of each terminal application and output it to an independent graphics buffer queue.

[0179] Here, an extended SurfaceFlinger is used to separate the rendering data of each terminal application and output it to an independent Graphic Buffer Queue.

[0180] In step S1103, the Guest terminal stores the data to be rendered for each terminal application through a graphics buffer queue, preparing for subsequent transmission.

[0181] In step S1104, the Guest end transmits the rendering frame data and window metadata to the Host end.

[0182] When the surface compositor detects a Surface belonging to multi-window mode during the compositing phase, it exports the Surface's YUV / RGB frame data (i.e., the frame data corresponding to the data to be rendered) from the GPU buffer. Lightweight encoding (such as H.264 intra-frame encoding or JPEG compression) can be optionally performed to reduce transmission bandwidth. The frame data is packaged with metadata (timestamp, resolution, rotation status, channel identifier (i.e., the aforementioned DisplayID), etc.) and sent to the host via a high-performance cross-system communication interface.

[0183] The Guest end transmits frame data and metadata to the Host end via shared memory, Socket, or Hyper-V channel. The Guest end uses a specific protocol to ensure low-latency, high-throughput data transmission and supports the concurrent operation of multiple terminal applications.

[0184] In step S1105, the App's window content is displayed on the Host side.

[0185] The host detects the communication channel. When the first data packet (i.e., frame data packet) is received, it performs the following operations based on the channel identifier and metadata: a new Windows native window (i.e., the application window mentioned above) is created; the developer sets the application window title to the App name and sets the application window icon to the App icon (the App icon can be obtained from the Guest system); the developer initializes the DirectX or OpenGL context and prepares to perform texture drawing; subsequently, the texture content of the application window is continuously updated through frame data to achieve smooth rendering.

[0186] When any app exits, the Guest notifies the Host to close the corresponding channel, and the Guest performs the operation of closing the corresponding channel; the Host destroys the corresponding application window and releases resources; the entire emulator system supports operations such as minimizing, maximizing, and closing application windows, and synchronizes the application window's state information back to the Guest.

[0187] Furthermore, to implement the terminal application installation process, taking any PC-based software management tool as a specific product example, this application embodiment also designs the following implementation process for installing the terminal application: First, the PC-based software management tool obtains the download address of the terminal application installation package by connecting to the backend server, and performs the download and installation operation of the terminal application locally on the electronic device (i.e., the PC), ultimately realizing the deployment and operation of the terminal application on the mobile phone. Figure 12 As shown, the process of installing a terminal application includes the following steps: In step S1201, the software management tool client sends a download request to the backend server. This download request is used to request the download address of the Android installation package (APK, Android Package) of the terminal application.

[0188] In step S1202, the backend server returns the download address to the software management tool client.

[0189] In step S1203, the software management tool client completes the APK download and installation.

[0190] In step S1204, the software management tool client creates a PC desktop shortcut for the terminal application.

[0191] In some embodiments, in order to construct a virtual display mechanism on the Guest side, the implementation process of the virtual display mechanism on the Guest side is further refined in this application. The following will be combined with... Figure 13 The Guest core components and class diagram shown illustrate the implementation process of the virtual display mechanism on the Guest side.

[0192] The first step is for the Guest client to call the Display Management Service (DMS, DisplayManagerService) to create a virtual display.

[0193] The Display Management Service manages all display devices. This can be achieved by calling the Display Manager Service's method to create a locked virtual display (Dms.createVirtualDisplayLocked()), where "Dms" represents the Display Manager Service, "createVirtualDisplayLocked()" is a method name, and "Locked" here indicates a locked state (meaning the method must be called within a locked thread / resource environment to ensure operational safety). Specifically, Dms.createVirtualDisplayLocked() is called to assign a unique display device identifier (DisplayDeviceID). In the implementation process, a virtual display adapter (VirtualDisplayAdapter, which manages the virtual display devices) is first created, then a virtual display device (VirtualDisplayDevice) object is constructed (i.e., the virtual display implementation class in AOSP). Finally, DMS registers the virtual display by calling the surface synthesizer's binder interface.

[0194] The second step involves the Guest layer interacting with the surface synthesizer (Native layer). First, DMS calls the surface compositor's `createDisplay()` method through the surface compositor interface (ISurfaceComposer (AIDL)). Then, the surface compositor creates a display device subclass: a virtual display device. The core responsibilities of the created virtual display device include managing its rendering output and layer compositing. Specifically, the created virtual display device has the following functions: compositing target, buffer queue management, resolution and DPI management, and secure content handling. The compositing target indicates that the created virtual display device is used to receive the compositing results from virtual layers (i.e., virtual composite images). The compositing results are no longer output to the physical screen but to the user-specified virtual display device. Buffer queue management refers to creating an independent buffer queue for the virtual display. The producer of the buffer queue is the surface compositor, and the consumer is the user-provided Surface (such as MediaCodec or SurfaceTexture). Resolution and DPI management means that the created virtual display device supports scaling, cropping, rotation, and other transformations. Secure content processing refers to the ability to configure a created virtual display device to allow the capture of Digital Rights Management (DRM) / secure content (indicated by the secure mode virtual display flag (VIRTUAL_DISPLAY_FLAG_SECURE)). The third step involves the Guest performing a data transfer operation, transferring data from the terminal application to the Surface.

[0195] During app rendering, the application draws the UI using Surfaces (e.g., using OpenGL, Canvas, etc.). Then, a surface compositor performs compositing: the surface compositor composites all layers onto the target frame buffer of the virtual display device. Data is then transferred within the target frame buffer: the surface compositor writes the composited frame image (i.e., the virtual composite image) to the buffer queue of the virtual display device; the other end of the buffer queue is connected to the user-provided Surface; data flows automatically without CPU copying (typically using a graphics allocator (e.g., GRALLOC) for shared memory).

[0196] During upper-layer processing, if the Surface is a media codec, the frame image is encoded into formats such as H.264 / VP9 and used for screen recording or streaming. If the Surface is a surface texture, it can be used for OpenGL rendering or to transmit the data to be rendered to the host (this method can be used to implement the multi-screen display scheme in this application embodiment).

[0197] On the Host side, in order to achieve efficient rendering of multiple windows and reverse mapping of input events, the following Host multi-window implementation process is designed in this application embodiment. The core tasks of the Host side include: receiving the independent App rendering stream from the Android Guest side; creating an independent native window for each App; achieving efficient rendering and reverse mapping of input events; managing the window lifecycle and implementing the multi-window process synchronously with the Guest side.

[0198] like Figure 14 As shown, the synchronous implementation of a multi-window process between the Host and Guest ends includes the following steps: In step S1401, the Guest sends a CreateVirtualDisplay command and passes the relevant parameters (width, height, dpi, fps) to the Host.

[0199] Here, the relevant parameters include: width (meaning the object's dimensions in the horizontal direction, used to describe the horizontal specifications of screens, images, containers, etc., such as "image width is 1080 pixels"); height (meaning the object's dimensions in the vertical direction, corresponding to width, such as "image height is 2340 pixels"); dots per inch (dpi) is a unit for measuring image resolution or screen display precision, with higher values ​​indicating clearer images; frames per second (fps) describes the smoothness of video, animation, or game visuals, with higher values ​​indicating smoother visuals.

[0200] In step S1402, the Host establishes a communication channel with the Guest.

[0201] In step S1403, the host parses the frame data and metadata.

[0202] In step S1404, the host determines whether it is a newly launched application.

[0203] In step S1405, if the judgment result is yes, the host creates a native window.

[0204] In step S1406, if the judgment result is negative, the Host searches for the original window.

[0205] In step S1407, the Host initializes the rendering context to implement graphics and image rendering.

[0206] In step S1408, the host renders the image to the application window.

[0207] In addition, the host can also pass through input events, such as Figure 15 As shown, the host can also pass through user input events through the following steps: In step S1501, the host captures mouse / keyboard input events.

[0208] In step S1502, the Host obtains the target application corresponding to the input event.

[0209] In step S1503, the Host encapsulates the input event into an Android Input event.

[0210] In step S1504, the Host sends the Android input event back to the Guest.

[0211] In this embodiment, the Host can implement a reverse mapping function for input events, enabling the App application interface to respond to mouse or keyboard input events.

[0212] To optimize virtual display performance, this application also introduces an HDMI display (HDMI Display) optimization scheme. The guest device can utilize a hardware compositor (HWC) to achieve hardware-level layer compositing and pass-through output, significantly reducing GPU load and power consumption, and providing a more stable display experience. For example... Figure 16 As shown, the specific process of the HDMI monitor optimization solution is as follows: First, the Guest side calls the surface compositor to create an HDMI display device: the surface compositor detects display change events on the HWC. Then, it calls the method to create a built-in display (createBuiltInDisplay()) or the method to create an external display (createExternalDisplay()) to create a virtual display instance. For HDMI, a subclass of the hardware compositor, HWComposer:ExternalDisplay, can be created.

[0213] Then, the Guest registers the display with the Display Manager Service (DMS).

[0214] The guest client calls the surface compositor to report new display information to DMS through the surface compositor client interface (ISurfaceComposerClient). The data transmitted by the surface compositor includes: display identifier (Display ID), type (e.g., HDMI type display device Display.TYPE_HDMI or external type display device Display.TYPE_EXTERNAL), resolution (resolved from EDID), and refresh rate.

[0215] Finally, the content is output through the created display device.

[0216] Output Mode 1: Mirror Mode (Default): The surface compositor copies the content of the primary display (primary display) to the HDMI monitor. HWC can directly copy layers using Device Composition, reducing GPU load.

[0217] Output Mode 2: Extended Mode: The HDMI display device can be used as a standalone monitor, capable of running different Activities. The terminal application needs to explicitly launch a specific Activity onto the designated monitor.

[0218] The technical solution of this application embodiment utilizes Android's virtual display (or HDMI display device) mechanism in an emulator environment to achieve independent windowed display of multiple apps within a single Android system instance (Guest end) and on the host machine (Host end) desktop. Compared to traditional single-window full-screen simulation or multiple independent emulator instances, this brings significant technical and user experience benefits. The beneficial effects are significant, mainly reflected in the following aspects: Firstly, it improves multitasking efficiency: Users can open multiple Android app windows simultaneously on the Host desktop, dragging, scaling, side-by-side display, and Alt+Tab switching just like operating native desktop applications. This function significantly enhances users' multitasking collaborative processing capabilities and is particularly suitable for use scenarios such as office work, social networking, and live streaming that require frequent app multitasking. Secondly, it reduces system resource consumption: Traditional multitasking solutions require launching multiple complete Android virtual machine instances, consuming a large amount of CPU, memory, and storage resources. This application embodiment only requires one Guest system, creating an independent virtual display channel for each app through virtual display, significantly reducing resource overhead and improving overall operating efficiency and device capacity. On the other hand, it enables a truly seamless windowed experience: each app runs as an independent native window on the host desktop (such as the Windows desktop), and can coexist seamlessly with other desktop applications (such as Chrome and Excel). It supports cross-application copy and paste, file drag and drop, and shortcut sharing. This design breaks away from the isolation caused by the black-box full-screen operation of traditional emulators. This technical solution achieves deep integration between the Android platform and the desktop ecosystem.

[0219] In summary, the embodiments of this application not only solve the shortcomings of current Android emulators in terms of multi-window functionality, resource utilization, and user experience, but also provide more flexible and efficient architectural support for the future development of Android emulators.

[0220] It is understood that in the embodiments of this application, if data related to user information or enterprise information is involved, when the embodiments of this application are applied to specific products or technologies, it is necessary to obtain user permission or consent, or to obfuscate this information in order to eliminate the correspondence between this information and the user; and the collection and processing of related data should strictly comply with the requirements of relevant laws and regulations when applied in practice, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0221] The following continues to describe the exemplary structure of the simulator-based data processing device 254 provided in the embodiments of this application as a software module. In some embodiments, such as Figure 2As shown, the simulator-based data processing device 254 includes: a display module 2541, used to display the simulator's operation interface on the current interface of the electronic device; the simulator includes a host end and a client end, the host end being simulator software running on the electronic device, and the client end being a virtual terminal environment created on the electronic device; a data processing module 2542, used to, in response to the operation of multiple terminal applications executed on the operation interface, assign a display identifier to each terminal application through the client end; and transmit the data to be rendered for each terminal application to the host end after mapping it with the corresponding display identifier; a window creation module 2543, used to create an application window for each terminal application based on the display identifier through the host end, and display the application window of each terminal application on the current interface of the electronic device; and a rendering module 2544, used to perform window texture rendering in the application window of each terminal application based on the data to be rendered.

[0222] In some embodiments, the data processing module 2542 is further configured to: construct a graphics buffer queue for each terminal application in a preset buffer based on the display identifier of each terminal application; map the data to be rendered of each terminal application to the corresponding display identifier and store it in its respective graphics buffer queue; and use the graphics buffer queue to transmit the data to be rendered of each terminal application to the host.

[0223] In some embodiments, the data processing module 2542 is further configured to: compress the data to be rendered for each terminal application to obtain compressed data to be rendered; obtain metadata of the data to be rendered; the metadata includes at least the display identifier of the terminal application; and package the compressed data to be rendered and the metadata into the graphics buffer queue.

[0224] In some embodiments, the window creation module 2543 is further configured to: in response to receiving the first data packet in the graphics buffer queue of any terminal application through the host, parse the first data packet to obtain the first data to be rendered and metadata; and create an application window for the terminal application based on the metadata.

[0225] In some embodiments, the apparatus further includes: a setting module, configured to, after creating an application window for the terminal application based on the metadata, obtain the application name and application icon of the terminal application from the client; set the window title of the application window to the application name of the terminal application; and set the window icon of the application window to the application icon of the terminal application.

[0226] In some embodiments, the data processing module 2542 is further configured to: create a communication channel for each terminal application; the communication channel is used to send data through a cross-system communication interface; the channel identifier of the communication channel is the display identifier of the terminal application; and transmit the data to be rendered in the graphics buffer queue of the terminal application to the host through the communication channel.

[0227] In some embodiments, the apparatus further includes: a program exit module, configured to, after rendering a window texture based on the data to be rendered, in response to a program exit operation for any terminal application, send a program exit notification message for any terminal application to the host via the client; and, in response to the program exit notification message, close the communication channel of any terminal application and stop displaying the application window of any terminal application via the host.

[0228] In some embodiments, the apparatus further includes: a window interaction module, configured to, after rendering a window texture based on the data to be rendered, respond to a window interaction operation for any terminal application via the host terminal to perform an operation response on the application window of any terminal application, and display the operation response result for the application window on the current interface of the electronic device; wherein the window interaction operation includes at least one of the following: window maximization operation, window minimization operation, and window closing operation; the window interaction operation is performed via an input device connected to the electronic device; and in response to the completion of the operation response for the window interaction operation, synchronize the operation response result to the client terminal.

[0229] In some embodiments, the window interaction module is further configured to: after rendering a window texture based on the data to be rendered, in response to a window interaction operation for any terminal application, encapsulate the window interaction operation into a client-side input operation via the host terminal, and send the client-side input operation to the client terminal; respond to the client-side input operation via the client terminal, and display the operation response result for the application window on the current interface of the electronic device; wherein the window interaction operation includes at least one of the following: window maximization operation, window minimization operation, and window closing operation; the window interaction operation is executed via an input device connected to the electronic device.

[0230] In some embodiments, the apparatus further includes: a virtual display device processing module, configured to, after assigning a display identifier to each terminal application via the client, create a virtual display device corresponding to the display identifier of the terminal application via the client for each terminal application; generate at least one virtual layer based on the data to be rendered of the terminal application; composite the at least one virtual layer to obtain a virtual composite image of the terminal application; and output the virtual composite image of the terminal application to the virtual display device; the data processing module 2542 is further configured to: map the virtual composite image of each terminal application to the corresponding display identifier and transmit it to the host.

[0231] In some embodiments, the virtual display device processing module is further configured to: create a virtual display adapter; the virtual display adapter is used to manage virtual display devices; and invoke the virtual display adapter to construct a virtual display device corresponding to the display identifier.

[0232] In some embodiments, the virtual display device includes a virtual built-in display device; the virtual display device processing module is further configured to: check the display change event of the hardware synthesizer; and in response to the display change event being a built-in display creation event, invoke the virtual display adapter to construct a virtual built-in display device corresponding to the display identifier.

[0233] In some embodiments, the apparatus further includes: an external display processing module, configured to, in response to a display change event being a create external display event, call an external display creation function to create an external display corresponding to the display identifier; the external display is used to display virtual composite images of different terminal applications, and the external display uses a hardware compositor to composite the at least one virtual layer, and uses the hardware compositor to display the virtual composite image on the external display.

[0234] In some embodiments, the virtual display device processing module is further configured to: construct a buffer queue for the terminal application in a preset frame buffer based on the display identifier of the terminal application; the buffer queue is connected to the virtual display device corresponding to the display identifier; store the virtual composite image of the terminal application in the buffer queue; and use the buffer queue to transmit the virtual composite image of the terminal application to the virtual display device.

[0235] In some embodiments, the data processing module 2542 is further configured to: in response to the data format of the virtual display device being a multimedia codec format, encode the virtual composite image into a first format corresponding to the multimedia codec format, and map the virtual composite image of the first format to the corresponding display identifier before transmitting it to the host; in response to the data format of the virtual display device being a surface texture format, call the graphics rendering program interface to map the virtual composite image to the corresponding display identifier before transmitting it to the host.

[0236] It should be noted that the description of the apparatus in this application embodiment is similar to the description of the method embodiment described above, and has similar beneficial effects as the method embodiment; therefore, it will not be repeated. For technical details not disclosed in this apparatus embodiment, please refer to the description of the method embodiment of this application for understanding.

[0237] This application provides an electronic device, including: a memory for storing computer-executable instructions or computer programs; and a processor for executing the computer-executable instructions or computer programs stored in the memory to implement the above-described method.

[0238] This application provides a computer program product, which includes computer-executable instructions or a computer program stored in a computer-readable storage medium; wherein, when the processor of an electronic device reads the computer-executable instructions or the computer program from the computer-readable storage medium and executes the computer-executable instructions or the computer program, the above-described method is implemented.

[0239] This application provides a computer-readable storage medium storing computer-executable instructions or a computer program. When the computer-executable instructions or the computer program are executed by a processor, the processor will perform the aforementioned method, for example, as... Figure 3 The method shown.

[0240] In some embodiments, the storage medium may be a computer-readable storage medium, such as a ferromagnetic random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), flash memory, magnetic surface memory, optical disc, or a compact disk-read-only memory (CD-ROM); or it may be a device that includes one or any combination of the above-mentioned memories.

[0241] In some embodiments, executable instructions may take the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0242] As an example, executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file containing other programs or data, for example, in one or more scripts within a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple collaborating files (e.g., files storing one or more modules, subroutines, or code sections). As an example, executable instructions may be deployed to execute on a single electronic device, or on multiple electronic devices located in one location, or on multiple electronic devices distributed across multiple locations and interconnected via a communication network.

[0243] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A simulator-based data processing method, characterized by, The method comprises: displaying an operation interface of a simulator on a current interface of an electronic device; the simulator comprises a host end and a client end, the host end is a simulator software running on the electronic device, and the client end is a virtual terminal environment created on the electronic device; in response to a running operation performed on the operation interface for a plurality of terminal applications, assigning a display identifier to each terminal application through the client end; and transmitting the to-be-rendered data of each terminal application to the host end after mapping the to-be-rendered data with the corresponding display identifier; creating an application window for each terminal application based on the display identifier through the host end, and displaying the application window of each terminal application on the current interface of the electronic device; in the application window of each terminal application, performing window texture rendering based on the to-be-rendered data.

2. The method of claim 1, wherein, The transmitting the to-be-rendered data of each terminal application to the host end after mapping the to-be-rendered data with the corresponding display identifier comprises: constructing a graphics buffer queue for each terminal application in a preset buffer based on the display identifier of each terminal application; storing the to-be-rendered data of each terminal application in the respective graphics buffer queue after mapping the to-be-rendered data with the corresponding display identifier; transmitting the to-be-rendered data of each terminal application to the host end using the graphics buffer queue.

3. The method of claim 2, wherein, The storing the to-be-rendered data of each terminal application in the respective graphics buffer queue after mapping the to-be-rendered data with the corresponding display identifier comprises: for each terminal application, compressing the to-be-rendered data of the terminal application to obtain compressed to-be-rendered data; obtaining meta information of the to-be-rendered data; the meta information at least comprises the display identifier of the terminal application; packaging the compressed to-be-rendered data and the meta information and storing them in the graphics buffer queue.

4. The method of claim 3, wherein, The creating an application window for each terminal application based on the display identifier through the host end comprises: in response to receiving a first data packet in the graphics buffer queue of any terminal application through the host end, analyzing the first data packet to obtain first to-be-rendered data and meta information; creating an application window for the terminal application based on the meta information.

5. The method of claim 4, wherein, After creating the application window for the terminal application based on the meta information, the method further comprises: obtaining a program name and a program icon of the terminal application from the client end; setting a window title of the application window as the program name of the terminal application, and setting a window icon of the application window as the program icon of the terminal application.

6. The method of claim 2, wherein, The transmitting the to-be-rendered data of each terminal application to the host end using the graphics buffer queue comprises: creating a communication channel for each terminal application; the communication channel is used for sending data through a cross-system communication interface; a channel identifier of the communication channel is the display identifier of the terminal application; transmitting the to-be-rendered data in the graphics buffer queue of the terminal application to the host end through the communication channel.

7. The method of claim 6, wherein, After the window texture rendering based on the to-be-rendered data, the method further comprises: in response to a program exit operation for any terminal application program, sending, by the client end, a program exit notification message for the any terminal application program to the host end; in response to the program exit notification message, closing, by the host end, a communication channel of the any terminal application program, and stopping displaying an application window of the any terminal application program.

8. The method of claim 1, wherein, After the window texture rendering based on the to-be-rendered data, the method further comprises: in response to a window interaction operation for any terminal application program, performing, by the host end, operation response on an application window of the any terminal application program, and displaying an operation response result for the application window on a current interface of the electronic device; wherein the window interaction operation comprises at least one of the following: a window maximization operation, a window minimization operation, and a window closing operation; the window interaction operation is executed through an input device connected to the electronic device; in response to completion of the operation response for the window interaction operation, synchronizing the operation response result to the client end.

9. The method of claim 1, wherein, After the window texture rendering based on the to-be-rendered data, the method further comprises: in response to a window interaction operation for any terminal application program, encapsulating, by the host end, the window interaction operation into a client end input operation, and sending the client end input operation to the client end; performing, by the client end, operation response on the client end input operation, and displaying an operation response result for the application window on a current interface of the electronic device; wherein the window interaction operation comprises at least one of the following: a window maximization operation, a window minimization operation, and a window closing operation; the window interaction operation is executed through an input device connected to the electronic device.

10. The method of claim 1, wherein, After allocating, by the client end, a display identifier for each terminal application program, the method further comprises: for each terminal application program, creating, by the client end, a virtual display device corresponding to the display identifier of the terminal application program; generating at least one virtual layer based on to-be-rendered data of the terminal application program; synthesizing the at least one virtual layer to obtain a virtual synthesized image of the terminal application program; outputting the virtual synthesized image of the terminal application program to the virtual display device; the mapping and transmission, to the host end, of the to-be-rendered data of each terminal application program and the corresponding display identifier, comprises: mapping and transmitting, to the host end, the virtual synthesized image of each terminal application program and the corresponding display identifier.

11. The method of claim 10, wherein, the creating, by the client end, of the virtual display device corresponding to the display identifier of the terminal application program, comprises: creating a virtual display adapter; the virtual display adapter is used for managing a virtual display device; calling the virtual display adapter to construct the virtual display device corresponding to the display identifier.

12. The method of claim 11, wherein, the virtual display device comprises a virtual built-in display device; the calling the virtual display adapter to construct the virtual display device corresponding to the display identifier, comprises: checking a display change event of a hardware compositor; in response to the display change event being a creation built-in display event, invoking the virtual display adapter to construct a virtual built-in display device corresponding to the display identifier.

13. The method of claim 12, wherein, The method further comprises: in response to the display change event being a creation external display event, invoking an external display creation function to create an external display corresponding to the display identifier; the external display is used to display a virtual composite image of different terminal application programs, and the external display is used to synthesize the at least one virtual layer through a hardware compositor, and the virtual composite image is displayed on the external display through the hardware compositor.

14. The method of claim 10, wherein, The method further comprises: based on the display identifier of the terminal application program, constructing a buffer queue for the terminal application program in a preset frame buffer; the buffer queue is connected to the virtual display device corresponding to the display identifier; storing the virtual composite image of the terminal application program into the buffer queue; transmitting the virtual composite image of the terminal application program to the virtual display device by using the buffer queue.

15. The method of claim 10, wherein, The method further comprises: in response to the data format of the virtual display device being a multimedia codec format, encoding the virtual composite image into a first format corresponding to the multimedia codec format, and transmitting the virtual composite image in the first format and the corresponding display identifier to the host end; in response to the data format of the virtual display device being a surface texture format, invoking a graphic rendering program interface to transmit the virtual composite image and the corresponding display identifier to the host end.

16. A simulator-based data processing apparatus, characterized by The apparatus comprises: a display module configured to display an operation interface of an emulator on a current interface of an electronic device; the emulator comprises a host end and a client end, the host end is an emulator software running on the electronic device, and the client end is a virtual terminal environment created on the electronic device; a data processing module configured to, in response to a running operation performed on the operation interface for a plurality of terminal application programs, assign a display identifier to each terminal application program through the client end, and transmit to-be-rendered data of each terminal application program and the corresponding display identifier to the host end; a window creation module configured to create an application window for each terminal application program based on the display identifier through the host end, and display the application window of each terminal application program on the current interface of the electronic device; a rendering module configured to perform window texture rendering in the application window of each terminal application program based on the to-be-rendered data.

17. An electronic device, comprising: The apparatus comprises: a memory configured to store computer executable instructions or computer programs; a processor configured to, when executing the computer executable instructions or computer programs stored in the memory, implement the data processing method based on the emulator according to any one of claims 1 to 15.

18. A computer-readable storage medium, characterized in that, Computer executable instructions or a computer program are stored, when the computer executable instructions or the computer program are executed by a processor, to cause the processor to implement the simulator-based data processing method of any one of claims 1 to 15.

19. A computer program product comprising computer-executable instructions or a computer program, characterized in that, The computer executable instructions or the computer program are executed by a processor to implement the simulator-based data processing method of any one of claims 1 to 15.

Images