Display processing method, device and equipment and readable storage medium

Abstract

The invention discloses a display processing method and device, equipment and a readable storage medium, and the method comprises the steps: displaying an operation interface of a simulator, the operation interface of the simulator comprising first media data; in response to a viewing operation on the first media data, displaying a first media display interface; the first media display interface is independent of the simulator; and displaying the first media data in the first media display interface. By adopting the media data display method and device, the display modes of the checked media data can be enriched in a simulator scene, so that the use experience of a user is improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and particularly to a display processing method, apparatus, device, and readable storage medium. Background Art

[0002] An emulator is a tool that emulates the operating system (such as the Android system) of one type of terminal device (such as a mobile phone) on another type of terminal device (such as a computer), enabling the running of application programs adapted to the other type of terminal device on this type of terminal device. For example, assuming that application A is only adapted to the Android system, if one needs to use application A on a computer, one needs to open the emulator in the computer and run application A in the emulator.

[0003] In the traditional technology, for the media data displayed in the operation interface of the emulator (which can refer to the media data in any reference program), the display method after being triggered for viewing is relatively single and strictly restricted by the emulator, resulting in a single interaction method between the user and the application programs running in the emulator, thereby reducing the user experience. Summary of the Invention

[0004] Embodiments of this application provide a display processing method, apparatus, device, and readable storage medium, which can enrich the display methods of the viewed media data in the emulator scenario, thereby enhancing the user experience.

[0005] On the one hand, embodiments of this application provide a display processing method, including:

[0006] Display the operation interface of the emulator, where the operation interface of the emulator contains first media data;

[0007] In response to a viewing operation on the first media data, display a first media display interface; the first media display interface is independent of the emulator;

[0008] Display the first media data in the first media display interface.

[0009] On the one hand, embodiments of this application provide a display processing apparatus, including:

[0010] A first display module, configured to display the operation interface of the emulator, where the operation interface of the emulator contains first media data;

[0011] A second display module, configured to display a first media display interface in response to a viewing operation on the first media data; the first media display interface is independent of the emulator;

[0012] A media display module, configured to display the first media data in the first media display interface.

[0013] In one embodiment, the first media data is the media data displayed in the application interface of the first application. The first application runs in the emulator, and the application interface of the first application is displayed in the operation interface of the emulator. The application operation content of the first application is also displayed in the application interface of the first application.

[0014] After the media display module displays the first media data in the first media display interface, the media display module is further specifically configured to:

[0015] During the process of the first media data being displayed in the first media display interface, in response to a trigger operation on the application operation content of the first application, display the trigger result content corresponding to the application operation content in the application interface of the first application. The trigger result content corresponding to the application operation content refers to the visual content obtained after triggering the application operation content.

[0016] In one embodiment, after the media display module displays the first media data in the first media display interface, the media display module is further specifically configured to:

[0017] In response to an interface adjustment operation on the first media display interface, adjust the display of the first media display interface to the media adjustment interface indicated by the interface adjustment operation.

[0018] In one embodiment, after the media display module displays the first media data in the first media display interface, the media display module is further specifically configured to:

[0019] In response to a display adjustment operation on the first media data in the first media display interface, switch the display state of the first media data in the first media display interface to the target display state indicated by the display adjustment operation; and,

[0020] Synchronously switch the display state of the first media data in the operation interface of the emulator to the target display state indicated by the display adjustment operation.

[0021] In one embodiment, the display position relationship between the first media display interface and the operation interface of the emulator includes any one of the following:

[0022] The first media display interface and the operation interface of the emulator are in an overlapping display relationship;

[0023] The first media display interface and the operation interface of the emulator are in a partitioned display relationship.

[0024] In one embodiment, after the media display module displays the first media data in the first media display interface, the media display module is further specifically configured to:

[0025] In response to a drag operation on the first media display interface, switch the display position of the first media display interface to the display position indicated by the drag operation.

[0026] In one embodiment, the operation interface of the simulator includes a media viewing control.

[0027] The viewing operation on the first media data includes any one of the following:

[0028] A triggering operation on the first media data;

[0029] An operation of dragging the first media data to a specified area;

[0030] A triggering operation on the media viewing control.

[0031] In one embodiment, the operation interface of the simulator further includes second media data.

[0032] After the media display module displays the first media data in the first media display interface, the media display module is further specifically configured to:

[0033] In response to a viewing operation on the second media data, display a second media display interface; the second media display interface is independent of the simulator and is synchronously displayed with the operation interface of the simulator;

[0034] Display the second media data in the second media display interface.

[0035] In one embodiment, the first media data is different from the second media data; the second media display interface is independent of the first media display interface and is synchronously displayed with the first media display interface.

[0036] In one embodiment, the second media display interface is determined based on the second application program to which the second media data belongs;

[0037] When the second application program to which the second media data belongs is the same as the first application program, the second media display interface is the same as the first media display interface;

[0038] When the second application program to which the second media data belongs is different from the first application program, the second media display interface is independent of the first media display interface and is synchronously displayed with the first media display interface.

[0039] In one embodiment, the second application program to which the second media data belongs is the same as the first application program;

[0040] The display method of displaying the second media data in the second media display interface includes any one of the following:

[0041] Switch the first media data displayed in the first media display interface to display the second media data;

[0042] During the process of displaying the first media data in the first media display interface, the second media data is synchronously displayed.

[0043] In one embodiment, the simulator runs on a computer device, and N media players are deployed on the computer device, where N is a positive integer, and one media player is used to play media data of one media type;

[0044] The specific implementation manner of the media display module for displaying the first media data in the first media display interface includes:

[0045] Play the first media data in the first media display interface through the media player corresponding to the target media type; the target media type refers to the media type to which the first media data belongs.

[0046] In one embodiment, the specific implementation manner of the media display module for displaying the first media data in the first media display interface includes:

[0047] Obtain the historical playback duration of the first media data when responding to the viewing operation of the first media data;

[0048] Based on the historical playback duration of the first media data, determine the playback start time point in the first media data;

[0049] Use the playback start time point as the playback start position to play and display the first media data in the first media display interface.

[0050] In one embodiment, the first media data is media data displayed in the application interface of the first application, the first application runs in the simulator, and the application interface of the first application is displayed in the operation interface of the simulator;

[0051] The display process of the media viewing control in the operation interface of the simulator includes:

[0052] In response to the trigger operation on the application interface of the first application, obtain the position coordinates of the triggered position;

[0053] If the triggered position is located in the display area of the first media data, display the media viewing control in the display area of the first media data.

[0054] In one embodiment, after obtaining the position coordinates of the triggered position, the second display module is further specifically configured to:

[0055] Obtain the interface content included in the application interface of the first application and the position coordinates of the triggered position;

[0056] Invoking a region prediction model based on interface content included in an application interface of the first application and location coordinates of a triggered location;

[0057] Performing regional prediction processing on the triggered position using a regional prediction model, and outputting a predicted probability that the triggered position is within the display area of the first media data;

[0058] If the predicted probability is greater than the probability threshold, determining that the triggered position is in the display area of the first media data;

[0059] If the predicted probability is less than the probability threshold, it is determined that the triggered position is not in the display area of the first media data.

[0060] In one embodiment, the specific implementation of the second display module displaying the first media display interface includes:

[0061] Performing a hash calculation on the first media data using a specified hash function to obtain a hash value of the first media data;

[0062] Traversing M active hash values based on the hash value of the first media data; M is a positive integer, and each active hash value refers to a hash value corresponding to an active media display interface, where an active media display interface refers to a media display interface that has displayed media data;

[0063] If the M active hash values include the hash value of the first media data, the active media display interface corresponding to the hash value of the first media data is displayed on top;

[0064] If the M active hash values do not include the hash value of the first media data, a first media presentation interface is created and displayed.

[0065] In one aspect, an embodiment of the present application provides a computer device, including: a processor and a memory;

[0066] The memory stores a computer program, and when the computer program is executed by the processor, the processor executes the method in the embodiment of the present application.

[0067] On one hand, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. The computer program includes program instructions. When the program instructions are executed by a processor, the method in the embodiment of the present application is executed.

[0068] In one aspect of the present application, a computer program product is provided, comprising a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, causing the computer device to perform a method provided in one aspect of an embodiment of the present application.

[0069] In an embodiment of the present application, for any media data (for example, first media data) displayed in the operation interface of the simulator, after the viewing operation is performed on the first media data, the present application can display a media display interface independent of the simulator (for example, first media display interface), and display the first media data in the first media display interface. In this way, when the user views any media data in the operation interface of the simulator, the display of the media data is not restricted by the operation interface of the simulator, and the viewed media data can be separated from the operation interface of the simulator and displayed in a media display interface independent of the simulator. Since the media display interface and the operation interface of the simulator are independent of each other, displaying the media data in the media display interface will not have any impact on the operation interface of the simulator, and the user can still perform corresponding operations on the operation interface of the simulator (for example, perform corresponding operations on an application running in the simulator), thereby improving the user's experience of using the simulator. In summary, the present application can enrich the display method of the viewed media data in the simulator scenario, thereby improving the user's experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0071] Figure 1 This is a schematic diagram of a scenario for viewing media data in a simulator scenario in a traditional solution provided in an embodiment of the present application;

[0072] Figure 2 is a schematic diagram of a media display system of a simulator provided in an embodiment of the present application;

[0073] Figure 3 This is a flow chart of a simulator-based display processing method provided in an embodiment of the present application;

[0074] Figure 4 This is a schematic diagram of a scenario for viewing media data in a simulator scenario provided by an embodiment of the present application;

[0075] Figure 5 This is a schematic diagram of a scenario for changing the display position of a media display interface provided by an embodiment of the present application;

[0076] Figure 6It is a schematic diagram of a scenario for adjusting the display angle of a media display interface provided by an embodiment of the present application;

[0077] Figure 7 It is a schematic diagram of a scenario for displaying a video in a media display interface provided by an embodiment of the present application;

[0078] Figure 8 It is a schematic flowchart of displaying a media viewing control for first media data in an operation interface provided by an embodiment of the present application;

[0079] Figure 9 It is a schematic flowchart of displaying second media data to be viewed provided by an embodiment of the present application;

[0080] Figure 10 It is a schematic diagram of a scenario for displaying second media data provided by an embodiment of the present application;

[0081] Figure 11 It is a schematic diagram of the logical architecture of a display solution provided by an embodiment of the present application;

[0082] Figure 12 It is a schematic diagram of the structure of a display processing device provided by an embodiment of the present application;

[0083] Figure 13 It is a schematic diagram of the structure of a computer device provided by an embodiment of the present application. Detailed implementation manners

[0084] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts shall fall within the protection scope of the present application.

[0085] In order to more clearly understand the technical solutions provided by the embodiments of the present application, the technical terms involved in the embodiments of the present application will be introduced here:

[0086] 1. Emulator:

[0087] It is a tool (i.e., an application program) of computer programs or computer software that can simulate another operating system. Its main purpose is to run software or games that can only run on a specific platform on different hardware or operating systems. The emulator creates a virtual environment by replicating the instruction set, hardware architecture, and peripheral interfaces of the target system, enabling source code or binary files to be executed on the host system while maintaining compatibility with the original environment. For example, an emulator can simulate and run (emulate) the operating system of a mobile device (such as a mobile phone) on a PC (Personal Computer), enabling applications adapted for mobile phones to be run on the PC. For instance, suppose there is an application A that is only adapted for the Android system. If you want to use application A on a PC, you need to open the emulator on the computer and simulate the operation of application A in the emulator.

[0088] 2. Operation Interface:

[0089] The operation interface refers to any operation interface in the emulator, which can be the operation interface provided by the emulator itself or the operation interface of an application program simulated and run in the emulator. Among them, the operation interface provided by the emulator itself is the basic component unit for the emulator to present visually. The emulator can consist of multiple operation interfaces, and the users of the emulator (such as a certain user, intelligent robot, etc.) can interact with the emulator through the operation interface of the emulator. The operation interface of an application program simulated and run in the emulator is the basic component unit for the application program simulated and run in the emulator to present visually. The application program can consist of multiple operation interfaces, and the users of the application program can interact with the application program through the operation interface of the application program. For the convenience of distinction in this application, the operation interface of an application program simulated and run in the emulator can be referred to as the application interface of this application program.

[0090] Exemplarily, the computer device where the emulator is located corresponds to a first operating system, and the application program simulated by the emulator is developed based on the specifications of a second operating system. The first operating system and the second operating system are two different operating systems. Therefore, the computer device needs to rely on the emulator to simulate the operating environment of the second operating system, so that the application program developed based on the specifications of the second operating system can be simulated and run on the computer device. For example, the application program can be a mobile application program simulated and run on the PC (Personal Computer) side through the emulator. The Windows operating system runs on the PC side, and the mobile application program is developed based on the specifications of the Android operating system on the mobile side. Android is an open-source operating system based on the Linux kernel, mainly used for mobile devices (such as smartphones and tablets) and other embedded devices. Android provides a rich application framework that allows developers to build various application programs and games.

[0091] Based on the above related introductions to the technologies such as the emulator and the operation interface, the present application proposes a display processing method based on the emulator. This display processing method of the emulator improves the display method of the media data viewed in the operation interface of the emulator. The design concept of the present application will be introduced below:

[0092] In practical applications, the user of the emulator can run the emulator on the computer device. The computer device will display the operation interface of the emulator on its own device screen. All application programs that the emulator can simulate and run will be displayed in this operation interface of the emulator. In this way, the user of the emulator can open any one or more application programs in the operation interface of the emulator to use the application program. After the user opens the application program, the emulator will simulate and run the application program. For any application program that is simulated and run, the computer device will display the application interface for this application program in the operation interface of the emulator, and display the application content related to this application program in this application interface. The application content can include different visually presented media data. For example, the application content can include at least one of the following media data: pictures, videos, audios, e-books, and texts, etc.

[0093] For various types of media data displayed in the application interface, the user can select any media data (e.g., a certain picture) for viewing. However, in the traditional technology, when the user selects a media data for viewing, the display method of the viewed media data is single and strictly restricted by the operation interface of the emulator. The display area of the viewed media data will cover the operation interface of the emulator. In this way, if the user wants to perform corresponding operations on other unviewed media data in the application interface, the user needs to first close the display area of the viewed media data. For ease of understanding, please also refer to Figure 1 , Figure 1 FIG. Figure 1 is a schematic diagram of a scenario for viewing media data in a traditional solution provided by an embodiment of the present application in an emulator scenario. Among them, as Figure 1 shown, the scenario is described by taking a computer device running an emulator and the user of the emulator being user a (hereinafter referred to as user a) as an example. When user a runs the emulator on the computer device, the computer device can display the operation interface 1001 of the emulator on its own device screen. As Figure 1 shown, in this operation interface 1001, there are application programs A, B, and C that can be simulated and run. User a can open any application program for running and using. Here, taking user a starting application program A as an example, the computer device can display the application interface 00A of application program A in this operation interface 1001 and display the application content of application program A in this application interface 00A. Among them, the application content includes picture 10A3 (i.e., a media data displayed in application interface 00A).

[0094] Here, it is assumed that the computer device is connected to a mouse. User a can trigger various operations in the operation interface 1001 through the mouse. For example, user a can move the control cursor M displayed on the device screen of the computer device to picture 10A3 through the mouse button, and when the control cursor M is on picture 10A3, double-click picture 10A3 by pressing the left mouse button twice. This double-click operation can be understood as user a performing a viewing operation on picture 10A3. The computer device can respond to this viewing operation and display a media display interface 000A for displaying this picture 10A3 on this operation interface 1001 and display this picture 10A3 in this media display interface 000A. Among them, this media display interface 000A is not independent of the emulator and cannot be separated from the emulator. In fact, this media display interface 000A can be understood as an interface provided by the emulator and can only be displayed in the operation interface 1001 of the emulator, as Figure 1As shown, the media display interface 000A is displayed in the operation interface 1001 and is limited by the display size of the operation interface 1001 (the maximum display size of the media display interface 000A can only reach the display size of the operation interface 1001), which is not conducive to viewing the picture 10A3 (for example: when there are many and miscellaneous elements in the picture 10A3, if the display size of the media display interface 000A is too small, it is very likely that the detailed content in the picture 10A3 cannot be seen clearly). Moreover, since both the media display interface 000A and the application interface 00A are in the operation interface 1001, when the display size of the media display interface 000A is relatively large, the media display interface 000A will also cover the application interface 00A (for example, as Figure 1 shown, the media display interface 000A basically completely covers the application interface 00A), so that when the user a views the picture 10A3, he / she cannot continue to perform corresponding operations on other application contents displayed in the application interface 00A (contents other than the picture 10A3 displayed in the application interface 00A). If the user a wants to continue browsing the application interface 00A and continue to perform corresponding operations on other application contents in the application interface 00A (for example, the application interface 00A contains a search control, and the user a triggers the search control to perform a search operation), then the user a needs to first close the media display interface 000A.

[0095] As can be seen from the embodiment shown in Figure 1 , in the traditional technology, when the user of the simulator views a certain media data in the operation interface, the viewed media data can only be displayed in the operation interface of the simulator (displayed through the interface provided by the simulator) and cannot be separated from the simulator. This not only causes the viewed media data to be limited by the display size of the operation interface of the simulator during display, but also may cause the user to be unable to continue browsing or operating the contents displayed in the application interface; the user needs to frequently close / open the display interface of the viewed media data to view the media data and browse / operate the contents displayed in the application interface, which greatly affects the user experience.

[0096] Based on this, the present application provides a display solution for the viewed media data, which can enrich the display mode of the viewed media data and improve the user experience. Among them, the display solution for the viewed media data designed in this solution can at least include three consecutive steps: 1. The user of the emulator runs the emulator on a computer device. The computer device displays the operation interface of the emulator on its own device screen and displays one or more application programs available for simulated operation in this operation interface; 2. The user starts a certain application program. The computer device displays the application interface for the started application program in the operation interface and displays the application content in this application interface. Among them, the application content may contain media data of various media types. The media types in the present application may at least include any one of the following: picture type, text type, video type, audio type. Then the media data contained in the application content can include pictures (belonging to the picture type), text (belonging to the text type), videos (belonging to the video type), and audio (belonging to the audio type). 3. The user selects a certain media data displayed in the application interface for viewing. 4. The computer device creates and displays a media display interface independent of the emulator and displays the selected media data in this media display interface. Among them, the independent of the emulator in the present application means not restricted by the emulator. In this way, the media display interface independent of the emulator can be not affected by the display size of the operation interface of the emulator. This media display interface can be of any size (needs to be smaller than the size of the device screen of the computer device) and any position, without being restricted in the operation interface of the emulator. In this way, since the operation interface of the emulator and the media display interface are independent of each other, the operations that can be performed in the operation interface and the operations that can be performed in the media display interface are also independent of each other. Then during the process of the user viewing the media data, the user can still browse the application content displayed in the application interface and perform corresponding operations on it.

[0097] It can be seen that the display solution for the viewed media data provided by the present application improves the traditional display mode of the viewed media data. The viewed media data does not need to be displayed in the operation interface of the emulator and will no longer cover the application content displayed in the application interface. The media display interface of the viewed media data and the operation interface of the emulator are independent of each other. The user can continue to perform corresponding operations in the operation interface of the emulator while viewing the displayed media data, realizing the synchronous display of the emulator and the media data. Thus, the user experience of using the emulator can be improved.

[0098] The display solution for the viewed media data provided by this application can be applied to a simulator, which can run on a computer device. The computer device can be a terminal device, which can include but is not limited to any of the following: smart phones, tablet computers, laptop computers, desktop computers, smart watches, smart home appliances, intelligent vehicle terminals, virtual reality (VR) devices, augmented reality (AR) devices, and aircraft, but is not limited thereto. Terminal devices are often equipped with display devices, which can be monitors, display screens, touch screens, etc. The touch screen can be a touch panel, a touch control panel, etc.

[0099] Based on the above-described solution, to better understand the solution and application scenario provided by this application, please refer to Figure 2 , Figure 2 which is a schematic diagram of a media display system of a simulator provided by an embodiment of this application. As Figure 2 shown, the media display system of the simulator can include a computer device 201 and a server 202, where:

[0100] 1) The computer device 201 can refer to the device used by the user of the object (the object using the simulator, such as a user, an intelligent robot, etc.). The user of the object can run the simulator on the computer device. For example, the computer device refers to a terminal device.

[0101] 2) The server 202 can be used to provide service support for the simulator. The provided service support includes, for example: the simulator can obtain relevant data of the operation interface of the simulator (referred to as operation interface data) from the server, and render the operation interface based on the operation interface data; another example is that the server can determine the display method after the media data in the operation interface is viewed for the simulator, and so on. The server mentioned in the embodiments of this application can be a single physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0102] The embodiments of this application do not limit the connection method between the computer device 201 and the server 202. The computer device 201 and the server 202 can establish a direct communication connection through wired communication, or the computer device 201 and the server 202 can establish an indirect communication connection through wireless communication. In addition, in Figure 2The number of the computer device 201 and the server 202 is taken as a single one only for example. In practical applications, it may include multiple distributed servers, which is hereby explained.

[0103] Figure 2 The media display system of the simulator in the illustrated embodiment is for more clearly explaining the technical solution of the embodiment of the present application, and does not constitute a limitation on the technical solution provided by the embodiment of the present application. Those of ordinary skill in the art can know that with the evolution of the system architecture and the emergence of new business scenarios, the technical solution provided by the embodiment of the present application is equally applicable to similar technical problems.

[0104] It should be noted that in the specific implementation manner of the present application, for relevant data such as user information and user data (such as a certain media data viewed by the user and the media data liked by the user), they all need to be obtained after the user manually authorizes and permits (that is, after the user agrees). That is to say, when the above embodiments of the present application are applied to specific products or technologies, the methods and related functions provided by the embodiments of the present application operate under the permission or consent of the user (the functions provided by the embodiments of the present application can be actively enabled by the user), and the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of the relevant regions.

[0105] Based on the above-described solution, the present application provides a more detailed display processing method. Next, the display processing method provided by the embodiment of the present application will be introduced from a visualization perspective.

[0106] Please refer to Figure 3 , Figure 3 , which is a schematic flowchart of a display processing method based on a simulator provided by an embodiment of the present application. The process of this method is also the solution process of the display scheme for the media data to be viewed. This display processing method can be applied to a simulator, and the simulator can run on a computer device. As Figure 3 shown, this display processing method may include but is not limited to the following steps S301 - step S303:

[0107] Step S301, display the operation interface of the simulator, and the operation interface of the simulator includes the first media data.

[0108] In this application, the operation interface of the emulator can be an operation interface provided by the emulator itself. Taking the computer device as a PC as an example, the operation interface of the emulator is the operation interface of the emulator running on the PC. Alternatively, the operation interface of the emulator can also refer to the operation interface of the application program running through the emulator in the computer device. Taking the computer device as a PC as an example, the operation interface of the emulator is the operation interface of the mobile application program simulated and run through the emulator on the PC. For the sake of distinction, this application understands the operation interface of the emulator as the operation interface provided by the emulator itself. For the operation interface of the application program running through the emulator in the computer device, this application refers to it as the application interface of the application program in the operation interface of the emulator. For example, in the Figure 1 illustrated embodiment, the interface 1001 can refer to the operation interface of the emulator on the computer device. The application interface 00A is the application interface of the application program A displayed when the application program A is running in the emulator. In fact, this application interface 00A can also be understood as the operation interface of the emulator.

[0109] The specific process of displaying the operation interface of the emulator may at least include the following three steps:

[0110] 1. Run the emulator in the computer device. The emulator is used to simulate the running environment of the second operating system.

[0111] Among them, the operating system running in the computer device can be called the first operating system. The application program is developed based on the specifications of the second operating system. Simulating the running environment of the second operating system in the computer device is a prerequisite for running the application program in the computer device. The emulator is a software tool installed in the computer device and can be used to simulate the running environment of the second operating system in the computer device. Thus, the emulator can be run in the computer device to simulate the running environment of the second operating system. Taking the application program as a mobile application program simulated and run on the PC as an example, the emulator can be a mobile emulator. The mobile emulator can be used to simulate the running environment of the mobile operating system on the PC, allowing the experience of the application program in the mobile device without actually owning a mobile device. An exemplary scenario running in the embodiment of this application can be: on a PC running the Windows operating system (i.e., the first operating system), the Android operating system (i.e., the second operating system) is simulated and run through the mobile emulator, and the application program developed based on the specifications of the Android operating system is downloaded, installed and opened for normal use.

[0112] Specifically, the emulator may include an application market program, the emulator host (the emulator Host side), and the emulator guest (the emulator Guest side). For example, when the emulator is a mobile emulator, the mobile emulator may include an application market program, the mobile emulator Host side, and the mobile emulator Guest side. Among them:

[0113] The application market program refers to the program in the emulator used to implement functions such as application search and application download; the application market program can also be used to manage the running states of the emulator Host side and the emulator Guest side.

[0114] The emulator Host side refers to the emulator software itself running on a computer device, as well as the supporting hardware abstraction layer, operating system, and user interface components; the emulator Host side can be responsible for creating and managing a virtual environment (i.e., the emulator Guest side), and providing a series of tools and interfaces so that the using object can interact with the virtual environment. For example, when the emulator Host side is the mobile emulator Host side, the mobile emulator Host side can be responsible for creating and managing a virtual mobile environment (i.e., the mobile emulator Guest side) so that the using object can interact with the virtual mobile environment.

[0115] The emulator Guest side refers to the virtual environment created by the emulator on a computer device. In this virtual environment, the running environment of a second operating system is simulated. In this virtual environment, application programs developed based on the specifications of the second operating system can be installed and run, and various operations and experiences can be carried out. For example, when the emulator Guest side is the mobile emulator Guest side, the virtual mobile environment created by the mobile emulator Guest side on a computer device can install and run mobile application programs, and the using object can perform various operations and experiences, just like using a real mobile device. In the embodiment of the present application, the mobile emulator Guest side is an Android operating system implemented based on a vbox (a virtual machine software) virtualization solution, and its manifestation form is a VDI (Virtual Desktop Infrastructure) file. It complies with the VDI file format specification, and the mobile emulator Host side loads this VDI file to implement loading and running this VDI file.

[0116] 2. Display the operation interface of the emulator.

[0117] Specifically, the emulator simulates the operating environment of the second operating system in the computer device, and can run application programs in the simulated operating environment of the second operating system, that is, application programs can be run in the emulator. The computer device can display these runnable application programs on the operation interface of the emulator for the user to select any application program to run.

[0118] 3. Display the operation interface of the application program simulated and run in the emulator.

[0119] Specifically, for any launched application program, the computer device can display the operation interface of the application program simulated and run in the operation interface of this emulator. In this application, for the convenience of distinction, the operation interface of the application program is called the application interface. In fact, this application interface can also be understood as the operation interface of the emulator because this application interface is displayed in the operation interface of this emulator and cannot be separated from the operation interface of the emulator.

[0120] It can be seen that in the operation interface of the emulator of this application, one or more application programs can be included. The user (which can be understood as the object using the computer device or the emulator, such as: user, intelligent robot, etc.) can launch a certain application program. Once a certain application program is launched, the emulator will display the application interface for this application program in the operation interface and display the application content of this application program in this application interface. The user can perform different operations on the application content. For example, the user selects to launch application program B (a game application) in the operation interface of the emulator. The emulator will display the application interface for this application program B in the operation interface and display the relevant game content of this application program B in this application interface. The user can log in to the game application and experience the game according to the displayed game content in this application interface. Another example, the user selects to launch video application C (application program C) in the operation interface of the emulator. The emulator will display the application interface for this application program C in the operation interface and display the relevant video content of this application program C in this application interface. The user can browse videos according to the video content displayed in this application interface.

[0121] For the application content displayed in a certain application interface, it can include media data of different media types. The media types can at least include but are not limited to: picture type, video type, audio type, text type, etc. Then the media data included in the application content can include but are not limited to: pictures under the picture type, videos under the video type, videos under the audio type, videos under the text type. The first media data in this application refers to a certain media data displayed in the application interface of the first application program. This first application program refers to any application program that is launched and run in the emulator.

[0122] Regarding the operation interface containing media data proposed in this application, it can refer to the operation interface of the application interface containing an application program. For example, since the application interface of the first application program is displayed in the operation interface of the emulator, and the application content of the first application program (including the first media data) is displayed in this application interface, this operation interface will contain the first media data. Therefore, this operation interface is actually the operation interface of the application interface containing the first application program.

[0123] In practical applications, after the computer device displays the operation interface of the emulator (including the application interface of the application program), the user can perform various different operations in this operation interface. For example: performing a start operation on a certain application program, performing an uninstall operation on a certain application program, after starting a certain application program, performing a viewing operation on a certain media data in the application interface of the operation interface, after starting a certain application program, performing an interaction operation on a certain media data in the application interface of the operation interface (such as: like, comment, forward, report, etc. interaction operations), performing a copy operation on a certain media data, and so on. Regarding the operations that can be performed in the operation interface of the emulator (including various operations performed in the application interface of the operation interface), this application can refer to them as emulator operations.

[0124] A communication connection can be established between the computer device and the control device. In this way, various emulator operations performed in the operation interface of the emulator can be triggered by the control device. The control device can include, but is not limited to, any one of the following: a mouse, a touchpad, and a handheld control component. Among them, a mouse is an external input device of a computer device, mainly used to control the position of the control cursor on the screen of the computer device, and operate on the content displayed in the operation interface through the buttons of the mouse (for example, it can include the left button and the right button of the mouse) and the scroll wheel device. For example, the user can use the mouse to control the control cursor on the screen of the computer device to move to a certain application program, and then the user can perform a start operation on this application program by clicking the left button of the mouse. A touchpad is an input device widely used on computer devices. It uses the sensing of finger movement to control the movement of the cursor and can be regarded as a substitute for a mouse. A handheld control component is a handheld input device that controls the position of the control cursor on the screen of the computer device by operating devices such as buttons and joysticks in the handheld control component. For example, the handheld control component can include a control handle.

[0125] Moreover, the embodiments of the present application do not limit the manner of establishing a communication connection between the control device and the computer device. The control device can be externally connected to the computer device, or the control device can be built into the computer device. The so-called external connection means that the control device is an external device that establishes a communication connection with the computer device. The control device can establish a communication connection with the computer device through a wired communication method, or the control device can establish a communication connection with the computer device through a wireless communication method; for example, a wired mouse can establish a communication connection with the computer device through a wired communication method (such as a physical cable), and a wireless mouse can establish a communication connection with the computer device through a wireless communication method (such as Bluetooth). The so-called built-in means that the control device is set on the computer device. For example, a touchpad is set on the computer device.

[0126] Step S302, in response to a viewing operation on the first media data, display a first media display interface; the first media display interface is independent of the emulator.

[0127] In the present application, based on the above, for any application interface displayed in the operation interface, the user can perform corresponding emulator operations on this application interface. For example: the user can view the media data in the application interface (the emulator operation performed is a viewing operation on the media data). For example, the user can select to view a certain picture, a certain video, or a certain text. Once the user wants to view a certain media data, the user can perform a viewing operation on the media data. Here, taking the user performing a viewing operation on the first media data as an example, the computer device responds to this viewing operation and displays a media display interface for displaying the first media data on the device screen of the computer device. To make a distinction, the media display interface for displaying the first media data in the present application is called the first media display interface.

[0128] Through Figure 1In the illustrated embodiment, in the prior art, for a certain piece of media data to be viewed, the media display interface for presenting the media data must be within the operation interface of the simulator and cannot be separated from the simulator itself. After viewing the media data, the display will only be enlarged within the operation interface of the simulator. The media display interface for presenting the media data cannot break through the limitations of the operation interface of the simulator, which greatly restricts the user's browsing experience of the media data. Moreover, the media display interface for presenting the media data will cover some application interfaces, and it is impossible to synchronously display the media data and perform simulator operations in the application interface. If the user wants to continue to perform a certain simulator operation in the application interface, the user needs to first close the currently displayed media display interface, which also has a greater impact on the user's experience of using the simulator. Based on this, for the media data to be viewed, the present application improves its display method. The present application will create an independent media display interface for the media data to be viewed. This media display interface can be overlapped with the operation interface of the simulator (the overlapping display can be full overlapping display or partial overlapping display), or can be partitioned or split-screen displayed with the operation interface of the simulator. However, regardless of whether the display position relationship between this media display interface and the operation interface of the simulator is an overlapping display relationship or a partitioned or split-screen display relationship, the display position of this media display interface can be dragged and changed (for example, dragging the media display interface from one display position to another by a mouse), which means that the media display interface in the present application will not affect the operation interface. Even if the media display interface covers the operation interface, the media display interface can be dragged to another position for display. That is to say, this media display interface and the operation interface of the simulator are independent of each other, and the operations that can be performed in the two interfaces are independent. During the process of displaying the media display interface, the user can still perform different simulator operations in the operation interface.

[0129] In summary, the media display interface created by this application for the viewed media data can be an interface independent of the emulator. To achieve independence from the emulator, this application can call the media player deployed in the computer device to create the media display interface. This media player can refer to a certain application that can only run on the operating system of the computer device, which can be a player built into the operating system or a third-party player downloaded on the computer device. Since this media player does not rely on the emulator to run, the media display interface it creates is an interface independent of the emulator and not restricted by the emulator. It supports synchronous or asynchronous display with the operation interface of the emulator. Moreover, various operations executable in this media display interface (such as operations that can be performed on the media data in the media display interface, for example, pausing the playback of media data, enhancing the clarity, playing at multiple speeds, etc.) can be executed in parallel with the emulator operations in the operation interface (such as operations that can be performed on the application content displayed in the operation interface, for example, liking and commenting on the application operation content; or, for example, pausing the playback of media data in the operation interface). The parallel execution in this application can mean simultaneous execution or sequential execution. For example, during the process of displaying media data in the media display interface, the user can like a certain application content in the operation interface while also pausing the playback of the displayed media data in the media display interface; for another example, during the process of displaying media data in the media display interface, the user can first like a certain application content in the operation interface and then pause the playback of the displayed media data in the media display interface.

[0130] That is to say, when media data is displayed in the media display interface, the emulator operations executable in the operation interface and the operations executable in the media display interface can be the same or different, and the emulator operations executable in the operation interface and the operations executable in the media display interface support simultaneous execution and also support sequential execution.

[0131] Based on this, for the first media display interface of the first media data, the display position relationship between it and the operation interface of the simulator can include any of the following: ① It is an overlapping display relationship with the operation interface of the simulator; ② It is a non-overlapping display relationship with the operation interface of the simulator, for example: it is a partitioned (or split-screen) display relationship. However, regardless of which display position relationship it is, the display position of the first media display interface or the operation interface of the simulator can be changed by dragging. For example, a user can perform a drag operation on the first media display interface to change the display position of the first media display interface. After the user performs a drag operation on the first media display interface, the computer device can respond to this drag operation and switch the display position of the first media display interface to the display position indicated by this drag operation (that is, the position where the first media display interface is located when the user stops dragging).

[0132] Step S303, display the first media data in the first media display interface.

[0133] In this application, the computer device can display the first media data in the first media display interface. For the first media display interface, it can be created by the media player deployed in the computer device. However, the media players deployed in the computer device are not single and can include multiple media players. Each media player can be used to play media data of a certain media type. For example, a picture player can be used to display / play pictures; another example is that a video player can be used to display / play videos; and yet another example is that an audio player can be used to display / play audio. This means that when creating a corresponding media display interface for any media data, it can be filtered based on the media type to which the media data belongs, and the media player corresponding to this media type is used to create a media display interface for it. Taking the example that there are N (N is a positive integer) media players deployed in the computer device, for the creation of the first media display interface, first obtain the media type to which the first media data belongs (referred to as the target media type in this application), then determine the media player corresponding to this target type from the N media players (that is: the media player used to play the media data of this target media type), and then the media player corresponding to this target media type creates the first media display interface for the first media data. Correspondingly, when displaying the first media data in the first media display interface, the first media data can also be passed to the media player corresponding to this target media type, and the media player corresponding to this target media type displays the first media data in this first media display interface.

[0134] To better understand the method provided in this application for displaying the viewed media data, the following will be described by way of example with reference to the accompanying drawings. Please also refer to Figure 4 ,Figure 4 This is a schematic diagram of a scenario for viewing media data in a simulator scenario provided by an embodiment of the present application. For comparison with Figure 1 the traditional media display method shown, the scenario shown in Figure 4 is still described by taking a computer device running a simulator and an object using the simulator as object a (hereinafter referred to as using object a) as an example. When using object a runs the simulator on the computer device, the computer device can display the operation interface 1001 of the simulator on its own device screen. As shown in Figure 4 In this operation interface 1001, there are application programs A, B, and C that can be simulated and run. Using object a can open any application program to run and use. Here, taking using object a to start application program A as an example, the computer device can display the application interface 00A of this application program A in this operation interface 1001, and display the application content of application program A in this application interface 00A. Among them, this application content contains picture 10A3 (i.e., a media data displayed in application interface 00A).

[0135] Here, it is assumed that the control device connected to the computer device is a mouse. Then, various simulator operations performed by using object a in the operation interface 1001 of the simulator can be triggered by the mouse. For example, using object a can move the control cursor M displayed on the screen of the computer device to picture 10A3 through the buttons of the mouse, and when the control cursor M is on picture 10A3, double-click picture 10A3 by pressing the left mouse button twice continuously. This double-click operation can be understood as using object a performing a viewing operation on picture 10A3. The computer device can respond to this viewing operation and display a media display interface 4001 for displaying this picture 10A3 on this operation interface 1001, and display this picture 10A3 in this media display interface 4001. As shown in Figure 4 This media display interface 4001 is an interface independent of the operation interface 1001 and can be displayed on the screen of the computer device together with the operation interface 1001. This media display interface 4001 is not limited by the display size of the operation interface 1001. For both the operation interface 1001 and the media display interface 4001, using object a can drag the interface to change the display position of the interface.

[0136] For example, in Figure 4In the illustrated embodiment, the media display interface 4001 and the operation interface 1001 are overlapped and displayed, and the media display interface 4001 will cover part of the display content in the application interface 00A. However, since the device screen of the computer device is much larger than the operation interface of the emulator, there is enough space on the device screen of the computer device for the media display interface 4001 to move. The media display interface 4001 no longer needs to be limited by the display size of the operation interface 1001. Based on this, for the user object a, in order to continue browsing the application content in the application interface 00A and continue to perform corresponding emulator operations on the application content in the application interface 00A, the user object a can move the display position of the media display interface 4001 or the operation interface 1001 so that the media display interface 4001 no longer covers the application content in the application interface 00A. For ease of understanding, the process of interface movement will be exemplarily described below in conjunction with the accompanying drawings. Please also refer to Figure 5 , Figure 5 FIG. Figure 5 is a schematic diagram of a scenario for changing the display position of the media display interface provided by an embodiment of the present application. Among them, in the embodiment as shown in Figure 5 , the computer device running the emulator is taken as the computer device 100a, and the device screen of the computer device 100a is taken as the interface 500 for description, and the operation interface 1001 will be displayed in this interface 500.

[0137] As shown in Figure 5 , in combination with the embodiment as shown in Figure 4 , after the computer device 100a responds to the viewing operation of the user object a on the picture 10A3, and displays the media display interface 4001 for displaying the picture 10A3 on the operation interface 1001 and displays the picture 10A3 in this media display interface 4001, the user object a moves the control cursor M to the media display interface 4001 through the mouse. When the control cursor M is within the media display interface 4001, the user object a performs a dragging operation on the media display interface 4001 through the mouse, and the dragging direction is the arrow direction as shown in Figure 5 . The computer device 100a can respond to this dragging operation of the user object a and change the display position of the media display interface 4001 following the dragging of the user object a in the dragging direction. When the user object a ends the dragging, the display position of the media display interface 4001 no longer changes, and the final position where this media display interface 4001 is located can be understood as the display position indicated by the dragging operation of the user object a. For example, as shown in Figure 5As shown, after the display position of the media display interface 4001 changes after being dragged, it will no longer overlap with the operation interface 1001 or the application interface 00A. At this time, during the process of displaying the picture 10A3 on the media display interface 4001, the user object a can still view the content displayed in the application interface 00A. Moreover, the user object a can continue to perform corresponding emulator operations on the content in the application interface 00A (for example, the user object a can perform a refresh operation to refresh the currently displayed picture 10A3 and view other media data in the application program A other than the picture 10A3).

[0138] Through Figure 4 And Figure 5 From the corresponding embodiments, it can be seen that the media display interface of the present application for displaying the media data viewed in the emulator is independent of the emulator itself and is not limited by the display size of the operation interface of the emulator. This means that during the process of displaying the media data, the user object can still continue to perform corresponding emulator operations on the operation interface, without affecting any operations on the operation interface of the emulator, which will well improve the user experience of the user object. Taking the first media data as an example, during the process of displaying the first media data on the first media display interface, the user object can synchronously perform corresponding emulator operations on other application contents displayed in the application interface of the first application program. Here, the other application contents can refer to the application contents in the application interface of the first application program other than the first media data, and the user object can operate on these application contents, that is: these application contents are operable contents. For the convenience of distinguishing from the first media data, the present application refers to the application contents in the application interface of the first application program other than the first media data as application operation contents. Then, during the process of displaying the first media data on the first media display interface, the user object can trigger any application operation content to perform corresponding operations. After the user object triggers the application operation content, the emulator can respond to this trigger operation and display the trigger result content corresponding to the application operation content in the application interface of the first application program. The trigger result content corresponding to this application operation content refers to the visual content obtained after triggering this application operation content. For example, the application operation content displayed in the application interface of the first application program includes a comment control for the first media data. The user object can trigger this comment control to comment on the first media data. The visual content after the user object triggers this comment control is a text input box. The user object can enter the content he wants to express about the first media data in this text input box. Then, this text input box is the trigger result content corresponding to this comment control. That is to say, different application operation contents can correspond to different trigger result contents because the visual contents after triggering different application operation contents are different.

[0139] Since the media display interface and the operation interface of the simulator in this application are independent of each other and neither depends on the other, when synchronously displaying the operation interface and the media display interface, the media display interface can also be independently controlled. For example: The user can change the display position of the media display interface; Another example is that the user can adjust the media display interface. Taking the first media display interface as an example, after the first media data is displayed in the first media display interface, the user can perform an interface adjustment operation on the first media display interface. This interface adjustment operation can include operations such as adjusting the display size and adjusting the display angle. After the user performs the interface adjustment operation on the first media display interface, the computer device can respond to this interface adjustment operation and adjust the display of the first media display interface to the media adjustment interface indicated by this interface adjustment operation. For example, when the interface adjustment operation is a size adjustment operation, the computer device can respond to the size adjustment operation on the first media display interface and adjust the display size of the first media display interface to the display size indicated by this size adjustment operation. This means that the media adjustment interface indicated by this interface adjustment operation is the first media display interface with the display size indicated by the size adjustment operation. Among them, the size adjustment operation in this application can include: size enlargement operation and size reduction operation. Another example is that when the interface adjustment operation is an angle adjustment operation, the computer device can respond to the angle adjustment operation on the first media display interface and adjust the tilt angle of the first media display interface to the tilt angle indicated by this angle adjustment operation. The tilt angle of the first media display interface in this application can refer to the tilt angle relative to the device screen of the computer device, and this tilt angle can be determined based on the symmetry line of the first media display interface in the vertical or horizontal direction.

[0140] Among them:

[0141] The horizontal direction can also be called the transverse direction, which refers to the left - right direction of the screen. Usually, the interface in this application is a rectangle. Then, this application can use a certain vertex on the device screen as the coordinate origin, and the two sides connected to this vertex as the coordinate axes (X - axis and Y - axis) to establish a coordinate system. The symmetry line parallel to the X - axis in the device screen can be used as the symmetry line of the device screen in the horizontal direction; Similarly, this application can use a certain vertex of the media display interface as the coordinate origin, and the two sides connected to this vertex as the coordinate axes (X - axis and Y - axis) to establish a coordinate system. The symmetry line parallel to the X - axis in the media display interface can be used as the symmetry line of the media display interface in the horizontal direction. Then, the included - angle formed between the symmetry line of the operation interface in the horizontal direction and the symmetry line of the media display interface in the horizontal direction can be determined, and this included - angle can be used as the tilt angle of this media display interface.

[0142] The vertical direction can also be referred to as the vertical orientation, which refers to the up-and-down direction of the screen. In this application, a certain vertex on the device screen can be used as the coordinate origin, and the two sides connected to this vertex can be used as the coordinate axes (X-axis and Y-axis) to establish a coordinate system. The symmetry line parallel to the Y-axis in the device screen can be used as the symmetry line of the device screen in the vertical direction; similarly, in this application, a certain vertex of the media display interface can be used as the coordinate origin, and the two sides connected to this vertex can be used as the coordinate axes (X-axis and Y-axis) to establish a coordinate system. The symmetry line parallel to the Y-axis in the device screen can be used as the symmetry line of the media display interface in the vertical direction. Then, the included angle formed between the symmetry line of the device screen in the vertical direction and the symmetry line of the media display interface in the vertical direction can be determined, and this included angle can be used as the tilt angle of the media display interface.

[0143] For the tilt angle of the media display interface, corresponding determination criteria can be formulated based on actual business requirements. This application is only an exemplary illustration for easy understanding, and this application does not limit the definition and determination method of the tilt angle of the media display interface.

[0144] During the process of this application displaying the first media display interface, it can synchronously display size adjustment controls for this first media display interface (including a size magnification control and a size reduction control), and can also include an angle adjustment control for the first media display interface. In this way, the user can trigger the size adjustment control to perform a size adjustment operation on the first media display interface, and can trigger the angle adjustment control to perform an angle adjustment operation on the first media display interface. For easy understanding, please also refer to Figure 6 , Figure 6 which is a schematic diagram of a scenario for adjusting the display angle of a media display interface provided by an embodiment of this application. As Figure 6 shown, in combination with the embodiment shown in Figure 5 , for the media display interface 4001, whether before or after the display position is changed, the computer device can display, in this media display interface 4001, a magnification control (i.e., a size magnification control for magnifying the display size of the media display interface 4001), a reduction control (i.e., a size reduction control for reducing the display size of the media display interface 4001), and a rotation control (i.e., an angle adjustment control for rotating the media display interface 4001, and the tilt angle of the media display interface can be changed by rotation). As Figure 6 shown, assuming that the user a clicks on the rotation control with the mouse, it can be considered that the user a has performed a triggering operation on the rotation control (i.e., an angle adjustment operation on the media display interface 4001), and the computer device 100a can respond to this operation and follow the rotation direction indicated by the rotation control (i.e., as Figure 6Rotate the media display interface 4001 in the counterclockwise rotation direction (as shown), such as Figure 6 As shown, after the media display interface 4001 is rotated, its tilt angle will change.

[0145] It should be noted that when the display position, display angle, and display size of the media display interface change, the media data displayed in this media display interface will also change dynamically with the change of the media display interface. For example: when the display size of the media display interface is enlarged, the display area of the media data displayed in this media display interface will also be enlarged accordingly.

[0146] Of course, in addition to being able to control the media display interface, the media data displayed in the media display interface can also be operated on separately. For example: when the media data displayed in the media display interface is a video or audio, pause play operation, speed play operation, slow play operation, volume adjustment operation (for example: volume increase operation, volume decrease operation, mute operation, etc.) can be performed on the video or audio; when the media data displayed in the media display interface is a picture, clarity adjustment operation (for example: clarity increase operation, clarity decrease operation, etc.) can be performed on the picture; when the media data displayed in the media display interface is text, font adjustment operation (for example: font increase operation, font decrease operation, font switching operation, etc.) can be performed on the text. After performing the corresponding operation on the media data, the display state of this media data may change accordingly. For example: after performing the pause play operation on the video, the display state of the video will switch from the play state to the static state (pause play state); after performing the clarity increase operation on the picture, the clarity of this picture will be higher, and the display state of the picture will switch from the blurred state (or low clarity state) to the clear state (high clarity state). For an operation that can change the display state of media data, this application calls it a display adjustment operation on media data. Once the display adjustment operation is performed, the display state of this media data will be switched. Taking the first media data as an example, after the first media data is displayed in the first media display interface, assuming that the user of the device performs a display adjustment operation on the first media data displayed in the first media display interface, the computer device can respond to this display adjustment operation and switch the display state of the first media data displayed in this first media display interface to the display state indicated by this display adjustment operation (this application refers to the display state indicated by the display adjustment operation as the target display state).

[0147] Through Figure 3 - Figure 6As can be seen from the corresponding embodiments, the media display solution based on the emulator provided by this application improves the display method of the media data to be viewed. During the process of running the emulator in the computer device and running the application through the emulator, the media data to be viewed can be displayed in the media display interface outside the operation interface of the emulator. This media display interface is independent of the emulator and, like the emulator, is displayed on the device screen of the computer device, which can bring a better user experience. For example, while viewing the displayed media data, the user can simultaneously perform corresponding emulator operations in the operation interface of the emulator, realizing the synchronization of emulator operations and the display of media data. The solution provided by the embodiments of this application can be applied to scenarios where it is necessary to break away from the limitations of the operation interface of the emulator to view the media data in the application, and has a more flexible media browsing method.

[0148] Based on the above description, it can be known that the operation interface of the emulator and the media display interface of the media data in this application support synchronous display. The synchronous display in this application can be understood as that the content displayed in the two interfaces is associated and synchronized. Since the content displayed in the media display interface is the media data viewed in the operation interface, this means that the synchronous display of the operation interface and the media display interface can be understood as: the display states of the media data displayed in the operation interface and the media data displayed in the media display interface are associated and synchronized. During the process of displaying media data in the media display interface, the user can adjust the display state of the media data displayed in the operation interface or the display state of the media data displayed in the media display interface. However, once the display state of this media data is adjusted, the display states of the media data displayed in the two interfaces will change accordingly. Taking the above first media data and the first media display interface as an example, after the first media data is displayed in the first media display interface, assuming that the user performs a display adjustment operation on the first media data to adjust the display state of the first media data, the computer device can respond to this operation and switch the display state of the first media data in the first media display interface to the target display state indicated by the display adjustment operation. In addition, the computer device can also synchronously switch the display state of the first media data displayed in the operation interface of the emulator (actually in the application interface of the first application) to this target display state. For example, when the first media data is a video, assuming that the display adjustment operation performed by the user on the video is a pause play operation, then the computer device will switch the display state of the video displayed in the first media display interface from the play state to the pause play state, and will also switch the display state of the video displayed in the operation interface from the play state to the pause play state.

[0149] Correspondingly, when the media data to be viewed is video or audio, when the video is displayed on the media display interface, it can be displayed according to the playback progress of the video shown in the operation interface. Taking the first media display interface and the first media data as an example, assuming that the first media data is video or audio, the process of displaying the first media data on the first media display interface can at least include: First, when responding to the viewing operation of the first media data, the historical playback duration of the first media data can be obtained. This historical playback duration can refer to the duration that the first media data has been played in the application interface of the operation interface. For example, if the first media data is a video with a total duration of 10s (i.e., 0 - 10s), when this video is played to the 5th second, the user performs a viewing operation on this video, and the time point when the computer device responds to this viewing operation can be the 5th second when this video is being played. At this time, this video has been played for 5s, and the historical playback duration of this video is 5s. Based on the historical playback duration of the first media data, the playback start time point can be determined in the first media data. This playback start time point is also the playback position point of the first media data when the viewing operation is performed on the first media data (for example, the 5th second of the above video). Then, using the playback start time point as the playback start position, the first media data can be played and displayed on the first media display interface.

[0150] For ease of understanding, please also refer to Figure 7 , Figure 7 FIG. is a schematic diagram of a scenario for displaying a video on a media display interface provided by an embodiment of the present application. Among them, in order to effectively combine and echo with the previous embodiments, as Figure 7 shown, the scenario still takes the computer device running the emulator as computer device 100a, the object using the emulator as object a (subsequently referred to as user object a), and the device screen of the computer device as interface 500 for illustration. When user object a runs the emulator on computer device 100a, computer device 100a can display the operation interface 1001 of the emulator on its own device screen. As Figure 7 shown, the operation interface 1001 of the emulator is displayed in interface 500. In this operation interface 1001, there are application programs A, B, and C that can be simulated and run. User object a can open any application program to run and use. Here, taking user object a starting application program B as an example, computer device 100a can display the application interface 00B of application program B in this operation interface 1001, and display the application content of application program B in this application interface 00B. Among them, this application content includes video 10A4 (i.e., a media data displayed in application interface 00B).

[0151] Here, it is assumed that the control device connected to the computer device 100a is a mouse. Then, various emulator operations performed by the user a in the operation interface 1001 of the emulator can be triggered by the mouse. For example, the user a can move the control cursor M displayed on the screen of the computer device to the video 10A4 through the mouse button, and when the control cursor M is on the video 10A4, double-click the video 10A4 by pressing the left mouse button twice. This double-click operation can be understood as the user a performing a viewing operation on the video 10A4. The computer device 100a can respond to this viewing operation and display a media display interface 7001 for presenting the video 10A4 on the operation interface 1001, and present the video 10A4 in this media display interface 7001.

[0152] Among them, when the computer device 100a presents the video 10A4 in the media display interface 7001, it can obtain the historical playback duration of the video 10A4 in the operation interface 1001 (for example, the historical playback duration is 1.5 s). Based on this historical playback duration, it can be determined that the playback start time point of the video 10A4 in the media display interface 7001 is the 1.5 s of the video 10A4. Then, the computer device 100a can start playing this video 10A4 from 1.5 s in the media display interface 7001.

[0153] Based on the above description, it can be known that the operation interface and the media display interface of the simulator in this application can also support asynchronous display. Corresponding to synchronous display, the asynchronous display in this application can be understood as that the content displayed in the two interfaces is not associated and synchronized. The asynchronous display of the operation interface and the media display interface can be understood as follows: for the same media data displayed in the operation interface and the media display interface, its display states in the two interfaces are not associated and synchronized, and synchronous management is not required. In other words, if the user performs a display adjustment operation on the media data in the operation interface, the computer device can adjust the display state of the media data displayed in this operation interface. However, the display state of this media data displayed in the media display interface can remain unchanged. Similarly, if the user performs a display adjustment operation on the media data in the media display interface, the computer device can also adjust the display state of the media data displayed in this media display interface. However, the display state of this media data displayed in the operation interface can remain unchanged. Moreover, when displaying media data in the media display interface, it is not necessary to refer to the display state of this media data in the operation interface, and this media data can be displayed in the media display interface with a new display state. For example, when playing a video in the media display interface, even if the video has been played to the 20th minute in the operation interface, the video can be played from the beginning in the media display interface; another example is that when displaying a picture in the media display interface, although the clarity of the picture displayed in the operation interface is 480p, the picture can be displayed with a clarity of 720p in the media display interface. Generally speaking, whether it is necessary to associate and synchronize the display states of the media data in the operation interface and the same media data displayed in the media display interface can be set accordingly based on actual business requirements.

[0154] It should be noted that the media display interface in this application supports synchronous or asynchronous closing with the operation interface of the simulator. Synchronous closing of the media display interface and the operation interface means that when one interface is closed and no longer displayed, the other interface will be closed and no longer displayed at the same time. For example, when media data is being displayed on the media display interface, if the user performs an operation to close the media display interface, the computer device will respond to this operation and close the media display interface. At the same time, it will also close the operation interface of the simulator. That is to say, when the media display interface and the operation interface are synchronously closed, an operation to close one of the interfaces by the user will result in both interfaces being closed. Asynchronous closing of the media display interface and the operation interface means that when one interface is closed and no longer displayed, the other interface will not be closed and no longer displayed at the same time. For example, when the same video is being displayed on both the operation interface of the simulator and the media display interface, assuming that the display states (i.e., playing states) of the videos in both interfaces are associated and synchronized, and when the video is played to the 10th minute, if the media display interface is closed and no longer displayed, this will not affect the operation interface of the simulator, which will continue to be displayed, and the video displayed on this operation interface will also continue to play.

[0155] Optionally, in some feasible embodiments, the media display interface can be synchronously closed when the operation interface is closed, but the operation interface is not affected by the media display interface. That is, when the operation interface is closed, the media display interface will be synchronously closed, but when the media display interface is closed, the operation interface will not be synchronously closed. In this way, the user can directly close the operation interface to also close the media display interface based on this operation interface. In addition, if the user accidentally closes a certain media display interface, since the operation interface is not synchronously closed, the user can reopen this media display interface through this operation interface to view the media, thereby improving the user experience.

[0156] Based on the above Figure 3According to the description in the corresponding embodiment, the present application can display a media viewing control for the first media data in the operation interface of the simulator. In this way, the viewing operation of the first media data can include any one of the following: ① A triggering operation on the first media data. In practical applications, the triggering operation on the first media data may refer to a double-click operation on the display area of the first media data. For example, when the control device is a mouse, move the control cursor to the display area of the first media data in the operation interface through the mouse, and then double-click the left mouse button twice at this time to implement the triggering operation on the first media data. ② An operation of dragging the first media data to a specified area. This specified area can be a preset area on the device screen of the computer device. For example, any area on the device screen of the computer device except the operation interface of the simulator can be used as this specified area. Once the first media data is dragged to the specified area, it can be considered that the viewing operation has been performed on the first media data. ③ A triggering operation on the media viewing control.

[0157] For the media viewing control, it can be displayed together with the media data while displaying the application content of the application program in the application interface. In this way, the user can directly trigger this media viewing control to view this media data; it can also be displayed after the user performs a preset specified operation. For ease of understanding, the following will take the media viewing control for displaying the first media data as an example to illustrate the display process of the media viewing control. Please refer to Figure 8 , Figure 8 which is a schematic flowchart of a method for displaying a media viewing control for the first media data in the operation interface provided by an embodiment of the present application. As Figure 8 shown, this process can at least include the following steps S801 - step S803:

[0158] Step S801, display the operation interface of the simulator, and the operation interface of the simulator includes the first media data.

[0159] Specifically, referring to the relevant description in the above step S301, the operation interface of the simulator displays the application interface of the first application program, and the first media data of the present application is the media data displayed in the application interface of the first application program.

[0160] Step S802, in response to a triggering operation on the application interface of the first application program, obtain the position coordinates of the triggered position.

[0161] Specifically, the triggering operation on the application interface of the first application in this application may refer to the triggering operation performed when the cursor is at any position on the application interface of the first application. For example, when the cursor is at a certain position on the application interface of the first application, a click operation is performed by pressing the right mouse button. When the user performs a triggering operation on the application interface of the first application, the computer device can obtain the position coordinates of the triggered position, which can be understood as the position where the cursor is located.

[0162] The position coordinates of the triggered position may refer to the position coordinates relative to the application interface. The process of obtaining the position coordinates of the triggered position in the application interface may at least include the following steps:

[0163] 1. Obtain the absolute coordinates of the triggered position on the device screen of the computer device.

[0164] For the device screen of the computer device, a coordinate system can be constructed with a certain vertex as the coordinate origin and the two sides connected to this vertex as the two coordinate axes (including the X-axis and the Y-axis). In this application, the coordinates (screen_x, screen_y) of the triggered position in this coordinate system can be obtained, and these coordinates can be used as the absolute coordinates of the triggered position on the device screen. Among them, when the computer device is a PC device, the Windows API GetCursorPos method can be called to obtain the absolute coordinates of the triggered position.

[0165] 2. Obtain the absolute coordinates of the triggered application interface on the device screen.

[0166] Similar to the device screen of the computer device, for the triggered application interface, a coordinate system can also be constructed with a certain vertex as the coordinate origin and the two sides connected to this vertex as the two coordinate axes (including the X-axis and the Y-axis). Then, for the vertex that serves as the origin in the application interface, the coordinates (simulator_rect.left, simulator_rect.top) of this vertex in the coordinate system constructed on the device screen can be obtained, and the coordinates of this vertex in the device screen can be used as the absolute coordinates of the triggered application interface on the device screen. Among them, when the computer device is a PC device, the GetWindowRect method can be called to obtain the coordinates of the vertex in the application interface in the coordinate system constructed on the device screen.

[0167] 3. Obtain the scaling ratio of the coordinates.

[0168] For the position coordinates of the triggered position in the application interface, this application refers to them as the relative coordinates in the application interface. Regarding the absolute coordinates of the triggered position on the device screen of the computer device and the relative coordinates on the application interface, this application can calculate and determine a scaling ratio (scale_factor) from the absolute coordinates to the relative coordinates. Among them, when the computer device is a PC device, the get_simulator_scale() method can be called to dynamically calculate and determine the current scaling ratio.

[0169] 4. Convert the absolute coordinates to relative coordinates based on the scaling ratio.

[0170] Based on the scaling ratio, the absolute coordinates of the triggered position can be calculated, so that the position coordinates (i.e., relative coordinates) of the triggered position in the coordinate system constructed by the application interface can be obtained. The relative coordinates of the triggered position can be expressed as (android_x, android_y), and the calculation method of android_x can be as shown in formula (1):

[0171] android_x = (screen_x - simulator_rect.left) / scale_factor Formula (1)

[0172] Among them, screen_x can be used to represent the X-axis coordinate in the absolute coordinates of the triggered position; simulator_rect.left can be used to represent the X-axis coordinate in the absolute coordinates of the application interface on the device screen; scale_factor is used to represent the scaling ratio.

[0173] The calculation method of android_y can be as shown in formula (2):

[0174] android_y = (screen_y - simulator_rect.top) / scale_factor Formula (2)

[0175] Among them, screen_y can be used to represent the Y-axis coordinate in the absolute coordinates of the triggered position; simulator_rect.left can be used to represent the Y-axis coordinate in the absolute coordinates of the application interface on the device screen; scale_factor is used to represent the scaling ratio.

[0176] After calculating the relative coordinates of the triggered position in the application interface, this relative coordinate can be used as the final position coordinate of the triggered position.

[0177] Step S803: If the triggered position is within the display area of the first media data, then display a media viewing control within the display area of the first media data.

[0178] Specifically, after obtaining the position coordinates of the triggered position, it is possible to determine whether the triggered position is within the display area of the first media data (i.e., the display area in the application interface) based on these position coordinates. If it is determined that the triggered position is within the display area of the first media data, then a media viewing control for the first media data can be displayed within the display area of the first media data. The specific process for determining whether the triggered position is within the display area of the first media data may at least include: First, it is possible to obtain the interface content included in the application interface of the first application (i.e., the application content displayed in the application interface described above, which will specifically include the first media data and the above-mentioned application operation content), as well as the position coordinates of the triggered position. Based on this interface content and the position coordinates of the triggered position, a multi-modal input for the region prediction model can be generated. Then, based on this multi-modal input, the region prediction model can be called (input this multi-modal input into the region prediction model), where the region prediction model can refer to any artificial intelligence model with a classification function, such as: ResNet-18 + BERT fusion model. Through the region prediction model, region prediction processing can be performed on the triggered position to output the prediction probability that the triggered position is within the display area of the first media data; if this prediction probability is greater than the probability threshold (this probability threshold can be set based on actual business requirements, for example: setting the probability threshold to 90%), then it can be determined that the triggered position is within the display area of the first media data; and if the prediction probability is less than this probability threshold, it is determined that the triggered position is not within the display area of the first media data.

[0179] It should be noted that when the region prediction model determines that the prediction probability is greater than the probability threshold, the region prediction model can output prediction border parameters for the first media data. These prediction border parameters refer to the relevant parameters of the region where the first media data is predicted by the region prediction model in the application interface. The region border formed by these prediction border parameters is the region in the application interface where the first media data is displayed.

[0180] It should be noted that, in order to improve the accuracy of the results output by the region prediction model, this application can pre-train the region prediction model so that the model performance of this region prediction model can increasingly meet the requirements. When the region prediction model meets the model convergence condition, the region prediction model that meets the model convergence condition can be used as the pre-trained region prediction model, and this pre-trained region prediction model can be used to perform region prediction processing. The pre-training process of the region prediction model can at least include the following steps: First, training samples for model training can be obtained. The training samples can include sample multi-modal inputs (the sample multi-modal inputs can be generated based on the position coordinates of the sample trigger position and the interface content of the sample application interface. The sample trigger position refers to the trigger position of the sample, and the sample application interface refers to the application interface of the sample), and the sample labels corresponding to the sample multi-modal inputs (the sample labels can be used to represent whether the sample trigger position is actually in the display area of the media data in the sample application interface. If it is, the sample label can be the value 1; if not, the sample label can be the value 0); 2. Transmit the sample multi-modal inputs to the untrained region prediction model. Through this region prediction model, the sample multi-modal inputs can be calculated and analyzed, and the sample prediction probability of itself can be output (that is, the probability that the sample trigger position is in the display area of the media data in the sample application interface); 3. Calculate the error between the sample label and the sample prediction probability, and use this error to optimize the model parameters in the region prediction model to obtain an optimized region prediction model. In the actual pre-training process, steps 1-3 can be the process of one round of training. The model training can be set to w rounds of training (w can be set based on actual business requirements, for example, set to 10, 100, etc.). After the region prediction model completes w rounds of training, it can be considered that this region prediction model meets the model convergence condition and a pre-trained region prediction model is obtained. Of course, in some training scenarios, after any round of training is completed, it can be determined whether the model performance of this region prediction model meets the requirements. If the model performance already meets the requirements, then it can no longer be trained, and this region prediction model that meets the requirements can be used as the pre-trained region prediction model. In summary, the model convergence condition in this application can be flexibly set based on actual needs. For example, it can be that the number of training rounds reaches a preset threshold; it can also be that the model performance meets the requirements. This application does not limit it.

[0181] In the embodiments of the present application, the display method of the viewed media data is improved. It can be realized that during the process of running a simulator in a computer device and running an application through the simulator, the viewed media data can be displayed in a media display interface outside the operation interface of the simulator. This media display interface is independent of the simulator and is displayed on the device screen of the computer device like the simulator, which can bring a better user experience to the user. Moreover, the media display method provided by the present application is more flexible, and it can be dynamically selected based on actual business needs whether to associate and synchronize the display state of the media data in the media display interface with the display state of the media data displayed in the operation interface, which can further enhance the user experience.

[0182] Based on the fact that the present application can support the display of media data independently of the operation interface of the simulator, since the media display interface no longer needs to be limited by the display size of the operation interface, the display area available for the media display interface is larger. Based on this, the present application can also support the synchronous display of multiple media display interfaces. In this way, multiple media display interfaces can synchronously display different media data, and the user can synchronously view multiple different media data, which can make the use of the simulator more convenient and further enhance the user experience. For ease of understanding, the following will be described with reference to the accompanying drawings. Please also refer to Figure 9 , Figure 9 is a schematic flowchart of a process for displaying the viewed second media data provided by an embodiment of the present application. This process can correspond to the process after the first media data is displayed in the first media display interface in the above Figure 3 shown embodiment. As Figure 9 shown, this process can at least include the following steps S901 - step S902:

[0183] Step S901, in response to a viewing operation on the second media data, display a second media display interface; the second media display interface is independent of the simulator and is synchronously displayed with the operation interface of the simulator.

[0184] Specifically, based on the above description, it can be known that in the operation interface of the simulator in the present application, any application can be started to run. Of course, during the running of a certain application, another application can also be started to run. It can be seen that the operation interface of the simulator can include the application interface of the first application, and this application interface will display the application content of the first application, and can also include the application interface of the second application, and this application interface will display the application content of the second application. The second media data in the present application can refer to a media data other than the first media data displayed in the application interface of the first application, or can refer to the media data displayed in the application interface of the first application.

[0185] During the process of displaying the first media data on the first media display interface, the user can continue to perform corresponding emulator operations on the operation interface. For example, the user can perform a viewing operation on the second media data displayed on the operation interface. After the user performs a viewing operation on the second media data displayed on the operation interface, the computer device can respond to this operation and display a media display interface (referred to as the second media display interface in this application) for displaying the second media data on the device screen of the computer device. Similar to the first media display interface, this second media display interface is also independent of the emulator and will be synchronously displayed with the operation interface of the emulator.

[0186] The second media display interface and the first media display interface in this application can be different interfaces. For example, this application can set the creation rule of the media display interface as: one media data corresponds to one media display interface. As long as any two media data are different, two media display interfaces need to be created to display the two media data respectively. In this case, the second media display interface will be independent of the first media display interface, and moreover, the second media display interface and the first media display interface are synchronously displayed.

[0187] The second media display interface and the first media display interface in this application can be the same interface. For example, this application can set the creation rule of the media display interface as: one application corresponds to one media display interface. Even if any two media data are different, but belong to the same application, only one media display interface needs to be created to display each media data in this application. In this case, the second media display interface needs to be determined based on the second application to which the second media data belongs. When the second application is the same as the first application, then this second media display interface is the same as the first media display interface, and there is no need to create a second media display interface again. The first media display interface can be determined as the second media display interface; when the second application is different from the first application, then a second media display interface needs to be re-created and displayed. This second media display interface is independent of the first media display interface and is synchronously displayed with the first media display interface.

[0188] Step S902, display the second media data in the second media display interface.

[0189] In this application, the computer device can display the second media data in the second media display interface. It should be noted that when the second media display interface and the first media display interface are the same interface, the first media data has already been displayed in the second media display interface (i.e., the first media display interface). At this time, the display method for the computer device to display the second media data in the second media display interface can include any of the following: ① Switch the first media data displayed in the first media display interface to display the second media data; ② During the process of displaying the first media data in the first media display interface, synchronously display the second media data. Among them, for the display method ②, when the first media data is displayed in the first media display interface, this first media data may be full-screen displayed in the first media display interface (that is, the entire interface in the first media display interface displays the relevant media content of the first media data). In this case, if it is necessary to synchronously display the second media data in the first media display interface, the display range of the first media data can be first reduced to leave a free area to display the second media data.

[0190] For ease of understanding the scenario of displaying the second media data, please also refer to Figure 10 , Figure 10 which is a schematic diagram of a scenario for displaying the second media data provided by an embodiment of this application. As Figure 10 shown, in combination with the embodiment shown in Figure 4 , for the picture 10A3 that the user object a has performed a viewing operation on, it can be understood as the first media data, and the media display interface 4001 can be understood as the first media display interface. Here, it is assumed that the operation interface 1001 of the emulator also contains a video 10A5. During the process of the media display interface 4001 displaying the picture 10A3, the user object a has performed a viewing operation on this video 10A5. Then, in this application, this video 10A5 can be understood as the second media data, and the computer device running the emulator will respond to this operation and display the media display interface (i.e., the second media display interface) for displaying this video 10A5. In this scenario, it is assumed that the creation rule for the media display interface in this application is: create a media display interface for an application to display the media data belonging to that application. Since the video 10A5 is also a media data belonging to the application A, then, although the video 10A5 and the picture 10A3 are not the same media data, but because they both belong to the application A, the media display interface for displaying this video 10A5 is still the media display interface 4001, and the computer device can synchronously display this video 10A5 in the media display interface 4001. For example, as Figure 10As shown, before the display of video 10A5, picture 10A3 is full-screen displayed in the media display interface 4001 (occupying the entire interface of the media display interface 4001). When the video 10A5 is being displayed, the computer device will reduce the display range of the picture 10A3 to leave enough area to display the video 10A5.

[0191] Regardless of the creation rule of the media display interface, in order to reduce or even avoid the repeated creation of the media display interface, this application can use a hash algorithm to generate an identifier for uniquely representing each created media display interface. In this way, when it is necessary to create a media display interface, it can be determined whether it has been created based on the already created media display interfaces, and if it is determined that this media display interface has not been created, then create and display this media display interface. Taking the creation and display of the first media display interface as an example: when the creation rule of the media display interface is "create a media display interface for a media data to display the media data belonging to it", the specific implementation process for creating and displaying the first media display interface can at least include: First, a specified hash function can be used to perform a hash calculation on the first media data, and thus the hash value of the first media data can be obtained; among them, the hash function is a function that maps data of any length to a fixed-length output (hash value). Through the hash algorithm, a data can be converted into another fixed-length data, and the data calculated through the hash algorithm can be called the hash value. Hash algorithms such as the SHA hash algorithm of the SHA series, the BLAKE hash algorithm of the BLAKE series, etc. This application can specify a unified hash function to perform hash calculations on each media data to achieve unity.

[0192] After calculating the hash value of the first media data, it is possible to traverse M (where M is a positive integer) active hash values based on the hash value of the first media data. Each active hash value refers to the hash value corresponding to an active media display interface. An active media display interface is a media display interface on which media data has been displayed. The hash value corresponding to an active media display interface refers to the hash value of the media data displayed on that interface, and this hash value can be used as the identifier of the active media display interface. If the hash value of the first media data is included in the M active hash values, it can be determined that the first media data has been viewed before, a corresponding media display interface has been created for the first media data, and the media display interface is also displayed on the device screen of the computer device. At this time, there is no need to repeat the creation of an interface for the first media data, and the active media display interface corresponding to the hash value of the first media data can be set to the top display (that is, it can be overlapped or completely cover any interface on the device screen for display). If the hash value of the first media data is not included in the M active hash values, it can be characterized that the first media display interface for the first media data has not been created currently, and it is necessary to create the corresponding first media display interface and display the first media display interface.

[0193] Similarly, when the creation rule of the media display interface is "create a media display interface for an application to display the media data belonging to the application", the specific implementation process of creating and displaying the first media display interface can at least include: First, a specified hash function can be used to perform a hash calculation on the first application, and thus the hash value of the first application can be obtained; after calculating the hash value of the first application, it is possible to traverse K (where K is a positive integer) active hash values based on the hash value of the first application. Each active hash value refers to the hash value corresponding to an active media display interface. An active media display interface is a media display interface on which media data in a certain application has been displayed. The hash value corresponding to an active media display interface refers to the hash value of the application responsible for display on that interface, and this hash value can be used as the identifier of the active media display interface. If the hash value of the first application is included in the K active hash values, it can be determined that some media data in the first application has been viewed before, a corresponding media display interface has been created for the first application, and the media display interface is also displayed on the device screen of the computer device. At this time, there is no need to repeat the creation of an interface for the first application, and the active media display interface corresponding to the hash value of the first application can be set to the top display (that is, it can be overlapped or completely cover any interface on the device screen for display). If the hash value of the first application is not included in the K active hash values, it can be characterized that the first media display interface for the first application has not been created currently, and it is necessary to create the corresponding first media display interface and display the first media display interface.

[0194] It should be noted that after any media display interface in this application is displayed, the user can perform an operation to close the display on it to close the display of this media display interface. When the media display interface is closed, the media data displayed in this media display interface will be correspondingly closed.

[0195] In the embodiments of this application, the display method of the media data to be viewed is improved. It can be realized that during the process of running an emulator in a computer device and running an application through the emulator, the media data to be viewed can be displayed in a media display interface outside the operation interface of the emulator. This media display interface is independent of the emulator and is displayed on the device screen of the computer device just like the emulator, which can bring a better user experience to the user. Moreover, the media display method provided in this application is more flexible, and different media data can be synchronously displayed in different media display interfaces, which means that the user can synchronously perform corresponding operations on different media data, which can well improve the user experience.

[0196] Further, to more clearly understand the logic of the media display solution based on the emulator (the display solution for the media data to be viewed) provided in this application, the logic of this solution will be briefly described below in conjunction with the logic architecture schematic diagram shown in Figure 11. Please also refer to Figure 11 , Figure 11 is a schematic diagram of the logic architecture of a display solution provided by an embodiment of this application. As Figure 11 shown, at least a trigger component, a content extraction component, an interface management component, and a rendering component can be included in this logic architecture. The functions implemented by each component will be briefly introduced below:

[0197] Trigger component: Based on the content described above, in this application, after the application interface of the application is displayed in the operation interface of the emulator, the application content of this application will be displayed in this application interface, and different media data will be included in this application content. The user can select any media data to view. Among them, a media viewing control for the media data can be displayed in the application interface. In this way, the user can view this media data by triggering the media viewing control. For the media viewing control, it can be displayed after the user performs a preset specified operation. For example, through the above Figure 8As can be seen from the illustrated embodiments, the specified operation is a triggering operation performed on the display area of the media data to be viewed in the application interface. It can be seen that if the media viewing control needs to be displayed after an object performs a preset specified operation, and this specified operation is a triggering operation performed on the display area of the media data, in this case, it is necessary to first determine whether the triggered position is within the display area of the media data. If it is within the display area of this media data in the application interface, the computer device can display the media viewing control for this media data. The triggering component can be used to detect the triggered position to determine whether it is within the display area of the media data in the application interface. Specifically, the triggering component will at least perform the following steps: 1. Obtain the position coordinates of the triggered position (this position coordinate refers to the relative coordinate in the application interface); 2. Obtain the interface content included in the application interface (that is, the application content of the application program displayed in this application interface); 3. Based on the position coordinates of the triggered position and the interface content, generate a multimodal input, and transmit this multimodal input to the region prediction model. This region prediction model will perform computational analysis on this multimodal input to obtain the prediction probability that the triggered position is within the display area of the media data in the application interface. When the prediction probability is greater than the probability threshold, the region prediction model will synchronously output the predicted border parameters of the media data in the application interface predicted by itself. The region border formed by these predicted border parameters is the display area where the media data is located in the application interface.

[0198] Content extraction component: The content extraction component can, based on the predicted border parameters transmitted by the triggering component, determine the region border formed by these predicted border parameters in the application interface, and extract the media data included in this region border. These media data will be used as the data to be rendered, and the data to be rendered can be stored in the buffer for subsequent rendering by the rendering component.

[0199] Interface management component: The interface management component can be used to manage the media display interface of the media data, mainly for detecting whether a certain media display interface is re-created, and, when it is determined that it is not re-created, creating a corresponding media display interface for a certain media data. In addition, the interface management component can also be used to destroy the created media display interface.

[0200] Rendering component: The rendering component is responsible for rendering the data to be rendered in the buffer to obtain the rendered data, and transmitting the rendered data to the corresponding media display interface for display.

[0201] Further, please refer to Figure 12 , Figure 12This is a schematic diagram of the structure of a display processing device provided in an embodiment of the present application. The display processing device may be a computer program running in a computer device, for example, the display processing device is an application software; the display processing device may be used to execute Figure 3 As shown in the method. Figure 12 As shown, the display processing device 1 may include: a first display module 11 , a second display module 12 and a media display module 13 .

[0202] A first display module 11 is used to display an operation interface of the simulator, where the operation interface of the simulator includes first media data;

[0203] The second display module 12 is configured to display a first media display interface in response to a viewing operation on the first media data; the first media display interface is independent of the simulator;

[0204] The media display module 13 is configured to display first media data in a first media display interface.

[0205] The specific implementation of the first display module 11, the second display module 12 and the media display module 13 can be found in the above Figure 3 The description of steps S301 to S303 in the corresponding embodiment will not be repeated here.

[0206] In one embodiment, the first media data is media data displayed in an application interface of a first application. The first application runs in a simulator, and the application interface of the first application is displayed in an operation interface of the simulator. The application interface of the first application also displays application operation content of the first application.

[0207] After the media display module 13 displays the first media data in the first media display interface, the media display module is further specifically configured to:

[0208] During the process of the first media data being displayed on the first media display interface, in response to the triggering operation on the application operation content of the first application, the trigger result content corresponding to the application operation content is displayed in the application interface of the first application. The trigger result content corresponding to the application operation content refers to the visual content obtained after the application operation content is triggered.

[0209] In one embodiment, after the media display module 13 displays the first media data in the first media display interface, the media display module 13 is further configured to perform the following steps:

[0210] In response to an interface adjustment operation on the first media display interface, the first media display interface is adjusted and displayed as a media adjustment interface indicated by the interface adjustment operation.

[0211] In one embodiment, after the media display module 13 displays the first media data in the first media display interface, the media display module 13 is further specifically configured to:

[0212] In response to a display adjustment operation on the first media data in the first media display interface, switch the display state of the first media data in the first media display interface to the target display state indicated by the display adjustment operation; and,

[0213] Synchronously switch the display state of the first media data in the operation interface of the simulator to the target display state indicated by the display adjustment operation.

[0214] In one embodiment, the display position relationship between the first media display interface and the operation interface of the simulator includes any one of the following:

[0215] The first media display interface and the operation interface of the simulator are in an overlapping display relationship;

[0216] The first media display interface and the operation interface of the simulator are in a partitioned display relationship.

[0217] In one embodiment, after the media display module 13 displays the first media data in the first media display interface, the media display module 13 is further specifically configured to:

[0218] In response to a drag operation on the first media display interface, switch the display position of the first media display interface to the display position indicated by the drag operation.

[0219] In one embodiment, the operation interface of the simulator includes a media viewing control;

[0220] The viewing operation on the first media data includes any one of the following:

[0221] A triggering operation on the first media data;

[0222] An operation of dragging the first media data to a specified area;

[0223] A triggering operation on the media viewing control.

[0224] In one embodiment, the operation interface of the simulator further includes second media data;

[0225] After the media display module 13 displays the first media data in the first media display interface, the media display module 13 is further specifically configured to:

[0226] In response to a viewing operation on the second media data, display a second media display interface; the second media display interface is independent of the simulator and is synchronously displayed with the operation interface of the simulator;

[0227] Display second media data in the second media display interface.

[0228] In one embodiment, the first media data is different from the second media data; the second media display interface is independent of the first media display interface and is synchronously displayed with the first media display interface.

[0229] In one embodiment, the second media display interface is determined based on a second application to which the second media data belongs;

[0230] When the second application to which the second media data belongs is the same as the first application, the second media display interface is the same as the first media display interface;

[0231] When the second application to which the second media data belongs is different from the first application, the second media display interface is independent of the first media display interface and is synchronously displayed with the first media display interface.

[0232] In one embodiment, the second application to which the second media data belongs is the same as the first application;

[0233] The display manner of displaying the second media data in the second media display interface includes any one of the following:

[0234] Switch the first media data displayed in the first media display interface to display the second media data;

[0235] Synchronously display the second media data during the process of displaying the first media data in the first media display interface.

[0236] In one embodiment, the simulator runs in a computer device, and N media players are deployed in the computer device, where N is a positive integer, and one media player is used to play media data of one media type;

[0237] The specific implementation manner of the media display module 13 for displaying the first media data in the first media display interface includes:

[0238] Play the first media data in the first media display interface through the media player corresponding to the target media type; the target media type refers to the media type to which the first media data belongs.

[0239] In one embodiment, the specific implementation manner of the media display module 13 for displaying the first media data in the first media display interface includes:

[0240] Obtain the historical playback duration of the first media data when responding to the viewing operation of the first media data;

[0241] Determine the playback start time point in the first media data based on the historical playback duration of the first media data;

[0242] Use the playback start time point as the playback start position and play and display the first media data in the first media display interface.

[0243] In one embodiment, the first media data is media data displayed in the application interface of the first application. The first application runs in a simulator, and the application interface of the first application is displayed in the operation interface of the simulator;

[0244] The display process of the media viewing control in the operation interface of the simulator includes:

[0245] In response to a trigger operation on the application interface of the first application, obtain the position coordinates of the triggered position;

[0246] If the triggered position is within the display area of the first media data, display the media viewing control in the display area of the first media data.

[0247] In one embodiment, after obtaining the position coordinates of the triggered position, the second display module 12 is further specifically configured to:

[0248] Obtain the interface content included in the application interface of the first application and the position coordinates of the triggered position;

[0249] Based on the interface content included in the application interface of the first application and the position coordinates of the triggered position, call the region prediction model;

[0250] Perform region prediction processing on the triggered position through the region prediction model and output the prediction probability that the triggered position is within the display area of the first media data;

[0251] If the prediction probability is greater than the probability threshold, determine that the triggered position is within the display area of the first media data;

[0252] If the prediction probability is less than the probability threshold, determine that the triggered position is not within the display area of the first media data.

[0253] In one embodiment, the specific implementation manner of the second display module 12 for displaying the first media display interface includes:

[0254] Perform hash calculation on the first media data using a specified hash function to obtain the hash value of the first media data;

[0255] Traversing M active hash values based on the hash value of the first media data; M is a positive integer, and each active hash value refers to a hash value corresponding to an active media display interface, where an active media display interface refers to a media display interface that has displayed media data;

[0256] If the M active hash values include the hash value of the first media data, the active media display interface corresponding to the hash value of the first media data is displayed on top;

[0257] If the M active hash values do not include the hash value of the first media data, a first media presentation interface is created and displayed.

[0258] The simulator-based media display solution provided by the present application improves the display method of the viewed media data, and can realize that in the process of running the simulator in the computer device and running the application through the simulator, the viewed media data can be displayed in a media display interface that is independent of the simulator's operating interface. This media display interface is independent of the simulator and is displayed on the device screen of the computer device like the simulator. This can bring a better user experience to the user. For example, the user can perform corresponding simulator operations in the simulator's operating interface while viewing the displayed media data, so that the simulator operation and the display of the media data are synchronized. The solution provided by the embodiment of the present application can be applicable to scenarios where it is necessary to view the media data within the application program without being restricted by the simulator's operating interface, and has a more flexible media browsing method.

[0259] Further, see Figure 13 , Figure 13 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. Figure 13 As shown, the above-mentioned computer device 8000 may include: a processor 8001, a network interface 8004 and a memory 8005. In addition, the above-mentioned computer device 8000 also includes: a user interface 8003, and at least one communication bus 8002. The communication bus 8002 is used to realize the connection and communication between these components. The user interface 8003 may include a display screen (Display), a keyboard (Keyboard), and the user interface 8003 may optionally include a standard wired interface and a wireless interface. The network interface 8004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 8005 may be a high-speed RAM memory, or a non-volatile memory (non-volatile memory), such as at least one disk memory. The memory 8005 may optionally also be at least one storage device located away from the aforementioned processor 8001. As Figure 13As shown, in the memory 8005 which is a computer-readable storage medium, an operating system, a network communication module, a user interface module, and a device control application program may be included.

[0260] In Figure 13 In the computer device 8000 shown, the network interface 8004 can provide network communication functions; the user interface 8003 is mainly used to provide an input interface for users; and the processor 8001 can be used to call the device control application program stored in the memory 8005 to achieve:

[0261] Display the operation interface of the emulator, and the operation interface of the emulator includes first media data;

[0262] In response to a viewing operation on the first media data, display a first media display interface; the first media display interface is independent of the emulator;

[0263] Display the first media data in the first media display interface.

[0264] It should be understood that the computer device 8000 described in the embodiments of the present application can execute the description of the display processing method in the corresponding embodiments mentioned above, and can also execute the description of the display processing device 1 in the corresponding embodiments mentioned above, which will not be elaborated here. In addition, the description of the beneficial effects of adopting the same method will not be elaborated either. Figure 3 to Figure 9 In Figure 12 In addition, it should be pointed out here that: the embodiments of the present application also provide a computer-readable storage medium, and the above computer-readable storage medium stores a computer program executed by the computer device 8000 for data processing mentioned above, and the above computer program includes program instructions. When the above processor executes the above program instructions, it can execute the description of the above display processing method in the corresponding embodiments mentioned above. Therefore, it will not be elaborated here. In addition, the description of the beneficial effects of adopting the same method will not be elaborated either. For the technical details not disclosed in the embodiments of the computer-readable storage medium involved in the present application, please refer to the description of the method embodiments of the present application.

[0265] Figure 3 to Figure 9 Figure 3 to Figure 9

[0266] ​The above computer-readable storage medium may be the display processing device provided in any of the foregoing embodiments or the internal storage unit of the above computer device, such as the hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the computer device. Further, the computer-readable storage medium may also include both the internal storage unit and the external storage device of the computer device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.

[0267] In one aspect of the present application, a computer program product is provided. The computer program product includes a computer program, and the computer program is stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device executes the method provided in one aspect of the embodiments of the present application.

[0268] In the description, claims and drawings of the embodiments of the present application, terms such as "first", "second", etc. are used to distinguish different objects, rather than to describe a specific order. In addition, the term "comprising" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, device, product or equipment that includes a series of steps or units is not limited to the listed steps or modules, but optionally further includes steps or modules not listed, or optionally further includes other step units inherent to these processes, methods, devices, products or equipment.

[0269] In the embodiments of the present application, the term "module" or "unit" refers to a computer program with a predetermined function or a part of a computer program, which works together with other related parts to achieve a predetermined goal, and can be fully or partially implemented by using software, hardware (such as a processing circuit or a memory), or a combination thereof. Similarly, one processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be a part of an overall module or unit that includes the function of the module or unit.

[0270] Those of ordinary skill in the art will appreciate that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the composition and steps of the examples have been generally described according to their functions in the above description. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0271] The methods and related devices provided by the embodiments of this application are described with reference to the method flowcharts and / or structural schematic diagrams provided by the embodiments of this application. Specifically, each process and / or block of the method flowchart and / or structural schematic diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for implementing the functions specified in Figure 1 one process or multiple processes and / or structural schematic Figure 1 one block or multiple blocks. These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device that implements the functions specified in Figure 1 one process or multiple processes and / or structural schematic Figure 1 one block or multiple blocks. These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in Figure 1 one process or multiple processes and / or structural schematic one block or multiple blocks.

[0272] The above-disclosed are only the preferred embodiments of this application. Of course, the scope of the rights of this application cannot be limited thereby. Therefore, equivalent changes made according to the claims of this application still fall within the scope covered by this application.

Claims

1. A display processing method, characterized in that, The method includes: Displaying an operation interface of the emulator, where the operation interface of the emulator contains first media data; In response to a viewing operation on the first media data, displaying a first media display interface; the first media display interface is independent of the emulator; Displaying the first media data in the first media display interface.

2. The method according to claim 1, characterized in that, The first media data is media data displayed in an application interface of a first application, the first application runs in the emulator, and the application interface of the first application is displayed in the operation interface of the emulator; The application operation content of the first application is also displayed in the application interface of the first application; After the first media data is displayed in the first media display interface, the method further includes: During the process of the first media data being displayed in the first media display interface, in response to a trigger operation on the application operation content of the first application, displaying trigger result content corresponding to the application operation content in the application interface of the first application, where the trigger result content corresponding to the application operation content refers to the visual content obtained after triggering the application operation content.

3. The method according to claim 1, wherein After the first media data is displayed in the first media display interface, the method further includes: In response to an interface adjustment operation on the first media display interface, adjusting the display of the first media display interface to a media adjustment interface indicated by the interface adjustment operation.

4. The method according to claim 1, characterized in that After the first media data is displayed in the first media display interface, the method further includes: In response to a display adjustment operation on the first media data in the first media display interface, switching the display state of the first media data in the first media display interface to a target display state indicated by the display adjustment operation; and, Synchronously switching the display state of the first media data in the operation interface of the emulator to the target display state indicated by the display adjustment operation.

5. The method according to claim 1, wherein The display position relationship between the first media display interface and the operation interface of the emulator includes any one of the following: The first media display interface and the operation interface of the emulator are in an overlapping display relationship; The first media display interface and the operation interface of the emulator are in a partitioned display relationship.

6. The method according to claim 1, characterized in that, After the first media data is displayed in the first media display interface, the method further includes: In response to a dragging operation on the first media display interface, switching the display position of the first media display interface to a display position indicated by the dragging operation.

7. The method according to claim 1, wherein The operation interface of the emulator contains a media viewing control; The viewing operation on the first media data includes any one of the following: A trigger operation on the first media data; An operation of dragging the first media data to a specified area; A trigger operation on the media viewing control.

8. The method according to claim 1, wherein The operation interface of the emulator further contains second media data; After the first media data is displayed in the first media display interface, the method further includes: In response to a viewing operation on the second media data, display a second media display interface; The second media display interface is independent of the emulator and is synchronously displayed with the operation interface of the emulator; Display the second media data in the second media display interface.

9. The method according to claim 8, characterized in that The first media data is different from the second media data; the second media display interface is independent of the first media display interface and is synchronously displayed with the first media display interface.

10. The method according to claim 8, characterized in that The second media display interface is determined based on a second application to which the second media data belongs; When the second application to which the second media data belongs is the same as the first application, the second media display interface is the same as the first media display interface; When the second application to which the second media data belongs is different from the first application, the second media display interface is independent of the first media display interface and is synchronously displayed with the first media display interface.

11. The method according to claim 10, wherein The second application to which the second media data belongs is the same as the first application; The display method of displaying the second media data in the second media display interface includes any one of the following: Switch the display of the first media data displayed in the first media display interface to the second media data; During the process of displaying the first media data in the first media display interface, synchronously display the second media data.

12. The method according to claim 1, wherein The emulator runs in a computer device, and N media players are deployed in the computer device, where N is a positive integer, and one media player is used to play media data of one media type; The displaying of the first media data in the first media display interface includes: Play the first media data in the first media display interface through the media player corresponding to the target media type; The target media type refers to the media type to which the first media data belongs.

13. The method according to claim 1, wherein The displaying of the first media data in the first media display interface includes: Obtain the historical playback duration of the first media data when responding to a viewing operation on the first media data; Based on the historical playback duration of the first media data, determine a playback start time point in the first media data; Use the playback start time point as the playback start position, and play and display the first media data in the first media display interface.

14. The method according to claim 7, wherein The first media data is media data displayed in the application interface of a first application, the first application runs in the emulator, and the application interface of the first application is displayed in the operation interface of the emulator; The display process of the media viewing control in the operation interface of the emulator includes: In response to a trigger operation on the application interface of the first application, obtain the position coordinates of the triggered position; If the triggered position is located in the display area of the first media data, display the media viewing control in the display area of the first media data.

15. The method according to claim 14, wherein After obtaining the position coordinates of the triggered position, the method further includes: Obtaining interface content contained in the application interface of the first application and the location coordinates of the triggered location; Invoking a region prediction model based on interface content included in an application interface of the first application and the location coordinates of the triggered location; performing region prediction processing on the triggered position using the region prediction model, and outputting a predicted probability that the triggered position is within the display region of the first media data; If the predicted probability is greater than a probability threshold, determining that the triggered position is in a display area of the first media data; If the predicted probability is less than the probability threshold, it is determined that the triggered position is not in the display area of the first media data.

16. The method according to claim 1, wherein The displaying of the first media display interface includes: Performing a hash calculation on the first media data using a specified hash function to obtain a hash value of the first media data; Traversing M active hash values based on the hash value of the first media data; M is a positive integer, and each active hash value refers to a hash value corresponding to an active media display interface, where the active media display interface refers to the media display interface that has displayed the media data; If the M active hash values include the hash value of the first media data, the active media display interface corresponding to the hash value of the first media data is displayed on top; If the M active hash values do not include the hash value of the first media data, a first media presentation interface is created and displayed.

17. A display processing device, characterized in that, The device comprises: A first display module is configured to display an operation interface of the simulator, wherein the operation interface of the simulator includes first media data; a second display module, configured to display a first media display interface in response to a viewing operation on the first media data; the first media display interface is independent of the simulator and is displayed synchronously with an operation interface of the simulator; The media display module is configured to display the first media data in the first media display interface.

18. A computer device, characterized in that, include: processor, memory, and network interface; The processor is connected to the memory and the network interface, wherein the network interface is used to provide a network communication function, the memory is used to store a computer program, and the processor is used to call the computer program so that the computer device executes the method according to any one of claims 1 to 16.

19. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor and executing the method according to any one of claims 1 to 16.

20. A computer program product, characterized in that, The computer program product comprises a computer program stored in a computer-readable storage medium. The computer program is suitable for being read and executed by a processor, so as to enable a computer device having the processor to perform the method according to any one of claims 1 to 16.

Images