Display method and device, electronic equipment and storage medium

Abstract

The invention relates to a display method and device, electronic equipment and a storage medium. The display method comprises the steps that when projection screen connection between a first device and a second device is successful, a preset projection screen interface is displayed on the first device, and the preset projection screen interface is generated by projection screen of a first virtual screen of the second device; a mirror image projection screen interface of the second device is displayed on an upper layer of the preset projection screen interface, the mirror image projection screen interface of the second device is generated by projection of a second virtual screen of the second device, and the second virtual screen is located on the upper layer of the first virtual screen. According to the method, a brand new screen projection form is provided for a user, the preset screen projection interface and the mirror image screen projection interface are mutually independent and coexist, diversification of the screen projection interfaces can be achieved, the user can give consideration to the preset screen projection interface and the mirror image screen projection interface and experience the two screen projection modes at the same time, the requirements of the user for the diversified screen projection modes can be met, and the user experience is improved. And the screen projection use scene can be enriched, so that the screen projection use experience of the user is improved.

Description

Technical Field

[0001] This disclosure relates to the field of screen projection technology, and in particular to a display method, device, electronic device and storage medium. Background Technology

[0002] With the increasing popularity of smart cars, users are demanding more diverse interconnectivity between terminal devices and smart cars. Screen mirroring is a common method of device interconnection. However, current technologies often result in a limited screen mirroring interface, preventing users from viewing other content on their terminal devices and thus failing to meet their needs for diverse mirroring options. Summary of the Invention

[0003] To overcome the problems existing in related technologies, this disclosure provides a display method, apparatus, electronic device, and storage medium.

[0004] According to a first aspect of the present disclosure, a display method is provided, the method comprising:

[0005] When the screen mirroring connection between the first device and the second device is successfully established, a preset screen mirroring interface is displayed on the first device. The preset screen mirroring interface is generated by the first virtual screen mirroring of the second device.

[0006] The mirrored screen projection interface of the second device is displayed on the upper layer of the preset screen projection interface. The mirrored screen projection interface of the second device is generated by the second virtual screen projection of the second device, and the second virtual screen is located on the upper layer of the first virtual screen.

[0007] In an exemplary embodiment, displaying the mirrored projection interface of the second device on the upper layer of the preset projection interface includes:

[0008] In response to a first control operation on the preset projection interface, a first control command associated with the first control operation is sent to the second device, wherein the first control operation is used to open the mirror projection interface of the second device;

[0009] The target window is displayed on the upper layer of the preset projection interface, and the mirrored projection interface is displayed in the target window. Both the target window and the second virtual screen are generated by the second device in response to the first control command.

[0010] In one exemplary embodiment, the method further includes:

[0011] In response to a second control operation on the mirrored screen projection interface, first location information is sent to the second device, wherein the first location information represents the triggering position of the second control operation in the target window;

[0012] An updated mirrored screen projection interface is displayed on the upper layer of the preset projection interface. The updated mirrored screen projection interface is generated by the updated second virtual screen projection, and the updated second virtual screen is determined by the second device based on the first location information.

[0013] In one exemplary embodiment, the method further includes:

[0014] When the target application is displayed in the mirrored screen projection interface, in response to the third control operation on the target window, a second control command associated with the third control operation is sent to the second device. The third control operation is used to switch the target application to be displayed by the preset screen projection interface.

[0015] Stop displaying the target window and display the target application in the preset projection interface; wherein, the preset projection interface for displaying the target application is generated by the updated first virtual screen projection, and the updated first virtual screen is determined by the second device in response to the second control command.

[0016] In one exemplary embodiment, the method further includes:

[0017] The adjusted mirrored projection interface is displayed on the upper layer of the preset projection interface;

[0018] The adjusted mirrored projection interface is generated by projection from the adjusted second virtual screen, which is obtained when the display orientation of the physical screen of the second device changes.

[0019] In one exemplary embodiment, the method further includes:

[0020] A screen mirroring connection is established with the first device through a preset application in the first device.

[0021] According to a second aspect of the present disclosure, a display method is provided, the method comprising:

[0022] When the second device and the first device successfully connect to the screen mirroring, the first virtual screen of the second device is projected onto the first device. The first virtual screen displays a preset screen mirroring interface so that the first device displays the preset screen mirroring interface.

[0023] The second virtual screen of the second device is projected onto the first device. The second virtual screen displays the mirrored projection interface of the second device. The second virtual screen is located on top of the first virtual screen, so that the first device displays the mirrored projection interface of the second device on the upper layer of the preset projection interface.

[0024] In an exemplary embodiment, projecting the second virtual screen of the second device onto the first device includes:

[0025] Receive a first control command sent by the first device, the first control command being used to instruct the second device to open the screen mirroring interface;

[0026] In response to the first control command, a target window is created on the upper layer of the preset projection interface of the first virtual screen, and a second virtual screen is created in the target window;

[0027] The first virtual screen and the second virtual screen of the target window will be projected onto the first device, so that the first device displays the target window on the upper layer of the preset projection interface, and displays the mirrored projection interface in the target window.

[0028] In an exemplary embodiment, creating a target window on top of a preset projection interface of the first virtual screen includes:

[0029] Based on the size of the physical screen of the second device, a first target size of the second virtual screen is determined, wherein the first target size is proportional to the size of the physical screen;

[0030] Based on the first target size, the preset size, and the display orientation of the physical screen, a second target size for the target window is determined;

[0031] Based on the second target size, the target window is created on top of the preset projection interface.

[0032] In an exemplary embodiment, determining the second target size of the target window based on the first target size, the preset size, and the display orientation of the physical screen includes:

[0033] If the physical screen is displayed in portrait orientation, the width value in the first target size is used as the width value in the second target size, and the sum of the length value in the first target size and the preset size is used as the height value in the second target size.

[0034] In an exemplary embodiment, determining the second target size of the target window based on the first target size, the preset size, and the display orientation of the physical screen includes:

[0035] If the physical screen is displayed in landscape orientation, the length value in the first target size is used as the width value in the second target size, and the sum of the width value in the first target size and the preset size is used as the height value in the second target size.

[0036] In one exemplary embodiment, the method further includes:

[0037] The system receives first location information sent by the first device, the first location information representing the trigger position of the second control operation in the target window, and the second control operation acting on the mirrored screen projection interface.

[0038] Based on the first location information, second location information is determined, wherein the second location information represents the trigger position of the second control operation on the physical screen of the second device;

[0039] Based on the second location information, update the second virtual screen to obtain the updated second virtual screen;

[0040] The updated second virtual screen is projected onto the first device, so that the first device displays the updated mirrored projection interface on the upper layer of the preset projection interface.

[0041] In an exemplary embodiment, determining the second location information based on the first location information includes:

[0042] Based on the first location information, a third location information is determined, wherein the third location information represents the triggering position of the second control operation in the second virtual screen;

[0043] Based on the third location information, the second location information is determined.

[0044] In one exemplary embodiment, the method further includes:

[0045] When the target application is displayed in the second virtual screen, a second control command sent by the first device is received. The second control command is used to instruct the second device to switch the target application to be displayed by the first virtual screen.

[0046] In response to the second control command, the target application is displayed in the first virtual screen to obtain an updated first virtual screen;

[0047] The updated first virtual screen is projected onto the first device, and the projection of the second virtual screen onto the first device is stopped, so that the first device stops displaying the target window and displays the target application in the preset projection interface.

[0048] In an exemplary embodiment, displaying the target application in the first virtual screen includes:

[0049] Based on the preset mapping relationship and the target application, the target display parameters associated with the target application are determined, wherein the preset mapping relationship represents the mapping relationship between the preset application and the preset display parameters;

[0050] Based on the target display parameters, the target application is displayed in the first virtual screen.

[0051] In one exemplary embodiment, the method further includes:

[0052] When the display orientation of the physical screen of the second device changes, the display orientation of the second virtual screen is adjusted to obtain an adjusted second virtual screen, wherein the display orientation of the adjusted second virtual screen is the same as that of the physical screen;

[0053] The adjusted second virtual screen is projected onto the first device, so that the first device displays the adjusted mirrored projection interface on the upper layer of the preset projection interface.

[0054] According to a third aspect of the present disclosure, a display device is provided, the device comprising:

[0055] The first display module is configured to display a preset projection interface on the first device when the first device and the second device successfully connect to the projection interface. The preset projection interface is generated by the projection of the first virtual screen of the second device.

[0056] The second display module is configured to display the mirrored projection interface of the second device on the upper layer of the preset projection interface. The mirrored projection interface of the second device is generated by projection from the second virtual screen of the second device, and the second virtual screen is located on the upper layer of the first virtual screen.

[0057] According to a fourth aspect of the present disclosure, a display device is provided, the device comprising:

[0058] The first screen mirroring module is configured to mirror the first virtual screen of the second device to the first device when the screen mirroring connection between the second device and the first device is successful. The first virtual screen displays a preset screen mirroring interface so that the first device displays the preset screen mirroring interface.

[0059] The second screen mirroring module is configured to mirror the second virtual screen of the second device onto the first device. The second virtual screen displays the mirrored screen mirroring interface of the second device. The second virtual screen is located on top of the first virtual screen, so that the first device displays the mirrored screen mirroring interface of the second device on the upper layer of the preset screen mirroring interface.

[0060] According to a fifth aspect of the present disclosure, an electronic device is provided, comprising:

[0061] processor;

[0062] Memory used to store processor-executable instructions;

[0063] The processor is configured to perform the method described in the first or second aspect of the embodiments of this disclosure.

[0064] According to a sixth aspect of the present disclosure, a non-transitory computer-readable storage medium is provided, wherein when instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the method described in the first or second aspect of the present disclosure.

[0065] The above-described method of this disclosure has the following beneficial effects: This method provides users with a brand-new screen casting mode. The preset screen casting interface and the mirrored screen casting interface are independent of each other and coexist, which can realize the diversification of screen casting interfaces. Users can take both the preset screen casting interface and the mirrored screen casting interface and experience two screen casting modes at the same time. This can meet users' needs for diversified screen casting modes and enrich the screen casting usage scenarios, thereby improving the user's screen casting experience.

[0066] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0067] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0068] Figure 1 This is a flowchart illustrating a display method according to an exemplary embodiment;

[0069] Figure 2 This is a schematic diagram illustrating a preset screen projection interface according to an exemplary embodiment;

[0070] Figure 3 This is a schematic diagram of screen projection of a first virtual screen according to an exemplary embodiment;

[0071] Figure 4 This is a schematic diagram illustrating a screen mirroring interface according to an exemplary embodiment;

[0072] Figure 5 This is a schematic diagram of screen projection of a second virtual screen according to an exemplary embodiment;

[0073] Figure 6 This is a flowchart illustrating a display method according to an exemplary embodiment;

[0074] Figure 7 This is a schematic diagram illustrating the display of a mirrored projection interface on a preset projection interface according to an exemplary embodiment;

[0075] Figure 8 This is a schematic diagram illustrating an updated screen mirroring interface according to an exemplary embodiment;

[0076] Figure 9 This is a schematic diagram illustrating the display of a target application in a preset projection interface according to an exemplary embodiment;

[0077] Figure 10 This is a schematic diagram of a mirrored projection interface in a preset projection interface according to an exemplary embodiment;

[0078] Figure 11 This is a flowchart illustrating a display method according to an exemplary embodiment;

[0079] Figure 12 This is a flowchart illustrating a display method according to an exemplary embodiment;

[0080] Figure 13 This is a schematic diagram illustrating the target window size, the second virtual screen size, and the physical screen size of the second device according to an exemplary embodiment;

[0081] Figure 14 This is a schematic diagram illustrating the adjustment of the display orientation of a second virtual screen according to an exemplary embodiment;

[0082] Figure 15 This is an interactive flowchart illustrating a display method according to an exemplary embodiment;

[0083] Figure 16 This is an interactive flowchart illustrating a display method according to an exemplary embodiment;

[0084] Figure 17 This is a block diagram illustrating a display device according to an exemplary embodiment;

[0085] Figure 18 This is a block diagram illustrating a display device according to an exemplary embodiment;

[0086] Figure 19 This is a block diagram illustrating an electronic device according to an exemplary embodiment. Detailed Implementation

[0087] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0088] In the exemplary embodiments of this disclosure, to overcome the technical problem that related technologies cannot meet users' needs for diverse screen projection modes, a display method is provided, comprising: when a first device and a second device successfully connect for screen projection, displaying a preset screen projection interface on the first device, the preset screen projection interface being generated by projection from a first virtual screen of the second device; and displaying a mirrored screen projection interface of the second device on a layer above the preset screen projection interface, the mirrored screen projection interface of the second device being generated by projection from a second virtual screen of the second device, the second virtual screen being located on top of the first virtual screen. This method provides users with a completely new screen projection mode, where the preset screen projection interface and the mirrored screen projection interface are independent of each other and coexist, enabling the diversification of screen projection interfaces, meeting users' needs for diverse screen projection modes, and enriching screen projection usage scenarios, thereby improving the user's screen projection experience.

[0089] In an exemplary embodiment of this disclosure, a display method is provided, applied to a first device. Figure 1 This is a flowchart illustrating a display method according to an exemplary embodiment, such as... Figure 1 As shown, it includes the following steps:

[0090] Step S101: When the screen mirroring connection between the first device and the second device is successfully established, a preset screen mirroring interface is displayed on the first device. The preset screen mirroring interface is generated by the first virtual screen of the second device.

[0091] In some implementations, the first device can be an electronic device such as a smart vehicle infotainment system (e.g., a central control screen, instrument panel screen, head-up virtual screen, rear-seat screen, rear-seat armrest screen, etc.), a tablet, a personal computer, a smart TV, or a smart screen, while the second device can be an electronic device such as a mobile phone, tablet, personal computer, or smart wearable device. Optionally, the screen size of the first device is larger than the screen size of the second device.

[0092] In some implementations, a screen mirroring connection is established with the first device through a preset application in the first device. When a user opens the preset application, they can send a screen mirroring connection request to the second device through the preset application. Upon receiving the screen mirroring connection request, the second device begins to establish a screen mirroring connection between the first and second devices. The screen mirroring connection between the first and second devices can be any device interconnection method, such as Universal Serial Bus (USB), Wireless Fidelity (WiFi), local area network (LAN), or near-field communication (NFC).

[0093] When the screen mirroring connection between the first device and the second device is successfully established, a preset screen mirroring interface is displayed on the display screen of the first device. This preset screen mirroring interface is an interactive display page suitable for the display screen of the second device. Any application projected from the second device to the first device can be displayed in the preset screen mirroring interface. This embodiment of the disclosure does not limit the content displayed in the preset screen mirroring interface; for example, the preset screen mirroring interface may display multiple service cards to help users quickly open some frequently used applications. In one example, Figure 2 This is a schematic diagram illustrating a preset screen mirroring interface according to an exemplary embodiment, such as... Figure 2 As shown, the preset screen mirroring interface displays multiple service cards, each corresponding to a different function.

[0094] The preset projection interface is generated by projecting the first virtual screen from the second device. The size of the first virtual screen is the same as or proportional to the size of the preset projection interface. Proportional means that the aspect ratios of the two are the same or their diameters are in a certain proportion. In one example, Figure 3 This is a schematic diagram illustrating the projection of a first virtual screen according to an exemplary embodiment, such as... Figure 3 As shown, a first virtual screen is created in the second device. The size of the first virtual screen is proportional to the display screen size of the first device, for example, their aspect ratios are the same. A preset projection interface is drawn in the first virtual screen. Then, an encoder bound to the first virtual screen is created. This encoder converts the image data of the first virtual screen into video stream data through the projection connection link and sends it to the first device. The first device decodes the video stream data and displays the corresponding image in full screen on its display screen, thus displaying the preset projection interface on the first device's display screen. Optionally, the first virtual screen can also be called an independent virtual screen.

[0095] Step S102: Display the mirrored screen projection interface of the second device on the upper layer of the preset screen projection interface. The mirrored screen projection interface of the second device is generated by the second virtual screen projection of the second device. The second virtual screen is located on the upper layer of the first virtual screen.

[0096] When a preset projection interface is displayed on the first device, a mirrored projection interface of the second device is displayed on top of the preset projection interface. This mirrored projection interface represents the live feed displayed on the physical screen of the second device. In some implementations, the mirrored projection interface of the second device is displayed as a floating window within the preset projection interface. In one example, Figure 4 This is a schematic diagram illustrating a screen mirroring interface according to an exemplary embodiment, such as... Figure 4 As shown, if the physical screen of the second device displays the desktop page, the mirrored screen mirroring interface of the second device is displayed on the upper layer of the preset screen mirroring interface, and the desktop page is also displayed in the mirrored screen mirroring interface. It should be noted that the preset screen mirroring interface and the mirrored screen mirroring interface are independent of each other, and the user can operate either one independently without affecting the other.

[0097] The mirrored screen projection interface is generated by projecting a second virtual screen from a second device. The size of the second virtual screen is the same as the size of the physical screen of the second device. Since the second virtual screen is located above the first virtual screen, the mirrored screen projection interface is displayed on top of the preset projection interface. In one example, Figure 5 This is a schematic diagram illustrating the projection of a second virtual screen according to an exemplary embodiment, such as... Figure 5 As shown, a second virtual screen is created in the second device. The size of the second virtual screen is proportional to the size of the physical screen of the second device, for example, their aspect ratios are the same. The real-time image displayed on the physical screen of the second device (i.e., the mirrored projection interface of the physical screen of the second device) is drawn on the second virtual screen in real time. Then, an encoder bound to the second virtual screen is created. This encoder converts the image data of the second virtual screen into video stream data through the projection connection link and sends it to the first device. The first device decodes the video stream data and displays the corresponding real-time image on the display screen of the first device, thus displaying the mirrored projection interface of the second device on the display screen of the first device. Optionally, the second virtual screen can also be called a mirrored virtual screen.

[0098] In some implementations, the mirrored screen projection interface on the first device can be dragged freely within the display area of ​​a preset projection interface. When the first device responds to the user's dragging operation, it sends drag trajectory information to the second device. The second device then moves the position of the second virtual screen according to the drag trajectory information, thereby moving the mirrored screen projection interface on the first device.

[0099] In the exemplary embodiments of this disclosure, when the first device and the second device successfully connect for screen mirroring, a preset screen mirroring interface is generated by the first virtual screen of the second device and displayed on the first device. A mirrored screen mirroring interface of the second device is then generated by the second virtual screen of the second device and displayed on top of the preset screen mirroring interface. This method provides users with a novel screen mirroring format. The preset screen mirroring interface and the mirrored screen mirroring interface are independent yet coexist, enabling diverse screen mirroring interfaces. Users can enjoy both the preset and mirrored screen mirroring interfaces simultaneously, experiencing both screen mirroring modes. This satisfies users' needs for diverse screen mirroring modes and enriches screen mirroring usage scenarios, thereby improving the user's screen mirroring experience.

[0100] In an exemplary embodiment of this disclosure, a display method is provided, applied to a first device. Figure 6 This is a flowchart illustrating a display method according to an exemplary embodiment, such as... Figure 6 As shown, it includes the following steps:

[0101] Step S601: When the screen mirroring connection between the first device and the second device is successfully established, a preset screen mirroring interface is displayed on the first device. The preset screen mirroring interface is generated by the first virtual screen of the second device.

[0102] For a detailed implementation of step S601, please refer to step S101, which will not be repeated here.

[0103] In step S602, in response to the first control operation on the preset projection interface, a first control command associated with the first control operation is sent to the second device. The first control operation is used to open the mirror projection interface of the second device.

[0104] The first control operation is a preset operation performed by the user to open the mirroring interface of the second device. The first control operation can be a touch operation, such as a preset gesture operation performed by the user on a preset display area in the preset mirroring interface, including double-click operation, long press operation, etc., or the first control operation can be a click operation performed by the user on a preset operation control in the preset mirroring interface; the first control operation can also be a voice command, such as the voice command "open mirroring of the second device" issued by the user.

[0105] Since the preset projection interface is generated by the projection of the first virtual screen of the second device, the user's control operation on the preset projection interface requires a response from the first virtual screen of the second device. Therefore, when the first device detects a first control operation on the preset projection interface, it sends a first control command associated with the first control operation to the second device to inform the second device that the first control operation has been detected, so that the second device can respond to the first control operation. In some embodiments, when the first control operation is a voice command, the first control command is that voice command; when the first control operation is a touch operation, the first control command is the position information of the first control operation in the preset projection interface, such as the coordinates of the preset operation control corresponding to the first control operation in the preset projection interface, so that the second device can respond to the first control operation based on the position information of the first control operation.

[0106] Step S603: Display the target window on the upper layer of the preset projection interface, and display the mirrored projection interface in the target window. The mirrored projection interface is generated by the second virtual screen of the second device. Both the target window and the second virtual screen are generated by the second device in response to the first control command. The second virtual screen is located on the upper layer of the first virtual screen.

[0107] When the second device receives the first control command, it responds to the first control command, that is, responds to the first control operation, and creates a target window in the upper layer of the first virtual screen. For example, it creates a floating view component above the main view of the first virtual screen, and creates a second virtual screen in the target window. The second virtual screen is set to mirror projection so that the mirror projection interface of the second device is displayed in the second virtual screen. For example, the function flag of the second virtual screen is set to VIRTUAL_DISPLAY_FLAG_AUTO_MIRROR. Then, the first virtual screen and the second virtual screen that will display the target window are projected onto the first device. The first device then displays the target window in the upper layer of the preset projection interface and displays the mirror projection interface in the target window.

[0108] In one example, when the first device is a smart car's central control screen and the second device is a mobile phone, Figure 7 This is a schematic diagram illustrating the display of a mirrored projection interface on a preset projection interface according to an exemplary embodiment, such as... Figure 7 As shown, after the first device and the second device successfully connect to the screen mirroring interface, the first device displays a preset screen mirroring interface. A service card in the preset screen mirroring interface displays a preset operation control "phone mirroring". When the user clicks the preset operation control, the target window is displayed on the upper layer of the preset screen mirroring interface, and then the mirrored screen mirroring interface is displayed in the target window. For example, if the real-time screen of the phone's physical screen is the desktop page, then the phone's desktop page is displayed in the target window of the first device.

[0109] In step S604, in response to the second control operation on the mirrored screen projection interface, first location information is sent to the second device, the first location information representing the trigger position of the second control operation in the target window.

[0110] The second control operation is a random operation performed by the user. This second control operation is a touch operation, such as a gesture applied by the user to any location within the mirrored screen interface, including single-click, double-click, long-press, and swipe operations. Since the mirrored screen interface is generated by the second virtual screen of the second device, the user's control operation on the mirrored screen interface requires a response from the second virtual screen of the second device. The second virtual screen is located in the target window above the first virtual screen. Therefore, when the first device detects a second control operation on the mirrored screen interface, the target window will also detect the second control operation. Consequently, the first device sends first location information to the second device, i.e., the trigger position of the second control operation within the target window, such as the coordinates of the touch point or touch trajectory within the target window.

[0111] In some implementations, a listener for touch events is registered on the view of the target window of the first device. When the user clicks or swipes on any area of ​​the target window, an onTouch event, i.e. a touch event, is triggered. Therefore, the listener can listen to the touch event and obtain the event information of the touch event, including but not limited to the horizontal and vertical coordinates of the touch event in the target window, pressing, releasing, etc., and use the horizontal and vertical coordinates of the touch event in the target window as the first position information.

[0112] Step S605: Display the updated mirrored screen projection interface on the upper layer of the preset screen projection interface. The updated mirrored screen projection interface is generated by the updated second virtual screen projection. The updated second virtual screen is determined by the second device based on the first location information.

[0113] After receiving the first location information, the second device responds to the second control operation based on the first location information and updates the display content of the second virtual screen. For example, if the second virtual screen is a mobile phone desktop page, and the second control operation is determined to be opening application 1 based on the first location information, then after responding to the second control operation, the second virtual screen is updated to display the page of application 1. The second device then projects the updated second virtual screen onto the first device, and the first device displays the updated mirrored projection interface on the upper layer of the preset projection interface.

[0114] In one example, when the first device is a smart car's central control screen and the second device is a mobile phone, Figure 8 This is a schematic diagram illustrating an updated screen mirroring interface according to an exemplary embodiment, such as... Figure 8As shown, when the mirrored screen of the first device displays the mobile phone desktop page, when the user clicks the icon of application 1 on the desktop page, the mirrored screen of the first device updates to display the page of application 1.

[0115] In step S606, when the target application is displayed in the mirrored screen projection interface, in response to the third control operation on the target window, a second control command associated with the third control operation is sent to the second device. The third control operation is used to switch the target application to be displayed by the preset screen projection interface.

[0116] The target application is any application supported by the second device for screen mirroring to the first device. The third control operation is a preset operation performed by the user to switch the target application from the mirrored screen mirroring interface to the preset screen mirroring interface. The third control operation can be a touch operation, such as a preset gesture operation performed by the user between the mirrored screen mirroring interface and the preset screen mirroring interface, including a swipe operation from the mirrored screen mirroring interface to the preset screen mirroring interface, or a gesture operation performed by the user on the functional area of ​​the target window, including double-click operation, long press operation, etc., or a click operation performed by the user on the preset operation control in the functional area of ​​the target window; the third control operation can also be a voice command, such as the user's voice commands such as "full screen display of the target application" or "zoom in on the mirrored screen mirroring".

[0117] Since the mirrored projection interface and the preset projection interface are generated by the second device, the user's third control operation on the target window in the preset projection interface requires a response from the second device. Therefore, when the first device detects a third control operation on the target window in the preset projection interface, it sends a second control command associated with the third control operation to the second device. This command informs the second device that the first device has detected the third control operation, enabling the second device to respond to the third control operation. In some implementations, when the third control operation is a voice command, the second control command is that voice command; when the third control operation is a touch operation, the second control command is the location information of the third control operation in the preset projection interface, such as the coordinates of the preset operation control corresponding to the third control operation in the preset projection interface, so that the second device responds to the third control operation based on the location information of the third control operation.

[0118] Step S607: Stop displaying the target window and display the target application in the preset projection interface; wherein, the preset projection interface for displaying the target application is generated by the updated first virtual screen projection, and the updated first virtual screen is determined by the second device in response to the second control command.

[0119] When the second device receives the second control command, it responds to the second control command, i.e., responds to the third control operation, updates the first virtual screen to display the target application, and then projects the updated first virtual screen onto the first device. Simultaneously, it stops projecting the second virtual screen onto the first device. The first device then stops displaying the target window and displays the target application in a preset projection interface. In some embodiments, when the second device displays the target application in the first virtual screen, the content of the target application interface remains unchanged; the target application interface is simply enlarged in length and / or width to match the first virtual screen. Therefore, when the first device displays the target application in the preset projection interface, the content remains unchanged. In some embodiments, if the target application displays a video playback page or a music playback page, when the second device updates the first virtual screen to display the target application, it displays the video playback page or music playback page in full screen. Therefore, the first device displays the video playback page or music playback page in full screen in the preset projection interface.

[0120] In one example, when the first device is a smart car's central control screen and the second device is a mobile phone, Figure 9 This is a schematic diagram illustrating the display of a target application in a preset projection interface according to an exemplary embodiment, such as... Figure 9 As shown, when the first device displays the target application in the mirroring interface, if the user clicks the magnification control in the target window, the target window will be closed and the target application will be displayed in the preset mirroring interface.

[0121] In one exemplary embodiment, the target window of the upper layer of the preset projection interface is stopped from being displayed, and the target window is displayed in the preset projection interface, with the size of the target window being enlarged to be the same as the size of the preset projection interface.

[0122] Step S608: Display the adjusted mirrored projection interface on the upper layer of the preset projection interface; wherein, the adjusted mirrored projection interface is generated by the adjusted second virtual screen projection, and the adjusted second virtual screen is obtained when the display direction of the physical screen of the second device changes.

[0123] Since the mirrored interface of the second virtual screen is a real-time view of the physical screen of the second device, when the display orientation of the physical screen of the second device changes, the display orientation of the second virtual screen also changes, resulting in a second virtual screen with an adjusted display orientation. The mirrored interface of the first device is generated by mirroring the second virtual screen. Therefore, the first device displays the adjusted mirrored interface on top of the preset mirrored interface. The adjusted mirrored interface refers to the mirrored interface with the adjusted display orientation. Display orientation includes both landscape and portrait display orientations.

[0124] In one example, when the first device is a smart car's central control screen and the second device is a mobile phone, Figure 10 This is a schematic diagram illustrating a mirrored screen mirroring interface in a preset screen mirroring interface according to an exemplary embodiment, such as... Figure 10 As shown, when the phone is displayed in portrait mode, the first device displays a mirrored projection interface in portrait mode on the upper layer of the preset projection interface (as shown in the image on the left). When the phone is switched from portrait to landscape mode, the first device displays a mirrored projection interface in landscape mode on the upper layer of the preset projection interface (as shown in the image on the right).

[0125] In an exemplary embodiment, at least one of the above steps S604-S605, S606-S607, and S608 may be omitted or replaced by other steps.

[0126] In the exemplary embodiments of this disclosure, the mirroring interface is dynamically activated through a first control operation, improving the flexibility of various screen mirroring modes and allowing users to flexibly activate the mirroring interface according to their individual needs, thereby enhancing the user's screen mirroring experience. Furthermore, the user can operate the mirroring interface on the first device, thus enabling reverse control of the second device from the first device. Moreover, the mirroring interface and the preset screen mirroring interface can interact, providing users with more screen mirroring options, further enriching screen mirroring application scenarios, and further enhancing the user's screen mirroring experience.

[0127] In an exemplary embodiment of this disclosure, a display method is provided, applied to a second device. Figure 11 This is a flowchart illustrating a display method according to an exemplary embodiment, such as... Figure 11 As shown, it includes the following steps:

[0128] Step S1101: When the second device and the first device successfully connect for screen mirroring, the first virtual screen of the second device is projected onto the first device, and the first virtual screen displays a preset screen mirroring interface so that the first device displays the preset screen mirroring interface.

[0129] In some implementations, the second device establishes a screen mirroring connection with the first device through a preset screen mirroring component, a preset application, or a preset service.

[0130] Step S1102: Project the second virtual screen of the second device onto the first device. The second virtual screen displays the mirrored projection interface of the second device. The second virtual screen is located on top of the first virtual screen, so that the first device displays the mirrored projection interface of the second device on the upper layer of the preset projection interface.

[0131] The specific implementation of steps S1101-S1102 in this embodiment is described in steps S101-S102, and will not be repeated here. This method provides users with a brand-new screen casting mode. By casting through a first virtual screen and a second virtual screen respectively, the preset screen casting interface and the mirrored screen casting interface in the first device are independent of each other and coexist, realizing the diversification of screen casting interfaces. Users can experience two screen casting modes at the same time, which can meet users' needs for diverse screen casting modes and enrich the screen casting usage scenarios, thereby improving the user's screen casting experience.

[0132] In an exemplary embodiment of this disclosure, a display method is provided, applied to a second device. Figure 12 This is a flowchart illustrating a display method according to an exemplary embodiment, such as... Figure 12 As shown, it includes the following steps:

[0133] Step S1201: When the second device and the first device successfully connect for screen mirroring, the first virtual screen of the second device is projected onto the first device, and the first virtual screen displays a preset screen mirroring interface so that the first device displays the preset screen mirroring interface.

[0134] For a detailed implementation of step S1201, please refer to step S101, which will not be repeated here.

[0135] Step S1202: Receive a first control command sent by the first device. The first control command is used to instruct the second device to open the screen mirroring interface.

[0136] For a detailed implementation of step S1202, please refer to step S602, which will not be repeated here.

[0137] In step S1203, in response to the first control command, a target window is created on the upper layer of the preset projection interface of the first virtual screen, and a second virtual screen is created in the target window. The second virtual screen displays the mirror projection interface of the second device.

[0138] For a detailed implementation of step S1203, please refer to step S603, which will not be repeated here.

[0139] In one exemplary embodiment, creating a target window on top of a preset projection interface of the first virtual screen includes:

[0140] Based on the size of the physical screen of the second device, a first target size of the second virtual screen is determined, and the first target size is proportional to the size of the physical screen.

[0141] Based on the first target size, the preset size, and the display orientation of the physical screen, determine the second target size of the target window;

[0142] Based on the second target size, a target window is created on top of the preset projection interface.

[0143] The first target size is the size of the second virtual screen. Since the second virtual screen is a real-time display of the physical screen of the second device, to avoid abnormal display of the mirrored projection interface, the first target size is proportional to the size of the physical screen of the second device. For example, if the physical screen of the second device is rectangular, the aspect ratio of the first target size is the same as the aspect ratio of the physical screen of the second device; if the physical screen of the second device is circular, the diameter of the first target size is proportional to the diameter of the physical screen of the second device. The preset size is the size of the reserved functional area of ​​the target window. For example, if the functional area of ​​the target window is located at the top or bottom of the target window, the preset size is the preset width value; if the functional area of ​​the target window is located on the left or right side of the target window, the preset size is the preset height value. The second target size is the size of the target window in the upper layer of the preset projection interface. The target window is drawn in the upper layer of the preset projection interface according to the second target size.

[0144] In some implementations, if the physical screen is displayed in portrait orientation, the width value in the first target size is used as the width value in the second target size, and the sum of the length value in the first target size and a preset size is used as the height value in the second target size. For example... Figure 10 As shown in the image on the left, the width of the target window is equal to the width of the second virtual screen, and the height of the target window is the sum of the height of the second virtual screen and the preset size.

[0145] In some implementations, if the physical screen is displayed in landscape orientation, the length value in the first target size is used as the width value in the second target size, and the sum of the width value in the first target size and a preset size is used as the height value in the second target size. For example... Figure 10 As shown in the image on the right, the width of the target window is equal to the length of the second virtual screen, and the height of the target window is the sum of the width of the second virtual screen and the preset size.

[0146] It should be noted that the target window is located within the first virtual screen; that is, the maximum height of the target window is the height of the first virtual screen, the maximum width of the target window is the width of the first virtual screen, and the target window does not exceed the first virtual screen. For example, as shown... Figure 10 As shown, if the display bar for displaying the time on the first device does not belong to the preset projection interface, since the target window in the second device is located within the first virtual screen, the target window in the first device is located within the preset projection interface. Therefore, on the first device, the target window cannot be moved to the top of the display bar for displaying the time.

[0147] Step S1204: The first and second virtual screens of the target window will be projected onto the first device, so that the first device displays the target window on the upper layer of the preset projection interface and displays the mirrored projection interface in the target window.

[0148] Step S1205: Receive first location information sent by the first device. The first location information represents the trigger position of the second control operation in the target window. The second control operation is applied to the mirrored screen projection interface.

[0149] For a detailed implementation of step S1205, please refer to step S604, which will not be repeated here.

[0150] Step S1206: Based on the first location information, determine the second location information, which represents the trigger position of the second control operation on the physical screen of the second device.

[0151] Since the second virtual screen displays a mirrored projection of the physical screen of the second device, the second control operation requires a response from the physical screen of the second device. Therefore, the trigger position of the second control operation in the target window is converted into the trigger position of the second control operation in the physical screen of the second device. That is, the first position information is converted into the second position information, and then the physical screen of the second device responds to the second control operation based on the second position information.

[0152] In some implementations, a third location information is determined based on the first location information, the third location information representing the trigger position of the second control operation in the second virtual screen; and the second location information is determined based on the third location information.

[0153] Since the difference between the size of the target window and the size of the second virtual screen is the size of the functional area of ​​the target window, the second device first converts the trigger position of the second control operation in the target window into the trigger position of the second control operation in the second virtual screen, and then converts the trigger position of the second control operation in the second virtual screen into the trigger position of the second control operation in the physical screen of the second device.

[0154] In one example, Figure 13 This is a schematic diagram illustrating the target window size, the second virtual screen size, and the physical screen size of the second device according to an exemplary embodiment, as shown below. Figure 13As shown, using pixels as the unit of size, the physical screen of the second device is 900×1600, the height of the first virtual screen is 900, the height of the target window on the first virtual screen is 900, and the size of the functional area of ​​the target window is 80. Therefore, the height of the second virtual screen is 820. Since the aspect ratio of the second virtual screen is the same as that of the physical screen of the second device, the aspect ratio of the second virtual screen is the ratio of 1600 to 900. Therefore, the width of the second virtual screen is approximately 820×9 / 16≈461. Thus, the size of the second virtual screen is 461×820. If the target window size is 461×900, then converting the trigger position of the second control operation in the target window to the trigger position of the second control operation in the second virtual screen means subtracting 80 from the coordinates of the trigger position in the height direction, while keeping the coordinates of the trigger position in the width direction unchanged. Converting the trigger position of the second control operation in the second virtual screen to the trigger position of the second control operation in the physical screen of the second device means enlarging the coordinates of the trigger position in the height direction by 1600 / 820≈1.95 times, and also enlarging the coordinates of the trigger position in the width direction by 900 / 461≈1.95 times.

[0155] Step S1207: Based on the second location information, update the second virtual screen to obtain the updated second virtual screen.

[0156] Based on the trigger position of the second control operation on the physical screen of the second device, the physical screen of the second device responds to the second control operation and updates the physical screen of the second device. The second virtual screen displays the mirrored projection interface of the second device, and the second virtual screen is updated synchronously to obtain the updated second virtual screen.

[0157] Step S1208: Project the updated second virtual screen onto the first device so that the first device displays the updated mirrored projection interface on the upper layer of the preset projection interface.

[0158] Step S1209: When the target application is displayed in the second virtual screen, a second control command sent by the first device is received. The second control command is used to instruct the second device to switch the target application to be displayed by the first virtual screen.

[0159] For a detailed implementation of step S1209, please refer to step S606, which will not be repeated here.

[0160] Step S1210: In response to the second control command, display the target application in the first virtual screen and obtain the updated first virtual screen.

[0161] For a detailed implementation of step S1210, please refer to step S607, which will not be repeated here.

[0162] In some implementations, displaying the target application in a first virtual screen includes:

[0163] Based on the preset mapping relationship and the target application, the target display parameters associated with the target application are determined. The preset mapping relationship represents the mapping relationship between the preset application and the preset display parameters.

[0164] Based on the target display parameters, the target application is displayed in the first virtual screen.

[0165] Because the display ratio and resolution of the target application change when it switches from the second virtual screen to the first virtual screen, the dots per inch (DPI) of the target application differs from that of the target application displayed on the second virtual screen. This can lead to display anomalies, including but not limited to abnormal font size, image size, and user interface position. To avoid these anomalies, a preset mapping relationship between preset applications and preset display parameters is pre-stored in the second device. The preset application can be its name or application identifier, and the preset display parameter is the display pixel density. Different preset applications correspond to different preset display parameters. When switching the target application to the first virtual screen, the target application is identified, the corresponding target display parameter is retrieved from the preset mapping relationship, the resize method of the first virtual screen is called to set the display parameters of the first virtual screen to the target display parameters, and then the target application is displayed on the first virtual screen.

[0166] Step S1211: Project the updated first virtual screen to the first device and stop projecting the second virtual screen to the first device, so that the first device stops displaying the target window and displays the target application in the preset projection interface.

[0167] In step S1212, when the display orientation of the physical screen of the second device changes, the display orientation of the second virtual screen is adjusted to obtain the adjusted second virtual screen, and the display orientation of the adjusted second virtual screen is the same as that of the physical screen.

[0168] The physical screen of the second device can be displayed in both landscape and portrait orientations (e.g., ...). Figure 10 (As shown). In some ways, Figure 14 This is a schematic diagram illustrating the adjustment of the display orientation of a second virtual screen according to an exemplary embodiment, such as... Figure 14As shown, a screen configuration listener is registered on the physical screen of the second device. When the display orientation of the physical screen of the second device rotates, the listener can detect the screen rotation event, including the screen rotation direction. If the display orientation of the second screen changes from portrait to landscape, the display orientation of the second virtual screen is synchronously adjusted to landscape, thus obtaining the adjusted second virtual screen. If the display orientation of the second screen changes from landscape to portrait, the display orientation of the second virtual screen is synchronously adjusted to portrait, thus obtaining the adjusted second virtual screen. After adjusting the display orientation of the second virtual screen, the size of the adjusted target window is determined based on the target window size determination method described in step S1203.

[0169] Step S1213: Project the adjusted second virtual screen onto the first device so that the first device displays the adjusted mirrored projection interface on the upper layer of the preset projection interface.

[0170] In an exemplary embodiment, at least one of the above steps S1205-S1208, S1209-S1211, and S1212-S1213 may be omitted or replaced by other steps.

[0171] In this embodiment, dynamically activating the screen mirroring interface enhances the flexibility of various screen mirroring modes, allowing users to flexibly activate the screen mirroring interface according to their individual needs and improving the user experience. Furthermore, users can operate the screen mirroring interface on the first device, thereby enabling reverse control of the second device from the first device. The screen mirroring interface and the preset screen mirroring interface can interact, and the screen mirroring interface on the first device can respond promptly to the physical screen of the second device. This provides users with more screen mirroring options, further enriching screen mirroring application scenarios and enhancing the user experience.

[0172] In an exemplary embodiment of this disclosure, a display method is provided. Figure 15 This is an interactive flowchart illustrating a display method according to an exemplary embodiment, such as... Figure 15 As shown, it includes the following steps:

[0173] Step S1501: When the second device and the first device successfully connect for screen mirroring, the second device projects the first virtual screen onto the first device, and the first virtual screen displays a preset screen mirroring interface.

[0174] Step S1502: The first device displays the preset projection interface.

[0175] In step S1503, the second device projects the second virtual screen onto the first device, and the second virtual screen displays the mirrored projection interface of the second device. The second virtual screen is located on top of the first virtual screen.

[0176] Step S1504: The first device displays the mirrored screen projection interface of the second device on the upper layer of the preset screen projection interface.

[0177] In an exemplary embodiment of this disclosure, a display method is provided. Figure 16 This is an interactive flowchart illustrating a display method according to an exemplary embodiment, such as... Figure 16 As shown, it includes the following steps:

[0178] Step S1601: When the second device and the first device successfully connect for screen mirroring, the second device projects the first virtual screen onto the first device, and the first virtual screen displays a preset screen mirroring interface.

[0179] In step S1602, the first device displays the preset projection interface.

[0180] In step S1603, the first device responds to the first control operation on the preset projection interface by sending a first control command associated with the first control operation to the second device. The first control operation is used to open the mirror projection interface of the second device.

[0181] In step S1604, the second device receives a first control command sent by the first device. The first control command is used to instruct the second device to open the screen mirroring interface.

[0182] In step S1605, the second device responds to the first control command, creates a target window on the upper layer of the preset projection interface of the first virtual screen, and creates a second virtual screen in the target window. The second virtual screen displays the mirror projection interface of the second device.

[0183] In step S1606, the second device projects the first and second virtual screens of the target window onto the first device.

[0184] In step S1607, the first device displays the target window on the upper layer of the preset projection interface, and displays the mirrored projection interface in the target window.

[0185] In step S1608, the first device responds to the second control operation on the mirrored screen projection interface by sending first location information to the second device. The first location information represents the trigger position of the second control operation in the target window.

[0186] In step S1609, the second device receives the first location information sent by the first device, the first location information representing the trigger position of the second control operation in the target window.

[0187] In step S1610, the second device determines the second location information based on the first location information. The second location information represents the trigger position of the second control operation in the second virtual screen.

[0188] Step S1611: The second device updates the second virtual screen based on the second location information to obtain the updated second virtual screen;

[0189] In step S1612, the second device projects the updated second virtual screen onto the first device.

[0190] In step S1613, the first device displays the updated mirrored screen projection interface on the upper layer of the preset screen projection interface.

[0191] In step S1614, when the first device displays the target application in the mirrored screen projection interface, in response to the third control operation on the target window, it sends a second control command associated with the third control operation to the second device. The third control operation is used to switch the target application to be displayed by the preset screen projection interface.

[0192] In step S1615, when the second device displays the target application in the second virtual screen, it receives a second control command sent by the first device. The second control command is used to instruct the second device to switch the target application to be displayed by the first virtual screen.

[0193] In step S1616, the second device responds to the second control command and displays the target application in the first virtual screen to obtain the updated first virtual screen;

[0194] In step S1617, the second device projects the updated first virtual screen onto the first device and stops projecting the second virtual screen onto the first device.

[0195] In step S1618, the first device stops displaying the target window and displays the target application in the preset projection interface.

[0196] In step S1619, when the display orientation of the physical screen of the second device changes, the second device adjusts the display orientation of the second virtual screen to obtain the adjusted second virtual screen, and the display orientation of the adjusted second virtual screen is the same as that of the physical screen.

[0197] In step S1620, the second device projects the adjusted second virtual screen onto the first device.

[0198] Step S1621: The first device displays the adjusted mirrored projection interface on the upper layer of the preset projection interface.

[0199] In an exemplary embodiment of this disclosure, a display device is provided, applied to a first device. Figure 17This is a block diagram illustrating a display device according to an exemplary embodiment, such as... Figure 17 As shown, it includes:

[0200] The first display module 1701 is configured to display a preset projection interface on the first device when the projection connection between the first device and the second device is successful. The preset projection interface is generated by the projection of the first virtual screen of the second device.

[0201] The second display module 1702 is configured to display the mirrored projection interface of the second device on the upper layer of the preset projection interface. The mirrored projection interface of the second device is generated by projection from the second virtual screen of the second device, which is located on the upper layer of the first virtual screen.

[0202] In one exemplary embodiment, the second display module 1702 is further configured to:

[0203] In response to a first control operation on a preset projection interface, a first control command associated with the first control operation is sent to the second device. The first control operation is used to open the mirror projection interface of the second device.

[0204] The target window is displayed on the upper layer of the preset projection interface, and the mirrored projection interface is displayed in the target window. Both the target window and the second virtual screen are generated by the second device in response to the first control command.

[0205] In one exemplary embodiment, the second display module 1702 is further configured to:

[0206] In response to a second control operation on the mirrored screen projection interface, a first location information is sent to the second device, the first location information representing the trigger position of the second control operation in the target window;

[0207] The updated mirrored screen projection interface is displayed on the upper layer of the preset screen projection interface. The updated mirrored screen projection interface is generated by the updated second virtual screen projection, and the updated second virtual screen is determined by the second device based on the first location information.

[0208] In one exemplary embodiment, the first display module 1701 is further configured to:

[0209] When the target application is displayed in the mirrored screen mirroring interface, in response to the third control operation on the target window, a second control command associated with the third control operation is sent to the second device. The third control operation is used to switch the target application to be displayed by the preset screen mirroring interface.

[0210] Stop displaying the target window and display the target application in a preset projection interface; wherein, the preset projection interface for displaying the target application is generated by the updated first virtual screen projection, and the updated first virtual screen is determined by the second device in response to the second control command.

[0211] In one exemplary embodiment, the second display module 1702 is further configured to:

[0212] The adjusted mirrored screen mirroring interface is displayed on top of the preset screen mirroring interface.

[0213] The adjusted mirrored projection interface is generated by the adjusted second virtual screen projection, which is obtained when the display orientation of the physical screen of the second device changes.

[0214] In one exemplary embodiment, the first display module 1701 is further configured to:

[0215] Establish a screen mirroring connection with the first device through a preset application in the first device.

[0216] In an exemplary embodiment of this disclosure, a display device is provided, which is applied to a second device. Figure 18 This is a block diagram illustrating a display device according to an exemplary embodiment, such as... Figure 18 As shown, it includes:

[0217] The first screen projection module 1801 is configured to project the first virtual screen of the second device to the first device when the screen projection connection between the second device and the first device is successful. The first virtual screen displays a preset screen projection interface so that the first device displays the preset screen projection interface.

[0218] The second projection module 1802 is configured to project the second virtual screen of the second device onto the first device. The second virtual screen displays the mirrored projection interface of the second device. The second virtual screen is located on top of the first virtual screen, so that the first device displays the mirrored projection interface of the second device on the upper layer of the preset projection interface.

[0219] In one exemplary embodiment, the second projection module 1802 is further configured to:

[0220] Receive a first control command sent by the first device, the first control command being used to instruct the second device to open the screen mirroring interface;

[0221] In response to the first control command, a target window is created on top of the preset projection interface of the first virtual screen, and a second virtual screen is created in the target window;

[0222] The first and second virtual screens of the target window will be projected onto the first device, so that the first device displays the target window on the upper layer of the preset projection interface, and displays the mirrored projection interface in the target window.

[0223] In one exemplary embodiment, the second projection module 1802 is further configured to:

[0224] Based on the size of the physical screen of the second device, a first target size of the second virtual screen is determined, and the first target size is proportional to the size of the physical screen.

[0225] Based on the first target size, the preset size, and the display orientation of the physical screen, determine the second target size of the target window;

[0226] Based on the second target size, a target window is created on top of the preset projection interface.

[0227] In one exemplary embodiment, the second projection module 1802 is further configured to:

[0228] If the physical screen is in portrait orientation, the width value in the first target size is used as the width value in the second target size, and the sum of the length value in the first target size and the preset size is used as the height value in the second target size.

[0229] In one exemplary embodiment, the second projection module 1802 is further configured to:

[0230] If the physical screen is in landscape orientation, the length value in the first target size is used as the width value in the second target size, and the sum of the width value in the first target size and the preset size is used as the height value in the second target size.

[0231] In one exemplary embodiment, the second projection module 1802 is further configured to:

[0232] Receive first location information sent by the first device, the first location information representing the trigger position of the second control operation in the target window, the second control operation acting on the mirrored screen projection interface;

[0233] Based on the first location information, a second location information is determined, which represents the trigger position of the second control operation on the physical screen of the second device.

[0234] Based on the second location information, update the second virtual screen to obtain the updated second virtual screen;

[0235] The updated second virtual screen is projected onto the first device, so that the first device displays the updated mirrored projection interface on the upper layer of the preset projection interface.

[0236] In one exemplary embodiment, the second projection module 1802 is further configured to:

[0237] Based on the first location information, the third location information is determined, and the third location information represents the trigger position of the second control operation in the second virtual screen;

[0238] Based on the third location information, the second location information is determined.

[0239] In one exemplary embodiment, the first projection module 1801 is further configured to:

[0240] When the target application is displayed in the second virtual screen, a second control command is received from the first device. The second control command is used to instruct the second device to switch the target application to be displayed on the first virtual screen.

[0241] In response to the second control command, the target application is displayed in the first virtual screen, and the updated first virtual screen is obtained;

[0242] The updated first virtual screen is projected onto the first device, and the projection of the second virtual screen onto the first device is stopped, so that the first device stops displaying the target window and displays the target application in the preset projection interface.

[0243] In one exemplary embodiment, the first projection module 1801 is further configured to:

[0244] Based on the preset mapping relationship and the target application, the target display parameters associated with the target application are determined. The preset mapping relationship represents the mapping relationship between the preset application and the preset display parameters.

[0245] Based on the target display parameters, the target application is displayed in the first virtual screen.

[0246] In one exemplary embodiment, the second projection module 1802 is further configured to:

[0247] When the display orientation of the physical screen of the second device changes, the display orientation of the second virtual screen is adjusted to obtain an adjusted second virtual screen. The display orientation of the adjusted second virtual screen is the same as that of the physical screen.

[0248] The adjusted second virtual screen is projected onto the first device so that the first device displays the adjusted mirrored projection interface on the upper layer of the preset projection interface.

[0249] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0250] Figure 19 This is a block diagram illustrating an electronic device 1900 according to an exemplary embodiment. The electronic device can be a first device or a second device.

[0251] Reference Figure 19The electronic device 1900 may include one or more of the following components: a processing component 1902, a memory 1904, a power supply component 1906, a multimedia component 1908, an audio component 1910, an input / output (I / O) interface 1912, a sensor component 1914, and a communication component 1916.

[0252] Processing component 1902 typically controls the overall operation of electronic device 1900, such as operations associated with display, telephone calls, data communication, camera operation, and recording operations. Processing component 1902 may include one or more processors 1920 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 1902 may include one or more modules to facilitate interaction between processing component 1902 and other components. For example, processing component 1902 may include a multimedia module to facilitate interaction between multimedia component 1908 and processing component 1902.

[0253] Memory 1904 is configured to store various types of data to support the operation of electronic device 1900. Examples of this data include instructions for any application or method operating on electronic device 1900, contact data, phone book data, messages, pictures, videos, etc. Memory 1904 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0254] Power supply component 1906 provides power to various components of electronic device 1900. Power supply component 1906 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 1900.

[0255] Multimedia component 1908 includes a screen that provides an output interface between the electronic device 1900 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 1908 includes a front-facing camera and / or a rear-facing camera. When the electronic device 1900 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0256] Audio component 1910 is configured to output and / or input audio signals. For example, audio component 1910 includes a microphone (MIC) configured to receive external audio signals when electronic device 1900 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 1904 or transmitted via communication component 1916. In some embodiments, audio component 1910 also includes a speaker for outputting audio signals.

[0257] I / O interface 1912 provides an interface between processing component 1902 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0258] Sensor assembly 1914 includes one or more sensors for providing state assessments of various aspects of electronic device 1900. For example, sensor assembly 1914 can detect the on / off state of electronic device 1900, the relative positioning of components such as the display and keypad of electronic device 1900, changes in position of electronic device 1900 or a component of electronic device 1900, the presence or absence of user contact with electronic device 1900, the orientation or acceleration / deceleration of electronic device 1900, and temperature changes of electronic device 1900. Sensor assembly 1914 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 1914 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 1914 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0259] Communication component 1916 is configured to facilitate wired or wireless communication between electronic device 1900 and other devices. Electronic device 1900 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 1916 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 1916 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0260] In an exemplary embodiment, the electronic device 1900 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0261] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 1904 including instructions, which can be executed by a processor 1920 of an electronic device 1900 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0262] A non-transitory computer-readable storage medium, wherein when instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform a display method, the display method including any of the methods described above, wherein the electronic device may be a first device or a second device.

[0263] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0264] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A display method, characterized in that, The method includes: When the screen mirroring connection between the first device and the second device is successfully established, a preset screen mirroring interface is displayed on the first device. The preset screen mirroring interface is generated by the first virtual screen mirroring of the second device. The mirrored screen projection interface of the second device is displayed on the upper layer of the preset screen projection interface. The mirrored screen projection interface of the second device is generated by the second virtual screen projection of the second device, and the second virtual screen is located on the upper layer of the first virtual screen.

2. The display method according to claim 1, characterized in that, The step of displaying the mirrored screen projection interface of the second device on the upper layer of the preset screen projection interface includes: In response to a first control operation on the preset projection interface, a first control command associated with the first control operation is sent to the second device, wherein the first control operation is used to open the mirror projection interface of the second device; The target window is displayed on the upper layer of the preset projection interface, and the mirrored projection interface is displayed in the target window. Both the target window and the second virtual screen are generated by the second device in response to the first control command.

3. The display method according to claim 2, characterized in that, The method further includes: In response to a second control operation on the mirrored screen projection interface, first location information is sent to the second device, wherein the first location information represents the triggering position of the second control operation in the target window; An updated mirrored screen projection interface is displayed on the upper layer of the preset projection interface. The updated mirrored screen projection interface is generated by the updated second virtual screen projection, and the updated second virtual screen is determined by the second device based on the first location information.

4. The display method according to claim 2, characterized in that, The method further includes: When the target application is displayed in the mirrored screen projection interface, in response to the third control operation on the target window, a second control command associated with the third control operation is sent to the second device. The third control operation is used to switch the target application to be displayed by the preset screen projection interface. Stop displaying the target window and display the target application in the preset projection interface; wherein, the preset projection interface for displaying the target application is generated by the updated first virtual screen projection, and the updated first virtual screen is determined by the second device in response to the second control command.

5. The display method according to claim 1, characterized in that, The method further includes: The adjusted mirrored projection interface is displayed on the upper layer of the preset projection interface; The adjusted mirrored projection interface is generated by projection from the adjusted second virtual screen, which is obtained when the display orientation of the physical screen of the second device changes.

6. The display method according to claim 1, characterized in that, The method further includes: A screen mirroring connection is established with the first device through a preset application in the first device.

7. A display method, characterized in that, The method includes: When the second device and the first device successfully connect to the screen mirroring, the first virtual screen of the second device is projected onto the first device. The first virtual screen displays a preset screen mirroring interface so that the first device displays the preset screen mirroring interface. The second virtual screen of the second device is projected onto the first device. The second virtual screen displays the mirrored projection interface of the second device. The second virtual screen is located on top of the first virtual screen, so that the first device displays the mirrored projection interface of the second device on the upper layer of the preset projection interface.

8. The display method according to claim 7, characterized in that, The step of projecting the second virtual screen of the second device onto the first device includes: Receive a first control command sent by the first device, the first control command being used to instruct the second device to open the screen mirroring interface; In response to the first control command, a target window is created on the upper layer of the preset projection interface of the first virtual screen, and a second virtual screen is created in the target window; The first virtual screen and the second virtual screen of the target window will be projected onto the first device, so that the first device displays the target window on the upper layer of the preset projection interface, and displays the mirrored projection interface in the target window.

9. The display method according to claim 8, characterized in that, Creating a target window on top of the preset projection interface of the first virtual screen includes: Based on the size of the physical screen of the second device, a first target size of the second virtual screen is determined, wherein the first target size is proportional to the size of the physical screen; Based on the first target size, the preset size, and the display orientation of the physical screen, a second target size for the target window is determined; Based on the second target size, the target window is created on top of the preset projection interface.

10. The display method according to claim 9, characterized in that, Determining the second target size of the target window based on the first target size, the preset size, and the display orientation of the physical screen includes: If the physical screen is displayed in portrait orientation, the width value in the first target size is used as the width value in the second target size, and the sum of the length value in the first target size and the preset size is used as the height value in the second target size.

11. The display method according to claim 9, characterized in that, Determining the second target size of the target window based on the first target size, the preset size, and the display orientation of the physical screen includes: If the physical screen is displayed in landscape orientation, the length value in the first target size is used as the width value in the second target size, and the sum of the width value in the first target size and the preset size is used as the height value in the second target size.

12. The display method according to claim 8, characterized in that, The method further includes: The system receives first location information sent by the first device, the first location information representing the trigger position of the second control operation in the target window, and the second control operation acting on the mirrored projection interface. Based on the first location information, a second location information is determined, wherein the second location information represents the trigger position of the second control operation on the physical screen of the second device; Based on the second location information, update the second virtual screen to obtain the updated second virtual screen; The updated second virtual screen is projected onto the first device, so that the first device displays the updated mirrored projection interface on the upper layer of the preset projection interface.

13. The display method according to claim 12, characterized in that, The step of determining the second location information based on the first location information includes: Based on the first location information, a third location information is determined, wherein the third location information represents the triggering position of the second control operation in the second virtual screen; Based on the third location information, the second location information is determined.

14. The display method according to claim 8, characterized in that, The method further includes: When the target application is displayed in the second virtual screen, a second control command sent by the first device is received. The second control command is used to instruct the second device to switch the target application to be displayed by the first virtual screen. In response to the second control command, the target application is displayed in the first virtual screen to obtain an updated first virtual screen; The updated first virtual screen is projected onto the first device, and the projection of the second virtual screen onto the first device is stopped, so that the first device stops displaying the target window and displays the target application in the preset projection interface.

15. The display method according to claim 14, characterized in that, Displaying the target application in the first virtual screen includes: Based on the preset mapping relationship and the target application, the target display parameters associated with the target application are determined, wherein the preset mapping relationship represents the mapping relationship between the preset application and the preset display parameters; Based on the target display parameters, the target application is displayed in the first virtual screen.

16. The display method according to claim 7, characterized in that, The method further includes: When the display orientation of the physical screen of the second device changes, the display orientation of the second virtual screen is adjusted to obtain an adjusted second virtual screen, wherein the display orientation of the adjusted second virtual screen is the same as that of the physical screen; The adjusted second virtual screen is projected onto the first device, so that the first device displays the adjusted mirrored projection interface on the upper layer of the preset projection interface.

17. A display device, characterized in that, The device includes: The first display module is configured to display a preset projection interface on the first device when the first device and the second device successfully connect to the projection interface. The preset projection interface is generated by the projection of the first virtual screen of the second device. The second display module is configured to display the mirrored projection interface of the second device on the upper layer of the preset projection interface. The mirrored projection interface of the second device is generated by projection from the second virtual screen of the second device, and the second virtual screen is located on the upper layer of the first virtual screen.

18. A display device, characterized in that, The device includes: The first screen projection module is configured to project the first virtual screen of the second device onto the first device when the second device and the first device successfully connect to screen projection. The first virtual screen displays a preset screen projection interface so that the first device displays the preset screen projection interface. The second screen mirroring module is configured to mirror the second virtual screen of the second device onto the first device. The second virtual screen displays the mirrored screen mirroring interface of the second device. The second virtual screen is located on top of the first virtual screen, so that the first device displays the mirrored screen mirroring interface of the second device on the upper layer of the preset screen mirroring interface.

19. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to perform the method as described in any one of claims 1-6 or 7-16.

20. A non-transitory computer-readable storage medium, characterized in that, When the instructions in the storage medium are executed by the processor of the electronic device, the electronic device is able to perform the method as described in any one of claims 1-6 or 7-16.

Images