Electronic device, method of casting, and medium

Abstract

The application relates to the technical field of terminals, and discloses an electronic device, a screen projection method thereof, and a medium. The screen projection method of the electronic device in the application comprises the following steps: when a first electronic device projects a display interface of an application displayed on a display screen of the first electronic device to a second electronic device, the first electronic device obtains, from the second electronic device, screen projection parameters of a region of the second electronic device that receives the screen projection, wherein the parameters can include the size, resolution, pixel density and the like of the screen projection region; then, the first electronic device adjusts the display interface of the application currently displayed on the display screen, adjusts the size, resolution, pixel density and the like of the display interface of the application to be suitable for the display region of the second electronic device, generates a new display interface, and then projects the adjusted new display interface to the screen projection region of the second electronic device that receives the screen projection, so that the layout of the display interface of the application after the screen projection is more suitable for the screen projection region of the second electronic device.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to a screen projection method, electronic device and medium thereof. Background Technology

[0002] With the widespread adoption of smart devices such as smartphones, the number of service applications (APPs) on these devices is also increasing. Users can switch between APPs on different devices while using one. For example, a user can switch their call service to the car's infotainment system while driving. Currently, this switching between devices is mainly achieved by adapting APPs to provide the same service across multiple devices or by projecting the application interface from the phone to other devices.

[0003] Adapting an app to provide the same service across multiple devices requires users to install the app on all their existing devices and log in to the same account to synchronize application data, resulting in a poor user experience. Furthermore, it places a significant workload on developers. When casting an app interface from a mobile phone to other devices, current technology ensures that the interface displayed on the cast device is identical to the phone's. However, if the phone and the cast device have different screen sizes, issues may arise such as an overly cluttered layout of elements on the cast interface and unused portions of the cast device's screen. For example... Figure 1 This illustration shows a scenario in the prior art where a music app 103 on a mobile phone 100 is projected onto the screen of a vehicle infotainment system 200. For example... Figure 1 As shown, after the user projects the music app 103 from the mobile phone 100 onto the screen of the car infotainment system 200, the music app 103 still uses the layout style of the music app 103 on the mobile phone 100 screen. Because the screen size of the mobile phone 100 is different from the screen size of the car infotainment system 200, some of the elements that the music app 103 can display on the screen of the mobile phone 100 cannot be fully displayed on the screen of the car infotainment system 200, and the other part will be squeezed together and look very bulky. In addition, some areas of the screen of the car infotainment system 200 are not utilized. Summary of the Invention

[0004] The purpose of this application is to provide a screen mirroring method, electronic device, and medium, which, without modifying the layout code of the content displayed on the electronic device (such as a music app), allows the layout of the displayed content to be adapted to the screen of the remote device after being mirrored to different remote devices.

[0005] The first aspect of this application provides a screen projection method, comprising: a first electronic device displaying a first display interface of a first application to a user on a display screen, and obtaining first projection parameters from a second electronic device, wherein the first projection parameters include the size of a first projection area used by the second electronic device to receive the projection;

[0006] The first electronic device adjusts the size of the first display interface and the layout of at least one display element in the first display interface according to the acquired first projection parameters, so as to generate a first projection interface of a size suitable for the first projection area.

[0007] The first electronic device projects its first screen onto the second electronic device.

[0008] In the embodiments of this application, the first electronic device is an electronic device that projects its screen onto the second electronic device. When projecting its screen onto the second electronic device, the first electronic device will keep the layout of the display interface of the first application (i.e., the first display interface) on its screen unchanged, while adjusting the layout of each display element in the first projection interface of the first application to be projected onto the projection area of ​​the second electronic device. For example, it will adjust the size, resolution, position, size, etc. of the first display interface, so that the interface layout of the first application after being projected is more suitable for the projection area of ​​the second electronic device (i.e., the first projection area).

[0009] For example, the first electronic device can be a mobile phone, and the second electronic device can be a vehicle infotainment system. The first display interface is the mobile phone screen, and the first projection area can be a portion of the vehicle infotainment system screen. Here, the mobile phone screen size can be a 6-inch portrait screen, while the projection parameters obtained by the mobile phone from the vehicle infotainment system indicate that the projection area size is a 7-inch landscape screen. The mobile phone then generates a projection interface (the first projection interface) based on the projection parameters and adjusts the display content of the music app, such as icons, buttons, and menus, within this interface. For example, changing the music app icons from two rows and three columns to one row and six columns allows the adjusted music app interface to fit the vehicle infotainment system's projection area, preventing the music app's content from appearing cluttered and ensuring the layout of the music app fits the projection area, thus avoiding blank areas.

[0010] In one possible implementation of the first aspect described above, the size of the first projection area of ​​the second electronic device is equal to or smaller than the display screen size of the second electronic device.

[0011] In the embodiments of this application, the projection area of ​​the vehicle-mounted system can be a 7-inch landscape screen, while the screen size of the vehicle-mounted system can be an 8-inch landscape screen. In other words, the projection area of ​​the vehicle-mounted system can be a partial area within the screen of the vehicle-mounted system.

[0012] In one possible implementation of the first aspect described above, the first projection parameters further include at least one of the following: the display resolution of the second electronic device, the pixel density, and the model of the second electronic device.

[0013] Considering the different types and usage scenarios of the devices being projected (i.e., the second electronic device), adjusting the interface layout of the application after projection can facilitate user operation in certain scenarios. Therefore, when sending projection parameters, the device being projected can simultaneously send its own model number or identifier, indicating its device type, to the projection electronic device (i.e., the first electronic device). For example, when a mobile phone projects its screen to a car infotainment system, the car infotainment system sends its model number to the phone. The phone determines that the receiving electronic device is the car infotainment system based on the received signal. To facilitate driver operation, when the mobile phone projects an app to the car infotainment system, some buttons can be placed near the steering wheel. For example, the audio and video call buttons in instant messaging apps can be moved to the left side of the projection area on the car infotainment system for easy access while driving. Similarly, when the driver searches for contacts on their mobile phone or instant messaging apps, the phone can remove irrelevant information such as tags and chat history from the instant messaging app interface and directly display the contact's avatar and name.

[0014] In one possible implementation of the first aspect described above, adjusting the size of the first display interface and the layout of at least one display element in the first display interface based on the acquired first projection parameters includes:

[0015] Adjust the size, resolution, and pixel density of the first projection interface to be the same as the size, resolution, and pixel density of the first projection area.

[0016] In one possible implementation of the first aspect described above, the first projection interface is generated on the virtual screen of the first electronic device. Thus, the first projection interface is invisible within the display screen of the first electronic device and does not affect the display of the first application on the display screen of the first electronic device.

[0017] In one possible implementation of the first aspect described above, the size of the first projection area of ​​the second electronic device is smaller than the screen size of the display of the first electronic device, and the first projection interface and the first display interface are simultaneously displayed on the display of the first electronic device. That is, when the first display interface is projected onto a smaller projection area, or when the screen of the second electronic device is smaller than the screen of the first electronic device, the first projection interface can be displayed in the display of the first electronic device in a picture-in-picture manner.

[0018] In one possible implementation of the first aspect described above, the first electronic device adjusts the layout of the display elements in the first display interface to generate the first projection interface in the following manner:

[0019] Modify the position of the displayed elements in the first display interface;

[0020] Scaling or scaling the display elements in the first display interface, such as changing the size of the display elements;

[0021] Rotate the display elements in the first display interface to change their orientation, for example, change the search bar from a horizontal rectangle to a vertical rectangle;

[0022] Modify how elements are displayed in the first display interface. For example, instead of displaying the music app icon by swiping up and down on the phone screen, display it by swiping left and right on the car's infotainment system screen.

[0023] Delete at least one display element in the first display interface. For example, when casting a music app from a mobile phone screen to a smartwatch, due to the screen size of the smartwatch, only the music section and buttons such as back, pause / start, and forward can be kept, and other display elements can be deleted.

[0024] One possible implementation of the first aspect mentioned above also includes:

[0025] The first electronic device responds to an operation command received from the second electronic device by modifying the display content in the first projection interface corresponding to the first display interface. The operation command is generated by the second electronic device in response to the user's operation on the display elements in the first projection interface on the display screen of the second electronic device.

[0026] After the first electronic device projects the content displayed on its first projection interface to the second electronic device, the user can perform touch actions on the first projection interface of the second electronic device. For example, the user can click a button on the first projection interface of the second electronic device to trigger a command. Upon receiving the command, the second electronic device sends it to the first electronic device, which then responds and executes the corresponding operation. For instance, after a mobile phone projects an instant messaging app onto a car's infotainment system, the user can perform interactive operations with the phone by clicking on the car's screen. For example, in the video call interface of a mobile instant messaging app, the video frame is displayed in landscape mode. After the mobile phone projects the video frame onto the car's infotainment system, the user can click the video adjustment button in the video call interface of the instant messaging app on the car's infotainment system to simultaneously adjust the video frames of both the car and mobile phones to portrait mode, thus improving the user's video call experience.

[0027] In one possible implementation of the first aspect described above, the first electronic device is capable of changing the display content of the first display interface in response to a user's operation on a first display element in the first display interface, and also changing the display of a second display element in the first display interface.

[0028] The first electronic device is also capable of responding to a user's operation on a second display element in a first display interface, further changing the display content in the first display interface, and changing the display of a third display element corresponding to the second display element in the first display interface; and

[0029] The first electronic device adjusts the size of the first display interface and the layout of at least one display element in the first display interface according to the acquired first projection parameters, so as to generate a first projection interface of a size suitable for the first projection area, including:

[0030] The first electronic device responds to a first operation instruction received from the second electronic device by modifying the display content of the first projection interface corresponding to the first display interface. The modification includes displaying a third display element in the first projection interface. The first operation instruction is generated by the second electronic device in response to the user's operation on the first display element in the first projection interface on the display screen of the second electronic device.

[0031] The first electronic device projects a first projection interface, including a third display element, onto the second electronic device.

[0032] For example, when the first electronic device is a mobile phone and the user wants to make an audio / video call through an instant messaging app, after the user clicks on the contact for the desired audio / video call (i.e., the first display element) on the phone, an audio / video call button (i.e., the second display element) appears on the phone's interface. After clicking the audio / video call button, an audio / video call selection menu (i.e., the third display element) appears on the instant messaging app's display interface (i.e., the first display interface). The user can then select either an audio or video call from the menu to initiate the call. When the electronic device to be projected from the phone is an in-vehicle infotainment system, to simplify the user's operation, after the phone projects the instant messaging app's contact interface onto the in-vehicle infotainment system, and the user clicks on the contact for the desired audio / video call (i.e., the first display element) on the in-vehicle infotainment system, the phone receives the instruction from the in-vehicle infotainment system and subsequently displays an interface including the audio / video call button and an interface including the audio / video call selection menu (i.e., the third display element). However, the phone only projects the interface including the audio / video call selection menu (i.e., the third display element) to the user, and not the interface including the audio / video call button.

[0033] In one possible implementation of the first aspect above, the first electronic device adjusts the size of the first display interface and the layout of at least one display element in the first display interface according to the acquired first projection parameters, to generate a first projection interface of a size suitable for the first projection area, including:

[0034] The first electronic device adds a fifth display element corresponding to the fourth display element to the first projection interface, wherein the first display interface includes the fourth display element but does not include the fifth display element; and

[0035] The method also includes:

[0036] In response to a second operation command received from the second electronic device, the first electronic device adjusts the layout of the fourth display element in the first projection interface, wherein...

[0037] The second operation command is generated by the second electronic device in response to the user's operation on the fifth display element in the first projection interface on the display screen of the second electronic device.

[0038] For example, in the video call interface of an instant messaging app on a mobile phone (i.e., the first electronic device), the video frame is displayed in landscape mode. If this is projected onto the car's infotainment system (i.e., the second electronic device), it is difficult for the user to see clearly. Therefore, when projecting from the mobile phone to the car's infotainment system, a video adjustment button (i.e., the fifth display element) for rotating the video frame (i.e., the fourth display element) is added to the instant messaging app's display interface to generate the first projection interface projected onto the car's infotainment system. Then, if the user wants to rotate the video frame to a portrait orientation, they can adjust it by clicking the video adjustment button in the car's call interface.

[0039] In one possible implementation of the first aspect above, the first electronic device adjusts the size of the first display interface and the layout of at least one display element in the first display interface according to the acquired first projection parameters, to generate a first projection interface of a size suitable for the first projection area, including:

[0040] The first electronic device obtains the layout configuration file corresponding to the first projection parameters;

[0041] According to the layout configuration file, the first electronic device modifies the first display interface of the first application to the first projection interface.

[0042] In one possible implementation of the first aspect described above, there is a correspondence between the layout configuration file and the application identifier, application version number, and projection parameters of the application on the first electronic device. Specifically, the same version of the same application corresponds to different projection parameters for different sized projection areas, and different projection parameters for the same version of the same application correspond to different configuration files.

[0043] In the embodiments of this application, layout configuration files can be stored in a tree-structured file directory. For example, the first-level subdirectory can be the identifier of the app; for a music app, the identifier could be MusicApp. Second-level subdirectories can be set under the first-level subdirectory; for example, for a music app, different versions, such as 1.0 and 1.1, can be used to set the second-level subdirectories. Finally, different configuration files can be set for different screen parameters. For music app version 1.0, if it supports two screen parameters, a first configuration file and a second configuration file can exist in its directory. By storing layout configuration files in a tree-structured file directory, the mobile phone can quickly find the corresponding layout configuration file based on the app to be projected and the screen parameters.

[0044] In one possible implementation of the first aspect above, the first electronic device obtains a layout configuration file corresponding to the first projection parameters, including:

[0045] The first electronic device obtains the first application identifier and the first application version number from the installation file of the first application;

[0046] The first electronic device selects the layout configuration file corresponding to the first screen projection parameter from multiple layout configuration files by matching the identifier of the first application, the version number of the first application, and the first screen projection parameter.

[0047] In the embodiments of this application, the first application identifier of the first application is uniquely determined, and there may be multiple versions of the first application. For each version of the first application and each screen projection parameter, a corresponding layout configuration file can be saved.

[0048] In one possible implementation of the first aspect described above, the layout configuration file includes identifiers of display elements in the first display interface and layout rules for the corresponding display elements.

[0049] In the embodiments of this application, the first electronic device can locate the display element of the first application by the identifier of the display element, for example, the id of the display element.

[0050] One possible implementation of the first aspect mentioned above also includes:

[0051] The first electronic device displays a second display interface of the second application to the user on the display screen and obtains second projection parameters from the second electronic device, wherein the second projection parameters include the size of the second projection area used by the second electronic device to receive projection.

[0052] The first electronic device adjusts the size of the second display interface and the layout of at least one display element in the second display interface according to the acquired second projection parameters, so as to generate a second projection interface with a size suitable for the second projection area.

[0053] The first electronic device simultaneously projects both the first and second projection interfaces onto the second electronic device.

[0054] In the embodiments of this application, for example, a mobile phone can simultaneously project a document app and a chat app that are open at the same time onto a tablet computer. The tablet computer's screen can display both the document app and the chat app simultaneously. This eliminates the hassle of users operating both the mobile phone and the tablet computer at the same time, allowing users to use the document app and chat app directly on the larger screen of the tablet computer without switching between the two apps, thus improving the user experience.

[0055] In one possible implementation of the first aspect described above, the first application is characterized as any one of a music application, an instant messaging application, a news application, a shopping application, or a video playback application.

[0056] A second aspect of this application provides a screen projection method for an electronic device, characterized in that it includes:

[0057] The second electronic device sends a first projection parameter to the first electronic device, wherein the first projection parameter includes the size of a first projection area used by the second electronic device to receive the projection.

[0058] The second electronic device displays the first projection interface sent by the first electronic device in the first projection area on the display screen. The first projection interface is generated by the first electronic device after adjusting the size of the first display interface of the first application displayed on the display screen of the first electronic device and the layout of at least one display element in the first display interface according to the first projection parameters.

[0059] The second electronic device detects the user's operation on the display elements of the first projection interface on the screen;

[0060] The second electronic device responds to the operation by generating an operation command and sending the operation command to the first electronic device. The operation command is used to instruct the first electronic device to perform a user operation and modify the displayed content on the first projection interface.

[0061] In one possible implementation of the second aspect described above, the size of the first projection area of ​​the second electronic device is equal to or smaller than the display screen size of the second electronic device.

[0062] In one possible implementation of the second aspect described above, the first projection parameters further include at least one of the following: the display resolution of the second electronic device, the pixel density, and the model of the second electronic device.

[0063] A third aspect of this application provides an electronic device, characterized in that it comprises:

[0064] Display screen;

[0065] Memory, which stores instructions;

[0066] The processor and memory are coupled, and when the program instructions stored in the memory are executed by the processor, the electronic device controls the display screen to perform the projection method provided in the first aspect above.

[0067] A third aspect of this application provides a readable medium storing instructions, characterized in that when the instructions are executed on the readable medium, the readable medium causes the readable medium to perform the screen projection method provided in the first aspect above. Attached Figure Description

[0068] Figure 1 An example is shown of an electronic device projecting its own music app onto the screen of a car's infotainment system;

[0069] Figure 2 An example of a screen mirroring system according to an embodiment of this application is shown;

[0070] Figure 3(a) shows an example of the interface of a music app on an electronic device;

[0071] Figure 3(b) shows an example of a music app on an electronic device being projected onto the screen of the vehicle's infotainment system;

[0072] Figure 3(c) shows an example of a music app on an electronic device being projected onto a tablet screen;

[0073] Figure 3(d) shows an example of simultaneously projecting a document app and a chat app from an electronic device onto a tablet screen;

[0074] Figure 4 This document demonstrates an example of a screen mirroring method for a music app on an electronic device to the screen of a car infotainment system.

[0075] Figure 5 An example of a tree structure showing the Overlay configuration file directory and its subdirectories is shown;

[0076] Figure 6 An example of the contents of an Overlay configuration file is shown;

[0077] Figure 7(a) shows an example of the layout of controls within the interface of a music app on an electronic device;

[0078] Figure 7(b) shows an example of the layout of controls within the interface of a music app on an electronic device;

[0079] Figure 8 This shows an example of a tree structure for the layout of controls in a music app;

[0080] Figure 9 This shows an example of a tree structure for the layout of controls in a music app;

[0081] Figure 10(a) shows an example of the layout of controls within the interface of a music app on an electronic device;

[0082] Figure 10(b) shows an example of the layout of controls within the interface of a music app after it is projected from an electronic device onto the screen of the vehicle's infotainment system.

[0083] Figure 11 This shows an example of the layout of controls within the music app's interface after the music app on an electronic device is projected onto the screen of a smartwatch;

[0084] Figure 12(a) illustrates another example of a screen projection system according to an embodiment of this application;

[0085] Figure 12(b) illustrates another example of a screen projection system according to an embodiment of this application;

[0086] Figure 13(a) shows an example of the control layout in the contacts interface of an instant messaging app on an electronic device;

[0087] Figure 13(b) shows an example of the control layout in the contact interface of an instant messaging app after the instant messaging app on an electronic device is projected onto the screen of the vehicle's infotainment system;

[0088] Figure 14(a) shows an example of the control layout within the search function interface of an instant messaging app on an electronic device;

[0089] Figure 14(b) shows an example of the control layout within the search function interface of an instant messaging app after it is projected from an electronic device onto the screen of an in-vehicle infotainment system.

[0090] Figure 15(a) shows an example of the control layout in the audio and video call interface of an instant messaging app on an electronic device;

[0091] Figure 15(b) shows an example of the control layout within the audio and video call interface of an instant messaging app after it is projected from an electronic device onto the screen of a vehicle's infotainment system.

[0092] Figure 16 This shows an example of the control layout within the contact video call interface of an instant messaging app on an electronic device;

[0093] Figure 17(a) shows an example of a layout where the video frame 1701 in the contact video call interface of an instant messaging APP on an electronic device is displayed in landscape mode;

[0094] Figure 17(b) shows an example of the layout where the video frame 1701 in the contact video call interface of the instant messaging APP is displayed in portrait mode after the instant messaging APP on the electronic device is projected onto the screen of the vehicle system.

[0095] Figure 17(c) shows an example of a layout where the video frame 1702 in the contact video call interface of an instant messaging APP on an electronic device is displayed in landscape mode;

[0096] Figure 17(d) shows an example of the layout where the video frame 1702 in the contact video call interface of the instant messaging APP is displayed in portrait mode after the instant messaging APP on the electronic device is projected onto the screen of the vehicle system.

[0097] Figure 17(e) shows an example layout where both video frames 1701 and 1702 are displayed in landscape mode in the contact video call interface of an instant messaging app on an electronic device.

[0098] Figure 17(f) shows an example of a layout where video frames 1701 and 1702 in the contact video call interface of the instant messaging app are displayed in portrait mode after the instant messaging app on the electronic device is projected onto the screen of the vehicle system.

[0099] Figure 18 A schematic diagram of the structure of an electronic device according to an embodiment of this application is shown;

[0100] Figure 19 A software structure block diagram of an electronic device according to an embodiment of this application is shown;

[0101] Figure 20 A block diagram of a system-on-a-chip (SoC) according to an embodiment of this application is shown. Detailed Implementation

[0102] The technical solutions of the embodiments of this application will be described below with reference to the accompanying drawings. In the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B; "and / or" in this text is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, in the description of the embodiments of this application, "multiple" refers to two or more than two.

[0103] According to some embodiments of this application, letters following reference numerals in the drawings, such as "100a", indicate references to elements having that particular reference numeral, while reference numerals without subsequent letters, such as "100", indicate general references to embodiments of the elements having that reference numeral.

[0104] The embodiments of this application will now be described in further detail with reference to the accompanying drawings.

[0105] Figure 2 A screen projection system is illustrated according to some embodiments of this application. For example... Figure 2 As shown, the projection system includes an electronic device 100, a remote device 200, and a cloud platform 300.

[0106] Electronic device 100 can obtain projection parameters from remote device 200 to adjust the layout of the displayed content when projecting to remote device 200. Then, based on the obtained projection parameters, it adjusts the layout of the displayed content to be projected so that the layout of the displayed content is suitable for the screen of remote device 200 after projection. It is understood that in some embodiments of this application, the projection parameters may be the screen size, resolution, pixel density, and device type of remote device 200. In other embodiments, the projection parameters may also be the size, resolution, pixel density, and device type of the projection area within the screen of remote device 200. Here, the projection area may be a partial area of ​​the screen of remote device 200 used to display the projected content within that area. Furthermore, different types of remote devices may have different requirements for the layout of the displayed content. For example, when projecting a mobile phone call service onto a vehicle infotainment system, the dial button can be placed on the left side of the vehicle's infotainment screen for the driver's convenience in making and receiving calls. Therefore, when casting, you can send the device type to the electronic device 100 to adjust the layout of the cast content. For example, the device type can include in-vehicle infotainment system, personal computer, smart TV, etc.

[0107] Figure 3(a) shows the interface of APP 103 (here, a music APP is used as an example, hereinafter referred to as music APP 103) on electronic device 100 (including the content that the user can see after swiping up on the screen). The content on the interface can be projected onto a remote device 200 (such as a car infotainment system and a tablet computer) that is larger than the screen of electronic device 100. Figures 3(b) and 3(c) show scenarios of projecting the music APP 103 on mobile phone 100 onto different types of remote devices 200 (such as car infotainment systems and tablet computers), respectively.

[0108] Specifically, as shown in Figure 3(b), mobile phone 100 can obtain screen projection parameters from vehicle infotainment system 200. Based on these parameters, mobile phone 100 generates a virtual screen with the same size as the screen of vehicle infotainment system 200. Then, mobile phone 100 displays the projected content (as shown in Figure 3(a)) on the virtual screen, and finally projects it onto vehicle infotainment system 200, which then displays the projected content in full screen. Figure 1 In comparison, as shown in Figure 3(b), after the music app 103 is projected from the electronic device 100 to the remote device 200, the display layout of each element in the music app 103 is more suitable for the screen of the vehicle system 200. This eliminates problems such as overly bloated element layouts and unused areas of the display screen of the remote device 200, resulting in a better user experience. The technical solution of this application, without modifying the layout code of the content displayed on the electronic device 100 (such as the music app 103), ensures that the layout of the displayed content is suitable for the screen of different remote devices 200 after projection.

[0109] As shown in Figure 3(c), when casting a music app 103 from an electronic device 100 (such as a mobile phone) to a remote device 200 (such as a tablet computer 200), the mobile phone 100 can obtain casting parameters from the tablet computer 200. Based on these parameters, the mobile phone 100 generates a virtual screen with the same size, resolution, and pixel density as the window 200A on the tablet computer 200 screen. The cast content (i.e., the interface of the music app 103) is then displayed on the virtual screen. Finally, the content displayed on the virtual screen is shown on the window 200A on the tablet computer 200 screen. It can be understood that in some embodiments, if the user modifies the size of the window 200A on the tablet computer 200 using zoom gestures, the tablet computer 200 can send the screen parameters of the changed window 200A to the mobile phone 100 in real time. This allows the mobile phone 100 to generate a virtual screen with different size and resolution parameters corresponding to the screen parameters, thereby dynamically adapting to the changes in the window 200A for casting.

[0110] Furthermore, in other embodiments, the display electronic device 100 (such as a mobile phone 100) can simultaneously project multiple apps onto a remote device 200, such as a tablet computer 200. For example, Figure 3(d) illustrates a scenario where a document app 106 and a chat app 107, both open on the mobile phone 100, are simultaneously projected onto the tablet computer 200. As shown in Figure 3(d), the mobile phone 100 receives the projection parameters for the corresponding window 200B on the tablet computer 200 from the tablet computer 200, then generates a virtual screen corresponding to the projection parameters of the window 200B. The size, resolution, and other parameters of this virtual screen are the same as those of the window 200B. The document app 106 is displayed on this virtual screen, and then the interface of the document app 106 displayed on the virtual screen is projected onto the window 200B of the tablet computer 200. Then, the mobile phone 100 receives the projection parameters of the window 200C on the tablet computer 200 from the tablet computer 200, and then generates a virtual screen with the projection parameters of the window 200C. The size, resolution and other parameters of the virtual screen are the same as those of the window 200C. The chat APP 107 is displayed on the virtual screen, and then the interface of the chat APP 107 on the virtual screen is projected onto the window 200C of the tablet computer 200. For example, in the figure, the user is using the chat APP 107 to make a video call.

[0111] The cloud 300 is used to generate and update layout rules, and sends the generated or updated layout rules to the electronic device 100. These layout rules are used to modify the different layouts of applications on screens of devices with different sizes. For example, if the screen size, resolution, and pixel density of electronic device 100 and remote device 200 are different, the layout of music app 103 displayed on the screens of electronic device 100 and remote device 200 will also be different. In some embodiments, layout rules can be set in a configuration file (such as the Overlay configuration file below). Electronic device 100 modifies the layout of the content to be projected by running the configuration file to adapt to the screen size, resolution, and pixel density of the remote device 200 being projected.

[0112] Specifically, the cloud 300 can collect and update the aforementioned configuration files independently. For example, the cloud 300 can collect configuration files for various applications from each electronic device 100 or from the developers of various applications (e.g., configuration files required from application developers when registering or listing in an app store; or, the cloud 300 can request each electronic device 100 to report the configuration files of the applications it has acquired). The cloud 300 can also send the latest collected application configuration files to the electronic device 100 based on a request from the electronic device 100 or through periodic push notifications. After obtaining the application configuration files, the electronic device 100 can save them to its local storage and use the obtained application configuration files to adjust the application layout when casting the application to the remote device 200. In addition, the cloud 300 can also send the configuration files corresponding to the applications that the remote device 200 supports for casting to the remote device 200 upon request.

[0113] Furthermore, it is understood that in other embodiments, the functions of the cloud 300 described above can also be implemented by the electronic device 100 or the remote device 200 without the involvement of the cloud. For example, the electronic device 100 can generate and obtain configuration files on its own, and then project the screen onto the remote device 200 based on the configuration files. For example, the electronic device 100 can access the layout configuration interface of an application through its own operating system's view system, create or update layout rules for its installed applications, and save the layout rules in the configuration file. Alternatively, the electronic device 100 can obtain the configuration files corresponding to its installed applications from an app store. After generating and obtaining the configuration files corresponding to the applications, the electronic device 100 can save them to its local storage. For another example, the remote device 200 can also generate configuration files. For example, the remote device 200 can generate layout rules adapted to its own screen size, resolution, and pixel density for applications that support screen projection, and save the layout rules in the configuration file stored in its own storage. For example, in the screen projection system shown in Figure 13(a) below, without the participation of the cloud or server, the electronic device 100 and the remote device 200 can achieve screen projection through wireless communication technologies (such as wireless fidelity (Wi-Fi) networks, Bluetooth, near field communication (NFC) and so on). For details, please refer to the descriptions in Figures 12(a) and 12(b).

[0114] It is understood that electronic device 100 and remote device 200 can be various computing devices capable of communicating with each other. For example, electronic device 100 and remote device 200 can include, but are not limited to, laptop computers, desktop computers, tablet computers, mobile phones, servers, wearable devices, head-mounted displays, mobile email devices, in-vehicle infotainment systems, portable game consoles, portable music players, e-reader devices, televisions with one or more processors embedded or coupled thereto, or other electronic devices capable of accessing networks. Here, "in-vehicle infotainment system" refers to a product installed in a car, which functionally enables information communication between people and vehicles, and between vehicles and the outside world (vehicle-to-vehicle, vehicle-to-electronic devices). In the following description, for simplicity, electronic device 100 is illustrated using a mobile phone 100 as an example, and remote device 200 is illustrated using an in-vehicle infotainment system 200.

[0115] Example 1

[0116] The technical solution of this application will be introduced below, taking the specific structure of the mobile phone 100 and the vehicle system 200 as an example, with the screen-casting music APP 103 as shown in Figures 3(a) and (b).

[0117] Continue to refer to Figure 2 The mobile phone 100 may include: SystemService 101, main screen 102, APP 103 running on the mobile phone 100, virtual screen 104, and OverlayApk 105 (Overlay application). SystemService 101 refers to a program, routine, or process that performs specified system functions to support other programs, especially low-level (close to hardware) programs. For example, in the Android system, the DisplayManage function of SystemService 101 can be used to control the main screen 102 and generate the virtual screen 104.

[0118] The main screen 102 is the physical screen of the phone 100.

[0119] APP 103 can be an application running on mobile phone 100 with a layout style. In this description, for ease of explanation, music APP 103 is used as an example to illustrate the technical solution of this application. In addition, it can be understood that in other embodiments of this application, the content displayed on the vehicle system 200 by mobile phone 100 is not limited to APP 103, but can also be the interface of any other application, program, etc. displayed on mobile phone 100.

[0120] The virtual screen 104 can be generated by SystemService 101 on the mobile phone 100, for example, through DisplayManage of SystemService 101. The virtual screen 104 is only a software-level simulated display screen. For example, SystemService 101 can generate a virtual screen 104 corresponding to the received parameters such as screen size, resolution, and pixel density of the vehicle's infotainment system 200 screen.

[0121] OverlayApk105, hereinafter referred to as Overlay Application 105, is used to obtain the Overlay configuration file from the cloud 300 and modify the layout of the APP 103 displayed in the virtual screen 104 according to the layout rules of the display elements (hereinafter referred to as controls) in APP 103 contained in the obtained Overlay configuration file.

[0122] The vehicle infotainment system 200 includes a client 201, a communication module 202, and a screen 203. The client 201 transmits the screen projection parameters of the vehicle infotainment system 200 to a mobile phone 100 connected to it via the communication module 202. The communication module 202 enables data communication between the vehicle infotainment system 200 and other electronic devices via wireless or wired communication methods, such as Wi-Fi, Bluetooth, RFID, and short-range wireless communication technologies. The screen 203 can display the interface of the APP 103 on the virtual screen 104 of the mobile phone 100.

[0123] Figure 4 According to an embodiment of this application, a technical solution for projecting a music app 103 from a mobile phone 100 onto the screen of a vehicle infotainment system 200 is shown. It is understood that the music app 103 described here is merely exemplary, and the technical solution of this application is applicable to various applications on the mobile phone 100, and is not limited thereto. Specifically, as... Figure 4 As shown, it includes:

[0124] 401: Mobile phone 100 communicates with vehicle infotainment system 200 via wireless communication. For example, mobile phone 100 can communicate with vehicle infotainment system 200 via wireless communication methods such as Bluetooth, WIFI, or NFC. In some embodiments, mobile phone 100 can also communicate with vehicle infotainment system 200 via wired communication, for example, mobile phone 100 can communicate with vehicle infotainment system 200 via data cable and Universal Serial Bus (USB) interface.

[0125] Before or after establishing a communication connection between mobile phone 100 and vehicle system 200, the user can enable the screen mirroring function of mobile phone 100, or mobile phone 100 can automatically enable the screen mirroring function.

[0126] 402: The client 201 installed on the vehicle infotainment system 200 sends the screen projection parameters such as the screen size, resolution, pixel density, and device type of the vehicle infotainment system 200 to the mobile phone 100.

[0127] In some embodiments, after the mobile phone 100 and the vehicle infotainment system 200 establish a communication connection, the mobile phone 100 can send an acquisition command to the vehicle infotainment system 200 to obtain screen projection parameters such as the screen size, resolution, pixel density, and device type of the vehicle infotainment system 200. Upon receiving the acquisition command, the vehicle infotainment system 200 sends the screen projection parameters back to the mobile phone 100. In other embodiments, after the mobile phone 100 and the vehicle infotainment system 200 establish a communication connection, the vehicle infotainment system 200 can proactively send screen projection parameters to the mobile phone 100.

[0128] Furthermore, it is understood that in other embodiments, the projection parameters may include only any two of screen size, resolution, and pixel density, and may not include device type. Screen size refers to the length of the screen's diagonal; for example, the screen size of a vehicle-mounted system 200 is 8 inches, meaning the diagonal length is 20 centimeters. Resolution is the size of pixels in the horizontal and vertical directions, calculated as vertical pixels * horizontal pixels, such as 1920px * 1080px. Pixel density refers to the number of pixels per inch of screen. The relationship between these three factors can be calculated using the formula: pixel density = Sqrt(horizontal pixels * horizontal pixels + vertical pixels * vertical pixels) / screen size, where Sqrt is the square root function. Therefore, the projection parameters may include only any two of screen size, resolution, and pixel density, with the third parameter calculated from the received parameters.

[0129] 403: Based on the screen projection parameters obtained from the vehicle's infotainment system 200, mobile phone 100 creates a virtual screen 104 and configures a screen ID (Display ID) for the virtual screen 104.

[0130] Specifically, mobile phone 100 can create a virtual screen 104 whose screen size, resolution, and pixel density are identical to those in the projection parameters obtained from the vehicle infotainment system 200. For example, in the Android system, mobile phone 100 can use the obtained screen size, resolution, and pixel density of the vehicle infotainment system 200 as parameters and use the `createVirtualDisplay(String, int, int, int, Surface, int)` method of the Android system service `DisplayManager` to create virtual screen 104.

[0131] In addition, during use, mobile phone 100 may create other virtual screens in response to other applications / processes. To distinguish virtual screen 104 from these virtual screens, mobile phone 100 configures a screen ID (Display ID) for each virtual screen when creating it. This allows mobile phone 100 to retrieve virtual screen 104 by searching for its screen ID when casting, and then display the music app 103 with its modified control layout on virtual screen 104.

[0132] Furthermore, it is understood that in this embodiment, since the size of the main screen 102 of the mobile phone 100 is generally smaller than the size of the screen 203 of the vehicle infotainment system 200, the main screen 102 of the mobile phone 100 cannot display the layout of the music app 103 that meets the projection parameters received from the vehicle infotainment system. Therefore, a virtual screen 104 is created to display the interface of the music app 103 that meets the projection parameters. In other embodiments, if the screen size of the remote device 200 to which the mobile phone 100 wants to project is smaller than the screen size of the mobile phone 100, in addition to displaying the application interface with the modified layout by creating a virtual screen as described above, a new window for the music app 103 with the modified layout can be created on the main screen 102 of the mobile phone 100 in a picture-in-picture format. This window and the original interface of the music app 103 are displayed simultaneously on the main screen 102, and then the newly created window is projected onto the screen 203 of the vehicle infotainment system 200. For details, please refer to the following text. Figure 11 The relevant description of the illustrated embodiment.

[0133] 404: When a user uses mobile phone 100, if the user opens music APP 103, the overlay application 105 of mobile phone 100 obtains the overlay configuration file corresponding to the screen parameters of vehicle system 200 from the multiple overlay configuration files of music APP 103.

[0134] It is understandable that for each app on mobile phone 100, mobile phone 100 stores multiple Overlay configuration files corresponding to different screen parameters. For example, music app 103 has three Overlay configuration files F1, F2, and F3, each corresponding to different screen parameters, such as screen size, resolution, and pixel density. If the screen parameters in the projection parameters sent by the vehicle system 200 are the same as configuration file F1, then the Overlay application 105 on mobile phone 100 will obtain the Overlay configuration file F1.

[0135] In some embodiments, the memory of the vehicle infotainment system 200 may pre-store the overlay configuration file of the music app 103 that supports screen casting. When the mobile phone 100 obtains the screen casting parameters from the vehicle infotainment system 200, it can simultaneously read the overlay configuration file of the music app 103 in the vehicle infotainment system 200, or the mobile phone 100 can update the overlay configuration file of the music app 103 from the vehicle infotainment system 200 to its own memory.

[0136] 405: Modify the layout of each control in the music app 103 based on the obtained Overlay configuration file, and then display the music app 103 with the modified layout in the virtual screen 104.

[0137] 406: Project the virtual screen 104 onto the screen 203 of the vehicle infotainment system 200.

[0138] 407: The vehicle infotainment system 200 displays the content on the virtual screen 104 on the screen 203.

[0139] Understandable. Figure 4 This describes the process by which mobile phone 100 adjusts the distribution of displayed content to fit the screen of vehicle infotainment system 200 after receiving the screen projection parameters from vehicle infotainment system 200. If the display area of ​​vehicle infotainment system 200 changes during screen projection, the vehicle infotainment system can resend the screen projection parameters to mobile phone 100. Mobile phone 100 then repeats the above steps to adjust the size and resolution of the virtual screen and then performs screen projection again.

[0140] The following is combined Figures 5 to 7(b) This document explains the Overlay configuration file and the technical solution for modifying the element layout in the music app 103 using the Overlay configuration file.

[0141] Generation and updating of Overlay configuration files

[0142] According to some embodiments of this application, the cloud 300 can generate and update Overlay configuration files. Developers can generate Overlay configuration files corresponding to each app on the cloud 300. Furthermore, for the same app, since the screen parameters of different remote devices 200 to be projected are different, such as screen size and resolution, multiple Overlay configuration files corresponding to multiple screen parameters can be generated. In addition, corresponding to app version updates, Overlay configuration files for the new version of the app can also be generated on the cloud 300.

[0143] For example, if the APP version is updated, when the cloud 300 and the mobile phone 100 are connected, the cloud 300 detects the difference between the Overlay configuration file stored in itself and the Overlay configuration file stored in the mobile phone 100. If the number and version of the Overlay configuration files stored in the cloud 300 are higher than those stored in the mobile phone 100, the cloud 300 can send the Overlay configuration files that are not available on the mobile phone 100 to the mobile phone 100 through wireless communication methods such as Bluetooth, WIFI, and near-field wireless communication technology, or through wired communication methods.

[0144] Overlay Configuration File Components and Usage

[0145] According to some embodiments of this application, on mobile phone 100, the Overlay configuration file is stored in a tree structure, for example, Figure 5 This shows the tree structure of the Overlay configuration file directory and its contained subdirectories. Specifically, as shown... Figure 5 As shown, in the Overlay configuration file directory, its first-level subdirectories are named after the app's identifier. For example, in some embodiments, the package name can be used as the app's identifier. For instance, for a music app, the package name is MusicApp, so MusicApp can be used as the music app's identifier. For a news app, the package name is NewsApp, so the news app's identifier can be NewsApp. Furthermore, it is understood that in other embodiments, other characters can also be used as the app's identifier, such as the identifiers generated by the cloud for each app; this is not limited here. Second-level subdirectories can be set under the first-level subdirectory of the Overlay configuration file. For example, for a music app, second-level subdirectories can be set for different versions of the music app, and the names of the second-level subdirectories can represent the version numbers of different versions, such as 1.0, 1.1, etc. For each version of the app, since the screen parameters of different remote devices 200 to be projected may be different, different configuration files can be set for the same version of the same app for different screen parameters. For example, for the music app version number 1.0, there are two screen parameters, where the first screen parameter has a first configuration file, and the second screen parameter has a second configuration file.

[0146] In the above embodiment, when the mobile phone 100 wants to cast the music APP 103, the overlay application 105 of the mobile phone 100 can search for the APP identifier (such as package name), version number of the music APP 103, and screen parameters (such as screen size, resolution and pixel density) of the vehicle system 200 in the casting parameters, retrieve the overlay configuration file corresponding to the current version of the music APP 103 and the received casting parameters in the tree-shaped overlay configuration file, and then load the retrieved overlay configuration file to adjust the layout of the controls in the music APP 103.

[0147] Figure 6 This shows the contents of an overlay configuration file. For example... Figure 6 As shown, each Overlay configuration file includes at least one layout rule, which specifies the rules for modifying the layout of controls in the interface of Music App 103. Specifically, the process of modifying the layout of controls in the app through the Overlay configuration file is as follows (continuing to use Music App 103 as an example):

[0148] After the Overlay application 105 loads the Overlay configuration file corresponding to the music app 103, the layout rules in the Overlay configuration file are used to locate and modify the interface of the music app 103.

[0149] As shown in Figure 7(a), the interface of the music app 103 consists of multiple controls, including icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, and navigation bars. Some controls can act as containers, including other controls. For example, as shown in Figures 7(a) and 7(b), the interface of the music app 103 includes a background interface 701, which is the bottom layer of the music app 103 controls. The background interface 701 contains a search bar 702, a title bar 703, a first category 704, a second category 705, and a bottom tab bar 706. The search bar 702 further includes a search box and a search button. The title bar 703 further includes five titles: "Artist," "Playlist," "Charts," "Benefits," and "Audiobook." The first column 704 displays the content under the "Playlist" title in the title bar 703. On the main screen 102 of the mobile phone 100, the first content column 704 of the music app 103 displays six content items in a 2x3 arrangement: "30 Selected Songs Daily," "New Songs Daily," "Nostalgic Selections," "Dynamic Rock," "Karaoke Selections," and "Movie Classics," as well as the titles "Playlist Recommendations" and "More." The second column 705 includes the titles "Charts" and "More," as well as the content of the second column 705 (not displayed on the main screen 102 of the mobile phone 100, but displayed on the screen 203 of the in-vehicle system 200). The bottom tab bar includes controls such as "Recommendations," "My," and "Radio" (not displayed on the main screen 102 of the mobile phone 100, but displayed on the screen 203 of the in-vehicle system 200).

[0150] In some embodiments, the various controls in the interface of the music app 103 can be distributed in a tree structure, and each control contains its own attributes, such as (Id, control type, index number, text information, etc.). Figure 8 and Figure 9 This illustrates a tree structure for the control layout in the music app 103. Specifically, as shown... Figure 8 As shown, the background interface 701 is the lowest-level control in the music app 103. Represented as a control tree, the background interface 701 can be set as the root node. Under the root node, there can be a search bar 702, a title bar 703, a first category 704, and a second category 705. Below the search bar 702, title bar 703, etc., specific display content can be set. For example, below the search bar 702, there can be a search box and a search button. Figure 9Taking "RelativeLayout" as an example, RelativeLayout represents a relative layout. If we consider RelativeLayout as a node in a tree structure, it includes three child nodes: "id / application_bg(ApplicationBgImageView)", "id / sliding_layout(SlidingUpPanelLayout)", and "id / tab_layout(LinearLayout)". Among them, id / application_bg(ApplicationBgImageView) represents the image layout, id / sliding_layout(SlidingUpPanelLayout) represents the sliding layout, and id / tab_layout(LinearLayout) represents the display box layout. In other words, the relative layout RelativeLayout, the image layout, and the display box layout are distributed in a tree structure.

[0151] The Overlay configuration file corresponding to Music App 103 contains layout rules for modifying the layout of various controls in the Music App 103 interface. For example, regarding the layout rules for the first column 704, the Overlay application 105 can first search for the controls whose layout needs to be modified based on their attribute IDs. For example, the attribute ID (layout ID="704") of the first column 704 is used to identify the first content column 704, and the Overlay application 105 can locate the first column 704 through (id="704"). Then, the Overlay application 105 modifies the layout of the controls included in the first column 704 according to the modification method specified in the layout rules. For example, the first column 704 shown in Figure 10(a) includes six content items: "30 Selected Songs Daily", "New Songs Daily", "Nostalgic Selections", "Dynamic Rock", "Karaoke Selections", and "Classic Films", as well as two titles: "Playlist Recommendations" and "More". Before modification, the items were arranged in two rows and three columns, with the two titles "Playlist Recommendations" and "More" located above the six content items. The Overlay configuration file, designed for the screen parameters of the vehicle's 200-screen infotainment system, stipulated that after screen mirroring, the layout of the six content items in the first column 704 needed to be changed from two rows and three columns to one row and six columns, with "More" placed on the left side of the same row, and the title "Playlist Recommendations" placed above the other controls. The configuration file also specified the position of the first column 704 after the layout adjustment. The Overlay application 105 adjusted the control layout in the first column 704 according to the Overlay configuration file, as shown in Figure 10(b).

[0152] For example, the Overlay application 105 can locate the search bar 702 using (id="702"). Then, the Overlay application 105 modifies the layout of the controls included in the search bar 702 according to the control modification method specified in the layout rules. For example, as shown in Figure 10(a), the search bar 702 includes a search box and a search button. When displayed on the main screen 102 of the mobile phone 100, the search bar 702 is located at the top of the interface. The Overlay configuration file specifies that, based on the screen parameters of the vehicle infotainment system 200 screen, the search box of the search bar 702 needs to be reduced in size after screen mirroring, and the position of the search bar 702 needs to be adjusted to the upper right side of the vehicle infotainment system 200 screen. The Overlay application 105 adjusts the control layout in the first column 704 according to the Overlay configuration file, and the adjusted layout is shown in Figure 10(b).

[0153] Here, the Overlay configuration file corresponding to the music app 103 contains a layout rule for the search bar 702. The positioning control method in this layout rule includes the control type of the search bar 702 (layout control type = "input box"). Here, control type = "input box" represents the control type of the search bar 702. When the music app 103 contains only one control with control type = "input box", the search bar 702 can be uniquely identified by positioning control type = "input box". After the Overlay application 105 positions the search bar 702 by control type = "input box", the Overlay application 105 obtains the modification control method in the layout rule of the search bar 702 and modifies the search bar 702 by modifying the control method. In the embodiment of this application, as shown in FIG10(b), in the virtual screen 104, the position of the search bar 702 of the music app 103 changes from filling the entire horizontal area to being on the right. Meanwhile, to adapt to the screen of the vehicle's infotainment system 200, a bottom tab bar 706, including "Recommended," "My," and "Radio," was set up at the top before the search bar 702. The position of the controls here can be determined by the values ​​of parameters such as left-aligned, top-aligned, and right-aligned. Left-aligned refers to the size of the top-left corner of the control on the x-axis, top-aligned refers to the size of the top-left corner of the control on the y-axis, and right-aligned refers to the size of the top-right corner of the control on the x-axis.

[0154] The following example uses the preset search content in the search bar 702 of the music app 103 to illustrate the process of locating controls and modifying their display content and font size through text information. The Overlay configuration file corresponding to the music app 103 contains another layout rule for the search bar 702. The method for locating controls in this layout rule includes the text information of the search content of the search bar 702 (layout text information = "Jay Chou"), as shown in Figure 10(a). Here, the text information = "Jay Chou" represents the preset search content of the search bar 702. When the music app 103 contains only one control with the text information = "Jay Chou", the search content of the search bar 702 can be uniquely determined by locating the text information = "Jay Chou". After the Overlay application 105 locates the search content of the search bar 702 through the text information "Jay Chou", the Overlay application 105 obtains the modification control method in the layout rules of the search bar 702 and modifies the search bar 702 through the modification control method. In the embodiment of this application, as shown in FIG10(b), in the virtual screen 104, the font size of the text in the search content of the music APP 103's search bar 702 is increased by one size. In some embodiments, the color of the text in the search content can also be changed.

[0155] The modifications to the layout of other controls in the music app 103 are similar to those in the first section 704 and the search bar 702, and will not be repeated here.

[0156] Mobile phone 100 projects the interface of music app 103, modified via the Overlay configuration file on virtual screen 104, onto screen 203 of vehicle infotainment system 102, as shown in Figure 10(b). The layout of music app 103 on screen 203 of vehicle infotainment system 102 differs from that on mobile phone 100 (as shown in Figure 10(a)). Since the screen of vehicle infotainment system 200 is landscape and that of mobile phone 100 is portrait, the layout of controls in the music app 103 interface is adjusted to better suit the screen 203 of vehicle infotainment system 200.

[0157] Example 2

[0158] The above embodiment describes a scheme for adjusting the layout of the projected interface when the mobile phone 100 projects its display content (e.g., the interface of an application) onto a device with a screen larger than that of the mobile phone 100. The following describes a scheme for adjusting the layout of the projected interface when the mobile phone 100 projects its display content onto a device with a screen smaller than that of the mobile phone 100. For ease of explanation, we will take projecting the music app 103 onto the screen of the smartwatch 400 as an example. Figure 11As shown. When the content displayed by mobile phone 100 is projected onto a device with a screen smaller than that of mobile phone 100, this can be achieved entirely by creating a virtual screen as described above, since the technology used is the same, and will not be described again here. Here, we introduce another implementation method: projecting the content onto the main screen in a picture-in-picture format. Specifically, the solutions for displaying content via picture-in-picture include:

[0159] like Figure 11 As shown, firstly, after the mobile phone 100 obtains the screen projection parameters of the smartwatch 400, the mobile phone 100 compares the screen parameters (screen size, resolution, pixel density) in the screen projection parameters with the screen parameters of the mobile phone 100 itself. If it is determined that the screen size, resolution, and pixel density of the smartwatch 400 are smaller than the screen size, resolution, and pixel density of the mobile phone 100 itself, it is determined to display the interface of the music APP 103 with the adjusted layout on the main screen 102 of the mobile phone 100 in a picture-in-picture manner.

[0160] Secondly, once the phone 100 confirms that the picture-in-picture mode is enabled in the full-screen window of the music app 103, the phone 100 generates a window in the main screen 102, and the screen size, resolution, and pixel density of this window are the same as those of the smartwatch 400.

[0161] Then, the Overlay application 105 of the mobile phone 100 loads the Overlay configuration file of the music app 103 corresponding to the screen parameters of the smartwatch 400, and displays the interface of the music app 103 with the adjusted layout in the window (e.g., Figure 11 The first window 1110 is a full-screen window, while the second window 1120 is displayed within a specific area of ​​the main screen of the phone 100. The second window 1120 is smaller than the first window 1110 and has the same screen size, resolution, and pixel density as the smartwatch 400. The content and layout of the first and second windows differ. The first window 1110 displays the interface of the music app 103 suitable for the screen 102 of the phone 100, while the second window 1120 displays the music app 103 interface with its layout adjusted according to the overlay configuration file of the corresponding smartwatch 400.

[0162] For example, due to the limited screen size of the smartwatch 400, the layout rules in the Overlay configuration file corresponding to the smartwatch 400 stipulate that only some controls in the first column 704 of the music APP 103 of the mobile phone 100 are displayed in the second window 1120 in a column-multiple-row format. In addition, corresponding operation buttons can be added for the smartwatch 400, such as the back button 401, the pause / start button 402, and the forward button 403.

[0163] Example 3

[0164] Figures 12(a) and 12(b) respectively show the... Figure 2 The screen projection systems shown are different. In the systems shown in Figures 12(a) and 12(b), the screen projection system only includes electronic device 100 and remote device 200. The specific screen projection method is different from... Figure 2 The difference in the illustrated embodiment is that the cloud 300 is not required; its functions are implemented by the electronic device 100 or the remote device 200. Specifically, taking the mobile phone 100 and the vehicle-mounted system 200 as examples, the screen projection process includes:

[0165] 1) Mobile phone 100 communicates with vehicle infotainment system 200 via the above-mentioned wireless communication method. Users can enable screen mirroring function on mobile phone 100, or mobile phone 100 can automatically enable screen mirroring function.

[0166] 2) The client 201 installed on the vehicle infotainment system 200 sends the screen projection parameters of the vehicle infotainment system 200, such as screen size, resolution, pixel density, and device type, to the mobile phone 100. For details, please refer to [link to relevant documentation]. Figure 4 Description of 402.

[0167] 3) Based on the screen projection parameters obtained from the vehicle's infotainment system 200, mobile phone 100 creates a virtual screen 104 and configures a screen ID (Display ID) for the virtual screen 104. See details... Figure 4 Description of 403.

[0168] 4) When a user is using mobile phone 100, if the user opens music APP 103, the Overlay application 105 of mobile phone 100 will obtain the Overlay configuration file corresponding to the screen projection parameters sent by vehicle system 200 from multiple Overlay configuration files of music APP 103.

[0169] In the embodiment shown in Figure 12(a), the Overlay application 105 of the mobile phone 100 searches for and retrieves the Overlay configuration file corresponding to the screen projection parameters sent by the vehicle system 200 from the multiple Overlay configuration files of the corresponding music APP 103 in its own memory 108.

[0170] In the embodiment shown in Figure 12(b), the Overlay configuration file corresponding to the music APP 103 can be stored in the memory 204 of the vehicle system 200. While obtaining the screen projection parameters of the vehicle system 200, the mobile phone 100 can send parameters such as the package name and version number of the music APP 103 to the vehicle system 200. The vehicle system 200 searches for and obtains the Overlay configuration file corresponding to the package name, version number and screen projection parameters from its own memory 204, and then sends the Overlay configuration file to the mobile phone 100.

[0171] 5) Modify the layout of each control in the music app 103 based on the obtained Overlay configuration file, and then display the music app 103 with the modified layout in the virtual screen 104. Project the virtual screen 104 onto the screen 203 of the vehicle system 200. The vehicle system 200 displays the content on the virtual screen 104 on the screen 203.

[0172] In another embodiment of this application, the mobile phone 100 can also adjust the layout of its own system application (e.g., desktop APP) according to the screen projection parameters of the vehicle system 200, so that the layout of the system application is suitable for the screen of the vehicle system 200 after being projected onto the vehicle system 200.

[0173] The above embodiment uses APP 103 as a music APP to illustrate the screen casting solution of this application. In practical applications, in order to make the screen-cast interface more suitable for the user's usage scenario or user habits, it is necessary to make some major adjustments to the application interface, such as deleting certain display elements, adjusting the arrangement and viewing method of display elements, etc. The following uses instant messaging APP 103 as an example to continue to illustrate the screen casting solution of this application.

[0174] Example 4

[0175] The following example illustrates the technical solution of this application by projecting the contact interface of the instant messaging APP 103 from a mobile phone 100 to the vehicle's infotainment system 200.

[0176] Typically, in the contact interface of instant messaging APP 103, contacts are presented in a row-by-row format. For example, as shown in Figure 13(a), the contact information in the content bar 1303 of instant messaging APP 103 is displayed in a single-column, multi-row format (list format), and each contact is located on the same row with the contact's avatar and contact's name.

[0177] When the instant messaging app 103's contact interface is projected from the mobile phone 100 onto an electronic device with a significantly different screen size or shape, such as a vehicle infotainment system 200 (where the screen is landscape while the mobile phone 100's screen is portrait), a situation arises where very little contact information is displayed on a single page, and some areas of the vehicle infotainment system 200's screen remain unused. Therefore, in this embodiment, when the mobile phone 100 projects the instant messaging app 103 onto the vehicle infotainment system 200, the layout of the contacts in the content bar 1303 of the instant messaging app 103 is altered, as shown in Figure 13(b). The contacts in the content bar 1303 of the instant messaging app 103 are displayed using a single-row, multi-column or multi-row, multi-column arrangement (grid layout), and the contact icons and names are arranged vertically. Therefore, based on the screen characteristics of the vehicle infotainment system 100, the screen space of the vehicle infotainment system 200 is fully utilized. At the same time, unlike the mobile phone 100 where users can only browse contact information by swiping up and down, after the screen is projected onto the screen of the vehicle infotainment system 200, users can also browse all contacts in the content bar 1303 of the instant messaging APP 103 by swiping up and down and left and right (as shown in Figure 13(b)).

[0178] The process of mobile phone 100 projecting the contact interface of instant messaging APP 103 to vehicle system 200 includes:

[0179] 1) Based on the screen projection parameters obtained from the vehicle's infotainment system 200, mobile phone 100 creates a virtual screen 104. The overlay application 105 of mobile phone 100 obtains the overlay configuration file of the corresponding instant messaging app 103. This step is related to... Figure 4 The steps described in 401-404 are the same, and will not be repeated here.

[0180] 2) The Overlay application 105 of mobile phone 100 modifies the layout of instant messaging APP 103 according to the Overlay configuration file of instant messaging APP 103. The specific modification process can be referred to the description above, and the specific technical implementation methods are basically the same.

[0181] For example, in the content bar 1303 of instant messaging app 103, which includes specific contact information, modifying its layout requires changing the original single-column layout to a multi-row, multi-column layout. Furthermore, the arrangement of contact avatars and names on the same row needs to be changed so that the contact avatar is placed above the contact name. The specific modification process is as follows:

[0182] Overlay application 105 can first search for the control whose layout needs to be modified based on the control's attribute ID. For example, the attribute ID (layout ID="1303") of content bar 1303 is used to identify content bar 1303, and Overlay application 105 can locate content bar 1303 through (id="1303"). Then, Overlay application 105 modifies the layout of content bar 1303. For example, as shown in Figure 13(a), on the screen of mobile phone 100, content bar 1303 includes 9 contacts: "Zhang San", "Li Si", "Wang Wu", "Zhao Si", "Li Yi", "Lao Li", "Lao Wang", "Lao Zhang" (not shown), and "Lao Zhao" (not shown). The above 9 contacts are arranged in nine rows and one column, and in each column, the contact's icon and name are arranged horizontally from left to right. In the modified layout of the virtual screen 104, as shown in Figure 13(b), the Overlay application 105 can modify the content bar 1303 into a three-row, four-column layout. The first column includes "Zhang San," "Li Si," "Wang Wu," and "Zhao Si," the second column includes "Li Yi," "Lao Li," "Lao Wang," and "Lao Zhang," and the third column includes "Lao Zhao." Furthermore, the icons and names of each contact are arranged vertically. The virtual screen 104 can simultaneously display two rows and three columns of contacts from the content bar 1303. Users can view the remaining contacts by swiping vertically and horizontally on the vehicle's infotainment system 200 screen.

[0183] Meanwhile, the Overlay application 105 modifies the layout of the title bar 1301 and search bar 1302 of the instant messaging app 103, placing them on the same horizontal line. For the bottom tab bar 1304, the Overlay application 105 sets it to be hidden.

[0184] 3) Mobile phone 100 projects virtual screen 104 onto vehicle system 200.

[0185] Example 5

[0186] The following example illustrates the technical solution of this application by projecting the search function interface of the instant messaging APP 103 from a mobile phone 100 to the vehicle's infotainment system 200.

[0187] As shown in Figure 14(a), in the default layout of the instant messaging APP 103, when the search function of the instant messaging APP 103 is opened without entering any search keywords, a customized search range 1402 is displayed, such as "Moments" or "Articles," and as shown, the customized search range is displayed in a 2x3 arrangement. Furthermore, the search bar 1401 and the voice input key 1403 are located above and below the customized search range 1402, respectively. However, when the instant messaging APP 103 is projected from the mobile phone 100 onto the vehicle's infotainment system 200, considering that users using the vehicle's infotainment system 200 have less entertainment intent and prefer accurate search results, the customized search range 1402 can be deleted during projection, and the voice input key 1403 can be placed near the search bar, as shown in Figure 14(a). Figure 14b As shown, this makes the entire search interface simpler and more convenient for users to perform search operations.

[0188] Specifically, the process of mobile phone 100 projecting the search function interface of instant messaging APP 103 onto vehicle system 200 includes:

[0189] 1) Based on the screen projection parameters obtained from the vehicle's infotainment system 200, mobile phone 100 creates a virtual screen 104. The overlay application 105 of mobile phone 100 obtains the overlay configuration file of the corresponding instant messaging app 103. The above steps are similar to... Figure 4 The steps described in 401-404 are the same, and will not be repeated here.

[0190] 2) The Overlay application 105 of mobile phone 100 modifies the layout of instant messaging APP 103 according to the Overlay configuration file of instant messaging APP 103. The specific modification process can be referred to the description above, and the specific technical implementation methods are basically the same.

[0191] For example, regarding the customized search scope 1402 in the search function interface of the instant messaging APP 103, which includes items such as "Moments" and "Articles," when modifying its layout, it needs to be set to not be displayed, making the search function interface of the instant messaging APP 103 on the screen of the vehicle system 100 more concise. At the same time, the voice input key 1403 should be moved before the search bar 1401 and kept always visible. The specific modification process is as follows:

[0192] Overlay application 105 first locates the customized search range 1402 and the voice input key 1403 using (id="1402") and (id="1403"). Then, Overlay application 105 modifies the layout of the customized search range 1402, for example, as follows: Figure 14aAs shown, on the screen of mobile phone 100, the customized search scope 1402 includes content such as "Moments," "Articles," and "Official Accounts," and is displayed in a two-row, three-column format below the search bar 1401; the voice input key 1403 is located below the customized search scope 1402. In the virtual screen 104, as shown in Figure 13(b), the Overlay application 105 can modify the voice input key 1403 to be positioned before the search bar 1401, placing it on the same horizontal line, left-aligned and top-aligned. The voice input key 1403 only includes the voice input icon, removing the icon's description, and its visibility attribute is modified to remain always visible, even when search results are displayed. For the customized search scope 1402, the Overlay application 105 disables its display.

[0193] 3) Mobile phone 100 projects virtual screen 104 onto vehicle system 200.

[0194] Figure 14(b) shows that on the screen of mobile phone 100, after the user enters the search term "Zhang" in the search bar 1401 and clicks search, the search results bar 1404 displays the search results in a multi-column arrangement. The search results may include: contacts, groups, and other content such as articles and photos (not shown), and each search result may include an icon and specific information. The steps for Overlay application 105 to modify the search results bar 1404 are also shown. Figure 14a As described above, to display search results more concisely on the screen of the vehicle infotainment system 200, the search results in the search results bar 1404 are displayed in a multi-row, multi-column format on the screen of the vehicle infotainment system 200. This can be achieved using the method described above. Figure 13b The grid layout in the settings allows you to change how search results are arranged. Additionally, it hides icons in the search results, displaying only the specific information for each result.

[0195] Example 6

[0196] The following example illustrates the technical solution of this application by projecting the audio and video call interface of the instant messaging APP 103 from a mobile phone 100 to the vehicle's infotainment system 200.

[0197] As shown in Figure 15(a), for the default audio and video call function of the instant messaging APP 103 on mobile phone 100, after the user clicks the contact icon of the instant messaging APP 103 (as shown in Figure 15(a)(1)), the user will enter the contact information interface (as shown in Figure 15(a)(2)). In the contact information interface, the user needs to click the button "Audio / Video Call 1504" to start an audio / video call. As shown in Figure 15(a)(3), after clicking the button "Audio / Video Call 1504", a selection menu will appear, prompting the user to select video call or audio call. Then, as shown in Figure 15(a)(4), if the user selects video call, the instant messaging APP 103 will enter the video call connection interface with the contact.

[0198] Based on the above description, it can be seen that when making audio and video calls through the instant messaging APP 103, the selection process is cumbersome. After clicking on a contact, users need to click the "Audio / Video Call 1504" button and select either video or audio call from the menu before entering the video or audio call connection interface. When users use the vehicle's infotainment system 200 to make audio and video calls using the instant messaging APP 103, this cumbersome operation will cause inconvenience. Therefore, when projecting the instant messaging APP 103 from the mobile phone 100 onto the vehicle's infotainment system 200, the audio and video call operation can be simplified by using simplified operation buttons or menu selection.

[0199] Specifically, Figures 15(b)(1) to 15(b)(3) illustrate a simplified screen mirroring method. As shown in Figure 15(b)(1), when the mobile phone 100 mirrors the instant messaging APP 103 to the vehicle's infotainment system 200, the user searches for the contact to make an audio / video call and clicks the contact icon. Then, the vehicle's infotainment system 200 sends the contact information selected by the user to the mobile phone 100. The screen of the mobile phone 100 displays the interfaces shown in Figures 15(a)(2) and 15(a)(3), while the interface mirrored to the vehicle's infotainment system 200 is only one, namely the interface shown in Figure 15(b)(2), which directly displays the buttons "Video Call 15041" and "Audio Call 15042". Thus, the user can directly click the button "Video Call 15041" or the button "Audio Call 15042" to enter the corresponding contact interface, as shown in Figure 15(b)(3).

[0200] Specifically, the process of mobile phone 100 projecting the contact information interface of instant messaging APP 103 to vehicle system 200 includes:

[0201] 1) Based on the screen projection parameters obtained from the vehicle's infotainment system 200, mobile phone 100 creates a virtual screen 104. The overlay application 105 of mobile phone 100 obtains the overlay configuration file of the corresponding instant messaging app 103. The above steps are similar to... Figure 4The steps described in 401-404 are the same, and will not be repeated here.

[0202] 2) The Overlay application 105 of mobile phone 100 modifies the layout of the contact information interface of instant messaging APP 103 according to the Overlay configuration file of instant messaging APP 103. The specific modification process can be referred to the description above, and the specific technical implementation methods are basically the same.

[0203] For example, the audio / video call button 1504 on the contact information interface of instant messaging app 103 includes options for both video and voice calls. When modifying its layout, it's necessary to set video and voice calls as separate buttons and replace audio / video call button 1504. The specific modification process is as follows:

[0204] Overlay application 105 can first locate audio / video call 1504 using (id="1504"). Then, Overlay application 105 modifies the layout of audio / video call 1504. On the screen of phone 100, after the user clicks on audio / video call 1504, the user will be prompted to choose whether to make a video call or a voice call. In virtual screen 104, Overlay application 105 first locates audio / video call 1504, and then obtains the video call and voice call functions it includes. For example, if the control type for video call and voice call is "button", and the control IDs can be "15041" and "15042" respectively, then audio / video call 1504 is replaced with video call "15041" and voice call "15042", and respective icons are added to video call and voice call. At the same time, after locating the contact icon and contact name 1501, Overlay application 105 modifies them to be arranged vertically, and then sets other contact information 1502 and sending messages 1503 to be hidden.

[0205] 3) Mobile phone 100 projects virtual screen 104 onto vehicle system 200.

[0206] When the user presses the video call button on the screen of the vehicle infotainment system 200, the instant messaging APP 103 of the vehicle infotainment system 200 enters the video call connection interface with the contact. At this time, the interface includes: the contact icon and contact name 1501, prompt information, and control bar 1505, including: cancel and switch to voice buttons.

[0207] Example 7

[0208] The following example illustrates the technical solution of this application by projecting the video call interface of the instant messaging APP 103 from a mobile phone 100 to the vehicle's infotainment system 200.

[0209] like Figure 16As shown, when mobile phone 100 makes a video call through instant messaging APP 103, the video frame 1602 of one party in the video call interface is displayed as a small window on the screen of mobile phone 100, while the video frame 1601 of the other party basically fills the entire screen. For vehicle system 200, if its screen is a large widescreen, for example, if the aspect ratio of the screen of vehicle system 200 exceeds 1.78, then in order to improve the video call experience, the size of the video frame 1602 when projecting can be increased. For example, the video frame 1601 and the video frame 1602 can be displayed in a split-screen manner on the screen of vehicle system 200, for example, by splitting the screen in a 2:1 or 1:1 ratio.

[0210] Specifically, the process of mobile phone 100 projecting the contact video call interface of instant messaging APP 103 to vehicle system 200 includes:

[0211] 1) Based on the screen projection parameters obtained from the vehicle's infotainment system 200, mobile phone 100 creates a virtual screen 104. The overlay application 105 of mobile phone 100 obtains the overlay configuration file of the corresponding instant messaging app 103. The above steps are similar to... Figure 4 The steps described in 401-404 are the same, and will not be repeated here.

[0212] 2) The Overlay application 105 on mobile phone 100 modifies the layout of the contact video call interface of instant messaging APP 103 according to the Overlay configuration file of instant messaging APP 103. The specific modification process can be referred to the description above, and the specific technical implementation methods are basically the same.

[0213] For example, in the contact video call interface of instant messaging APP 103, video frame 1602 is usually located within video frame 1601. When modifying its layout, based on the screen size of the vehicle infotainment system 200, video frames 1601 and 1602 can be changed to a split-screen display. The specific modification process is as follows:

[0214] Overlay application 105 can first locate the contact video call interface 1600 of instant messaging APP 103 via (id="1600"), and then modify the layout of this interface. On the screen of phone 100, such as... Figure 16As shown, video frame 1602 is displayed as a small window in the upper right corner of the screen of mobile phone 100. In virtual screen 104, Overlay application 105 sets the contact video call interface 1600 to a split-screen display format, and then positions video frames 1601 and 1602 using (id="1601") and (id="1602"), displaying video frames 1601 and 1602 in the contact video call interface of instant messaging APP 103 from left to right. For example, if the aspect ratio of the screen of vehicle system 200 exceeds 1.78, video frames 1601 and 1602 can be displayed in a split-screen format on the screen of vehicle system 200 with a display ratio of 2:1.

[0215] 3) Mobile phone 100 projects virtual screen 104 onto vehicle system 200.

[0216] In some embodiments, based on the screen size of the vehicle infotainment system 200, video frames 1602 and 1601 can be displayed in a 1:1 split ratio (the screen of the vehicle infotainment system 200 is an ultra-wide screen with an aspect ratio exceeding 2.33), or they can be displayed in the same way as on the screen of the mobile phone 100, with video frame 1602 displayed as a small window within video frame 1601. Furthermore, the contact video call interface may also include a control bar 1603, which may include buttons for switching cameras, hanging up, and switching to voice calls.

[0217] Example 8

[0218] The following is another technical solution for projecting the video call interface of the instant messaging APP 103 from mobile phone 100 to the vehicle's infotainment system 200.

[0219] As shown in Figure 17(a), the contact video call interface of the instant messaging APP 103 on mobile phone 100 and... Figure 16 The same as in, and Figure 16 The difference lies in the fact that in the video call interface of the instant messaging APP 103 on mobile phone 100, the video frame 1701 is displayed in landscape mode. If the video frame 1701 is still displayed in landscape mode after the mobile phone 100 projects it onto the vehicle system 200, it will affect the user's video call experience. Therefore, after projection, the mobile phone 100 has video adjustment buttons 1703 and 1704 configured in the video frames 1701 and 1702 respectively. As shown in Figure 17(b), after the user clicks the video adjustment button 1703 in the video frame 1701 of the instant messaging APP 103 in the vehicle system 200, the video frames 1701 of the vehicle system 200 and the video frames 1701 of the mobile phone 100 are simultaneously adjusted to portrait mode.

[0220] The process of mobile phone 100 projecting the contact video call interface of instant messaging APP 103 to vehicle system 200 includes:

[0221] 1) Based on the screen projection parameters obtained from the vehicle's infotainment system 200, mobile phone 100 creates a virtual screen 104. The overlay application 105 of mobile phone 100 obtains the overlay configuration file of the corresponding instant messaging app 103. The above steps are similar to... Figure 4 The steps described in 401-404 are the same, and will not be repeated here.

[0222] 2) The Overlay application 105 on mobile phone 100 modifies the layout of the contact video call interface of instant messaging APP 103 according to the Overlay configuration file of instant messaging APP 103. The specific modification process can be referred to the description above, and the specific technical implementation methods are basically the same.

[0223] For example, in the contact video call interface of instant messaging APP 103, the video frame 1701 is displayed in landscape mode. When modifying its layout, a video adjustment button 1703 is added inside the video frame 1701, allowing the user to click this button to change the orientation of the video frame 1701. The specific modification process is as follows:

[0224] Overlay application 105 can first locate the video frame 1701 using (id="1701"). Then, Overlay application 105 modifies the layout of this interface. On the screen of mobile phone 100, as shown in Figure 17(a), the video frame 1701 is displayed in landscape mode. In virtual screen 104, Overlay application 105 adds a video adjustment button 1703 to the video frame 1701 and sets its position to top left. The control type of the video adjustment button 1703 can be "button".

[0225] 3) Mobile phone 100 projects virtual screen 104 onto vehicle infotainment system 200. When the user clicks the video adjustment button 1703 on the screen of vehicle infotainment system 200, vehicle infotainment system 100 sends the command triggered by clicking video adjustment button 1703 to mobile phone 100. After receiving the command, mobile phone 100 obtains the "rotation" attribute of the video frame 1701 of instant messaging APP 103 on its own screen, and modifies the attribute to rotate the video frame 1701 to portrait mode, for example, by changing the content of the "rotation" attribute from "horizontal" to "vertical". At the same time, the overlay application 105 of mobile phone 100 also modifies the "rotation" attribute of the video frame 1701 of instant messaging APP 103 in virtual screen 104, so that the video frame 1701 of instant messaging APP 103 in virtual screen 104 is also rotated to portrait mode. Afterwards, the instant messaging APP 103 in the virtual screen 104 will also be updated in real time on the screen of the vehicle system 100.

[0226] The same function can be achieved by adding a video adjustment button 1704 to the video frame 1702 using the method described above. As shown in Figure 17(c), in the contact video call interface of the instant messaging APP 103 on the mobile phone 100, the video frame 1702 is displayed in landscape mode. After the contact video call interface of the instant messaging APP 103 on the mobile phone 100 is projected onto the vehicle system 200, the user can click the video adjustment button 1704 of the video frame 1702 of the contact video call interface of the instant messaging APP 103 on the vehicle system 200, as shown in Figure 17(d), to adjust the video frame 1702 of the vehicle system 200 and the video frame 1702 of the mobile phone 100 to be displayed in portrait mode at the same time.

[0227] In some other embodiments, as shown in FIG17(e), in the contact video call interface of the instant messaging APP 103 on the mobile phone 100, both video frames 1701 and 1702 are displayed in landscape mode. After the contact video call interface of the instant messaging APP 103 on the mobile phone 100 is projected onto the vehicle system 200, the Overlay application 105 of the mobile phone 100 sets a video adjustment button 1705 in the contact video call interface of the instant messaging APP 103. The user can adjust the video frames 1701 and 1702 of the vehicle system 200 and the mobile phone 100 to be displayed in portrait mode by clicking the video adjustment button 1705 in the contact video call interface of the instant messaging APP 103 on the vehicle system 200, as shown in FIG17(f).

[0228] Figure 18 A schematic diagram of a mobile phone 100 is shown.

[0229] like Figure 18As shown, the mobile phone 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0230] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the mobile phone 100. In other embodiments of this application, the mobile phone 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0231] Processor 110 may include one or more processing units, such as application processors (APs), modem processors, graphics processing units (GPUs), image signal processors (ISPs), controllers, video codecs, digital signal processors (DSPs), baseband processors, and / or neural network processing units (NPUs). These different processing units may be independent devices or integrated into one or more processors.

[0232] The controller can generate operation control signals based on the instruction opcode and timing signals to complete the control of instruction fetching and execution.

[0233] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0234] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0235] The MIPI interface can be used to connect the processor 110 to peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI) and a display serial interface (DSI). In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to enable the mobile phone 100 to take pictures. The processor 110 and the display screen 194 communicate via the DSI interface to enable the mobile phone 100 to display.

[0236] USB interface 130 is an interface that conforms to the USB standard specification, specifically it can be a Mini USB interface, Micro USB interface, USB Type C interface, etc.

[0237] It is understood that the interface connection relationships between the modules illustrated in the embodiments of this application are merely illustrative and do not constitute a structural limitation on the mobile phone 100. In other embodiments of this application, the mobile phone 100 may also adopt different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0238] The charging management module 140 receives charging input from the charger. The power management module 141 connects the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, internal memory 121, display screen 194, camera 193, and wireless communication module 160, etc. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage current, impedance).

[0239] The wireless communication function of mobile phone 100 can be realized through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor.

[0240] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals.

[0241] The mobile communication module 150 can provide solutions for wireless communication applications including 2G / 3G / 4G / 5G on the mobile phone 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low-noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 1. In some embodiments, at least some functional modules of the mobile communication module 150 may be housed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 may be housed in the same device.

[0242] The modem processor may include a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs sound signals through an audio device (not limited to speaker 170A, receiver 170B, etc.) or displays images or videos through the display screen 194. In some embodiments, the modem processor may be a separate device. In other embodiments, the modem processor may be independent of the processor 110 and may be housed in the same device as the mobile communication module 150 or other functional modules.

[0243] The wireless communication module 160 can provide solutions for wireless communication applications on the mobile phone 100, including wireless local area networks (WLAN) (such as Wi-Fi), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR). The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signal, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0244] In the embodiments of this application, after the mobile phone 100 communicates with the vehicle system 200 through the wireless communication module 160, it obtains parameters such as screen size, resolution, pixel density, and device type from the vehicle system 200.

[0245] In some embodiments, antenna 1 of mobile phone 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, enabling mobile phone 100 to communicate with networks and other devices via wireless communication technology. The wireless communication technology may include Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technologies, etc. The GNSS may include the Global Positioning System (GPS), the Global Navigation Satellite System (GLONASS), the BeiDou Navigation Satellite System (BDS), the Quasi-Zenith Satellite System (QZSS), and / or satellite-based augmentation systems (SBAS).

[0246] The mobile phone 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0247] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a miniature LED, a microLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, mobile phone 100 may include one or N displays 194, where N is a positive integer greater than 1.

[0248] In the embodiments of this application, the display screen 194 can be used to display a virtual screen 104 generated based on parameters such as the screen size, resolution, pixel density, and device type of the vehicle infotainment system 200. When the size of the virtual screen 104 is smaller than that of the display screen 194, the display screen 194 can display the virtual screen 104 in a picture-in-picture manner.

[0249] The mobile phone 100 can achieve shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor.

[0250] The external storage interface 120 can be used to connect an external storage card, such as a Micro SD card, to expand the storage capacity of the mobile phone 100. The external storage card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, music, video, and other files can be saved on the external storage card.

[0251] Internal memory 121 can be used to store computer executable program code, which includes instructions. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.), etc. The data storage area may store data created during the use of mobile phone 100 (such as audio data, phonebook, etc.). Furthermore, internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc. Processor 110 executes various functional applications and data processing of mobile phone 100 by running instructions stored in internal memory 121 and / or instructions stored in memory located in the processor. Internal memory 121 can be used to store overlay configuration files corresponding to application programs.

[0252] The mobile phone 100 can achieve audio functions such as music playback and recording through the audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

[0253] The audio module 170 is used to convert digital audio information into analog audio signals for output, and also to convert analog audio input into digital audio signals. The audio module 170 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 170 may be located in the processor 110, or some functional modules of the audio module 170 may be located in the processor 110.

[0254] Touch sensor 180K, also known as a "touch device," can be located on display screen 194. The touch sensor 180K and display screen 194 together form a touchscreen, also known as a "touchscreen." Touch sensor 180K detects touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 194. In other embodiments, touch sensor 180K may also be located on the surface of mobile phone 100, in a different position than display screen 194.

[0255] Keypad 190 includes a power button, volume buttons, etc. Keypad 190 can be a mechanical keypad or a touch keypad. Mobile phone 100 can receive keypad input and generate key signal inputs related to user settings and function control of mobile phone 100.

[0256] Motor 191 can generate vibration alerts. Indicator 192 can be an indicator light, used to indicate charging status, battery level changes, messages, missed calls, notifications, etc. SIM card interface 195 is used to connect a SIM card.

[0257] Figure 19 This is a software structure block diagram of a mobile phone 100 according to an embodiment of this application.

[0258] A layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the Android runtime and system libraries, and the kernel layer.

[0259] The application layer can include a series of application packages.

[0260] Figure 19 As shown, the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and SMS.

[0261] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0262] Figure 19 As shown, the application framework layer may include a window manager, content provider, view system, phone manager, resource manager, notification manager, etc.

[0263] The window manager is used to manage windowed applications. It can retrieve screen size, determine the presence of a status bar, lock the screen, and capture screenshots, among other things.

[0264] Content providers store and retrieve data, making that data accessible to applications. This data may include videos, images, audio, made and received phone calls, browsing history and bookmarks, phone books, etc.

[0265] A view system includes visual controls, such as controls for displaying text and controls for displaying images. View systems can be used to build applications. An interface can consist of one or more views. For example, an interface including a text notification icon can include views for displaying text and views for displaying images. For instance, the aforementioned mobile phone 100 can use the view system to create or update layout rules for its installed applications and save these layout rules in a configuration file.

[0266] In the embodiments of this application, the view system can be used to modify the layout style of the music APP103 corresponding to the Overlay configuration file based on the obtained Overlay configuration file.

[0267] The phone manager is used to provide communication functions for the mobile phone 100. For example, it manages call status (including connection, hang-up, etc.).

[0268] The file explorer provides applications with various resources, such as localized strings, icons, images, layout files, video files, and more.

[0269] In embodiments of this application, the resource manager can also be used to store overlay configuration files.

[0270] The notification manager allows applications to display notifications in the status bar. These notifications can be used to deliver informational messages and can disappear automatically after a short pause, requiring no user interaction. For example, the notification manager can be used to notify users of completed downloads or message alerts. The notification manager can also display notifications as icons or scrolling text in the top status bar, such as notifications from background applications, or as dialog boxes on the screen. Examples include displaying text messages in the status bar, emitting sounds, vibrating electronic devices, and flashing indicator lights.

[0271] AndroidRuntime consists of core libraries and a virtual machine. AndroidRuntime is responsible for the scheduling and management of the Android system.

[0272] The core library consists of two parts: one part is the functionalities that need to be called by the Java language, and the other part is the Android core library.

[0273] The application layer and application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0274] System libraries can include multiple functional modules. For example: surface manager, media libraries, 3D graphics processing libraries (e.g., OpenGL ES), 2D graphics engines (e.g., SGL), etc.

[0275] The Surface Manager is used to manage the display subsystem and provides the blending of 2D and 3D layers for multiple applications.

[0276] The media library supports playback and recording of various common audio and video formats, as well as still image files. It supports multiple audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG.

[0277] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0278] A 2D graphics engine is a graphics engine for 2D drawing.

[0279] The kernel layer is the layer between hardware and software. The kernel layer contains at least the display driver, camera driver, audio driver, and sensor driver.

[0280] Now for reference Figure 20 The diagram shown is a block diagram of a system 2000 according to an embodiment of this application. Figure 20 An example system 500 according to several embodiments is schematically illustrated. In one embodiment, system 2000 may include one or more processors 2004, system control logic 2008 connected to at least one of the processors 2004, system memory 2012 connected to system control logic 2008, non-volatile memory (NVM) 2016 connected to system control logic 2008, and network interface 2020 connected to system control logic 2008.

[0281] In some embodiments, processor 2004 may include one or more single-core or multi-core processors. In some embodiments, processor 2004 may include any combination of general-purpose processors and special-purpose processors (e.g., graphics processors, application processors, baseband processors, etc.).

[0282] NVM / Memory 2016 may include one or more tangible, non-transitory readable media for storing data and / or instructions. In some embodiments, NVM / Memory 2016 may include any suitable non-volatile memory such as flash memory and / or any suitable non-volatile storage device, such as at least one of HDD (Hard Disk Drive), CD (Compact Disc) drive, and DVD (Digital Versatile Disc) drive. In some embodiments, NVM / Memory 2016 may include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute the present application as described above. Figure 4 All or part of the steps of the method shown.

[0283] NVM / Storage 2016 may include a portion of the storage resources on the device on which System 2000 is installed, or it may be accessible by the device, but is not necessarily part of the device. For example, NVM / Storage 2016 may be accessed over a network via Network Interface 2020.

[0284] Specifically, system memory 2012 and NVM / memory 2016 may each include a temporary copy and a permanent copy of instruction 2024. Instruction 2024 may include: when executed by at least one of processors 2004, causing system 2000 to implement, as Figure 4 The instructions for the method shown. In some embodiments, the instructions 2024, hardware, firmware and / or their software components may additionally / alternatively be located in the system control logic 2008, the network interface 2020 and / or the processor 2004.

[0285] Network interface 2020 may include a transceiver for providing a radio interface to system 2000, thereby enabling communication with any other suitable device (such as a front-end module, antenna, etc.) via one or more networks. In some embodiments, network interface 2020 may be integrated into other components of system 2000. For example, network interface 2020 may be integrated into at least one of processor 2004, system memory 2012, NVM / memory 2016, and firmware device with instructions (not shown). System 2000 may further include input / output (I / O) device 2032. I / O device 2032 may include a user interface enabling a user to interact with system 2000; the peripheral component interface is designed to allow peripheral components to also interact with system 2000.

[0286] Through the above description of the embodiments, those skilled in the art will understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0287] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another apparatus, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0288] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0289] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0290] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0291] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for projecting a screen onto an electronic device, characterized in that, include: The first electronic device displays the first display interface of the first application to the user on the display screen, and obtains the first projection parameters from the second electronic device, wherein the first projection parameters include the size of the first projection area used by the second electronic device to receive the projection and the model of the second electronic device; Based on the first projection parameters obtained, the first electronic device creates a virtual screen or picture-in-picture window with the same size as the first projection area when the size of the first projection area is smaller than the screen size of the display of the first electronic device. Based on the user's usage habits of the display elements in the projection interface and the model of the second electronic device, the layout of at least one display element in the first display interface is adjusted to generate a first projection interface suitable for the size of the first projection area. The first electronic device projects the first projection interface onto the second electronic device; When the first electronic device detects a modification window operation on the first projection area, it obtains the modified first projection parameters, wherein the modified first projection parameters include the model of the second electronic device and the size of the modified first projection area used by the second electronic device to receive projection. According to the modified first projection parameters, when the size of the first projection area is smaller than the screen size of the display of the first electronic device, the first electronic device creates a virtual screen or picture-in-picture window corresponding to the size of the modified first projection area. Based on the user's usage habits of the display elements in the projection interface and the model of the second electronic device, the layout of at least one display element in the first display interface is adjusted to generate a modified first projection interface suitable for the size of the modified first projection area. The first electronic device projects the modified first projection interface onto the second electronic device.

2. The method according to claim 1, characterized in that, The size of the first projection area of ​​the second electronic device is equal to or smaller than the size of the display screen of the second electronic device.

3. The method according to claim 1, characterized in that, The first projection parameters also include at least one of the display resolution and pixel density of the second electronic device.

4. The method according to claim 1, characterized in that, The first electronic device adjusts the layout of the display elements in the first display interface in the following way: Modify the position of the displayed elements in the first display interface; Scaling the display elements in the first display interface; Rotate the display elements in the first display interface; Modify the viewing method of the displayed elements in the first display interface; Delete at least one display element in the first display interface.

5. The method according to claim 1, characterized in that, Also includes: The first electronic device, in response to an operation instruction received from the second electronic device, modifies the display content in the first projection interface corresponding to the first display interface, wherein the operation instruction is generated by the second electronic device in response to the user's operation on the display elements in the first projection interface on the display screen of the second electronic device.

6. The method according to claim 1, characterized in that, The first electronic device is capable of changing the displayed content of the first display interface in response to a user's operation on a first display element in the first display interface, and the change includes displaying a second display element in the first display interface. The first electronic device is also capable of responding to a user's operation on the second display element in the first display interface, further changing the display content in the first display interface, and the change includes displaying a third display element corresponding to the second display element in the first display interface; and The first electronic device adjusts the size of the first display interface and the layout of at least one display element in the first display interface according to the acquired first projection parameters, to generate a first projection interface of a size suitable for the first projection area, including: The first electronic device, in response to a first operation instruction received from the second electronic device, modifies the display content of the first projection interface corresponding to the first display interface, wherein the modification includes displaying a third display element in the first projection interface, wherein the first operation instruction is generated by the second electronic device in response to the user's operation on the first display element in the first projection interface on the display screen of the second electronic device; The first electronic device projects a first projection interface, including the third display element, onto the second electronic device.

7. The method according to claim 1, characterized in that, The first electronic device adjusts the size of the first display interface and the layout of at least one display element in the first display interface according to the acquired first projection parameters, to generate a first projection interface of a size suitable for the first projection area, including: The first electronic device adds a fifth display element corresponding to the fourth display element to the first projection interface, wherein the first display interface includes the fourth display element but does not include the fifth display element; and The method further includes: The first electronic device, in response to a second operation command received from the second electronic device, adjusts the layout of the fourth display element in the first projection interface, wherein... The second operation instruction is generated by the second electronic device in response to the user's operation on the fifth display element in the first projection interface on the display screen of the second electronic device.

8. The method according to claim 1, characterized in that, The first electronic device adjusts the size of the first display interface and the layout of at least one display element in the first display interface according to the acquired first projection parameters, to generate a first projection interface of a size suitable for the first projection area, including: The first electronic device obtains a layout configuration file corresponding to the first projection parameters; The first electronic device modifies the first display interface of the first application to the first projection interface according to the layout configuration file.

9. The method according to claim 8, characterized in that, There is a correspondence between the layout configuration file and the application identifier, version number, and projection parameters of the application on the first electronic device. The same version of the same application corresponds to different projection parameters for different sized projection areas, and different projection parameters for the same version of the same application correspond to different configuration files.

10. The method according to claim 8, characterized in that, The first electronic device obtains a layout configuration file corresponding to the first projection parameters, including: The first electronic device obtains the first application identifier and the first application version number from the installation file of the first application; The first electronic device selects the layout configuration file corresponding to the first screen projection parameter from multiple layout configuration files by matching the identifier of the first application, the version number of the first application, and the first screen projection parameter.

11. The method according to claim 8, characterized in that, The layout configuration file includes the identifiers of the display elements in the first display interface and the layout rules corresponding to the display elements.

12. The method according to any one of claims 1 to 11, characterized in that, Also includes: The first electronic device displays a second display interface of the second application to the user on the display screen and obtains second projection parameters from the second electronic device, wherein the second projection parameters include the size of the second projection area used by the second electronic device to receive projection; The first electronic device adjusts the size of the second display interface and the layout of at least one display element in the second display interface according to the acquired second projection parameters, so as to generate a second projection interface with a size suitable for the second projection area. The first electronic device simultaneously projects the first projection interface and the second projection interface onto the second electronic device.

13. The method according to any one of claims 1 to 11, characterized in that, The first application can be any one of the following: music application, instant messaging application, news application, shopping application, or video playback application.

14. A method for projecting a screen onto an electronic device, characterized in that, include: The second electronic device sends a first projection parameter to the first electronic device, wherein the first projection parameter includes the size of the first projection area used by the second electronic device to receive the projection and the model of the second electronic device; The second electronic device displays the first projection interface sent by the first electronic device in the first projection area on the display screen. The first projection interface is generated by the first electronic device according to the first projection parameters, when the size of the first projection area is smaller than the screen size of the display screen of the first electronic device, by creating a virtual screen or picture-in-picture window with the same size as the first projection area, and adjusting the layout of at least one display element in the first display interface according to the user's usage habits of the display elements in the projection interface and the model of the second electronic device. The second electronic device detects the user's operation on the display elements of the first projection interface on the display screen; The second electronic device generates an operation instruction in response to the operation and sends the operation instruction to the first electronic device, wherein the operation instruction is used to instruct the first electronic device to modify the displayed content in the first projection interface in accordance with the user operation.

15. The method according to claim 14, characterized in that, The size of the first projection area of ​​the second electronic device is equal to or smaller than the size of the display screen of the second electronic device.

16. The method according to claim 14, characterized in that, The first projection parameters also include at least one of the display resolution and pixel density of the second electronic device.

17. An electronic device, characterized in that, include: Display screen; Memory, which stores instructions; A processor coupled to a memory, wherein when program instructions stored in the memory are executed by the processor, the electronic device controls the display screen to perform the projection method according to any one of claims 1 to 16.

18. A readable medium storing instructions, characterized in that, When the instruction is executed on the readable medium, the readable medium causes the screen projection method as described in any one of claims 1 to 16 to be performed.

Images