Screen projection method, system and electronic device

Abstract

A screen projection method, system and electronic device. In the method, a user can drag a screen projection window in a remote device to a local device. For example, the user can use a mouse to drag the screen projection window in the remote device to the local device. The user can drag the screen projection window from the local device back to the remote device. The user can adjust the size of the screen projection window displayed in the local device. By implementing the technical solutions provided in the present application, the user operation is simple when projecting the screen.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to screen projection methods, systems and electronic devices applied to electronic devices. Background Technology

[0002] Currently, there are increasingly more scenarios where electronic devices are interconnected. One example is multi-screen collaboration. In this scenario, electronic device 1 can project content displayed on its screen (e.g., a video playback window, or a chat window) onto electronic device 2. In this way, electronic device 2 (e.g., a tablet) can display the content shown on the screen of electronic device 1 (e.g., a mobile phone).

[0003] In existing technology, users can project a window from electronic device 1 onto the display screen of electronic device 2, and then return the projected window from electronic device 2 to electronic device 1. However, this operation is cumbersome, resulting in a poor user experience.

[0004] Therefore, how to conveniently project a window from electronic device 1 onto electronic device 2, and how to return a window projected onto electronic device 2 to electronic device 1, are problems that urgently need to be solved. Summary of the Invention

[0005] This application provides a screen mirroring method, system, and electronic device. The screen mirroring method allows users to drag a window from one electronic device to another, and vice versa. This allows users to easily and conveniently mirror a window from one electronic device to another, improving the user experience.

[0006] In a first aspect, a screen mirroring method is provided. This method can be applied to a screen mirroring system, which may include a first electronic device and a second electronic device. The first electronic device and the second electronic device establish a communication connection. The method may include: the first electronic device detecting a first drag operation, the first drag operation being used to move a first screen mirroring window from the first electronic device to the screen of the second electronic device; in response to the first drag operation, the first electronic device acquiring a first image of the first screen mirroring window, the first image including the display content of the first screen mirroring window that has been dragged out of the screen of the first electronic device; the first electronic device sending the first image to the second electronic device; and the second electronic device displaying the first image on the screen of the second electronic device.

[0007] Using the screen mirroring method described in the first aspect above, users can drag windows from the first electronic device to the second electronic device. Users can simply drag and drop windows from the first electronic device to the second. This simplifies the user experience and enhances user engagement.

[0008] In conjunction with the first aspect, in one possible implementation, the first image includes a portion or all of the display content of the first projection window. This allows a user to project a portion of the first projection window from the first electronic device to the second electronic device, or to project the entire first projection window to the second electronic device.

[0009] In conjunction with the first aspect, in one possible implementation, the first electronic device sends a first image to a second electronic device, including: the first electronic device sending the first image and first location information to the second electronic device; the first location information is used to determine the position of the first image displayed on the screen of the second electronic device. In this way, the second electronic device can determine the specific position where the first image is displayed.

[0010] In conjunction with the first aspect, in one possible implementation, the second electronic device displays a first image on the display screen of the second electronic device, comprising: the second electronic device displays the first image on the display screen of the second electronic device according to first position information.

[0011] In conjunction with the first aspect, in one possible implementation, the first electronic device sends the first image to the second electronic device, including: the first electronic device overlays the first image onto a transparent background image to obtain an overlaid image; the transparent background image is generated by the first electronic device according to the size of the display screen of the second electronic device, and the size of the transparent background image is the same as the size of the display screen of the second electronic device; the transparent background image is not displayed on the first electronic device; and the first electronic device sends the overlaid image to the second electronic device.

[0012] In conjunction with the first aspect, in one possible implementation, the method may further include: a first electronic device detecting a second drag operation, the second drag operation being used to move a second projection window displayed on the display screen of the first electronic device from the display screen of the first electronic device to the display screen of the second electronic device; in response to the second drag operation, the first electronic device acquiring a second image of the second projection window, and the first electronic device overlaying the second image onto a transparent background image to obtain an overlaid image, the second image including the display content of the second projection window that was dragged off the display screen of the first electronic device.

[0013] In this way, when the first electronic device projects multiple windows onto the second electronic device, the images of all the windows can be overlaid on a transparent background image before being sent to the second electronic device. This way, the first electronic device only needs to send one image to the second electronic device, saving transmission bandwidth and improving transmission efficiency.

[0014] In conjunction with the first aspect, in one possible implementation, before the first electronic device detects the first drag operation, the method may further include: the first electronic device determining a first relative position between the first electronic device and the second electronic device, the first drag operation being used to move the first projection window from the display screen of the first electronic device to the display screen of the second electronic device in a first direction, the first direction being the direction of the second electronic device relative to the first electronic device indicated by the first relative position.

[0015] The first phase position includes the second electronic device being located to the right of the first electronic device, the second electronic device being located to the left of the first electronic device, the second electronic device being located above the first electronic device, and the second electronic device being located below the first electronic device.

[0016] The first drag operation is used to move the first projection window from the display screen of the first electronic device to the display screen of the second electronic device in a first direction, including:

[0017] The first direction is the right side of the second electronic device relative to the first electronic device, as indicated by the first relative position. The first drag operation is used to move the first projection window from the display screen of the first electronic device to the right of the display screen of the second electronic device.

[0018] Alternatively, the first direction is the left side of the second electronic device relative to the first electronic device, as indicated by the first relative position, and the first drag operation is used to move the first projection window from the display screen of the first electronic device to the left of the display screen of the second electronic device;

[0019] Alternatively, the first direction is indicated by the first relative position, with the second electronic device positioned above the first electronic device, and the first drag operation is used to move the first projection window upward from the display screen of the first electronic device to the display screen of the second electronic device;

[0020] Alternatively, the first direction is indicated by the first relative position, where the second electronic device is below the first electronic device, and the first drag operation is used to move the first projection window down from the display screen of the first electronic device to the display screen of the second electronic device.

[0021] In this embodiment, the movement speed of the first drag operation has a component in the first direction, which can be referred to as the first drag operation moving in the first direction. The first electronic device can detect the movement speed of the first movement operation. For example, the first electronic device can determine the speed of the first movement operation by the movement speed of the mouse.

[0022] In this way, the direction of movement of the first drag operation is related to the first relative position, which is more in line with the operating habits of most users and can improve the user experience.

[0023] In conjunction with the first aspect, in one possible implementation, the first relative position is set by the user in the first electronic device; or the first electronic device obtains it based on the actual position of the first electronic device and the actual position of the second electronic device. In this way, the first electronic device can determine its relative position to the second electronic device.

[0024] In conjunction with the first aspect, in one possible implementation, the method may further include: in response to a first drag operation, a first electronic device sends second position information of a first mouse pointer to a second electronic device, the second position information being used to determine the position of the first mouse pointer on the display screen of the second electronic device; the second electronic device displays the first mouse pointer on the display screen of the second electronic device according to the second position information.

[0025] In this way, the user can drag the mouse pointer from the first electronic device to the second electronic device.

[0026] In conjunction with the first aspect, in one possible implementation, a second mouse pointer is displayed on the screen of the second electronic device. The method may further include: when the first mouse pointer is displayed on the screen of the second electronic device, the second mouse pointer is hidden based on second position information. This avoids the simultaneous display of two mouse pointers on the second electronic device, which could interfere with user operation.

[0027] In conjunction with the first aspect, in one possible implementation, after the second electronic device displays the first image, the method may further include: the second electronic device receiving a first operation, the first operation acting on the first image; the second electronic device sending a first instruction to the first electronic device, the first instruction instructing the first electronic device to respond to the first operation; and the first application to which the first projection window in the first electronic device belongs responding to the first operation.

[0028] The first operation may include clicking the maximize control in the first projection window, clicking the minimize control in the first projection window, clicking the control used to close the first projection window, or moving any item in the first projection window. It is understood that the first operation can be any control clicked by the user in the first projection window, and is not limited to the controls described above.

[0029] The first instruction may carry the operation type of the first operation and the location information of the first operation.

[0030] In this way, users can project a screen from the first electronic device to the second electronic device, and can also control the projected window from the second electronic device, such as moving the projected window back to the first electronic device, closing the projected window, maximizing the projected window, and minimizing the projected window, etc.

[0031] In conjunction with the first aspect, in one possible implementation, after the second electronic device displays the first image, the method further includes: the second electronic device receiving a second operation, the second operation acting on a first window, the first window being a window of a second application in the second electronic device; and the second application of the second electronic device responding to the second operation.

[0032] In this way, when a projection window is displayed on the second electronic device, the user does not need to close the projection window and can directly operate the application window on the second electronic device.

[0033] In conjunction with the first aspect, in one possible implementation, after the second electronic device displays the first image, the method further includes: the second electronic device receiving a third operation, the third operation acting on a transparent window, the transparent window being the window where the overlaid image is located, the transparent window being the focus window; the second electronic device determining the area where the third operation acts based on the location of the third operation; the second electronic device sending a second instruction to the first electronic device, the second instruction being used to instruct the first electronic device to respond to the third operation; and the first application to which the first projection window in the first electronic device belongs responding to the third operation.

[0034] The third operation can include clicking the maximize control in the first projection window, clicking the minimize control in the first projection window, clicking the control used to close the first projection window, or moving any item in the first projection window. It is understood that the third operation can be any control clicked by the user in the first projection window, and is not limited to the controls described above.

[0035] The second instruction can carry the operation type and location information of the third operation.

[0036] In this way, when the second electronic device displays images of one or more projection windows, the second electronic device can determine which projection window the user's operation is applied to.

[0037] In conjunction with the first aspect, in one possible implementation, after the second electronic device displays the first image, the method further includes: the second electronic device receiving a fourth operation, the fourth operation acting on a focus window; if the focus window is a local window of the second electronic device, then the application described in the local window of the second electronic device responds to the fourth operation; if the focus window is a projection window, and the projection window is the window where the first image is located, then the second electronic device sends a third instruction to the first electronic device, the third instruction being used to instruct the first electronic device to respond to the fourth operation. The first application to which the first projection window in the first electronic device belongs responds to the fourth operation.

[0038] In this way, the second electronic device can determine whether the window being operated on by the user is a local window or a projected window.

[0039] Secondly, a screen mirroring method is provided, which can be applied to a screen mirroring system. The screen mirroring system may include a first electronic device and a second electronic device, and the first electronic device and the second electronic device establish a communication connection. The method may include: the second electronic device detecting a fifth operation; the display screen of the second electronic device displaying an overlay image; the overlay image being sent to the second electronic device by the first electronic device; the overlay image being obtained by overlaying a first image onto a transparent background image; the first image being an image of a first screen mirroring window acquired by the first electronic device; and the first screen mirroring window being a window of a first application in the first electronic device; the second electronic device sending a fourth instruction to the first electronic device; the fourth instruction being used to instruct the first application in the first electronic device to respond to the fifth operation.

[0040] The fifth operation can include clicking the maximize control in the first projection window, clicking the minimize control in the first projection window, clicking the control used to close the first projection window, or moving any item in the first projection window. It is understood that the first operation can be any control clicked by the user in the first projection window, and is not limited to the controls described above.

[0041] The fourth instruction can carry the operation type and position information of the fifth operation.

[0042] In this way, users can project a screen from the first electronic device to the second electronic device, and can also control the projected window from the second electronic device, such as moving the projected window back to the first electronic device, closing the projected window, maximizing the projected window, and minimizing the projected window, etc.

[0043] In conjunction with the second aspect, in one possible implementation, before the electronic device detects the fifth operation, the method may further include: a first electronic device detecting a first drag operation, the first drag operation being used to move a first projection window in the first electronic device from the display screen of the first electronic device to the display screen of a second electronic device; in response to the first drag operation, the first electronic device acquiring a first image of the first projection window, the first image including the display content of the first projection window that has been dragged off the display screen of the first electronic device; the first electronic device overlaying the first image onto a transparent background image to obtain an overlaid image; the first electronic device sending the overlaid image to the second electronic device; and the second electronic device displaying the overlaid image on the display screen of the second electronic device.

[0044] The transparent background image is generated by the first electronic device according to the size of the display screen of the second electronic device, and the size of the transparent background image is the same as the size of the display screen of the second electronic device; the transparent background image is not displayed in the first electronic device.

[0045] Thirdly, a screen mirroring method is provided, which is applied to a first electronic device. The method may include: the first electronic device detecting a first drag operation, the first drag operation being used to move a first screen mirroring window from the display screen of the first electronic device to the display screen of a second electronic device, the second electronic device establishing a communication connection with the first electronic device; in response to the first drag operation, the first electronic device acquiring a first image of the first screen mirroring window, the first image including the display content of the first screen mirroring window that has been dragged off the display screen of the first electronic device; and the first electronic device sending the first image to the second electronic device, the first image being used to display on the display screen of the second electronic device.

[0046] Using the screen mirroring method provided in the third aspect above, users can drag windows from the first electronic device to the second electronic device. Users can simply drag and drop windows from the first electronic device to the second. This simplifies the user experience and enhances the overall user experience.

[0047] In conjunction with the third aspect, in one possible implementation, the first image includes a portion or all of the display content of the first projection window. This allows a user to project a portion of the first projection window from the first electronic device to the second electronic device, or to project the entire first projection window to the second electronic device.

[0048] In conjunction with the third aspect, in one possible implementation, the first electronic device sending the first image to the second electronic device may include: the first electronic device sending the first image and first location information to the second electronic device; the first location information is used to determine the position of the first image displayed on the screen of the second electronic device. In this way, the second electronic device can determine the specific position where the first image is displayed.

[0049] In conjunction with the third aspect, in one possible implementation, and in conjunction with the first aspect, in one possible implementation, the first electronic device sends the first image to the second electronic device, comprising: the first electronic device overlaying the first image onto a transparent background image to obtain an overlaid image; the transparent background image being generated by the first electronic device according to the size of the display screen of the second electronic device, the size of the transparent background image being the same as the size of the display screen of the second electronic device; the transparent background image not being displayed on the first electronic device; and the first electronic device sending the overlaid image to the second electronic device.

[0050] In conjunction with the third aspect, in one possible implementation, the method may further include: a first electronic device detecting a second drag operation, the second drag operation being used to move a second projection window displayed on the display screen of the first electronic device from the display screen of the first electronic device to the display screen of the second electronic device; in response to the second drag operation, the first electronic device acquiring a second image of the second projection window, and the first electronic device overlaying the second image onto a transparent background image to obtain an overlaid image, the second image including the display content of the second projection window that has been dragged off the display screen of the first electronic device.

[0051] In this way, when the first electronic device projects multiple windows onto the second electronic device, the images of all the windows can be overlaid on a transparent background image before being sent to the second electronic device. This way, the first electronic device only needs to send one image to the second electronic device, saving transmission bandwidth and improving transmission efficiency.

[0052] In conjunction with the third aspect, in one possible implementation, before the first electronic device detects the first drag operation, the method may further include: the first electronic device determining a first relative position between the first electronic device and the second electronic device; the first drag operation being used to move the first projection window from the display screen of the first electronic device to the display screen of the second electronic device in a first direction, the first direction being the direction of the second electronic device relative to the first electronic device indicated by the first relative position.

[0053] The first phase position includes the second electronic device being located to the right of the first electronic device, the second electronic device being located to the left of the first electronic device, the second electronic device being located above the first electronic device, and the second electronic device being located below the first electronic device.

[0054] The first drag operation is used to move the first projection window from the display screen of the first electronic device to the display screen of the second electronic device in a first direction, including:

[0055] The first direction is the right side of the second electronic device relative to the first electronic device, as indicated by the first relative position. The first drag operation is used to move the first projection window from the display screen of the first electronic device to the right of the display screen of the second electronic device.

[0056] Alternatively, the first direction is the left side of the second electronic device relative to the first electronic device, as indicated by the first relative position, and the first drag operation is used to move the first projection window from the display screen of the first electronic device to the left of the display screen of the second electronic device;

[0057] Alternatively, the first direction is indicated by the first relative position, with the second electronic device positioned above the first electronic device, and the first drag operation is used to move the first projection window upward from the display screen of the first electronic device to the display screen of the second electronic device;

[0058] Alternatively, the first direction is indicated by the first relative position, where the second electronic device is below the first electronic device, and the first drag operation is used to move the first projection window down from the display screen of the first electronic device to the display screen of the second electronic device.

[0059] In this embodiment, the movement speed of the first drag operation has a component in the first direction, which can be referred to as the first drag operation moving in the first direction. The first electronic device can detect the movement speed of the first movement operation. For example, the first electronic device can determine the speed of the first movement operation by the movement speed of the mouse.

[0060] In this way, the direction of movement of the first drag operation is consistent with the first relative position, which is more in line with the operating habits of most users and can improve the user experience.

[0061] In conjunction with the third aspect, in one possible implementation, the first relative position is set by the user in the first electronic device; or the first electronic device obtains it based on the actual positions of the first electronic device and the second electronic device. In this way, the first electronic device can determine its relative position to the second electronic device.

[0062] In conjunction with the third aspect, in one possible implementation, the method may further include: in response to the first drag operation, the first electronic device sends second position information of the first mouse pointer to the second electronic device, the second position information being used to determine the position of the first mouse pointer on the display screen of the second electronic device.

[0063] Alternatively, in one possible implementation, the first drag operation can be the same as the second drag operation, that is, the user can drag the first projection window and the first mouse pointer together to the second electronic device.

[0064] In this way, the user can drag the mouse pointer from the first electronic device to the second electronic device.

[0065] In conjunction with the third aspect, in one possible implementation, the first electronic device receives a first instruction sent by the second electronic device, the first instruction being used to instruct the first electronic device to respond to a first operation, the first operation being applied to a first image displayed in the second electronic device; the first application to which the first projection window in the first electronic device belongs responds to the first operation.

[0066] The first operation may include clicking the maximize control in the first projection window, clicking the minimize control in the first projection window, clicking the control used to close the first projection window, or moving any item in the first projection window. It is understood that the first operation can be any control clicked by the user in the first projection window, and is not limited to the controls described above.

[0067] The first instruction may carry the operation type of the first operation and the location information of the first operation.

[0068] In this way, users can project a screen from the first electronic device to the second electronic device, and can also control the projected window from the second electronic device, such as moving the projected window back to the first electronic device, closing the projected window, maximizing the projected window, and minimizing the projected window, etc.

[0069] In conjunction with the third aspect, in one possible implementation, after the second electronic device displays the first image, the method further includes: the second electronic device receiving a third operation, the third operation acting on a transparent window, the transparent window being the window where the overlaid image is located, the transparent window being the focus window; the second electronic device determining the area where the third operation acts on the first image based on the location of the third operation; the second electronic device sending a second instruction to the first electronic device, the second instruction being used to instruct the first electronic device to respond to the third operation; and the first application to which the first projection window in the first electronic device belongs responding to the third operation.

[0070] The third operation can include clicking the maximize control in the first projection window, clicking the minimize control in the first projection window, clicking the control used to close the first projection window, or moving any item in the first projection window. It is understood that the third operation can be any control clicked by the user in the first projection window, and is not limited to the controls described above.

[0071] The second instruction can carry the operation type and location information of the third operation.

[0072] In this way, when the second electronic device displays images of one or more projection windows, the second electronic device can determine which projection window the user's operation is applied to.

[0073] In conjunction with the third aspect, in one possible implementation, after the second electronic device displays the first image, the method further includes: the second electronic device receiving a fourth operation, the fourth operation acting on a focus window; if the focus window is a local window of the second electronic device, then the application described in the local window of the second electronic device responds to the fourth operation; if the focus window is a projection window, and the projection window is the window where the first image is located, then the second electronic device sends a third instruction to the first electronic device, the third instruction being used to instruct the first electronic device to respond to the fourth operation. The first application to which the first projection window in the first electronic device belongs responds to the fourth operation.

[0074] In this way, the second electronic device can determine whether the window being operated on by the user is a local window or a projected window.

[0075] Fourthly, a screen mirroring method is provided, which can be applied to a second electronic device. The method may include: the second electronic device receiving a first image sent by a first electronic device, the first image being obtained by the first electronic device from a first screen mirroring window in the first electronic device when receiving a first drag operation, the first image including the display content of the first screen mirroring window being dragged out of the display screen of the first electronic device; the first drag operation being used to move the first screen mirroring window from the first electronic device to the second electronic device, the second electronic device establishing a communication connection with the first electronic device; and the second electronic device displaying the first image on the display screen of the second electronic device.

[0076] Using the screen mirroring method provided in the fourth aspect above, users can drag windows from the first electronic device to the second electronic device. Users can simply drag and drop windows from the first electronic device to the second. This simplifies the user experience and enhances user engagement.

[0077] In conjunction with the fourth aspect, in one possible implementation, the first image includes a portion or all of the display content of the first projection window. This allows a user to project a portion of the first projection window from the first electronic device to the second electronic device, or to project the entire first projection window to the second electronic device.

[0078] In conjunction with the fourth aspect, in one possible implementation, the second electronic device receives a first image sent by the first electronic device, including: the second electronic device receiving the first image and first position information sent by the first electronic device, wherein the first position information is used to determine the position of the first image displayed on the screen of the second electronic device.

[0079] In conjunction with the fourth aspect, in one possible implementation, the second electronic device displays the first image on its display screen, comprising: the second electronic device displaying the first image on its display screen according to first position information. In this way, the second electronic device can determine the specific position where the first image is displayed.

[0080] In conjunction with the fourth aspect, in one possible implementation, the method may further include: a second electronic device receiving an overlay image sent by the first electronic device, the overlay image being obtained by superimposing a first image and a second image onto a transparent background image; the second image being obtained by the first electronic device from a second projection window in the first electronic device when receiving a second drag operation, the second image including the display content of the second projection window that has been dragged out of the display screen of the first electronic device, the second drag operation being used to move the second projection window from the display screen of the first electronic device to the display screen of the second electronic device.

[0081] In this way, the second electronic device can display images from multiple projection windows of the first electronic device, and can simultaneously project images from multiple projection windows through a single transparent window (the transparent window is the window where the overlaid image is located).

[0082] In conjunction with the fourth aspect, in one possible implementation, the method may further include: a second electronic device receiving a second position message sent by a first electronic device, the second position message being used to determine the position of a first mouse pointer in the first electronic device on the display screen of the second electronic device; and the second electronic device displaying the first mouse pointer on its display screen based on the second position information. This allows the user to drag the mouse pointer from the first electronic device to the second electronic device.

[0083] In conjunction with the fourth aspect, in one possible implementation, a second mouse pointer is displayed on the screen of the second electronic device. The method may further include: when the second electronic device displays the first mouse pointer based on the second position information, it hides the second mouse pointer. This avoids displaying two mouse pointers simultaneously in the second electronic device, thus preventing interference with user operation.

[0084] In conjunction with the fourth aspect, in one possible implementation, after the second electronic device displays the first image, the method may further include: the second electronic device receiving a first operation, the first operation acting on the first image; the second electronic device sending a first instruction to the first electronic device, the first instruction instructing the first electronic device to respond to the first operation. The first operation may include clicking a maximize control in the first projection window, clicking a minimize control in the first projection window, clicking a control for closing the first projection window, or moving any item in the first projection window. It is understood that the first operation can be any control clicked by the user in the first projection window, and is not limited to the controls described above.

[0085] The first instruction may carry the operation type of the first operation and the location information of the first operation.

[0086] In this way, users can project a screen from the first electronic device to the second electronic device, and can also control the projected window from the second electronic device, such as moving the projected window back to the first electronic device, closing the projected window, maximizing the projected window, and minimizing the projected window, etc.

[0087] In conjunction with the fourth aspect, in one possible implementation, after the second electronic device displays the first image, the method further includes: the second electronic device receiving a second operation, the second operation acting on a first window, the first window being a window of a second application in the second electronic device; and the second application of the second electronic device responding to the second operation.

[0088] In this way, when a projection window is displayed on the second electronic device, the user does not need to close the projection window and can directly operate the application window on the second electronic device.

[0089] In conjunction with the fourth aspect, in one possible implementation, after the second electronic device displays the first image, the method further includes: the second electronic device receiving a third operation, the third operation acting on a transparent window, the transparent window being the window where the overlaid image is located, the transparent window being the focus window; the second electronic device determining the area where the third operation acts on the first image based on the position of the third operation; the second electronic device sending a second instruction to the first electronic device, the second instruction being used to instruct the first electronic device to respond to the third operation; and the first application to which the first projection window in the first electronic device belongs responding to the third operation.

[0090] The third operation can include clicking the maximize control in the first projection window, clicking the minimize control in the first projection window, clicking the control used to close the first projection window, or moving any item in the first projection window. It is understood that the third operation can be any control clicked by the user in the first projection window, and is not limited to the controls described above.

[0091] The second instruction can carry the operation type and location information of the third operation.

[0092] In this way, when the second electronic device displays images of one or more projection windows, the second electronic device can determine which projection window the user's operation is applied to.

[0093] In conjunction with the fourth aspect, in one possible implementation, after the second electronic device displays the first image, the method further includes: the second electronic device receiving a fourth operation, the fourth operation acting on a focus window; if the focus window is a local window of the second electronic device, then the application described in the local window of the second electronic device responds to the fourth operation; if the focus window is a projection window, and the projection window is the window where the first image is located, then the second electronic device sends a third instruction to the first electronic device, the third instruction being used to instruct the first electronic device to respond to the fourth operation. The first application to which the first projection window in the first electronic device belongs responds to the fourth operation.

[0094] In this way, the second electronic device can determine whether the window being operated on by the user is a local window or a projected window.

[0095] Fifthly, a screen projection system is provided, which may include a first electronic device and a second electronic device, wherein the first electronic device is used to execute a screen projection method in any possible implementation of the third aspect above; and the second electronic device is used to execute a screen projection method in any possible implementation of the fourth aspect above.

[0096] In a sixth aspect, an electronic device is provided, which may include: one or more processors and one or more memories; the one or more memories are respectively coupled to the one or more processors; the one or more memories are used to store computer program code, the computer program code including computer instructions; when the computer instructions are executed on the processor, the electronic device causes the electronic device to execute a screen projection method in any of the possible implementations of the third aspect above.

[0097] In a seventh aspect, an electronic device is provided, which may include: one or more processors and one or more memories; the one or more memories are respectively coupled to the one or more processors; the one or more memories are used to store computer program code, the computer program code including computer instructions; when the computer instructions are executed on the processor, the electronic device causes the electronic device to execute a screen projection method in any of the possible implementations of the fourth aspect above.

[0098] Eighthly, a computer storage medium is provided, including computer instructions that, when executed on an electronic device, cause the electronic device to perform the screen projection method in any possible implementation of any of the above aspects.

[0099] Ninthly, a computer program product is provided that, when the computer program product is run on an electronic device, causes the electronic device to execute the screen projection method in any possible implementation of any of the above aspects. Attached Figure Description

[0100] Figure 1This is a schematic diagram of a screen projection system provided in an embodiment of this application;

[0101] Figure 2 This is a schematic diagram of a screen mirroring interface mentioned in an embodiment of this application;

[0102] Figure 3 This is a schematic diagram of a screen projection interface provided in an embodiment of this application;

[0103] Figures 4-12 This is a schematic diagram of the interface of a remote device and a local device during the screen projection process provided in an embodiment of this application;

[0104] Figures 13A-13B This is a schematic diagram of a screen projection scenario provided in an embodiment of this application;

[0105] Figure 14 This is a schematic diagram of another screen projection scenario provided in an embodiment of this application;

[0106] Figure 15 This is a schematic flowchart of a screen mirroring method provided in an embodiment of this application;

[0107] Figure 16 This is a schematic diagram showing the boundary between the display screens of the remote device and the local device provided in an embodiment of this application;

[0108] Figures 17A-17B This is a schematic diagram of the boundaries of a set of windows provided in an embodiment of this application;

[0109] Figure 18 This is a schematic diagram of a screen projection interface provided in an embodiment of this application;

[0110] Figure 19 This is a schematic diagram of the interface where a portion of the projection window in the remote device is projected onto the local device, as provided in the embodiments of this application.

[0111] Figure 20 This is a schematic diagram of window overlay provided in an embodiment of this application;

[0112] Figure 21 This is a schematic flowchart of a screen mirroring method provided in an embodiment of this application;

[0113] Figure 22 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0114] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this application refers to and includes any or all possible combinations of one or more of the listed items.

[0115] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature, and in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0116] The screen mirroring involved in this application refers to the transmission of a window displayed on one electronic device (such as an audio playback window, a video playback window, an instant messaging software chat window, etc.) to another electronic device for display, achieving the effect of displaying a window of one electronic device on another. The screen mirroring involved in this application can include wired screen mirroring and wireless screen mirroring. Wired screen mirroring can establish a connection between multiple electronic devices through a high-definition multimedia interface (HDMI) and transmit window data through an HDMI cable; wireless screen mirroring can establish a connection between multiple electronic devices through a wireless local area network (e.g., Wi-Fi) and transmit window data through the wireless local area network.

[0117] The screen projection system described in this application may include at least two electronic devices. See also Figure 1 , Figure 1 This is a system architecture diagram of the screen projection system 10 provided in this application embodiment. For example... Figure 1 As shown, the screen mirroring system 10 may include electronic device 100 and electronic device 200. Electronic device 100 may also be referred to as a remote device. Electronic device 200 may be referred to as a local device. Electronic device 100 can be a screen mirroring sender (Source), and electronic device 200 can be a screen mirroring receiver (Sink). That is, a window in electronic device 100 can be mirrored to electronic device 200. The window that electronic device 100 mirrors to electronic device 200 can be called a screen mirroring window.

[0118] In this embodiment, electronic device 100 can be referred to as a first electronic device, and electronic device 200 can be referred to as a second electronic device. The first electronic device has a display screen, and the second electronic device also has a display screen. The first projection window displayed by the first electronic device is the first projection window displayed on the display screen of the first electronic device. The first image of the first projection window displayed by the second electronic device is the first image of the first projection window displayed on the display screen of the second electronic device.

[0119] The electronic device 100 has at least the ability to project content. Optionally, the electronic device 100 may include an application layer, an application framework layer, and a driver layer.

[0120] The application layer can contain a series of application packages. For example... Figure 1 As shown, the application package can include applications (also known as apps) such as camera, calendar, map, screen mirroring, music, SMS, gallery, call, navigation, Bluetooth, and video.

[0121] Optionally, electronic device 100 can achieve screen mirroring capability through a screen mirroring application. The screen mirroring application can provide a screen mirroring connection user interface (UI) and a screen mirroring display UI. The screen mirroring connection UI can be used to set up the connection between screen mirroring devices, for example, establishing a screen mirroring connection between electronic device 100 and electronic device 200 in the screen mirroring connection UI. The screen mirroring display UI can display the devices that can be screen mirrored, as well as display and set the relative positions between the devices, etc.

[0122] Optionally, the system of the electronic device 100 may provide a screen projection control for the electronic device 100 to achieve screen projection capabilities.

[0123] like Figure 1 As shown, the application framework layer provides application programming interfaces (APIs) and programming frameworks for applications in the application layer. The application framework layer includes some predefined functions.

[0124] like Figure 1As shown, the application framework layer of electronic device 100 may include screen mirroring services and data management. The screen mirroring services may include window capture services, layer overlay services, layer encoding services, and keyboard and mouse management services. The window capture service can be used to capture window image data from electronic device 100 to be mirrored onto electronic device 200. The layer overlay service can be used to overlay one or more mirrored window images. The layer encoding service is used to encode the overlaid mirrored window image data. The keyboard and mouse management service is used to manage keyboard and mouse operations received by electronic device 100. Data management may include stream transmission management, touch data management, keyboard and mouse event management, and touch area management. The stream transmission management can be used to manage the encoded data of the mirrored windows in electronic device 100 after being encoded by the layer encoding service. The touch data management can be used to manage touch data received by electronic device 100 from the user or touch data sent by electronic device 200 to electronic device 100. Keyboard and mouse event management can be used to manage keyboard and mouse events received by electronic device 100 and keyboard and mouse events sent to electronic device 100 by electronic device 200, such as a user dragging a projection window to the right with the mouse. Touch area management is used to manage the touch area received by electronic device 100.

[0125] like Figure 1 As shown, the electronic device 100 may include a driver layer, which is a layer between hardware and software. The driver layer can be used to discover devices that can be connected for screen mirroring (referred to as device discovery), as well as for authentication (referred to as device authentication) and connection (referred to as device connection) between screen mirroring devices. The aforementioned device discovery, device authentication, and device connection can be accomplished by one driver or multiple drivers in the driver layer, which is not limited here.

[0126] The electronic device 200 has at least screen mirroring reception capability. Optionally, the electronic device 200 may include an application layer, an application framework layer, and a driver layer.

[0127] The application layer can contain a series of application packages. For example... Figure 1 As shown, the application package can include applications (also known as apps) such as camera, calendar, map, screen mirroring, music, SMS, gallery, call, navigation, Bluetooth, and video.

[0128] Optionally, the electronic device 200 can achieve screen mirroring reception capability through a screen mirroring application.

[0129] Optionally, the system of the electronic device 200 may provide a screen projection control for the electronic device 200 to achieve screen projection receiving capability.

[0130] like Figure 1As shown, the application framework layer provides application programming interfaces (APIs) and programming frameworks for applications in the application layer. The application framework layer includes some predefined functions.

[0131] like Figure 1 As shown, the application framework layer of the electronic device 200 may include screen mirroring services and data management.

[0132] Among them, such as Figure 1 As shown, the screen mirroring service can include window Z-order service, layer overlay service, layer decoding service, and keyboard and mouse management service, etc. The window Z-order service can be used to set the Z-axis order of windows in the electronic device 200 and the screen mirroring windows received by the electronic device 200, and display the windows (windows in the electronic device 200 and screen mirroring windows received by the electronic device 200) according to this Z-axis order. The layer overlay service can be used to overlay multiple windows. The layer decoding service can be used to decode the received encoded screen mirroring window data.

[0133] Among them, such as Figure 1 As shown, data management can include stream reception management, touch data management, keyboard and mouse event management, and touch area management. Stream reception management can be used to receive encoded data from the projection window sent by electronic device 100 to electronic device 200. Touch data management can be used to manage the touch data received by electronic device 200 from the user. Keyboard and mouse event management can be used to manage keyboard and mouse events received by electronic device 200 or keyboard or mouse events sent by electronic device 100 to electronic device 200. The touch area received by electronic device 200 is also shown.

[0134] like Figure 1 As shown, the electronic device 200 may include a driver layer, which is a layer between hardware and software. The driver layer can be used to discover devices that can be connected for screen mirroring (referred to as device discovery), as well as for authentication (referred to as device authentication) and connection (referred to as device connection) between screen mirroring devices. The aforementioned device discovery, device authentication, and device connection can be accomplished by one driver or multiple drivers in the driver layer, which is not limited here.

[0135] Examples of electronic devices 100 include, but are not limited to, electronic devices running iOS, Android, Microsoft, HarmonyOS, or other operating systems. Optionally, the first electronic device 100 can be a mobile phone, tablet computer, personal digital assistant (PDA), or desktop computer. The second electronic device 200 can be a television, tablet computer, or desktop computer. In some feasible embodiments, the screen projection system of this application may also include a mouse for dragging and dropping the projection window from electronic device 100 to electronic device 200.

[0136] It is understood that the remote device in this application embodiment can be an electronic device that sends the projection window; that is, in this application embodiment, the electronic device that projects the window to another device is called the remote device. For example, Figure 1 The electronic device 100 shown in the figure. In this embodiment, the local device can be an electronic device that receives and displays a projection window sent by another electronic device. That is, in this embodiment, the electronic device capable of displaying the content projected by another electronic device is referred to as the local device. For example, Figure 1 Electronic device 200 shown in the figure.

[0137] In some embodiments, a remote device can project content displayed on another remote device onto a local device. The local device can then display the projected content in full screen. For example... Figure 2 As shown, computer 20A can cast a video to tablet 20B, and tablet 20B can display the cast video in real time. However, the cast video from computer 20A will be displayed full-screen on tablet 20B. Therefore, when a user wants to operate an application on tablet 20B, they need to first hide the cast content displayed on tablet 20B and then find the application they want to operate, which is cumbersome and results in a poor user experience.

[0138] In some embodiments, a remote device can project an application window onto a local device, which can then display the projected window in full screen or as a window. For example... Figure 3As shown, mobile phone 30A can project the application interface 301 (projection window) onto computer 30B, and the projection window (i.e., window 301) can be displayed in the user interface 302 of computer 30B. When the projection window of mobile phone 30A is projected onto computer 30B, the user needs to find and open the projection window on the phone, establish a connection with computer 30B, and after projection is initiated, the projection window can be displayed on computer 30B. When the user needs to return the projection window from computer 30B to mobile phone 30A, the user needs to find and close the projection window on mobile phone 30A. Thus, the operation of projection window from remote device to local device, and from local device to remote device, is relatively cumbersome and results in a poor user experience.

[0139] To simplify screen mirroring and improve user experience, this application provides a screen mirroring method and electronic device. In the screen mirroring method provided by this application, a user can drag a screen mirroring window from a remote device to a local device. For example, a user can use a mouse to drag a screen mirroring window from a remote device to the local device. The user can drag the screen mirroring window back from the local device to the remote device. The user can adjust the size of the screen mirroring window displayed on the local device.

[0140] The following detailed description, with reference to the accompanying drawings, provides a screen projection method based on embodiments of this application. This application uses a computer as the remote device and a tablet as the local device for illustration. Figures 4-11 This diagram illustrates the interface of a screen mirroring window from computer 400 to tablet 500.

[0141] like Figure 4 As shown, Figure 4 The image shows a computer 400 and a tablet 500. It is understood that the computer 400 and tablet 500 can establish a communication connection. The computer 400 can display a user interface 40A. This user interface 40A can display icons for several applications, such as the recycle bin icon 401, the This PC icon 402, the screen mirroring icon 403, and the WeChat icon 404. The computer 400 can perform screen mirroring via screen mirroring 403. The computer 400 may have a touch area 405A. Optionally, the computer 400 can be connected to a mouse 600.

[0142] like Figure 4 As shown, tablet 500 can display a user interface 50A. This user interface 50A can display icons for several applications, such as the settings icon 501, the fitness and health icon 502, the screen mirroring icon 503, the memo icon 504, and so on. Tablet 500 can achieve screen mirroring reception capability through screen mirroring 503.

[0143] It is understandable that the content displayed in the user interface 40A of computer 400 may not be limited to... Figure 4The content shown is not limited to the content displayed in the user interface 50A of the tablet 500. Figure 4 The content shown in the image.

[0144] Optionally, the PC 400 can also send screens via a screen mirroring control. The Tablet 500 can also receive screens via a screen mirroring control.

[0145] Users can enable the screen mirroring function (i.e., screen mirroring sending and / or screen mirroring receiving function) on the computer 400 and tablet 500. For example, users can turn on screen mirroring 403 on the computer 400 and screen mirroring 503 on the tablet 500.

[0146] Users can select a screen mirroring receiving device on computer 400. When a user clicks the screen mirroring icon 403 in user interface 40A, computer 400 can display a window for selecting a screen mirroring receiving device.

[0147] like Figure 5 As shown, computer 400 can display a user interface 40B, which may display a window 4001. This window 4001 is used to select a screen mirroring receiving device. For example, one or more screen mirroring receiving devices may be present in window 4001, such as tablet 500, computer 100, computer 200, and mobile phone 300.

[0148] In window 4001, users can select one or more devices as the screen mirroring receiving device for computer 400. Users can select tablet 500 as the screen mirroring receiving device by clicking control 4002. Users can select computer 100 as the screen mirroring receiving device by clicking control 4003. Users can select computer 200 as the screen mirroring receiving device by clicking control 4004. Users can select mobile phone 300 as the screen mirroring receiving device by clicking control 4005. The following explanation uses the example of a user selecting tablet 500 as the screen mirroring receiving device.

[0149] Understandable, Figure 5 The window 4001 shown for selecting a screen mirroring receiving device is merely an example. This application does not limit the shape of the window used to select a screen mirroring receiving device or the content contained within the window (e.g., controls, text, icons, etc.).

[0150] Optionally, after selecting the receiving device, the user can also set the relative positions of the remote and local devices in the screen mirroring application (i.e., screen mirroring 403) on computer 400. The user can determine the direction in which the screen mirroring window is dragged by setting the relative positions of the remote and local devices. For example, if the user sets tablet 500 to the right of computer 400, the user can drag the screen mirroring window from computer 400 to the right onto tablet 500. If the user sets tablet 500 to the top of computer 400, the user can drag the screen mirroring window from computer 400 upwards onto tablet 500.

[0151] like Figure 6 As shown, the computer 400 can display a user interface 40C. The user interface 40C can display a window 4001. The user has selected the tablet 500 as the screen mirroring receiving device. The user can click on control 4006, and in response to the user's operation, the computer 400 can display a window for setting the relative positions of the remote and local devices.

[0152] Understandably, users can select the screen mirroring receiving device and click control 4006 using mouse 600. Users can also select the screen mirroring receiving device and click control 4006 using touch area 405.

[0153] like Figure 7 As shown, the computer 400 may display a user interface 40D. The user interface 40D may include a window 5001 for setting up remote and local devices. For example, a user can select to place the tablet 500 to the right of the computer 400 by clicking control 5002. A user can also select to place the tablet 500 to the left of the computer 400 by clicking control 5003. A user can also select to place the tablet 500 above the computer 400 by clicking control 5004. A user can also select to place the tablet 500 below the computer 400 by clicking control 5005. The following explanation uses the example of a user selecting to place the tablet 500 to the right of the computer 400 as an example.

[0154] It is understandable that the user sets the relative positions of the remote and local devices in window 5001. The actual positions of the remote and local devices may differ from the relative positions set by the user. For example, the user selects to place tablet 500 to the right of computer 400. Tablet 500 can actually be placed to the left, above, or below computer 400.

[0155] like Figure 8 As shown, computer 400 can display a user interface 40D. After the user selects the relative position of tablet 500 and computer 400 in window 5001 of user interface 40D, they can click control 5006 to indicate completion of settings. After computer 400 detects the user clicking control 5006, it can display user interface 40E.

[0156] Understandable, Figure 7 and Figure 8 The window 5001 shown is merely an example for setting the relative positions of the remote and local devices. This application does not limit the shape or content of the window used to set the relative positions of the remote and local devices.

[0157] like Figure 9 As shown, computer 400 can display a user interface 40E. User interface 40E can display a prompt box 9001. This prompt box 9001 is used to prompt the user on how to project the screen from computer 400 to tablet 500. For example, the prompt box 9001 may contain the prompt text "Please drag the window to be projected to the right to the screen receiving device".

[0158] It is understood that the embodiments of this application do not limit the shape of the prompt box 9001 or the specific content in the prompt box 9001.

[0159] Once the user selects the receiving device (also known as the local device), the user can project the screen from the remote device onto the local device. For example, the user can drag the screen projection window from computer 400 to the right onto tablet 500.

[0160] like Figure 10 As shown, the computer 400 can display a user interface 40F. This user interface 40F can contain a window 1001. The user can project the window onto the tablet 500 by dragging it to the right. For example, the user drags window 1001 to the right using the mouse pointer 1002.

[0161] Understandable, Figure 10 When the user sets the relative positions of the computer 400 and the tablet 500, the selected position is tablet 500 to the right of computer 400. However, tablet 500 can be placed to the right, left, above, below, etc. of computer 400, and this embodiment does not limit this.

[0162] When the user drags window 1101 to the right onto the computer 400 screen, and continues to drag to the right, window 1101 will gradually appear on the tablet 500.

[0163] like Figure 11 As shown, computer 400 can display a user interface 40G. The user interface 40G can contain a window 1101. Window 1101 is... Figure 10 This is part of window 1001. A user interface 50B can be displayed on the tablet 500. The user interface 50B displays window 1102 and mouse pointer 1002. Window 1102 is... Figure 10Another part of window 1001. That is... Figure 10 Window 1001 is partially displayed on computer 400 and partially on tablet 500. Mouse pointer 1002 moves from computer 400 to tablet 500. The user can use mouse pointer 1002 to continue dragging window 1102 to the right.

[0164] Users can continue to drag the projection window to the right until it is fully displayed on the receiving device.

[0165] like Figure 12 As shown, the computer 400 can display a user interface 40H, and the tablet 500 can display a user interface 50C. The projection window from the computer 400 is fully displayed on the tablet 500. Window 1201 displayed on the tablet 500 is the projection window from the tablet 400.

[0166] Furthermore, in one possible implementation, the user can drag the window 1201 back into the computer 400.

[0167] In some embodiments, the computer 400 can establish screen mirroring connections with multiple devices and send screen mirroring windows to multiple devices.

[0168] For example, such as Figure 13A As shown, computer 400 can project a screen onto tablet 500 or computer 700. Computer 400 can display windows 1301 and 1302. Users can drag window 1302 to the right onto tablet 500 or to the left onto computer 700.

[0169] like Figure 13B As shown, window 1301 in computer 400 has been projected onto tablet 500, and tablet 500 can display window 1301. Window 1302 in computer 400 has been projected onto computer 700, and computer 700 can display window 1302.

[0170] Optionally, in some embodiments, the remote device can project multiple windows onto a local device. For example, computer 400 can project multiple windows onto tablet 500.

[0171] For example, a user can drag window 1301 from computer 400 to tablet 500, and then drag window 1302 to tablet 500. That is, computer 400 can project both window 1301 and window 1302 onto tablet 500.

[0172] In other cases, the remote or local device may have multiple displays. The following example uses a remote device with two displays. Users can configure the relative positions of the remote device's multiple displays to the local device.

[0173] For example, such as Figure 14 As shown, the remote device has two displays, referred to as Display 1 and Display 2. If the user sets the local device to the right of Display 2 and Display 1 to the left of Display 2, then the user can drag a window from Display 2 to the right onto the local device, and drag a window from Display 2 to the left onto Display 1. The user can also drag a window from Display 1 to the right onto Display 2, and then continue dragging it to the right onto the local device.

[0174] Understandably, multiple windows on a remote device can be projected onto the local device. If the remote device can display N windows simultaneously, the user can project each of those N windows onto the local device one by one. The number N depends on the performance of the remote device; generally, N is at least greater than 3.

[0175] In this embodiment, a user can drag a projection window from a remote device to the local device. Specifically, the remote device can determine whether to trigger a projection operation based on the user's movement of the projection window. When the user moves the projection window to the first boundary (e.g., the right boundary) of the remote device's display screen, the remote device begins to capture an image of the first portion of the projection window and sends it to the local device, which can receive and display this first portion of the image. If the remote device determines that the local device is to the right of the remote device, then the first boundary is the right boundary.

[0176] Figure 15 A schematic flowchart illustrating a screen mirroring method according to an embodiment of this application is shown. Figure 15 As shown, the screen mirroring method can specifically include the following steps:

[0177] S101. The remote device establishes a communication connection with the local device.

[0178] There are various ways for a remote device to establish a communication connection with a local device. For example, the remote device and the local device can communicate via a wireless local area network (e.g., wireless fidelity, Wi-Fi). They can also establish a communication connection via a USB cable. This application does not limit the method by which the remote device and the local device establish a communication connection.

[0179] In the embodiments of this application, the remote device can be referred to as the first electronic device, and the local device can be referred to as the second electronic device.

[0180] S102. The remote device receives user operation U1 applied to the first projection window at the first moment. The first moment is the moment when the first boundary of the first projection window is detected to be in the first position.

[0181] Optionally, before step S102, both the remote device and the local device need to enable the screen mirroring function.

[0182] Furthermore, in one possible implementation, the remote device and the local device need to establish a screen mirroring connection. This connection can be used to determine the two devices to be mirrored. Through this connection, the remote device can determine which window to mirror onto the local device. Users can follow the steps described above. Figures 4-6 The steps shown establish a screen mirroring connection between the remote device and the local device. The remote device can be one of the aforementioned devices. Figures 4-12 The computer 400 shown in the image. The local device can be the one described above. Figures 4-12 The plate 500 shown in the figure.

[0183] Furthermore, the remote device may have the above-mentioned features. Figure 1 The diagram shows a device discovery module, a device authentication module, and a device connection module. The device discovery module is used to search for and find devices that can establish a screen mirroring connection. This module can find devices that are on the same Wi-Fi network as the remote device and have screen mirroring receiving capabilities. When the user selects a screen mirroring receiving device (i.e., the local device), the device authentication module initiates authentication between the remote device and the screen mirroring receiving device. After successful authentication, the device connection module establishes a screen mirroring connection between the remote device and the screen mirroring receiving device.

[0184] Furthermore, users can also set the relative positions of the remote and local devices on the remote device. The remote device determines the direction in which the user drags the projection window based on this relative position. For example, a user can follow the above... Figure 7 The relative positions of the remote and local devices are set as shown in the diagram. If the user sets the relative position so that the local device is to the right of the remote device, the user can drag the first projection window from the remote device to the right until it is on the local device. If the user sets the relative position so that the local device is to the left of the remote device, the user can drag the first projection window from the remote device to the left until it is on the local device. If the user sets the relative position so that the local device is above the remote device, the user can drag the first projection window from the remote device upwards until it is on the local device. If the user sets the relative position so that the local device is below the remote device, the user can drag the first projection window from the remote device downwards until it is on the local device.

[0185] It is understood that the actual relative positions of the remote device and the local device can be the same as or different from the relative positions set by the user. That is, the user can set the relative positions of the remote device and the local device according to their actual relative positions, or they can set them differently. This application does not limit this aspect.

[0186] Optionally, in one possible implementation, the remote device can obtain the actual location of both the remote device and the local device. The actual location of the remote device can be obtained from its positioning module; the actual location of the local device can be sent from the local device to the remote device. The remote device can determine the actual relative position of the two devices based on their respective locations. The remote device can then determine the direction in which the user drags the projection window based on this relative position. For example, if the local device is to the right of the remote device, the user can drag the first projection window from the remote device to the right until it is on the local device. If the local device is to the left of the remote device, the user can drag the first projection window from the remote device to the left until it is on the local device. If the local device is above the remote device, the user can drag the first projection window from the remote device upwards until it is on the local device. If the actual relative position is that the local device is below the remote device, the user can drag the first projection window on the remote device downwards until the first projection window is dragged to the local device.

[0187] User operation U1 can be an operation in which the user drags the first projection window from the remote device to the local device. At the first moment, the first boundary of the first projection window is in the first position.

[0188] The first boundary of the first projection window is determined by the direction in which the user drags the projection window to the local device, as determined by the remote device. If the remote device determines that the user can drag the projection window to the right, then the first boundary of the first projection window is its right boundary. If the remote device determines that the user can drag the projection window to the left, then the first boundary of the first projection window is its left boundary. If the remote device determines that the user can drag the projection window upwards, then the first boundary of the first projection window is its top boundary. If the remote device determines that the user can drag the projection window downwards, then the first boundary of the first projection window is its bottom boundary.

[0189] When the first boundary of the first projection window is the right boundary of the first projection window, the first position can be the right boundary of the remote device's display screen. When the first boundary of the first projection window is the left boundary of the first projection window, the first position can be the left boundary of the remote device's display screen. When the first boundary of the first projection window is the top boundary of the first projection window, the first position can be the top boundary of the remote device's display screen. When the first boundary of the first projection window is the bottom boundary of the first projection window, the first position can be the bottom boundary of the remote device's display screen.

[0190] In the embodiments of this application, when the content of the display screen in an electronic device (e.g., a remote device or a local device) is displayed in a top-down forward direction, the upper boundary of the display screen can be referred to as the upper boundary of the display screen; the lower boundary of the display screen can be referred to as the lower boundary of the display screen; the left boundary of the display screen can be referred to as the left boundary of the display screen; and the right boundary of the display screen can be referred to as the right boundary of the display screen.

[0191] For example, Figure 16 When the content on the display screen of the computer 400 shown is displayed from top to bottom, the top edge of the computer 400 display screen can be called the top edge of the computer 400 display screen; the bottom edge of the computer 400 display screen can be called the bottom edge of the computer 400 display screen; the left edge of the computer 400 display screen can be called the left edge of the computer 400 display screen; and the right edge of the computer 400 display screen can be called the right edge of the computer 400 display screen.

[0192] Figure 16 In the tablet 500 shown, when the tablet 500 is placed horizontally and the content on the display screen of the tablet 500 is displayed from top to bottom, the upper boundary of the display screen of the tablet 500 can be called the upper boundary of the display screen of the tablet 500; the lower boundary of the display screen of the tablet 500 can be called the lower boundary of the display screen of the tablet 500; the left boundary of the display screen of the tablet 500 can be called the left boundary of the display screen of the tablet 500; and the right boundary of the display screen of the tablet 500 can be called the right boundary of the display screen of the tablet 500.

[0193] In this embodiment of the application, the boundary in the window that is closest to the right edge of the display screen in the electronic device can be called the right edge of the window; the boundary in the window that is closest to the left edge of the display screen in the electronic device can be called the left edge of the window; the boundary in the window that is closest to the top edge of the display screen in the electronic device can be called the top edge of the window; and the boundary in the window that is closest to the bottom edge of the display screen in the electronic device can be called the bottom edge of the window.

[0194] Understandably, windows displayed on electronic devices can be rectangular or square, or other shapes such as circles, rhombuses, ellipses, etc., or they can be irregular shapes. For rectangular or square windows, the four boundaries of the window are the four sides of the rectangular or square window. For other shapes or irregular shapes, the four boundaries of the window can be the four points closest to the four boundaries of the display screen, or the straight lines containing the four points that are parallel to the boundaries of the display screen.

[0195] For example, such as Figure 17A As shown, window 1701 in computer 400 is a rectangular window. The edge of window 1701 that is closest to the right boundary is the right boundary of window 1701; the edge of window 1701 that is closest to the left boundary is the left boundary of window 1701; the edge of window 1701 that is closest to the top boundary is the top boundary of window 1701; and the edge of window 1701 that is closest to the bottom boundary is the bottom boundary of window 1701.

[0196] For example, such as Figure 17B As shown, window 1702 in computer 400 is an irregularly shaped window. The right boundary of window 1702 can be defined as either point H1, which is closest to the right edge of the computer 400's display screen, or a line L1 containing point H1 parallel to the right edge of the computer 400's display screen. Similarly, the top boundary of window 1702 can be defined as either point H2, which is closest to the top edge of the computer 400's display screen, or a line L2 containing point H2 parallel to the top edge of the computer 400's display screen. The left boundary of window 1702 can be defined as either point H3, which is closest to the left edge of the computer 400's display screen, or a line L3 containing point H3 parallel to the left edge of the computer 400's display screen. The bottom boundary of window 1702 can be defined as either point H4, which is closest to the bottom edge of the computer 400's display screen, or a line L4 containing point H4 parallel to the bottom edge of the computer 400's display screen.

[0197] For example, the explanation will be given with the first boundary of the first projection window as its right boundary and the first position as the right boundary of the remote device. At the first moment, that is, when the right boundary of the first projection window is at the right boundary of the remote device, the remote device can receive user operation U1. User operation U1 can be the user dragging the first projection window to the right.

[0198] like Figure 18As shown, computer 400 can be referred to as the remote device, and tablet 500 can be referred to as the local device. Window 1001 in computer 400 can be the first projection window. When the right edge of window 1001 is at the right edge of the computer 400 display screen, the user can continue to drag window 1001 to the right, dragging window 1001 onto tablet 500.

[0199] It is understandable that, optionally, user operation U1 can be a continuous operation, and the remote device can also receive user operation U1 before the first moment. For example, the user can keep dragging the first projection window to the right, and at the first moment, the user is still dragging the first projection window to the right.

[0200] Alternatively, in another possible implementation, the first boundary of the first projection window remains in the first position, and the user only begins to drag the first projection window onto the local device at the first moment. This application does not limit the scope of this implementation.

[0201] S103. The remote device acquires an image of the first part of the first projection window, wherein the first part is the portion of the first projection window that is not displayed on the remote device during the first time period starting from the first moment.

[0202] Starting from the first moment, the user continuously moves the first projection window from the remote device towards the local device within the first time period. The remote device can determine the portion of the first projection window that is not displayed on the remote device based on the distance moved by the user's operation U1. Then, the remote device can acquire the image of the first portion of the first projection window.

[0203] The remote device can acquire the movement distance of the user's operation U1. For example, if the user controls the mouse pointer to move the first projection window by sliding their finger across the touchscreen or touch panel on the remote device, the sensors in the touchscreen or touch panel on the remote device can detect the movement distance of the user's finger. The movement distance of the user's finger detected by the remote device can be the movement distance of the user's operation U1. If the user controls the mouse pointer to move the first projection window by using the mouse, the sensors in the mouse can detect the movement distance of the mouse, and the mouse can send the movement distance to the remote device. The remote device can use the movement distance of the mouse as the movement distance of the user's operation U1.

[0204] The remote device can determine, based on user operations, which portion of the first projection window can be displayed on the remote device's screen and which portion is not displayed on the remote device's screen.

[0205] In one possible implementation, the remote device can determine the first portion of the first projection window that is not displayed on the remote device based on the first boundary and the movement distance of the user operation U1.

[0206] like Figure 19 As shown, window 1101 displayed in computer 400 is... Figure 18 The window 1001 shown is a portion of what is displayed on the computer 400. The window 1102 shown on the tablet 500 is... Figure 18 The portion of window 1001 shown is not displayed in computer 400. When the user drags to the right... Figure 18 The window 1001 shown is moved a distance of d1. A portion of window 1001 not displayed on computer 400 is... Figure 19 The window 1102 shown is the area from the right boundary of window 1002 to the left by d1.

[0207] After the remote device determines that the first portion of the first projection window is not displayed on the remote device, it can acquire an image of that first portion. The remote device can generate an image of the first portion of the first projection window, referred to as the first portion image. This first portion image contains the content displayed in the first portion. For example, Figure 19 The window 1102 shown is displayed as an image on the tablet 500.

[0208] In one possible implementation, the remote device can also obtain the size of the local device's display screen. The remote device can also adaptively adjust the size of the first portion of the image based on the size of the local device's display screen.

[0209] Optionally, the remote device can also obtain the position information of the first part of the image. This position information can be the coordinates of the vertices in the first part of the image. For example, if the first part of the image is rectangular or square, then the position information of the first image can be the coordinates of the four vertices of the first part of the image, such as (x1, y1), (x2, y2), (x3, y3), and (x4, y4).

[0210] Furthermore, the coordinates of the vertices in the first part of the image can be coordinates in the remote device coordinate system or coordinates in the local device coordinate system. In this embodiment, the remote device coordinate system refers to a coordinate system established with a first point in the remote device as the origin. This first point can be the top-left vertex of the remote device's display screen. The local device coordinate system refers to a coordinate system established with a second point in the local device as the origin. This second point can be the top-left vertex of the local device's display screen.

[0211] Furthermore, if the coordinates of the vertices in the first part of the image are in the coordinate system of the remote device, the local device can convert the coordinates of the vertices in the first part of the image into coordinates in the local device's coordinate system based on the obtained screen size of the remote device. If the coordinates of the vertices in the first part of the image are in the coordinate system of the local device, that is, the remote device can obtain the coordinates of the vertices in the first part of the image in the remote device's coordinate system, the remote device can then convert the coordinates of the vertices in the first part of the image into coordinates in the local device's coordinate system based on the obtained screen size of the local device.

[0212] Alternatively, in one possible implementation, the location information may be the coordinates of the top-left corner of the first part of the image and the size of the first part of the image (e.g., the width and height of the first part of the image, or the resolution, etc.).

[0213] It is understood that the position information of the first part of the image is used to determine the position of the first part of the image displayed on the screen of this device. This application embodiment does not limit the specific content included in this position information.

[0214] In one possible implementation, the remote device may include, for example: Figure 1 The window capture service is shown in the image. A remote device can use this window capture service to acquire the first portion of the image.

[0215] S104. The remote device sends the first part of the image and the location information of the first part of the image to the local device.

[0216] The remote device can send the first part of the image and the location information of the first part of the image to the local device.

[0217] In one possible implementation, the remote device may include, for example: Figure 1 The image shows a layer encoding service module. The remote device can use this layer encoding service module to encode the first part of the image and send it to the local device.

[0218] In one possible implementation, the remote device can generate a transparent background image with the same size as the local device's display screen. The remote device overlays the first portion of the image onto the transparent background image to obtain the overlaid image. The remote device then sends the overlaid image to the local device.

[0219] For example, such as Figure 20 As shown, the image in the first part can be Figure 20 The image in window 1102 can have a transparent background image. Figure 20Image 2001 is displayed in the tablet 500. The size of image 2001 can be the same as the size of the display screen in the local device 500. The computer 400 can overlay the image of window 1102 onto image 2001 and send it to the tablet 500.

[0220] Furthermore, in one possible implementation, the remote device may include, for example... Figure 1 The image shows a layer overlay service module. Remote devices can use this module to overlay the first part of the image with a transparent background image to obtain the overlaid image.

[0221] Optionally, in step S104, the remote device can send the first part of the image to the local device, but does not need to send the location information of the first part of the image.

[0222] S105. The local device displays the image of the first part based on the location information of the first part of the image.

[0223] The local device can receive a first portion of the image and its location information from the remote device. Then, based on the location information of the first portion of the image, the local device displays that first portion of the image.

[0224] It is understandable that the screen size on the remote device and the screen size on the local device may differ. Therefore, the size of the first part of the image sent from the remote device to the local device can be adaptively adjusted according to the size of the local device's screen. For example, when the local device is a computer, the size of the first part of the image is a first size; if the local device is a mobile phone, the size of the first part of the image is a second size. When the computer's screen size is larger than the mobile phone's screen size, the first size is larger than the second size.

[0225] In one possible implementation, if the local device receives the first portion of image encoded data, the local device can decode the encoded data. Specifically, the local device may include, for example: Figure 1 The screen projection decoding service module shown allows the local device to decode the received image encoding data from the first part.

[0226] Alternatively, in one possible implementation, the local device may further include, for example: Figure 1 The layer Z-order service module shown allows the local device to set the display order of the first part of the image and the windows in the local device.

[0227] S106. At the second moment, the remote device receives the user's user operation U2. The second moment is the moment when the mouse pointer is detected to be in the second position.

[0228] At the second moment, the mouse pointer is at the second position, and the remote device can receive the user's user operation U2. The second position is located at the second boundary of the remote device, which is determined by the direction in which the user drags the projection window onto the local device, as determined by the remote device. If the remote device determines that the user can drag the projection window to the right onto the local device, then the second boundary is the right boundary of the remote device's display screen. If the remote device determines that the user can drag the projection window to the left onto the local device, then the second boundary is the left boundary of the remote device's display screen. If the remote device determines that the user can drag the projection window upwards onto the local device, then the second boundary is the upper boundary of the remote device's display screen. If the remote device determines that the user can drag the projection window downwards onto the local device, then the second boundary is the lower boundary of the remote device's display screen.

[0229] For example, a user can drag the first projection window to the right by controlling the mouse pointer until it is on the local device. When the mouse pointer is at the right edge of the remote device, the user can continue to drag the first projection window to the right by controlling the mouse pointer.

[0230] It is understandable that both the first and second positions are located at the second boundary of the remote device's display screen.

[0231] Understandably, in one possible implementation, if the mouse pointer is located elsewhere in the first projection window except for the first boundary, and the user drags the first projection window with the mouse pointer, at a first moment, the first boundary of the first projection window reaches the second boundary of the remote device's display screen. Then, the user continues to drag the first projection window, and at a second moment, the mouse pointer reaches the second boundary of the remote device's display screen. That is, the second moment can be a moment after the first moment.

[0232] It is understandable that, optionally, in one possible implementation, if the mouse pointer is at the first boundary of the first projection window, when the user drags the first projection window to the right, the mouse pointer and the first boundary of the first projection window can simultaneously reach the second boundary of the remote device's display screen. Alternatively, the first boundary of the first projection window is always at the first position on the remote device's display screen, i.e., on the second boundary of the remote device's display screen. The user places the mouse pointer on the first boundary of the first projection window and then drags the first projection window to the right using that mouse pointer. In this case, the first moment and the second moment are the same moment, that is, steps S102 and S106 are executed simultaneously. The remote device may not execute steps S103 and S104, and the local device may not execute step S105.

[0233] It is understandable that user operation U1 and user operation U2 can be two identical operations; that is, user operation U1 can be the user dragging the first projection window from the remote device to the local device. User operation U2 can also be the user dragging the first projection window from the remote device to the local device. For example, when the mouse pointer is at any position in the first projection window except for the first boundary (see [link to relevant documentation]). Figure 18 When the user drags the first projection window from the remote device to the local device, the operation of dragging the first projection window from the first position of the first boundary of the remote device's display screen to the second position of the mouse pointer is called user operation U1, and the operation of dragging the first projection window from the second position of the mouse pointer to the first position of the remote device's display screen is called user operation U2.

[0234] Furthermore, user operation U1 and user operation U2 can be continuous operations, that is, after the remote device receives user operation U1, it continuously receives user operation U2.

[0235] Optionally, user operation U1 and user operation U2 can also be the same operation. For example, when the mouse pointer is at the first boundary of the first projection window, the user drags the first projection window from the remote device to the local device. The operation of the user dragging the first projection window from the first boundary of the first projection window to the local device when the mouse pointer is at the second boundary of the remote device's display screen can be called user operation U1, or user operation U2.

[0236] S107a. The remote device acquires an image of the second part of the first projection window. The second part is a portion of the first projection window that is not displayed on the remote device at the second moment. The image of the second part contains the content of the image of the first part.

[0237] Starting from the second moment, the user continuously moves the first projection window from the remote device towards the local device during the second time period. The remote device can determine, based on the distance moved by the user's operation U2, that the second portion of the first projection window is not displayed on the remote device. Then, the remote device can acquire the image of the second portion of the first projection window.

[0238] It is understandable that if the mouse pointer is at the first boundary of the first projection window, then the second part of the first projection window can be the first part of the first projection window described in step S103 above. If the mouse pointer is not at the first boundary of the first projection window, then the second part of the first projection window can include the first part of the first projection window.

[0239] The second time period can be the period from when the remote device detects the user's operation U2 to when the remote device stops detecting the user's operation U2, that is, the period from when the user starts dragging the first projection window from the second position of the mouse pointer on the remote device until the user stops dragging the first projection window. The second time period can also be the period from when the remote device detects the mouse pointer at the second position on the remote device until the first projection window is fully displayed on the local device. Alternatively, the second time period can be a preset time period set by the remote device system, whereby the remote device sends an image of the portion of the first projection window not displayed on the remote device every preset time period. This embodiment of the application does not limit this.

[0240] Step S107a can be referred to the description in step S103, and will not be repeated here.

[0241] S107b: The remote device determines the position information of the mouse pointer in the third position in the local device based on the second position and the movement information of the user operation U2.

[0242] The user can move the mouse pointer from a second position to a third position using user operation U2. The remote device can obtain the position information of the mouse pointer when it is at the second position. The remote device can also obtain the movement information of user operation U2. Based on the position information of the second position and the movement information of user operation U2, the remote device can determine the position information of the third position, such as coordinates.

[0243] The movement information of user operation U2 can include the distance and direction of movement. The remote device can acquire this movement information. For example, if a user controls the mouse pointer to move the first projection window by sliding their finger across the touchscreen or touch panel of the remote device, the sensors in the touchscreen or touch panel can detect the distance and direction of the user's finger movement within a second time period. The distance and direction of the user's finger movement detected by the remote device can be the movement distance and direction of user operation U2. If the user controls the mouse pointer to move the first projection window by using the mouse, the sensors in the mouse can detect the mouse movement distance, and the mouse can send the movement distance and direction within a second time period to the remote device. The remote device can use the movement distance and direction of the mouse within the second time period as the movement distance and direction of user operation U2.

[0244] It is understandable that steps S107a and S107b can be executed simultaneously, or step S107b can be executed before step S107a.

[0245] S108. The remote device sends the second part of the image and the location information of the second part of the image, as well as the location information of the third location, to the local device.

[0246] The remote device can send the second part of the image and the location information of the second part of the image, as well as the location information of the third position of the mouse pointer, to the local device.

[0247] In one possible implementation, the remote device may include, for example: Figure 1 The layer encoding service module is shown in the image. The remote device can use this layer encoding service module to encode the second part of the image and send it to the local device.

[0248] In one possible implementation, the remote device can generate a transparent background image with the same size as the local device's display screen. The remote device then overlays the second part of the image onto the transparent background image to obtain a superimposed image. The remote device then sends the superimposed image to the local device.

[0249] Step S108 can be referred to the description in step S104, and will not be repeated here.

[0250] Understandably, the remote device can simultaneously send the second part of the image and the location information of the second part of the image, as well as the location information of the third location, to the local device, or it can send the second part of the image and the location information of the second part of the image, as well as the location information of the third location, to the local device one by one in multiple sessions.

[0251] Specifically, in one possible implementation, the image of the second part and the location information of the second part and the location information of the third part can be included in the same message, that is, the remote device can send the image of the second part and the location information of the second part and the location information of the third part to the local device through a single message.

[0252] Optionally, in one possible implementation, the location information of the second part of the image and the second part of the image can be included in the first message, and the location information of the third location can be included in the second message. The remote device sends the location information of the second part of the image and the second part of the image to the local device through the first message. The remote device sends the location information of the third location to the local device through the second message.

[0253] Optionally, in one possible implementation, the image of the second part is included in the third message, the location information of the image of the second part can be included in the fourth message, and the location information of the third location can be included in the fifth message. The remote device sends the image of the second part to the local device via the third message. The remote device sends the location information of the image of the second part to the local device via the fourth message. The remote device sends the location information of the third location to the local device via the fifth message.

[0254] S109. Based on the location information of the second part of the image, the local device displays the second part of the image and displays the mouse pointer at the third position.

[0255] The local device can receive the second part of the image and the position information of the second part of the image sent by the remote device. The local device can display the second part of the image on its display screen according to the position information of the second part of the image. The local device can also receive the position information of the third position of the mouse pointer and display the mouse pointer at the third position on its display screen.

[0256] In one possible implementation, if the local device receives the second portion of the image encoded data, the local device can decode the encoded data. Specifically, the local device may include, for example: Figure 1 The screen projection decoding service module shown allows the local device to decode the received second part of the image encoded data.

[0257] Alternatively, in one possible implementation, the local device may further include, for example: Figure 1 The layer Z-order service module shown allows the local device to set the display order of the second part of the image and the windows in the local device.

[0258] Optionally, in one possible implementation, if the local device originally displays a mouse pointer, when the remote device sends the position information of the third position of the mouse pointer of the remote device, the local device can display the mouse pointer in the local device according to the position information of the third position.

[0259] S110. At the third moment, the remote device receives the user's user operation U3. The third moment is the moment when the first projection window is fully displayed on the local device.

[0260] At the third moment, the remote device can receive the user's operation U3, which could be the user dragging the first projection window. For example, the user continues to drag the first projection window to the right.

[0261] User operation U3 can be the same operation as user operation U1 and user operation U2. The user can continuously perform user operation U1, user operation U2 and user operation U3. That is, there can be no time interval between user operation U1 and user operation U2, and there can be no time interval between user operation U2 and user operation U3.

[0262] In this embodiment of the application, user operation U1 can be referred to as the first drag operation, user operation U2 can also be referred to as the first drag operation, and user operation U3 can be referred to as the first drag operation. Alternatively, the first drag operation includes user operation U1, user operation U2, and user operation U3, that is, the user can be said to have completed the first drag operation only after continuously executing user operation U1, user operation U2, and user operation U3.

[0263] It is understandable that the first projection window displayed on the local device is the image of the first projection window. Therefore, when the user drags the first projection window on the local device, what is actually being dragged is the image of the first projection window.

[0264] S111. The remote device acquires the image of the first projection window and determines the position information of the mouse pointer at the fourth position based on the movement information of the user operation U3.

[0265] When the user has dragged the entire first projection window from the remote device to the local device, the remote device can obtain the first projection window.

[0266] The user can move the mouse pointer from the third position to the fourth position using user operation U3. The remote device can obtain the position information of the mouse pointer when it is at the third position. The remote device can also obtain the movement information of user operation U3. Based on the position information of the third position and the movement information of user operation U3, the remote device can determine the position information of the fourth position, such as coordinates.

[0267] Step S111 can be referred to the descriptions in steps S107a and S107b, and will not be repeated here.

[0268] S112. The remote device sends the image of the first projection window and the location information of the image of the first projection window, as well as the location information of the fourth position, to the local device.

[0269] The remote device can send the image of the first projection window, the position information of the image of the first projection window, and the position information of the fourth position of the mouse pointer to the local device.

[0270] In one possible implementation, the remote device may include, for example: Figure 1The layer encoding service module is shown in the image. The remote device can use this layer encoding service module to encode the image of the first projection window and send it to the local device.

[0271] In one possible implementation, the remote device can generate a transparent background image with the same size as the local device's display screen. The remote device overlays the image of the first projection window onto the transparent background image to obtain the overlaid image. The remote device then sends the overlaid image to the local device.

[0272] Step S112 can be referred to the description in step S108, and will not be repeated here.

[0273] S113. Based on the position information of the image in the first projection window, the local device displays the image of the first projection window and displays the mouse pointer at the fourth position.

[0274] This device can receive the image of the first projection window and its position information from the remote device. The local device can then display the image of the first projection window on its own screen according to the image's position information. The local device can also receive the position information of the fourth location of the mouse pointer and display that mouse pointer at the fourth location on its own screen.

[0275] In one possible implementation, if the local device receives encoded data of the image from the first projection window, the local device can decode the encoded data. Specifically, the local device may include, for example: Figure 1 The screen projection decoding service module shown allows the local device to decode the image encoded data received from the first screen projection window.

[0276] Alternatively, in one possible implementation, the local device may further include, for example: Figure 1 The layer Z-order service module shown allows the local device to set the image of the first projection window and the display order of windows in the local device.

[0277] In one possible implementation, the user can drag the first projection window from the local device back to the remote device. The mouse pointer can also be moved from the local device to the remote device.

[0278] Understandably, users can follow the steps described above to drag the second, third, and Nth projection windows from the remote device onto the local device. In other words, multiple windows from the remote device can be projected onto the local device via drag-and-drop operations.

[0279] In one possible implementation, when a remote device projects multiple windows onto a local device, the remote device can overlay the images of the multiple windows onto a transparent background image before sending the overlaid image to the local device. For example, the remote device can overlay the images of the first and second projected windows onto a transparent background image before sending the overlaid image to the local device. It's understood that the user first moves the first projected window from the remote device to the local device, and then moves the second projected window from the remote device to the local device. When the remote device moves the first projected window first, it only needs to overlay the image of the first projected window onto the transparent background image before sending it to the local device. Once the image of the first projected window is displayed on the local device, the user can then move the second projected window from the remote device to the local device. When the user continues to move the second projection window from the remote device to the local device, the remote device can overlay the images of the first projection window and the second projection window onto the transparent background image, and then send the overlaid image to the local device.

[0280] In this embodiment, the image of the first projection window can be referred to as the first image, the image of the first portion of the first projection window can be referred to as the first image, and the position information of the first portion of the image can be referred to as the first position information. The image of the second portion of the first projection window can be referred to as the first image, and the position information of the second portion of the image can be referred to as the first position information. The superimposed image can be referred to as the superimposed image.

[0281] In one possible implementation, when the local device displays its own mouse pointer, and the remote device's mouse pointer moves to the local device, the local device hides its own mouse pointer if it can display the remote device's mouse pointer. Here, the remote device's mouse pointer can be referred to as the first mouse pointer, and the local device's mouse pointer can be referred to as the second mouse pointer.

[0282] This application provides a screen mirroring method that allows users to drag a screen mirroring window from a remote device to their local device, or vice versa. This simple drag-and-drop operation provides a good user experience.

[0283] Furthermore, in some scenarios, after the local device displays the first projection window from the remote device, the user can also operate the projection window or the application window on the local device. Specific steps can be as follows: Figure 21 As shown.

[0284] like Figure 21 As shown in the embodiments of this application, a screen mirroring method may further include the following steps:

[0285] S201. The local device receives user operation U4, and the local device displays the image M1 of the first projection window.

[0286] The local device can receive user operation U4, and the local device can detect the position of the user operation U4 on the display screen.

[0287] The local device can display an image M1 from the first projection window. This image M1 is sent to the local device by the remote device.

[0288] Optionally, the local device may also display windows containing one or more local applications of the local device.

[0289] S202. The local device determines whether the user operation U4 is applied to the image M1 of the first projection window; if yes, then execute step S203a; if no, then execute step S203b.

[0290] The local device can determine whether user operation U4 is applied to image M1 of the first projection window based on the position of user operation U4 on the local device's display screen and the position information of image M1 in the first projection window. Specifically, if the coordinates of the four vertices of image M1 in the first projection window are (x1, y1), (x2, y2), (x3, y3), and (x4, y4), and the coordinates of the position of user operation U4 on the display screen are (xa, ya), if the coordinates (xa, ya) are within the area formed by the four vertices (x1, y1), (x2, y2), (x3, y3), and (x4, y4), then the local device determines that user operation A is applied to image M1 of the first projection window. Then the local device can execute step S203a. If the coordinates (xa, ya) are not within the area formed by the four vertices (x1, y1), (x2, y2), (x3, y3), and (x4, y4), then the local device determines that user operation A is applied to the local device. Then the local device can execute step S203b.

[0291] S203a. The local device sends a first instruction to the remote device, the first instruction being used to instruct the remote device to respond to user operation A.

[0292] The local device can send a first instruction to the remote device, which instructs the remote device to respond to the user operation U4. The first instruction may carry the type of the user operation U4 (click operation, movement operation, etc.) and the position information of the user operation U4 on the display screen. The user operation U4 can be an operation of clicking a first control, such as clicking a control in the first projection window to enlarge, shrink, or close the first projection window, etc. The user operation U4 can also be an operation of moving the first projection window.

[0293] S203b, Local device responds to user operation U4.

[0294] When the local device determines that the user operation U4 is applied to the local device, the local device can respond to the user operation U4. If the user operation U4 is applied to the window of application 1, the local device can notify application 1 to refresh the window of application 1 in response to the user operation U4. For example, application 1 can maximize the window of the first application.

[0295] S204. In response to user operation U4, the remote device obtains the image M2 of the first projection window and the position information of the image M2 of the first projection window.

[0296] If the user minimizes the first projection window, the remote device can acquire an image of the minimized first projection window. That is, image M2 of the minimized first projection window can be the image of the minimized first projection window. The remote device can also acquire the position information of the minimized first projection window.

[0297] If the user maximizes the first projection window, the remote device can acquire an image of the maximized first projection window. That is, image M2 of the first projection window can be the image of the maximized first projection window. The remote device can also acquire the position information of this maximized first projection window.

[0298] If the user operates U4 to move the first projection window, the remote device can obtain the image M2 of the first projection window and the position information of the moved first projection window. It is understood that the image M2 of the first projection window can have the same content displayed as the image M1 of the first projection window.

[0299] S205. The remote device sends the image M2 of the first projection window and the location information of the image M2 of the first projection window to the local device.

[0300] The remote device can send the image M2 of the first projection window and the location information of the image M2 of the first projection window to the local device. For details, please refer to the description in step S104 above, which will not be repeated here.

[0301] S206. The local device displays the image M2 of the first projection window according to the position information of the image M2 of the first projection window.

[0302] The local device can receive the image M2 of the first projection window and its location information from the remote device. Then, based on the location information of the image M2 of the first projection window, the local device displays the image M2 of the first projection window.

[0303] In this embodiment, both image M1 and image M2 can be referred to as the first image. If user operation U4 is applied to image M1 in the first projection window, then user operation U4 can be referred to as the first operation. If user operation U4 is applied to a local window of the local device, then user operation U4 can be referred to as the second operation.

[0304] In this way, users can operate the first projection window on their local device. Users can operate on application windows on their local device without needing to terminate projection from the remote device or hide the first projection window. This simplifies user operation and does not affect the normal use of the local device.

[0305] The exemplary electronic device 10 provided in the embodiments of this application will be introduced first below.

[0306] Figure 22 This is a schematic diagram of the structure of the electronic device 10 provided in the embodiments of this application. The electronic device 10 can be a remote device or a local device in the embodiments of this application.

[0307] The following describes the embodiment using electronic device 10 as an example. It should be understood that electronic device 10 may have more or fewer components than shown in the figures, may combine two or more components, or may have different component configurations. The various components shown in the figures can be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application-specific integrated circuits.

[0308] Electronic device 10 may include: processor 110, external memory interface 120, internal memory 121, universal serial bus (USB) interface 130, charging management module 140, power management module 141, battery 142, antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, sensor module 180, button 190, motor 191, indicator 192, camera 193, display screen 194, and subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0309] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the electronic device 10. In other embodiments of this application, the electronic device 10 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0310] Processor 110 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0311] The controller can serve as the nerve center and command center of the electronic device 10. The controller can generate operation control signals based on the instruction opcode and timing signals to control the fetching and execution of instructions.

[0312] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0313] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0314] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple I2C buses. The processor 110 can couple to the touch sensor 180K, charger, flash, camera 193, etc., through different I2C bus interfaces. For example, the processor 110 can couple to the touch sensor 180K through the I2C interface, enabling the processor 110 and the touch sensor 180K to communicate through the I2C bus interface, thereby realizing the touch function of the electronic device 10.

[0315] The I2S interface can be used for audio communication. In some embodiments, the processor 110 may include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to enable communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the I2S interface to enable the function of answering phone calls through a Bluetooth headset.

[0316] The PCM interface can also be used for audio communication, sampling, quantizing, and encoding analog signals.

[0317] The UART interface is a universal serial data bus used for asynchronous communication.

[0318] The MIPI interface can be used to connect the processor 110 to peripheral devices such as the display 194 and the camera 193. The MIPI interface includes the camera serial interface (CSI) and the display serial interface (DSI).

[0319] The GPIO interface can be configured via software. The GPIO interface can be configured as either control signals or data signals.

[0320] The SIM interface can be used to communicate with the SIM card interface 195 to transmit data to or read data from the SIM card.

[0321] USB interface 130 is an interface that conforms to the USB standard specification, specifically it can be a Mini USB interface, Micro USB interface, USB Type C interface, etc. USB interface 130 can be used to connect a charger to charge electronic device 10, and it can also be used for data transfer between electronic device 10 and peripheral devices.

[0322] It is understood that the interface connection relationships between the modules illustrated in the embodiments of the present invention are merely illustrative and do not constitute a structural limitation on the electronic device 10. In other embodiments of this application, the electronic device 10 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0323] The charging management module 140 is used to receive charging input from the charger. The charger can be a wireless charger or a wired charger.

[0324] The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 to power the processor 110, internal memory 121, external memory, display 194, camera 193, and wireless communication module 160, etc.

[0325] The wireless communication function of electronic device 10 can be implemented through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor, and baseband processor.

[0326] Antennas 1 and 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 10 can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with a tuning switch.

[0327] The mobile communication module 150 can provide solutions for wireless communication, including 2G / 3G / 4G / 5G, applied to the electronic device 10. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via the antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to the modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via the antenna 1. In some embodiments, at least some functional modules of the mobile communication module 150 may be housed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 may be housed in the same device.

[0328] The modem processor may include a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs sound signals through audio devices (not limited to speaker 170A, receiver 170B, etc.) or displays images or videos through the display screen 194. In some embodiments, the modem processor may be a separate device. In other embodiments, the modem processor may be independent of the processor 110 and may be housed in the same device as the mobile communication module 150 or other functional modules.

[0329] The wireless communication module 160 can provide solutions for wireless communication applications on the electronic device 10, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0330] In some embodiments, antenna 1 of electronic device 10 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, enabling electronic device 10 to communicate with networks and other devices via wireless communication technology. The wireless communication technology may include Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time-Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology, etc. The GNSS may include Global Positioning System (GPS), Global Navigation Satellite System (GLONASS), BeiDou Navigation Satellite System (BDS), Quasi-Zenith Satellite System (QZSS), and / or Satellite Based Augmentation Systems (SBAS).

[0331] Electronic device 10 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connecting the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0332] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Mini LED, a MicroLED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 10 may include one or N displays 194, where N is a positive integer greater than 1.

[0333] Electronic device 10 can perform shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor.

[0334] The ISP (Image Signal Processor) is used to process data fed back from the camera 193. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can also perform algorithmic optimization of image noise, brightness, and skin tone. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 193.

[0335] Camera 193 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In some embodiments, the electronic device 10 may include one or N cameras 193, where N is a positive integer greater than 1.

[0336] A digital signal processor (DSP) is used to process digital signals. Besides digital image signals, it can also process other digital signals. For example, when the electronic device 10 selects a frequency, the DSP can perform Fourier transforms on the frequency energy.

[0337] Video codecs are used to compress or decompress digital video. Electronic device 10 may support one or more video codecs. Thus, electronic device 100 can play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG 2, MPEG 3, MPEG 4, etc.

[0338] An NPU (Neural Processing Unit) is a neural network (NN) computing processor that, by borrowing from the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, rapidly processes input information and can continuously learn on its own. NPUs enable intelligent cognitive applications in electronic devices, such as image recognition, facial recognition, speech recognition, and text understanding.

[0339] Internal memory 121 may include one or more random access memory (RAM) and one or more non-volatile memory (NVM).

[0340] Random access memory can include static random-access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM, such as fifth-generation DDR SDRAM, which is generally called DDR5 SDRAM), etc.

[0341] Non-volatile memory can include disk storage devices and flash memory.

[0342] Flash memory can be classified according to its operating principle, including NOR FLASH, NAND FLASH, 3D NAND FLASH, etc.; according to the level of the storage cell, including single-level cell (SLC), multi-level cell (MLC), triple-level cell (TLC), quad-level cell (QLC), etc.; and according to the storage specification, including universal flash storage (UFS) and embedded multimedia card (eMMC), etc.

[0343] The random access memory can be directly read and written by the processor 110. It can be used to store executable programs (such as machine instructions) of the operating system or other running programs, as well as user and application data.

[0344] Non-volatile memory can also store executable programs and user and application data, and can be pre-loaded into random access memory for direct reading and writing by the processor 110.

[0345] Electronic device 10 can implement audio functions such as music playback and recording through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

[0346] Audio module 170 is used to convert digital audio information into analog audio signal output, and also to convert analog audio input into digital audio signal. Audio module 170 can also be used for encoding and decoding audio signals.

[0347] The speaker 170A, also known as a "loudspeaker," is used to convert audio electrical signals into sound signals. The electronic device 10 can listen to music or make hands-free calls through the speaker 170A.

[0348] The receiver 170B, also known as the "earpiece," is used to convert audio electrical signals into sound signals. When the electronic device 10 receives a telephone call or voice message, the receiver 170B can be brought close to the ear to hear the voice.

[0349] Microphone 170C, also known as a "microphone" or "voice transducer," is used to convert sound signals into electrical signals. When making a phone call or sending a voice message, the user can speak by bringing their mouth close to microphone 170C, inputting the sound signal into microphone 170C. Electronic device 10 may have at least one microphone 170C. In some embodiments, electronic device 10 may have two microphones 170C, which, in addition to collecting sound signals, can also perform noise reduction. In other embodiments, electronic device 10 may also have three, four, or more microphones 170C, which can collect sound signals, reduce noise, identify the sound source, and perform directional recording, etc.

[0350] The 170D headphone jack is used to connect wired headphones. The 170D headphone jack can be a USB 130 interface or a 3.5mm Open Mobile Terminal Platform (OMTP) standard interface, a CTIA (Cellular Telecommunications Industry Association of the USA) standard interface.

[0351] The pressure sensor 180A is used to sense pressure signals and can convert pressure signals into electrical signals.

[0352] The gyroscope sensor 180B can be used to determine the motion attitude of the electronic device 10.

[0353] The 180C barometric pressure sensor is used to measure barometric pressure.

[0354] The magnetic sensor 180D includes a Hall sensor. The electronic device 10 can use the magnetic sensor 180D to detect the opening and closing of the flip cover.

[0355] The 180E accelerometer can detect the magnitude of acceleration in various directions (typically three axes) of an electronic device 10. When the electronic device 10 is stationary, it can detect the magnitude and direction of gravity. It can also be used to identify the posture of the electronic device and is applied to applications such as screen orientation switching and pedometers.

[0356] Distance sensor 180F is used to measure distance.

[0357] The proximity sensor 180G may include, for example, a light-emitting diode (LED) and a light detector, such as a photodiode. The LED may be an infrared LED. The electronic device 10 emits infrared light outward through the LED. The electronic device 10 uses a photodiode to detect infrared reflected light from nearby objects.

[0358] The ambient light sensor 180L is used to sense the ambient light intensity. The electronic device 10 can adaptively adjust the brightness of the display screen 194 based on the sensed ambient light intensity. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also work with the proximity sensor 180G to detect whether the electronic device 10 is in a pocket to prevent accidental touches.

[0359] The fingerprint sensor 180H is used to collect fingerprints.

[0360] The 180J temperature sensor is used to detect temperature.

[0361] Touch sensor 180K, also known as a "touch panel," can be located on display screen 194. The touch sensor 180K and display screen 194 together form a touchscreen, also known as a "touchscreen." Touch sensor 180K detects touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 194. In other embodiments, touch sensor 180K may also be located on the surface of electronic device 10, in a different position than display screen 194.

[0362] Buttons 190 include a power button, volume buttons, etc. Buttons 190 can be mechanical buttons or touch-sensitive buttons. The electronic device 10 can receive button input and generate key signal inputs related to user settings and function control of the electronic device 10.

[0363] Motor 191 can generate vibration alerts. Motor 191 can be used for incoming call vibration alerts or for touch vibration feedback.

[0364] Indicator 192 can be an indicator light, used to indicate charging status, power changes, or to indicate messages, missed calls, notifications, etc.

[0365] The SIM card interface 195 is used to connect a SIM card. The electronic device 10 interacts with the network through the SIM card to realize functions such as making calls and data communication.

[0366] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0367] As used in the above embodiments, depending on the context, the term "when..." can be interpreted as meaning "if...", "after...", "in response to determining...", or "in response to detecting...". Similarly, depending on the context, the phrase "when determining..." or "if (the stated condition or event) is interpreted as meaning "if determining...", "in response to determining...", "when (the stated condition or event) is detected", or "in response to detecting (the stated condition or event)".

[0368] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive), etc.

[0369] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A screen projection method, characterized in that, The method is applied to a screen projection system, which includes a first electronic device and a second electronic device, wherein the first electronic device and the second electronic device establish a communication connection, and the method includes: The first electronic device detects a first drag operation, which is used to move the first projection window displayed on the screen of the first electronic device from the screen of the first electronic device to the screen of the second electronic device; In response to the first drag operation, the first electronic device acquires a first image of the first projection window, the first image including the display content of the first projection window that has been dragged out of the display screen of the first electronic device; The first electronic device detects a second drag operation, which is used to move the second projection window displayed on the display screen of the first electronic device from the display screen of the first electronic device to the display screen of the second electronic device; In response to the second drag operation, the first electronic device acquires a second image of the second projection window, the second image including the display content of the second projection window that has been dragged out of the display screen of the first electronic device; The first electronic device overlays the first image and the second image onto a transparent background image to obtain an overlaid image; the transparent background image is generated by the first electronic device according to the size of the display screen of the second electronic device, the size of the transparent background image is the same as the size of the display screen of the second electronic device, and the transparent background image is not displayed on the first electronic device. The first electronic device sends the overlaid image to the second electronic device; The second electronic device displays the overlaid image on its display screen.

2. The method according to claim 1, characterized in that, The first image contains part or all of the display content of the first projection window.

3. The method according to claim 1, characterized in that, Before the first electronic device detects the second drag operation, the method includes: The first electronic device sends the first image and the first location information to the second electronic device; the first location information is used to determine the position of the first image displayed on the screen of the second electronic device.

4. The method according to claim 3, characterized in that, After the first electronic device sends the first image and the first location information to the second electronic device, the method further includes: the second electronic device displays the first image on the display screen of the second electronic device according to the first location information.

5. The method according to claim 1, characterized in that, Before the first electronic device detects the first drag operation, the method further includes: The first electronic device determines a first relative position between the first electronic device and the second electronic device, and the first drag operation is used to move the first projection window from the display screen of the first electronic device to the display screen of the second electronic device in a first direction, wherein the first direction is the direction of the second electronic device relative to the first electronic device indicated by the first relative position.

6. The method according to claim 5, characterized in that, The first drag operation is used to move the first projection window from the display screen of the first electronic device to the display screen of the second electronic device in a first direction, including: The first direction is the right side of the second electronic device relative to the first electronic device, as indicated by the first relative position, and the first drag operation is used to move the first projection window from the display screen of the first electronic device to the right of the display screen of the second electronic device. Alternatively, the first direction is the left side of the second electronic device relative to the first electronic device as indicated by the first relative position, and the first drag operation is used to move the first projection window from the display screen of the first electronic device to the left of the display screen of the second electronic device. Alternatively, the first direction is the position above the second electronic device relative to the first electronic device, as indicated by the first relative position, and the first drag operation is used to move the first projection window upward from the display screen of the first electronic device to the display screen of the second electronic device. Alternatively, the first direction is the position of the second electronic device relative to the first electronic device as indicated by the first relative position, and the first drag operation is used to move the first projection window down from the display screen of the first electronic device to the display screen of the second electronic device.

7. The method according to claim 6, characterized in that, The first relative position is set by the user in the first electronic device; or the first electronic device obtains it based on the actual position of the first electronic device and the actual position of the second electronic device.

8. The method according to claim 1, characterized in that, The method further includes: In response to the first drag operation, the first electronic device sends the second position information of the first mouse pointer on the display screen of the first electronic device to the second electronic device, the second position information being used to determine the position of the first mouse pointer on the display screen of the second electronic device; The second electronic device displays the first mouse pointer on its display screen based on the second location information.

9. The method according to claim 8, characterized in that, The second electronic device displays a second mouse pointer on its screen, and the method further includes: When the second electronic device displays the first mouse pointer on its screen based on the second location information, it hides the second mouse pointer.

10. The method according to claim 1, characterized in that, After the second electronic device displays the overlaid image on its display screen, the method further includes: The second electronic device receives the first operation, which is applied to the first image; The second electronic device sends a first instruction to the first electronic device, the first instruction being used to instruct the first electronic device to respond to the first operation; The first application to which the first projection window belongs in the first electronic device responds to the first operation.

11. The method according to any one of claims 1-10, characterized in that, After the second electronic device displays the overlaid image on its display screen, the method further includes: The second electronic device receives a second operation, which is performed in a first window, where the first window is the window of a second application in the second electronic device; The second application of the second electronic device responds to the second operation.

12. A screen projection method, characterized in that, The method includes: The first electronic device detects a first drag operation, which is used to move the first projection window from the display screen of the first electronic device to the display screen of the second electronic device, and the second electronic device establishes a communication connection with the first electronic device; In response to the first drag operation, the first electronic device acquires a first image of the first projection window, the first image including the display content of the first projection window that has been dragged out of the display screen of the first electronic device; The first electronic device detects a second drag operation, which is used to move the second projection window displayed on the display screen of the first electronic device from the display screen of the first electronic device to the display screen of the second electronic device; In response to the second drag operation, the first electronic device acquires a second image of the second projection window, the second image including the display content of the second projection window that has been dragged out of the display screen of the first electronic device; The first electronic device overlays the first image and the second image onto a transparent background image to obtain an overlaid image; the transparent background image is generated by the first electronic device according to the size of the display screen of the second electronic device, the size of the transparent background image is the same as the size of the display screen of the second electronic device, and the transparent background image is not displayed on the first electronic device. The first electronic device sends the overlaid image to the second electronic device.

13. The method according to claim 12, characterized in that, Before the first electronic device detects the second drag operation, the method further includes: The first electronic device sends the first image and the first location information to the second electronic device; the first location information is used to determine the position of the first image displayed on the screen of the second electronic device.

14. The method according to claim 13, characterized in that, Before the first electronic device detects the first drag operation, the method further includes: The first electronic device determines a first relative position between the first electronic device and the second electronic device, and the first drag operation is used to move the first projection window from the display screen of the first electronic device to the display screen of the second electronic device in a first direction, wherein the first direction is the direction of the second electronic device relative to the first electronic device indicated by the first relative position.

15. The method according to claim 14, characterized in that, The first drag operation is used to move the first projection window from the display screen of the first electronic device to the display screen of the second electronic device in a first direction, including: The first direction is the right side of the second electronic device relative to the first electronic device, as indicated by the first relative position, and the first drag operation is used to move the first projection window from the display screen of the first electronic device to the right of the display screen of the second electronic device. Alternatively, the first direction is the left side of the second electronic device relative to the first electronic device as indicated by the first relative position, and the first drag operation is used to move the first projection window from the display screen of the first electronic device to the left of the display screen of the second electronic device. Alternatively, the first direction is the position above the second electronic device relative to the first electronic device, as indicated by the first relative position, and the first drag operation is used to move the first projection window upward from the display screen of the first electronic device to the display screen of the second electronic device. Alternatively, the first direction is the position of the second electronic device relative to the first electronic device as indicated by the first relative position, and the first drag operation is used to move the first projection window down from the display screen of the first electronic device to the display screen of the second electronic device.

16. The method according to claim 12, characterized in that, The method further includes: In response to the first drag operation, the first electronic device sends second position information of the first mouse pointer to the second electronic device, the second position information being used to determine the position of the first mouse pointer on the display screen of the second electronic device.

17. The method according to any one of claims 12-16, characterized in that, After the first electronic device sends the overlaid image to the second electronic device, the method further includes: The first electronic device receives a first instruction sent by the second electronic device. The first instruction is used to instruct the first electronic device to respond to a first operation, which is performed on the first image displayed in the second electronic device. The first application to which the first projection window belongs in the first electronic device responds to the first operation.

18. A screen projection method, characterized in that, include: The second electronic device receives an overlay image sent by the first electronic device. The overlay image is obtained by superimposing a first image and a second image onto a transparent background image. The first image is obtained by the first electronic device from a first projection window when receiving a first drag operation. The first image includes the display content of the first projection window that has been dragged out of the display screen of the first electronic device. The first drag operation is used to move the first projection window from the display screen of the first electronic device to the display screen of the second electronic device. The second image is obtained by the first electronic device from a second projection window when receiving a second drag operation. The second image includes the display content of the second projection window that has been dragged out of the display screen of the first electronic device. The second drag operation is used to move the second projection window from the display screen of the first electronic device to the display screen of the second electronic device. The transparent background image is generated by the first electronic device according to the size of the display screen of the second electronic device. The size of the transparent background image is the same as the size of the display screen of the second electronic device. The transparent background image is not displayed in the first electronic device. The second electronic device and the first electronic device have established a communication connection. The second electronic device displays the overlaid image on its display screen.

19. The method according to claim 18, characterized in that, The method further includes: The second electronic device receives a first image and first location information sent by the first electronic device, wherein the first location information is used to determine the position of the first image displayed on the screen of the second electronic device.

20. The method according to claim 19, characterized in that, After the second electronic device receives the first image and first location information sent by the first electronic device, the method further includes: The second electronic device displays the first image on its display screen based on the first location information.

21. The method according to claim 18, characterized in that, The method further includes: The second electronic device receives a second location message sent by the first electronic device. The second location message is used to determine the position of the first mouse pointer in the first electronic device on the display screen of the second electronic device. The second electronic device displays the first mouse pointer on its display screen based on the second location information.

22. The method according to claim 21, characterized in that, The second electronic device displays a second mouse pointer on its screen, and the method further includes: When the second electronic device displays the first mouse pointer on its screen based on the second location information, it hides the second mouse pointer.

23. The method according to claim 18, characterized in that, After the second electronic device displays the overlay image, the method further includes: The second electronic device receives the first operation, which is applied to the first image; The second electronic device sends a first instruction to the first electronic device, the first instruction being used to instruct the first electronic device to respond to the first operation.

24. The method according to any one of claims 18-23, characterized in that, After the second electronic device displays the overlay image, the method further includes: The second electronic device receives a second operation, which is performed in a first window, where the first window is the window of a second application in the second electronic device; The second application of the second electronic device responds to the second operation.

25. A screen projection system, characterized in that, The screen projection system includes a first electronic device and a second electronic device, which are connected in communication. The first electronic device is used to execute the screen projection method as described in any one of claims 12-17, and the second electronic device is used to execute the screen projection method as described in any one of claims 18-24.

26. An electronic device, characterized in that, include: One or more processors and one or more memories; the one or more memories are respectively coupled to the one or more processors; the one or more memories are used to store computer program code, the computer program code including computer instructions; when the computer instructions are executed on the processor, the electronic device causes the electronic device to perform the screen projection method as described in any one of claims 12-17 and / or claims 18-24.

27. A computer-readable storage medium, characterized in that, Includes computer instructions that, when executed on an electronic device, cause the electronic device to perform the screen projection method as described in any one of claims 12-17 or 18-24.

28. A computer program product containing instructions, characterized in that, When the computer instructions are executed on the electronic device, the electronic device causes the electronic device to perform the screen projection method as described in any one of claims 12-17 or 18-24.

Images