Mirror image control method and device, computer equipment and storage medium

Abstract

The invention provides a mirror image control method and device, computer equipment and a storage medium. The method comprises the steps that under the condition that a target device displays a mirror image, an interaction event aiming at the mirror image is monitored, original coordinates of the interaction event on the target device are obtained, and the mirror image is a mirror image of a display picture on a source device; determining a target coordinate of the interaction event on the source device according to the original coordinate; and sending the interaction event carrying the target coordinate to the source device, so that the source device responds to the interaction event. According to the mirroring operation method provided by the invention, direct control of a user on the source equipment through the target equipment can be realized, the operation is convenient, the user experience is greatly improved, and the mirroring operation method has relatively high security.

Description

Technical Field

[0001] The present disclosure relates to the field of mirror control technology, and particularly to a mirror control method, apparatus, computer device, and storage medium. Background Art

[0002] Screen mirroring technology is a currently popular technology that can mirror the mobile phone screen to large-screen devices such as computers and TVs, that is, generate a mirror image of the mobile phone on the target device, thereby facilitating the user to view the content displayed on the mobile phone.

[0003] In related technologies, it is usually impossible to directly control the mobile phone through the mirror image, but to indirectly control the mobile phone by deploying simulated controls such as simulated buttons and simulated mice, which is inconvenient to operate and the user experience is poor. Summary of the Invention

[0004] To overcome the problems existing in related technologies, the present disclosure provides a mirror control method, apparatus, computer device, and storage medium.

[0005] In a first aspect of the present disclosure, a mirror control method is provided, and the method includes:

[0006] When a mirror image is displayed on a target device, listen for interaction events for the mirror image and obtain the original coordinates of the interaction events on the target device, where the mirror image is a mirror image of the content displayed on a source device;

[0007] Determine the destination coordinates of the interaction event on the source device according to the original coordinates;

[0008] Send the interaction event carrying the destination coordinates to the source device so that the source device responds to the interaction event.

[0009] Optionally, the determining the destination coordinates of the interaction event on the source device according to the original coordinates includes:

[0010] Determine the destination coordinates of the interaction event on the source device according to the width of the mirror image, the height of the mirror image, the width of the content displayed on the source device, the height of the content displayed on the source device, and the original coordinates.

[0011] Optionally, the determining the destination coordinates of the interaction event on the source device according to the width of the mirror image, the height of the mirror image, the width of the content displayed on the source device, and the height of the content displayed on the source device includes:

[0012] Use the product of the ratio of the width of the mirror image to the width of the display screen on the source device and the abscissa of the original coordinate as the abscissa of the target coordinate, and use the product of the ratio of the height of the mirror image to the height of the display screen on the source device and the ordinate of the original coordinate as the ordinate of the target coordinate.

[0013] Optionally, sending the interaction event carrying the target coordinate to the source device to enable the source device to respond to the interaction event includes:

[0014] Send the interaction event carrying the target coordinate to the source device, so that the source device maps the interaction event to the system layer of the source device based on reflection technology to perform a simulation operation.

[0015] Optionally, sending the interaction event carrying the target coordinate to the source device to enable the source device to respond to the interaction event includes:

[0016] Based on the soft bus, send the interaction event carrying the target coordinate to the source device to enable the source device to respond to the interaction event, where the soft bus is used to determine the optimal communication medium between the source device and the target device according to the communication factors in the current environment, and establish an information transmission channel between the source device and the target device based on the optimal communication medium.

[0017] Optionally, sending the interaction event carrying the target coordinate to the source device to enable the source device to respond to the interaction event includes:

[0018] Replace the original coordinate carried in the interaction event with the target coordinate, and send the replaced interaction event to the source device to enable the source device to respond to the interaction event.

[0019] Optionally, the method is executed by the system layer of the target device.

[0020] A second aspect of the present disclosure provides a mirror control method, the method includes:

[0021] When the target device displays a mirror image, receive and respond to the interaction event carrying the target coordinate sent by the target device;

[0022] Wherein, the mirror image is a mirror image of the display screen on the source device, the target coordinate represents the coordinate of the interaction event on the source device, and the target coordinate is determined by the target device listening to the interaction event for the mirror image and according to the original coordinate of the obtained interaction event on the target device.

[0023] Optionally, receiving and responding to the interaction event carrying the destination coordinates sent by the target device includes:

[0024] Receiving the interaction event carrying the destination coordinates sent by the target device, and mapping the interaction event to the system layer of the source device based on reflection technology to perform a simulation operation.

[0025] Optionally, the method is executed by the system layer of the source device.

[0026] A third aspect of the present disclosure provides a mirror control device, and the device includes:

[0027] An event acquisition module, configured to listen for an interaction event for the mirror and obtain the original coordinates of the interaction event on the target device when the mirror is displayed on the target device, where the mirror is a mirror of the display screen on the source device;

[0028] A coordinate determination module, configured to determine the destination coordinates of the interaction event on the source device according to the original coordinates;

[0029] A sending module, configured to send the interaction event carrying the destination coordinates to the source device, so that the source device responds to the interaction event.

[0030] Optionally, when the coordinate determination module is configured to determine the destination coordinates of the interaction event on the source device according to the original coordinates, it is configured to:

[0031] Determine the destination coordinates of the interaction event on the source device according to the width of the mirror, the height of the mirror, the width of the display screen on the source device, the height of the display screen on the source device, and the original coordinates.

[0032] Optionally, when the coordinate determination module is configured to determine the destination coordinates of the interaction event on the source device according to the width of the mirror, the height of the mirror, the width of the display screen on the source device, and the height of the display screen on the source device, it is configured to:

[0033] Use the product of the ratio of the width of the mirror to the width of the display screen on the source device and the abscissa of the original coordinates as the abscissa of the destination coordinates, and use the product of the ratio of the height of the mirror to the height of the display screen on the source device and the ordinate of the original coordinates as the ordinate of the destination coordinates.

[0034] Optionally, when the sending module is configured to send the interaction event carrying the destination coordinates to the source device so that the source device responds to the interaction event, it is configured to:

[0035] Send the interaction event carrying the destination coordinates to the source device, so that the source device maps the interaction event to the system layer of the source device based on reflection technology to perform a simulation operation.

[0036] Optionally, the sending module is used to send the interaction event carrying the destination coordinates to the source device. When the source device responds to the interaction event, it is used for:

[0037] Based on the soft bus, send the interaction event carrying the destination coordinates to the source device, so that the source device responds to the interaction event. Wherein, the soft bus is used to determine the optimal communication medium between the source device and the target device according to the communication factors in the current environment, and establish an information transmission channel between the source device and the target device based on the optimal communication medium.

[0038] Optionally, the sending module is used to send the interaction event carrying the destination coordinates to the source device. When the source device responds to the interaction event, it is used for:

[0039] Replace the original coordinates carried in the interaction event with the destination coordinates, and send the replaced interaction event to the source device, so that the source device responds to the interaction event.

[0040] Optionally, the device is applied to the system layer of the target device.

[0041] A fourth aspect of the present disclosure provides a mirror control device, the device includes:

[0042] An event response module, configured to receive and respond to an interaction event carrying destination coordinates sent by the target device when the target device displays a mirror;

[0043] Wherein, the mirror is a mirror of the display screen on the source device, the destination coordinates represent the coordinates of the interaction event on the source device, and the destination coordinates are determined by the target device listening to the interaction event for the mirror and according to the original coordinates of the obtained interaction event on the target device.

[0044] Optionally, when the event response module is used to receive and respond to the interaction event carrying the destination coordinates sent by the target device, it is used for:

[0045] Receive the interaction event carrying the destination coordinates sent by the target device, and map the interaction event to the system layer of the source device based on reflection technology to perform a simulation operation.

[0046] Optionally, the device is applied to the system layer of the source device.

[0047] The fifth aspect of the present disclosure provides a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the method described in any one of the first aspect or the second aspect is implemented.

[0048] The sixth aspect of the present disclosure provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the method described in any one of the first aspect or the second aspect is implemented.

[0049] The technical solutions provided by the embodiments of the present disclosure may include the following beneficial effects:

[0050] In the embodiments of the present disclosure, when the target device displays a mirror, it listens for interaction events for the mirror, determines the destination coordinates of the interaction event on the source device according to the original coordinates of the interaction event on the target device, and sends the interaction event carrying the destination coordinates to the source device, so that the source device responds to the interaction event carrying the destination coordinates. Among them, the mirror control method does not need to rely on virtual buttons, virtual mice, nor does it need to rely on third-party applications or virtual touch layers. It can directly control the mobile phone by listening to the interaction events sent on the target device, which is convenient to operate, greatly improves the user experience, and has high security. At the same time, since the interaction event sent from the target device to the source device carries the determined destination coordinates, the source device can directly respond to the interaction event without repackaging or other special processing of the interaction event, thus realizing a seamless control on the source device side (for the source device, this interaction event is no different from the interaction event generated by the user directly operating the source device), reducing the resource occupancy and operating power consumption on the source device side. It should be noted that in the mirror projection technology, the source device is usually responsible for resource operation and service processing, while the target device is usually only responsible for image display. Therefore, reducing the operation resource occupancy and operating power consumption on the source device side is particularly important.

[0051] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The accompanying drawings herein are incorporated into the specification and form a part of the present disclosure, showing embodiments consistent with the present disclosure and used together with the specification to explain the principles of the present disclosure.

[0053] Figure 1 It is a schematic diagram of a mirror shown in some exemplary embodiments.

[0054] Figure 2 It is a flowchart of a mirror control method shown in some exemplary embodiments.

[0055] Figure 3 is a flow chart of another mirror control method shown in some exemplary embodiments.

[0056] Figure 4 is a timing diagram of a mirror control method shown in some exemplary embodiments.

[0057] Figure 5 is a block diagram of a mirror control device showing some exemplary embodiments.

[0058] Figure 6 is a block diagram of another mirror control device shown in some exemplary embodiments.

[0059] Figure 7 The figure is a hardware structure diagram of a computer device shown in some exemplary embodiments. DETAILED DESCRIPTION

[0060] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0061] The terms used in this disclosure are for the purpose of describing specific embodiments only and are not intended to limit the disclosure. The singular forms of "a", "said" and "the" used in this disclosure and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0062] It should be understood that although the terms first, second, third, etc. may be used in the present disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present disclosure, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0063] Screen projection technology is currently a popular technology. It can display the mobile phone screen on large-screen devices such as computers and TVs, that is, create a mirror image of the mobile phone on the target device, making it easier for users to watch the screen displayed on the mobile phone.

[0064] In the related art, it is usually impossible to directly control a mobile phone through mirroring. Instead, the control of the mobile phone is indirectly achieved by deploying simulated controls such as simulated buttons and simulated mice, which is inconvenient to operate and results in a poor user experience. In addition, in some related technologies, a transparent floating window is run or other methods are used to establish a transparent touch control layer on the device (in the specification of the present disclosure, this control layer is referred to as a virtual touch control layer) to implement simulated touch operations. However, this solution increases the number of background applications on the mobile phone, affects the device performance and increases power consumption. At the same time, it also increases the operation latency, further affecting the user experience.

[0065] In view of this, the present disclosure provides a mirror control method, device, computer device and storage medium.

[0066] Please refer to Figure 1 , which is a schematic diagram of a mirror involved in some embodiments of the present disclosure. Among them, a mobile phone (source device) is connected to a tablet computer (target device), and the display screen is transmitted to the tablet computer, and a mirror of the mobile phone display screen is generated on the tablet computer. Among them, the display screen may be the content displayed on the entire mobile phone screen, or the screen corresponding to one or more application software running on the mobile phone, or a partial screen of the mobile phone or a partial screen of one or more application software running on the mobile phone. The present disclosure does not limit this.

[0067] Next, the mirror control method provided by the embodiments of the present disclosure will be described in detail from the perspective of the target device side and the source device side.

[0068] The first aspect of the present disclosure provides a mirror control method applied to the target device side. Please refer to Figure 2 , which includes the following steps:

[0069] Step S201, when the mirror is displayed on the target device, listen for interaction events for the mirror and obtain the original coordinates of the interaction events on the target device, where the mirror is a mirror of the display screen on the source device.

[0070] The original coordinates may be the relative position of the interaction event on the screen of the target device. Preferably, they may be the relative position of the interaction event on the mirror. An interaction event is an event generated by a user's interaction with the mirror, such as a click event, a double - click event, a swipe event, etc. The target device monitors the interaction events for the mirror and, when an interaction event occurs, obtains the content of the interaction event (including the original coordinates of the interaction event on the target device). As an example, the framework of the execution method may monitor the interaction events based on the Android standard interface and, after detecting an interaction event, obtain this interaction event; as another example, the sensors of the target device (such as a gyroscope, a microphone, etc.) may be monitored, and the events indicated by the actions collected by the sensors may be automatically recognized (such as capturing a "shake" event through the gyroscope).

[0071] Step S202: Determine the destination coordinates of the interaction event on the source device according to the original coordinates.

[0072] Among them, the destination coordinates may be coordinates relative to the entire screen of the source device or coordinates relative to the displayed screen on the source device. In practical applications, if the displayed screen corresponding to the mirror on the source device is not the content displayed on the entire screen of the source device, then in one case, the source device may generate a coordinate system for the displayed screen corresponding to the mirror. At this time, preferably, the destination coordinates may be coordinates relative to the displayed screen on the source device; in another case, when the source device does not generate a separate coordinate system for the displayed screen on the source device, then preferably, the destination coordinates may be coordinates relative to the entire screen of the source device. It should be understood that in the following text, the case where the destination coordinates are the coordinates of the displayed screen on the source device may be used to illustrate the embodiments, but its specific setting may vary according to different actual situations, and the present disclosure does not limit this.

[0073] Exemplarily, one endpoint of the mirror (such as the upper - left vertex or the lower - left vertex of its circumscribed rectangle) may be used as the origin to establish a coordinate system, and the corresponding endpoint on the displayed screen of the source device may be used as the origin to establish another coordinate system. The step of determining the destination coordinates of the interaction event on the source device according to the original coordinates may include:

[0074] Determine the destination coordinates of the interaction event on the source device according to the width of the mirror, the height of the mirror, the width of the displayed screen on the source device, the height of the displayed screen on the source device, and the original coordinates.

[0075] Specifically, the step of determining the destination coordinates of the interaction event on the source device according to the width of the mirror, the height of the mirror, the width of the displayed screen on the source device, and the height of the displayed screen on the source device includes:

[0076] Use the product of the ratio of the width of the mirror image to the width of the display screen on the source device and the abscissa of the original coordinate as the abscissa of the target coordinate, and use the product of the ratio of the height of the mirror image to the height of the display screen on the source device and the ordinate of the original coordinate as the ordinate of the target coordinate.

[0077] That is, the target coordinates (x, y) can be calculated based on formula (1), where phoneWidth is the actual width of the display screen of the source device, phoneHeight is the actual height of the display screen of the source device, projectionWidth is the actual width of the mirror image, projectionHeight is the actual height of the mirror image, input(x) is the abscissa of the interaction event in the mirror image (the origin of the coordinate system can be selected according to the actual situation, but preferably, the position of the origin of the mirror image coordinate system in the mirror image and the position of the origin of the source device coordinate system in the display screen of the source device should have a corresponding proportional relationship), and input(y) is the ordinate of the interaction event in the mirror image. Based on this, it is possible to determine the target coordinates with less computing resources, and still be able to determine the accurate target coordinates even when the mirror image has been scaled or stretched. It should be understood that the target coordinates calculated based on the above formula are the coordinates of the interaction event in the display screen of the source device.

[0078]

[0079] Exemplarily, a coordinate system can be established with an endpoint of the target device screen (such as the upper left vertex or the lower left vertex of its circumscribed rectangle) as the origin, and another coordinate system can be established with the corresponding endpoint on the source device screen as the origin. In this example, the relative position of the interaction event in the mirror image can be determined by means of the relative position of the mirror image in the target device screen and the position of the interaction event, and this relative position can be used as the target coordinate.

[0080] Step S203, send the interaction event carrying the target coordinates to the source device so that the source device responds to the interaction event.

[0081] Among them, the specific manner of responding to the interaction event can be determined according to the preset configuration of the source device. For example, an equivalent operation of the interaction event can be implemented on the source device; for another example, the source device can directly map the received interaction event to the system layer of the source device based on the Reflection technology, so that the system layer responds to the interaction event, thereby realizing a seamless interaction on the source device side, that is:

[0082] Sending the interaction event carrying the target coordinates to the source device so that the source device responds to the interaction event includes:

[0083] Send the interaction event carrying the target coordinates to the source device, so that the source device maps the interaction event to the system layer of the source device based on reflection technology to perform simulation operations.

[0084] Based on this, for the source device, this interaction event is no different from the interaction event generated by the user directly operating the source device. Therefore, there is no need to modify the underlying code of the system, nor to translate or repackage the event, which is highly efficient without occupying additional system resources.

[0085] There can be multiple ways for the interaction event to carry the target coordinates. Exemplarily, the target coordinates and the interaction event can be encapsulated into the same data packet; additionally, the target coordinates can be used to replace the part corresponding to the original coordinates in the interaction event, so that the interaction event is equivalent to the interaction event generated by the user directly operating the source device, thereby achieving complete imperceptibility on the source device side (that is, sending the interaction event carrying the target coordinates to the source device so that the source device responds to the interaction event includes replacing the original coordinates carried in the interaction event with the target coordinates and sending the replaced interaction event to the source device so that the source device responds to the interaction event).

[0086] In the embodiments of the present disclosure, when the target device is displaying a mirror, it listens for interaction events for the mirror, determines the target coordinates of the interaction event on the source device based on the original coordinates of the interaction event on the target device, and sends the interaction event carrying the target coordinates to the source device, so that the source device responds to the interaction event carrying the target coordinates. Among them, the mirror control method does not need to rely on virtual buttons, virtual mice, nor on third-party applications or virtual touch layers, and can directly control the mobile phone by listening for interaction events sent on the target device, which is convenient to operate, greatly improves the user experience, and has high security; at the same time, since the interaction event sent from the target device to the source device carries the determined target coordinates, the source device can directly respond to the interaction event without repackaging or other special processing of the interaction event, thereby achieving imperceptible control on the source device side and reducing the resource occupancy and operating power consumption on the source device side. It should be noted that in mirror projection technology, the source device is usually responsible for resource operation and service processing, while the target device is usually only responsible for image display. Therefore, reducing the occupancy of computing resources and operating power consumption on the source device side is particularly important. It can be seen that the mirror control method provided by the present disclosure has high practical value.

[0087] In some embodiments of the present disclosure, in step S203, the target device may send the interaction event to the source device based on multiple methods.

[0088] Exemplarily, sending the interaction event carrying the destination coordinate to the source device to enable the source device to respond to the interaction event includes:

[0089] Based on the soft bus, sending the interaction event carrying the destination coordinate to the source device to enable the source device to respond to the interaction event, where the soft bus is used to determine the optimal communication medium between the source device and the target device according to the communication factors in the current environment, and establish an information transmission channel between the source device and the target device based on the optimal communication medium.

[0090] Among them, the soft bus is a software bus capable of establishing a communication connection, which can be a distributed soft bus or a self-configured and packaged communication connection framework. Preferably, the soft bus can have the following functions. The soft bus can automatically discover other devices configured with the soft bus and establish a communication connection at the system layer of these devices; at the same time, the soft bus is used to determine the optimal communication medium (such as connecting through Bluetooth or through a local area network) between the target terminal and the source device according to the communication factors in the current environment (such as the local area network connection quality, Bluetooth connection quality), and establish an information transmission channel between the target terminal and the source device based on the optimal communication medium. Preferably, the information transmission channel between the target terminal and the source device can be maintained or switched in real time according to the communication factors between these two terminals. For example, if the target terminal and the source device decide through an algorithm that the local area network is optimal, they can choose to establish an information transmission channel based on the local area network and perform information synchronization; if the target terminal and the source device decide through an algorithm that there is already an information transmission channel between these devices, they can reuse the information transmission channel; if the target terminal and the source device decide through an algorithm that multicast can be used to transmit information at this time, they can choose to establish an information transmission channel based on multicast and perform information synchronization; when the local area network connection between the target terminal and the source device is disconnected, if the target terminal and the source device can be connected through Bluetooth, they can choose to establish an information transmission channel based on Bluetooth and perform information synchronization. Based on this, an efficient and secure interaction event sending process can be realized.

[0091] Exemplarily again, a picture transmission stream and a control transmission stream can be established between the system layer of the device where the mirror is located and the system layer of the source device;

[0092] Among them, the screen transmission stream is used to transmit the display content of the source device to the device where the mirror is located, so that the device where the mirror is located presents the display content of the source device to the user; the control transmission stream is used to send the interaction event and the destination coordinate to the source device, so that the source device performs a simulation operation according to the interaction event and the destination coordinate. Based on this, stable transmission of the video stream and the control stream can be achieved, and there is a relatively small communication delay.

[0093] In some embodiments of the present disclosure, the method is executed by the system layer of the target device. Thus, the framework for executing the method can directly call the system layer interface (such as the view event listening system based on Android native) to obtain the interaction event, and directly establish a transmission channel (such as the soft bus channel or the two transmission streams described above) between the system layer of the target device and the system layer of the source device. The display screen on the source device and the interaction event of the user with the mirror do not need to be transmitted through the public network and the third-party server (this method has relatively low security and is prone to leakage of user privacy), and it is also not necessary to monitor or transmit events by establishing a virtual touch layer or through the interfaces provided by the developer mode and debugging tools of the system. It has relatively high efficiency and high security at the same time.

[0094] The second aspect of the present disclosure provides a mirror control method applied to the source device side. Please refer to Figure 3 , which includes the following steps:

[0095] Step S301, when the target device displays the mirror, receive and respond to the interaction event sent by the target device carrying the destination coordinate;

[0096] Among them, the mirror is a mirror of the display screen on the source device, and the destination coordinate represents the coordinate of the interaction event on the source device. The destination coordinate is determined by the target device listening to the interaction event for the mirror and according to the original coordinate of the obtained interaction event on the target device.

[0097] The original coordinate can be the relative position of the interaction event with respect to the screen of the target device, or the relative position of the interaction event with respect to the mirror. The interaction event is an event generated by the user's interaction with the mirror, such as a click event, a double-click event, a swipe event, etc. The target device listens to the interaction event for the mirror and, when an interaction event occurs, obtains the content of the interaction event (including the original coordinate of the interaction event on the target device).

[0098] The specific way to respond to the interaction event can be determined according to the preset configuration of the source device. For example, an equivalent operation of the interaction event can be implemented on the source device; for another example, the source device can directly map the received interaction event to the system layer of the source device based on the Reflection technology, so that the system layer responds to the interaction event, thereby realizing a seamless interaction on the source device side, that is:

[0099] Receiving and responding to the interaction event carrying the destination coordinate sent by the target device includes:

[0100] Receiving the interaction event carrying the destination coordinate sent by the target device, and mapping the interaction event to the system layer of the source device based on the Reflection technology to perform a simulation operation.

[0101] Based on this, for the source device, this interaction event is no different from the interaction event generated by the user directly operating the source device. Therefore, there is no need to modify the underlying code of the system, nor to translate or repackage the event, which has high efficiency while not occupying additional system resources.

[0102] In some embodiments of the present disclosure, the method is executed by the system layer of the source device. Thus, the framework of the execution method can directly call the system layer interface to respond to the interaction event, and directly establish a transmission channel between the system layer of the target device and the system layer of the source device (such as the soft bus channel or the two transmission streams described in the foregoing embodiments). The display screen on the source device and the interaction event of the user with the mirror do not need to be transmitted through the public network and the third-party server (with low security and easy leakage of user privacy), which has high efficiency and high security at the same time.

[0103] In the embodiments of the present disclosure, when the target device displays a mirror, it listens for interaction events for the mirror, determines the destination coordinates of the interaction event on the source device based on the original coordinates of the interaction event on the target device, and sends the interaction event carrying the destination coordinates to the source device, so that the source device responds to the interaction event carrying the destination coordinates. Among them, the mirror control method does not need to rely on virtual buttons, virtual mice, nor does it need to rely on third-party applications or virtual touch layers. It can directly control the mobile phone by listening for interaction events sent on the target device, which is convenient to operate, greatly improves the user experience, and has high security. At the same time, since the interaction event sent from the target device to the source device carries the determined destination coordinates, the source device can directly respond to the interaction event without repackaging or other special processing of the interaction event, thus realizing seamless control on the source device side and reducing the resource occupancy and operating power consumption on the source device side. It should be noted that in the mirror projection technology, the source device is usually responsible for resource operation and service processing, while the target device is usually only responsible for image display. Therefore, reducing the operation resource occupancy and operating power consumption on the source device side is particularly important. It can be seen that the mirror control method provided by the present disclosure has high practical value. In addition, for the specific details and beneficial effects of the embodiments of the second aspect, reference can be made to the corresponding parts in the embodiments of the first aspect, which will not be elaborated here.

[0104] Next, in combination with the Figure 4 sequence diagram of the mirror control method shown below, at least one embodiment of the present disclosure will be further described.

[0105] First, the mobile phone (source device) projects the current display screen onto the tablet computer (target device), so that a mirror corresponding to the display screen appears on the target device (for example, Figure 1 the schematic diagram shown).

[0106] When the user operates on the mirror (such as clicking on the screen), the interaction event will be captured by the system layer of the tablet computer. Next, the tablet computer determines the destination coordinates of this interaction event corresponding to the display screen of the mobile phone based on the original coordinates corresponding to the interaction event in the mirror, and uses these destination coordinates to replace the original coordinates in the interaction event to encapsulate an interaction event carrying the destination coordinates.

[0107] Next, the tablet computer sends the encapsulated interaction event to the system layer of the mobile phone based on the cross-device data sending technology (see the example above).

[0108] After receiving the interaction event, the mobile phone parses the interaction event and reflects this interaction event to the system layer based on the reflection technology, so as to directly simulate the operation performed by the user on the mobile phone side.

[0109] For the technical details and technical effects involved in the above steps, please refer to the foregoing embodiments and will not be elaborated herein.

[0110] Corresponding to the embodiments of the foregoing method, the present disclosure also provides embodiments of an apparatus and a terminal to which the apparatus is applied.

[0111] A third aspect of the present disclosure provides a mirror control apparatus, please refer to Figure 5 , the apparatus includes:

[0112] An event acquisition module 501, configured to listen for an interaction event for the mirror and acquire the original coordinates of the interaction event on the target device when the mirror is displayed on the target device, where the mirror is a mirror of the display screen on the source device;

[0113] A coordinate determination module 502, configured to determine the destination coordinates of the interaction event on the source device according to the original coordinates;

[0114] A sending module 503, configured to send the interaction event carrying the destination coordinates to the source device so that the source device responds to the interaction event.

[0115] Optionally, when the coordinate determination module 502 is used to determine the destination coordinates of the interaction event on the source device according to the original coordinates, it is configured to:

[0116] Determine the destination coordinates of the interaction event on the source device according to the width of the mirror, the height of the mirror, the width of the display screen on the source device, the height of the display screen on the source device, and the original coordinates.

[0117] Optionally, when the coordinate determination module 502 is used to determine the destination coordinates of the interaction event on the source device according to the width of the mirror, the height of the mirror, the width of the display screen on the source device, and the height of the display screen on the source device, it is configured to:

[0118] Take the product of the ratio of the width of the mirror to the width of the display screen on the source device and the abscissa of the original coordinates as the abscissa of the destination coordinates, and take the product of the ratio of the height of the mirror to the height of the display screen on the source device and the ordinate of the original coordinates as the ordinate of the destination coordinates.

[0119] Optionally, when the sending module 503 is used to send the interaction event carrying the destination coordinates to the source device so that the source device responds to the interaction event, it is configured to:

[0120] Send the interaction event carrying the destination coordinates to the source device, so that the source device maps the interaction event to the system layer of the source device based on the reflection technology to perform a simulation operation.

[0121] Optionally, the sending module 503 is used to send the interaction event carrying the destination coordinates to the source device. When the source device responds to the interaction event, it is used for:

[0122] Based on the soft bus, send the interaction event carrying the destination coordinates to the source device, so that the source device responds to the interaction event. Wherein, the soft bus is used to determine the optimal communication medium between the source device and the target device according to the communication factors in the current environment, and establish an information transmission channel between the source device and the target device based on the optimal communication medium.

[0123] Optionally, the sending module 503 is used to send the interaction event carrying the destination coordinates to the source device. When the source device responds to the interaction event, it is used for:

[0124] Replace the original coordinates carried in the interaction event with the destination coordinates, and send the replaced interaction event to the source device, so that the source device responds to the interaction event.

[0125] Optionally, the device is applied to the system layer of the target device.

[0126] A fourth aspect of the present disclosure provides a mirror control device. Please refer to Figure 6 , the device includes:

[0127] An event response module 601, configured to receive and respond to an interaction event carrying destination coordinates sent by the target device when the target device displays a mirror;

[0128] Wherein, the mirror is a mirror of the display screen on the source device, the destination coordinates represent the coordinates of the interaction event on the source device, and the destination coordinates are determined by the target device listening to the interaction event for the mirror and according to the original coordinates of the obtained interaction event on the target device.

[0129] Optionally, when the event response module 601 is used to receive and respond to the interaction event carrying the destination coordinates sent by the target device, it is used for:

[0130] Receive the interaction event carrying the destination coordinates sent by the target device, and map the interaction event to the system layer of the source device based on the reflection technology to perform a simulation operation.

[0131] Optionally, the device is applied to the system layer of the source device.

[0132] For the implementation processes of the functions and roles of the various modules in the above device, please refer to the implementation processes of the corresponding steps in the above method for details, which will not be elaborated here.

[0133] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can be referred to the partial description of the method embodiment. The device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed to multiple network modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the present disclosure solution. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0134] In a fifth aspect, the embodiment of the mirror control device provided by the present disclosure can be applied to a computer device. Please refer to the attached Figure 7 , which exemplarily shows a hardware schematic diagram of a computer device. For example, the device 700 can be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0135] The device 700 may include one or more of the following components: a processing component 701, a memory 702, a power component 703, a multimedia component 704, an audio component 705, an input / output (I / O) interface 706, a sensor component 707, and a communication component 708.

[0136] The processing component 701 generally controls the overall operation of the device 700, such as operations associated with display, telephone call, data communication, camera operation, and recording operation. The processing component 701 may include one or more processors 709 to execute instructions to complete all or part of the steps of the above method. In addition, the processing component 701 may include one or more modules to facilitate the interaction between the processing component 701 and other components. For example, the processing component 701 may include a multimedia module to facilitate the interaction between the multimedia component 704 and the processing component 701.

[0137] The memory 702 is configured to store various types of data to support the operation of the device 700. Examples of such data include instructions for any application or method operating on the device 700, contact data, phone book data, messages, pictures, videos, and the like. The memory 702 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, a magnetic disk, or an optical disk.

[0138] The power component 703 provides power to the various components of the device 700. The power component 703 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 700.

[0139] The multimedia component 704 includes a screen that provides an output interface between the device 700 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can not only sense the boundaries of the touch or swipe actions, but also detect the duration and pressure associated with the touch or swipe operation. In some embodiments, the multimedia component 704 includes a front camera and / or a rear camera. When the device 700 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each of the front camera and the rear camera can be a fixed optical lens system or have a focal length and optical zoom capabilities.

[0140] The audio component 705 is configured to output and / or input audio signals. For example, the audio component 705 includes a microphone (MIC) that is configured to receive external audio signals when the device 700 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals can be further stored in the memory 702 or transmitted via the communication component 708. In some embodiments, the audio component 705 further includes a speaker for outputting audio signals.

[0141] The I / O interface 706 provides an interface between the processing component 701 and a peripheral interface module, which can be a keyboard, a click wheel, buttons, etc. These buttons can include, but are not limited to: a home button, a volume button, a power-on button, and a lock button.

[0142] The sensor assembly 707 includes one or more sensors for providing a status assessment of various aspects of the device 700. For example, the sensor assembly 707 can detect the on / off state of the device 700, the relative positioning of components, such as the display and keypad of the device 700. The sensor assembly 707 can also detect a change in the position of the device 700 or a component of the device 700, the presence or absence of user contact with the device 700, the orientation or acceleration / deceleration of the device 700, and the temperature change of the device 700. The sensor assembly 707 can also include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 707 can also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 707 can also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0143] The communication component 708 is configured to facilitate communication between the device 700 and other devices in a wired or wireless manner. The device 700 can access a wireless network based on communication standards, such as WiFi, 2G or 3G, 4G or 5G, or a combination thereof. In an exemplary embodiment, the communication component 708 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 708 further includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0144] In an exemplary embodiment, the device 700 can be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components for performing the mirror control method of the above computer device.

[0145] In a sixth aspect, in an exemplary embodiment of the present disclosure, there is also provided a non-transitory computer-readable storage medium including instructions, such as a memory 702 including instructions, and the above instructions can be executed by a processor 709 of the device 700 to complete the mirror control method of the above computer device. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0146] The above describes specific embodiments of the present disclosure. Other embodiments are within the scope of the appended claims. In some cases, the acts or steps recited in the claims may be performed in a different order than in the embodiments and still achieve the desired result. Additionally, the processes depicted in the figures do not necessarily require the particular order or sequential order shown to achieve the desired result. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0147] Those skilled in the art will readily conceive of other implementations of the present disclosure after considering the specification and practicing the invention herein. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include known common knowledge or conventional technical means in the technical field not claimed in the present disclosure. The specification and examples are only to be considered exemplary, and the true scope and spirit of the present disclosure are pointed out by the following claims.

[0148] It should be understood that the present disclosure is not limited to the exact structures described above and shown in the figures, and various modifications and changes can be made without departing from its scope. The scope of the present disclosure is only limited by the appended claims.

[0149] The foregoing is only a preferred embodiment of the present disclosure and is not intended to limit the present disclosure. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present disclosure shall be included within the scope of protection of the present disclosure.

Claims

1. A mirror control method, characterized in that, The method includes: When the mirror is displayed on the target device, listening for interaction events for the mirror and obtaining the original coordinates of the interaction events on the target device, where the mirror is a mirror of the display screen on the source device; Determining the destination coordinates of the interaction event on the source device according to the original coordinates; Sending the interaction event carrying the destination coordinates to the source device so that the source device responds to the interaction event.

2. The mirror control method according to claim 1, wherein The determining the destination coordinates of the interaction event on the source device according to the original coordinates includes: Determining the destination coordinates of the interaction event on the source device according to the width of the mirror, the height of the mirror, the width of the display screen on the source device, the height of the display screen on the source device, and the original coordinates.

3. The mirror control method according to claim 2, characterized in that The determining the destination coordinates of the interaction event on the source device according to the width of the mirror, the height of the mirror, the width of the display screen on the source device, and the height of the display screen on the source device includes: Taking the product of the ratio of the width of the mirror to the width of the display screen on the source device and the abscissa of the original coordinates as the abscissa of the destination coordinates, and taking the product of the ratio of the height of the mirror to the height of the display screen on the source device and the ordinate of the original coordinates as the ordinate of the destination coordinates.

4. The mirror control method according to claim 1, wherein The sending the interaction event carrying the destination coordinates to the source device so that the source device responds to the interaction event includes: Sending the interaction event carrying the destination coordinates to the source device so that the source device maps the interaction event to the system layer of the source device based on reflection technology to perform an analog operation.

5. The mirror control method according to claim 1, wherein The sending the interaction event carrying the destination coordinates to the source device so that the source device responds to the interaction event includes: Based on the soft bus, sending the interaction event carrying the destination coordinates to the source device so that the source device responds to the interaction event, where the soft bus is used to determine the optimal communication medium between the source device and the target device according to the communication factors in the current environment, and establish an information transmission channel between the source device and the target device based on the optimal communication medium.

6. The mirror control method according to claim 1, wherein The sending the interaction event carrying the destination coordinates to the source device so that the source device responds to the interaction event includes: Replacing the original coordinates carried in the interaction event with the destination coordinates, and sending the replaced interaction event to the source device so that the source device responds to the interaction event.

7. The mirror control method according to claim 1, wherein The method is executed by the system layer of the target device.

8. A mirror control method, characterized in that, The method includes: When the mirror is displayed on the target device, receiving and responding to the interaction event carrying the destination coordinates sent by the target device; Where the mirror is a mirror of the display screen on the source device, the destination coordinates represent the coordinates of the interaction event on the source device, and the destination coordinates are determined by the target device listening for the interaction event for the mirror and according to the original coordinates of the interaction event obtained on the target device.

9. The mirror control method according to claim 8, wherein Receiving and responding to an interaction event carrying a destination coordinate sent by the target device includes: Receiving the interaction event carrying the destination coordinate sent by the target device, and mapping the interaction event to the system layer of the source device based on reflection technology to perform a simulation operation.

10. The mirror control method according to claim 8, characterized in that, The method is executed by the system layer of the source device.

11. A mirror control device, characterized in that, The device includes: An event acquisition module, configured to, when a mirror image is displayed on the target device, monitor an interaction event for the mirror image and obtain the original coordinate of the interaction event on the target device, where the mirror image is a mirror image of the display screen on the source device; A coordinate determination module, configured to determine the destination coordinate of the interaction event on the source device according to the original coordinate; A sending module, configured to send the interaction event carrying the destination coordinate to the source device, so that the source device responds to the interaction event.

12. A mirror control device, characterized in that, The device includes: An event response module, configured to, when a mirror image is displayed on the target device, receive and respond to an interaction event carrying a destination coordinate sent by the target device; where the mirror image is a mirror image of the display screen on the source device, the destination coordinate represents the coordinate of the interaction event on the source device, and the destination coordinate is determined by the target device monitoring the interaction event for the mirror image and according to the obtained original coordinate of the interaction event on the target device.

13. A computer device, characterized in that, It includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the method according to any one of claims 1 to 10 is implemented.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, the method according to any one of claims 1 to 10 is implemented.

Images