Device control method and terminal device

Abstract

This application provides a device control method and a terminal device, and relates to the field of terminal technology. The method includes: generating a touch slide event in response to a touch slide operation performed by a user by sliding from outside the projection window to inside the projection window after the destination projection device displays a projection window of the source projection device; converting the touch slide event into an edge slide event of the source projection device when the starting touch point of the touch slide operation is within a target area, and then sending the edge slide event to the source projection device, where the target area is an area formed by extending the edge of the projection window outward a preset distance; and updating the screen display content based on the edge slide event after the source projection device receives the edge slide event. According to the technical solution provided in this application, the convenience of performing a reverse control operation using a bezel-less screen gesture in a projection scenario can be improved, thereby improving the user experience.

Description

[Technical field]

[0001] This application claims priority to Chinese Patent Application No. 202011548741.5, filed with the China National Intellectual Property Office on December 24, 2020, and entitled “DEVICE CONTROL METHOD AND TERMINAL DEVICE,” which is incorporated herein by reference in its entirety.

[0002] This application relates to the field of terminal technology, and in particular to a device control method and a terminal device. [Background technology]

[0003] With the rapid development of terminal technology, device interconnection technology has been widely used. Through device interconnection technology, users can conveniently transfer content on one terminal device (specifically, the source projection device) to another terminal device (specifically, the destination projection device). For example, using the projection function, a projection window of a mobile phone (specifically, a mirror image of the screen content of the mobile phone) can be displayed on a tablet, and a user may make gesture operations in the projection window to control the mobile phone in reverse.

[0004] At present, the screen-to-body ratio of terminal devices is increasing, and bezel-less screens have become mainstream. For the convenience of users, major manufacturers have developed related bezel-less screen gestures. For example, related control functions (e.g., navigation bar functions) can be realized using edge slide gestures that slide inward from the edge of the screen. However, there is a problem that the operation is inconvenient when the source projection device is controlled by the bezel-less screen gesture on the destination projection device. Summary of the Invention

[0005] In view of this, the present application provides a device control method and a terminal device that improve the convenience of performing reverse control operations using bezel-less screen gestures in a projection scenario, thereby improving the user experience.

[0006] In order to achieve the above object, according to a first aspect of the present invention, an embodiment of this application provides a device control method applied to a system including a destination projection device and a source projection device. The method includes:

[0007] The destination projection device displays the projection window of the source projection device.

[0008] The destination projection device generates a first touch-slide event in response to a first touch-slide action performed by the user by sliding from outside the projection window to inside the projection window.

[0009] When a starting touch point of the first touch-slide action is located within the target area, the destination projection device converts the first touch-slide event into an edge-slide event for the source projection device and sends the edge-slide event to the source projection device, the target area being an area formed by extending outward from an edge of the projection window a preset distance.

[0010] After receiving the edge slide event, the source projection device updates the screen display content based on the edge slide event.

[0011] According to the device control method provided in this embodiment, the destination projection device may convert a touch-slide event generated by a touch-slide operation performed by a user by sliding from outside the projection window to inside the projection window into an edge-slide event of the source projection window. In this way, the source projection device may detect an edge-slide gesture based on the complete edge-slide event, which can improve the success rate of the edge-slide gesture of the source projection device being triggered by the user on the destination projection device, thereby improving the convenience of the reverse control operation and the user experience.

[0012] In a possible implementation of the first aspect, the first touch slide event includes a series of touch events generated by the first touch slide motion, and converting the first touch slide event into an edge slide event of the projection window includes converting touch events whose coordinates are located within the target area and are within the first touch slide event into screen edge events of the originating projection device, and generating an edge slide event based on the screen edge events and the touch events whose coordinates are located within the projection window and are within the first touch slide event.

[0013] In the above implementation, the touch event in the target area is converted into a screen edge event of the originating projection device. Because the target area includes the touch down event, a complete edge slide event may be obtained based on the screen edge event obtained through the conversion and the touch event corresponding to the touch slide operation in the projection window, so that the user can successfully trigger the edge slide gesture of the originating projection device.

[0014] In a possible implementation of the first aspect, the first touch slide event includes a series of touch events generated by a first touch slide motion, and converting the first touch slide event into an edge slide event of the projection window includes generating a touch down event based on a target event, where the target event is a first touch event generated by the touch slide motion in the projection window and the touch down event is a screen edge event of the originating projection device, and generating an edge slide event based on the touch down event and a touch event located within the projection window and within the first touch slide event.

[0015] In the above implementation, a touch down event belonging to the screen edge event of the originating projection device is generated based on the first touch event in the projection window, and then a complete edge slide event can be obtained based on the touch down event and the touch event in the projection window, so that the user can successfully trigger the edge slide gesture of the originating projection device.

[0016] In a possible implementation of the first aspect, generating a touch down event based on the target event includes inserting a touch down event before the target event, where coordinates of the touch down event are the same as coordinates of the target event.

[0017] In a possible implementation of the first aspect, generating the touch down event based on the target event includes converting the target event into a touch down event.

[0018] In a possible implementation of the first aspect, the method further includes:

[0019] When the starting touch point of the first touch-slide action is located outside the target area, the destination projection device updates the screen display content of the destination projection device in response to the first touch-slide event.

[0020] In the above implementation, when the starting touch point of the first touch-slide operation is located outside the target area, the destination projection device uses the operation as an operation for the screen display content of the destination projection device and responds to the operation, thereby improving the accuracy of the event response result and thereby improving the user experience.

[0021] In a possible implementation of the first aspect, the method further includes:

[0022] In response to a second touch-slide action performed by the user on the source projection device, an edge slide gesture is detected by the source projection device for a second touch-slide event generated by the second touch-slide action using the first region as a screen edge region.

[0023] In response to a third touch-slide action performed by the user by sliding inward from the projection window, the destination projection device generates a corresponding third touch-slide event to the source projection device and transmits the third touch-slide event to the source projection device.

[0024] The source projection device uses the second region as a screen edge region, and after receiving the third touch-slide event, detects an edge-slide gesture for the third touch-slide event, where the first region and the second region are both regions formed by extending inward from a screen edge of the source projection device, and the range of the second region is greater than the range of the first region.

[0025] If an edge sliding gesture is detected, the source projection device executes a control function corresponding to the detected edge sliding gesture and updates the screen display content.

[0026] In the above implementation, the source projection device uses the first region as a screen edge region to detect an edge slide gesture for a touch slide event triggered by a user on the source projection device, and uses a second region larger than the first region as a screen edge region to detect an edge slide gesture for the received touch slide event, thus improving the success rate of the edge slide gesture of the source projection device being triggered by a user on the destination projection device, thereby improving the reliability and user experience of the reverse control operation.

[0027] In a possible implementation of the first aspect, the edge slide gesture is a gesture to go back to a previous level, a gesture to go back to the home screen, or a gesture for a recent task list.

[0028] In a possible implementation of the first aspect, the method further includes:

[0029] The source projection device transmits the screen display contents to the destination projection device.

[0030] After receiving the on-screen display content, the destination projection device displays the on-screen display content within the projection window.

[0031] In the above implementation, after updating the screen display content, the source projection device transmits the screen display content to the destination projection device, so that the destination projection device can update the content of the projection window in time.

[0032] According to a second aspect, an embodiment of this application provides a device control method applied to a destination projection device, the method includes: displaying a projection window of a source projection device; generating a first touch slide event in response to a touch slide action performed by a user by sliding from outside the projection window to inside the projection window; converting the touch slide event into an edge slide event of the source projection device when a starting touch point of the touch slide action is located within a target area, and sending the edge slide event to the source projection device, where the target area is an area formed by extending outward from an edge of the projection window a preset distance.

[0033] In a possible implementation of the second aspect, the touch slide event includes a series of touch events generated by a touch slide motion, and converting the touch slide events into edge slide events of the projection window includes converting touch events whose coordinates are located within the target area and are within the touch slide event into screen edge events of the originating projection device, and generating edge slide events based on the screen edge events and the touch events whose coordinates are located within the projection window and are within the touch slide event.

[0034] In a possible implementation of the second aspect, the touch slide event includes a series of touch events generated by a touch slide motion, and converting the touch slide event into an edge slide event of the projection window includes generating a touch down event based on a target event, where the target event is a first touch event generated by the touch slide motion in the projection window and the touch down event is a screen edge event of the originating projection device, and generating an edge slide event based on the touch down event and a touch event located within the projection window and within the touch slide event.

[0035] In a possible implementation of the second aspect, generating a touch down event based on the target event includes inserting a touch down event before the target event, where coordinates of the touch down event are the same as coordinates of the target event.

[0036] In a possible implementation of the second aspect, generating the touch down event based on the target event includes converting the target event into a touch down event.

[0037] In yet another possible implementation of the second aspect, the method further includes updating a screen display content of the destination projection device in response to a touch-slide event when an initiating touch point of the touch-slide operation is located outside the target area.

[0038] In yet another possible implementation of the second aspect, the method further includes receiving screen display content corresponding to the touch slide event and transmitted by the source projection device, and displaying the screen display content within the projection window.

[0039] According to a third aspect, an embodiment of the present application provides a device control method applied to a source projection device, the method includes: when a touch-slide action performed by a user on the source projection device is received, using a first region as a screen edge region to detect an edge-slide gesture for a touch-slide event generated by the touch-slide action, and when a touch-slide event transmitted by a destination projection device is received, using a second region as a screen edge region to detect an edge-slide gesture for the received touch-slide event, where both the first region and the second region are regions formed by extending inward from a screen edge of the source projection device, and a coverage of the second region is greater than a coverage of the first region; and performing a control function corresponding to the detected edge-slide gesture to update a screen display content.

[0040] According to a fourth aspect, an embodiment of this application provides a device control method applied to a destination projection device, the device including: a display module configured to display a projection window of a source projection device; an input module configured to receive a first touch slide event in response to a touch slide operation performed by a user by sliding from outside the projection window to inside the projection window; a processing module configured to generate a touch slide event in response to the touch slide operation and convert the touch slide event into an edge slide event of the source projection device when a starting touch point of the touch slide operation is located within a target area, the target area being an area formed by extending a preset distance outward from an edge of the projection window; and a communication module configured to transmit the edge slide event to the source projection device.

[0041] In a possible implementation of the fourth aspect, the touch slide event includes a series of touch events generated by a touch slide motion. The processing module is specifically configured to convert touch events whose coordinates are located within the target area and are within the touch slide event into screen edge events of a source projection device, and generate edge slide events based on the screen edge events and the touch events whose coordinates are located within the projection window and are within the touch slide event.

[0042] In a possible implementation of the fourth aspect, the touch slide event includes a series of touch events generated by a touch slide motion. The processing module is specifically configured to generate a touch down event based on a target event, where the target event is a first touch event generated by the touch slide motion in the projection window, and the touch down event is a screen edge event of the originating projection device, and to generate an edge slide event based on the touch down event and a touch event located in the projection window and within the touch slide event.

[0043] In a possible implementation of the fourth aspect, the processing unit is specifically configured to insert a touch down event before a target event, where the coordinates of the touch down event are the same as the coordinates of the target event.

[0044] In a possible implementation of the fourth aspect, the processing unit is specifically configured to convert a target event into a touch down event.

[0045] In a possible implementation of the fourth aspect, the processing unit is further configured to, in response to a touch-slide event, update a screen display content of the destination projection device using the display module, in particular when a starting touch point of the touch-slide operation is located outside the target area.

[0046] In a possible implementation of the fourth aspect, the communication module is further configured to receive screen display content corresponding to the touch slide event and transmitted by the source projection device.

[0047] The display module is further configured to display the screen display content within the projection window.

[0048] According to a fifth aspect, an embodiment of the present application provides a device control apparatus for a source projection device, the apparatus including: an input module, a processing module, a communication module, and a display module.

[0049] The processing module is configured, when the input module receives a touch-slide action performed by a user on the source projection device, to use the first region as a screen edge region and detect an edge-slide gesture for a touch-slide event generated by the touch-slide action.

[0050] The processing module is further configured such that, when the communication module receives a touch slide event transmitted by the destination projection device, it uses the second region as a screen edge region and detects an edge slide gesture for the received touch slide event, wherein the first region and the second region are both regions formed by extending inward from a screen edge of the source projection device, and the coverage of the second region is greater than the coverage of the first region.

[0051] The processing module is further configured to execute a control function corresponding to the detected edge slide gesture and update the screen display content using the display module.

[0052] According to a sixth aspect, an embodiment of the present application provides a terminal device including a memory and a processor, the memory being configured to store a computer program, the processor being configured to execute a method according to the second aspect or the third aspect when invoking the computer program.

[0053] According to a seventh aspect, an embodiment of the present application provides a computer readable storage medium storing a computer program which, when executed by a processor, implements a method according to the second or third aspect.

[0054] According to an eighth aspect, an embodiment of the present application provides a computer program product, which when executed on a terminal device enables the terminal device to perform a method according to the second aspect or the third aspect.

[0055] According to a ninth aspect, an embodiment of the present application provides a chip system including a processor coupled to a memory, the processor executing a computer program stored in the memory to implement a method according to the second or third aspect. The chip system may be a single chip or a chip module including multiple chips.

[0056] The beneficial effects of the second to ninth aspects can be understood by referring to the relevant description in the first aspect, and the details will not be described again here. [Brief description of the drawings]

[0057] [Figure 1] FIG. 1 is a schematic diagram of a projection scenario according to an embodiment of the present application.

[0058] [Diagram 2] FIG. 2 is a schematic diagram of a touch-slide operation according to one embodiment of the present application.

[0059] [Diagram 3] FIG. 2 is a schematic diagram of the structure of a terminal device according to an embodiment of this application;

[0060] [Figure 4] 1 is a schematic diagram of a software architecture of a terminal device according to one embodiment of this application.

[0061] [Diagram 5] 1 is a schematic diagram of a target area according to an embodiment of the present application.

[0062] [Figure 6] FIG. 2 is a schematic diagram of an event handling process according to an embodiment of the present application.

[0063] [Figure 7] FIG. 2 is a schematic diagram of an event coordinate transformation according to an embodiment of the present application.

[0064] [Figure 8] FIG. 2 is a schematic diagram of an event processing result according to an embodiment of the present application;

[0065] [Figure 9] FIG. 13 is a schematic diagram of another event processing result according to an embodiment of the present application;

[0066] [Figure 10A] FIG. 2 is a schematic diagram of a device control result according to an embodiment of the present application. [Figure 10B] FIG. 2 is a schematic diagram of a device control result according to an embodiment of the present application.

[0067] [Figure 11]1 is a schematic diagram of several user interfaces according to embodiments of the present application. [Figure 12] 1 is a schematic diagram of several user interfaces according to embodiments of the present application. [Figure 13] 1 is a schematic diagram of several user interfaces according to embodiments of the present application.

[0068] [Figure 14] FIG. 2 is a schematic diagram of a screen edge region according to an embodiment of the present application.

[0069] [Figure 15] FIG. 2 is a schematic diagram of another event handling process according to an embodiment of the present application.

[0070] [Figure 16A] FIG. 13 is a schematic diagram of another device control result according to an embodiment of the present application. [Figure 16B] FIG. 13 is a schematic diagram of another device control result according to an embodiment of the present application.

[0071] [Figure 17] 11A-11C are schematic diagrams of several other user interfaces according to embodiments of the present application. [Figure 18] 11A-11C are schematic diagrams of several other user interfaces according to embodiments of the present application.

[0072] [Figure 19] FIG. 2 is a schematic diagram of the structure of a device control device according to an embodiment of this application. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0073] Hereinafter, the embodiments of this application will be described with reference to the accompanying drawings in the embodiments of this application. The terms used in the implementation of the embodiments of this application are only used to describe specific embodiments of this application, and are not intended to limit this application.

[0074] 1 is a schematic diagram of a communication scenario according to an embodiment of this application. As shown in FIG. 1, the projection scenario includes a source projection device 100 and a destination projection device 200.

[0075] The source projection device 100 and the destination projection device 200 may be terminal devices such as mobile phones, tablet computers, notebook computers, desktop computers, wearable devices, smart TVs, etc. The same terminal device may be used as the source projection device 100 and the destination projection device 200. There may be one or more source projection devices 100 and one or more destination projection devices 200. Specifically, one source projection device 100 may project screen display contents to multiple destination projection devices 200 for display, and one destination projection device 200 may simultaneously display screen display contents of multiple source projection devices 100. In FIG. 1, an example in which the projection scenario includes two terminal devices is used for illustration, in which the source projection device 100 is a mobile phone and the destination projection device 200 is a tablet computer.

[0076] The source projection device 100 and the destination projection device 200 may be connected by a wireless communication method, such as Wireless Fidelity (Wi-Fi), Bluetooth (BT), near field communication (NFC), etc. Alternatively, the source projection device 100 and the destination projection device 200 may be connected by wire, for example, through a data line.

[0077] After the source projection device 100 and the destination projection device 200 establish a connection, the source projection device 100 may transmit screen display contents to the destination projection device 200, and the destination projection device 200 may display the screen display contents of the source projection device 100 in a projection window. The projection window may be displayed as a floating window. The destination projection device 200 may also feed back an operation event (e.g., a tap event or a touch-slide event) triggered by a user in the projection window to the source projection device 100. The source projection device 100 may update the screen display contents based on the received operation event. The display contents in the projection window of the destination projection device 200 are updated accordingly. That is, the user may control the source projection device 100 on the destination projection device 200 inversely. For example, a user may control the source projection device 100 to simulate an edge slide gesture (which may also be referred to as a screen gesture, bezel-less screen gesture, screen gesture navigation, screen navigation gesture, etc.) of the source projection device 100 in the projection window of the destination projection device 200 by sliding inward from a window edge of the projection window to perform a corresponding function.

[0078] The process by which the terminal device detects an edge slide gesture is described below.

[0079] To detect the edge slide gesture, the terminal device predetermines (e.g., determines when the terminal device is powered on) a hot zone (i.e., a screen edge region) for detecting the edge slide gesture, and this screen edge region is invisible to the user. When a touch-slide action of the user is detected, the terminal device detects the edge slide gesture based on the screen edge region. The slide start position of the edge slide gesture needs to be located within the screen edge region.

[0080] FIG. 2 is a schematic diagram of a touch-slide operation according to one embodiment of this application. As shown in FIG. 2, a screen edge area A is an area formed by extending inward from a screen edge, and may include a left edge area, a right edge area, a top edge area, and / or a bottom edge area. The width of the edge area (specifically, the distance between the inner edge and the outer edge (screen edge)) may be the same, and the specific size may be, for example, 2 mm or another value.

[0081] Edge slide gestures may include navigation bar function gestures triggered by sliding inward left, inward right, or inward up to implement navigation bar functions; quick switch application gestures triggered by sliding left or right along the bottom edge region or sliding upward in an arc from the bottom edge region to quickly switch to the previous or next application; split screen window enable gestures triggered by sliding inward from the left or right edge of the screen and pausing to enable a split screen application list; and edge slide gestures to implement other functions.

[0082] A user may select the gesture navigation mode from the system navigation mode setting function to enable the navigation bar function gesture recognition function and the quick switch application gesture recognition function of the terminal device, or may select another navigation mode, such as the 3-key navigation mode, from the system navigation mode setting function to disable the navigation bar function gesture recognition function and the quick switch application gesture recognition function of the terminal device. In order to facilitate the description of the technical solutions in the embodiments of this application, unless otherwise specified below, the terminal device performs system navigation in the gesture navigation mode, i.e., the gesture navigation mode is in an enabled state.

[0083] Specifically, the navigation bar gestures may include a back to previous level gesture, a back to home screen gesture, and a gesture for a recent task list. A back to previous level gesture may be triggered by sliding inward a specific distance from the left edge region or the right edge region. A back to previous level gesture may be triggered by sliding upward a specific distance from the bottom edge region. A gesture for a recent task list may be triggered by sliding upward a specific distance from the bottom edge region and pausing, or by sliding upward a further distance.

[0084] For ease of explanation, in this embodiment of the application, a navigation bar function gesture is used as an example for explanation purposes. Unless otherwise specified below, an edge slide gesture represents a navigation bar function gesture.

[0085] As shown in FIG. 2, the arrows indicate touch-slide actions. When the terminal device detects an edge-slide gesture, it determines whether the starting position of the touch-slide action (i.e., the starting touch point) is located within the screen edge region A. Only the touch-slide action whose starting position is located within the screen edge region A may be considered as an edge-slide gesture. As shown in the figure, the starting position of the touch-slide action indicated by the arrow 1 is located within the screen edge region A, and the edge-slide gesture may be triggered. The starting position of the touch-slide action indicated by the arrow 2 is located outside the screen edge region A. Therefore, it is considered that the touch-slide action is not performed from the screen edge, and the edge-slide gesture is not triggered.

[0086] When the terminal device is used as the source projection device 100, the user may initiate an inward slide in a window edge region of the projection window on the destination projection device 200 to trigger an edge slide gesture of the source projection device 100. As shown in FIG. 2, the window edge region B of the projection window corresponds to the screen edge region A of the source projection device 100. For example, the screen display content of the source projection device 100 is displayed on the projection window in 1:1 mode. The size of the window edge region B is the same as the size of the screen edge region A.

[0087] In actual operation, because the window edge area B is small, when a user performs an operation on the destination projection device 200, it is difficult for the start position of the touch slide operation to be within the window edge area B, and the cases shown by arrows 3 and 4 often occur. Therefore, it is easy to occur that the edge slide gesture of the source projection device 100 is not triggered. Based on the above problem, an embodiment of this application provides a device control method for improving the success rate of a user triggering the edge slide gesture of the source projection device 100 on the destination projection device 200, thereby improving the reliability of the reverse control operation and the user experience. The device control method may be applied to the above terminal device. Hereinafter, the structure of the terminal device will be described first using a mobile phone as an example.

[0088] FIG. 3 is a schematic diagram of the structure of a terminal device according to one embodiment of this application.

[0089] The terminal 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headset jack 170D, a sensor module 180, buttons 190, a motor 191, indicators 192, a camera 193, a display 194, a subscriber identification module (SIM) card interface 195, and the like. The sensor module 180 may include a pressure sensor 180A, a gyro sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, an optical proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, and the like.

[0090] It will be understood that the structure shown in this embodiment of the application does not constitute a specific limitation on the terminal. In some other embodiments of the application, the terminal device may include more or fewer components than those shown in the figure, may combine some components, may split some components, or may have a different component arrangement. The components shown in the figure may be implemented using hardware, software, or a combination of software and hardware.

[0091] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, a neural-network processing unit (NPU), etc. The different processing units may be separate components or may be integrated into one or more processors.

[0092] The controller may be the nerve center and command center of the terminal device, and may generate operation control signals according to the instruction operation code and the time sequence signal to complete the complete control of instruction fetching and instruction execution.

[0093] The memory may be disposed within the processor 110 and is configured to store instructions and data. In some embodiments, the memory within the processor 110 is a cache. The memory may store instructions or data that are used or are used periodically by the processor 110. If the processor 110 needs to use the instructions or data again, the processor may retrieve the instructions or data directly from the memory. This avoids repeated accesses, reduces the latency of the processor 110, and improves the efficiency of the system.

[0094] In some embodiments, the processor 110 may include one or more interfaces, which may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM), a universal serial bus (USB) interface, or the like.

[0095] The charging management module 140 is configured to receive a charging input from a charger. The charger may be a wireless charger or a wired charger. In some embodiments of wired charging, the charging management module 140 may receive the charging input of the wired charger via the USB port 130. In some embodiments of wireless charging, the charging management module 140 may receive the wireless charging input through a wireless charging coil of the terminal device. The charging management module 140 may further provide power to the terminal device by using the power management module 141 while charging the battery 142.

[0096] The power management module 141 is configured to couple the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives inputs from the battery 142 and / or the charging management module 140 and provides power to the processor 110, the internal memory 121, the external memory, the display 194, the camera 193, the wireless communication module 160, etc. The power management module 141 may be further configured to monitor parameters such as battery capacity, battery cycle count, battery health (leakage and impedance), etc. In some other implementations, the power management module 141 may alternatively be disposed in the processor 110. In some other embodiments, the power management module 141 and the charging management module 140 may alternatively be disposed in the same device.

[0097] The wireless communication function of the terminal device may be implemented using antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, a modem processor, a baseband processor, and the like.

[0098] Antenna 1 and Antenna 2 are configured to transmit and receive electromagnetic signals. Each antenna in the terminal device may be configured to cover one or more communication frequency bands. To improve antenna utilization, different antennas may be further multiplexed. For example, Antenna 1 may be multiplexed as a diversity antenna in a wireless local area network. In some other embodiments, the antennas may be used in combination with tuning switches.

[0099] The mobile communication module 150 may be applied to a terminal device and provide a wireless communication solution including 2G / 3G / 4G / 5G, etc. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 may receive electromagnetic waves via the antenna 1, filter, amplify, etc., and transmit the electromagnetic waves to the modem processor for demodulation. The mobile communication module 150 may further amplify the signal modulated by the modem processor and convert the signal to electromagnetic waves for emission via the antenna 1. In some embodiments, at least some functional modules of the mobile communication module 150 may be disposed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 may be disposed in the same device as at least some modules of the processor 110.

[0100] The wireless communication module 160 may be applied to a terminal device and provide wireless communication solutions including wireless local area networks (WLAN) (e.g., wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC) technology, infrared (IR) technology, etc. The wireless communication module 160 may be one or more devices integrating at least one communication processor module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, performs frequency modulation and filtering on the electromagnetic wave signals, and transmits the processed signals to the processor 110. The wireless communication module 160 further receives signals to be transmitted from the processor 110, performs frequency modulation and amplification on the signals, and converts the signals into electromagnetic waves for emission via the antenna 2.

[0101] In some embodiments, antenna 1 and mobile communication module 150 in the terminal device are coupled, and antenna 2 and wireless communication module 160 in the terminal device are coupled, so that the terminal device can communicate with a network and another device using a wireless communication technology, which may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time division-synchronous code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, IR technology, etc. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GNSS), a BeiDou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS), and / or a satellite based augmentation system (SBAS).

[0102] The terminal device may implement display functions using a GPU, a display 194, an application processor, etc. The GPU is a microprocessor for image processing and is connected to the display 194 and the application processor. The GPU is configured to perform mathematical and geometric calculations and render images. The processor 110 may include one or more GPUs that execute program instructions that generate or change display information.

[0103] The display 194 is configured to display images, videos, etc. The display 194 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a mini-LED, a micro-LED, a micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the terminal device may include one or N displays 194, where N is a positive integer greater than 1.

[0104] The terminal device can implement the photographing function by using an ISP, a camera 193, a video codec, a GPU, a display 194, an application processor, and the like.

[0105] The external memory interface 120 may be configured to connect to an external storage card, for example a microSD card, to expand the storage capabilities of the electronic device. The external storage card communicates with the processor 110 via the external memory interface 120 to implement data storage functions. For example, music, video, and other files are stored on the external storage card.

[0106] The internal memory 121 may be configured to store computer executable program code. The executable program code includes instructions. The processor 110 operates the instructions stored in the internal memory 121 to perform various functional applications and data processing of the terminal device. The internal memory 121 may include a program storage area and a data storage area. The program storage area may store an operating system, applications required by at least one function (e.g., a sound playback function or an image playback function), and the like. The data storage area may store data (audio data, a phone book, and the like) created during the use of the terminal device. Additionally, the internal memory 121 may include a high-speed random access memory and may further include a non-volatile memory, for example, at least one magnetic disk storage device, a flash memory, or a universal flash storage (UFS).

[0107] The terminal device may implement audio functions, such as music playback and recording, via an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headset jack 170D, an application processor, etc.

[0108] 4 is a schematic diagram of a software architecture of a terminal device according to an embodiment of this application. As mentioned above, the terminal device in a projection scenario includes a source projection device 100 and a destination projection device 200. The functions performed by the source projection device 100 and the destination projection device 200 are different, and the corresponding software architectures are also different. This figure shows an example of the software architecture of the source projection device 100 and the destination projection device 200.

[0109] It will be understood that this figure only shows the software architecture when the terminal device is used separately as the source projection device 100 and the destination projection device 200. When the terminal device has the functions of both the source projection device 100 and the destination projection device 200, the software architecture of the terminal device is a set of the software architectures of the source projection device 100 and the destination projection device 200. Additionally, the software architectures of the source projection device 100 and the destination projection device 200 are similar. For ease of explanation, unless otherwise specified, the terminal device may represent the source projection device 100 and the destination projection device 200 below.

[0110] The software system of the terminal device may be divided into several layers. As shown in FIG. 4, in some embodiments, the software system may be divided from top to bottom into an application layer, a framework layer, and a driver layer.

[0111] The application layer may include a series of applications (APPs). The applications may include a projection management APP related to projection, and may further include other APPs such as camera, gallery, phone, map, browser, music, video and message.

[0112] The projection management APP may include a device connection APP configured to manage the connection between the terminal devices. The projection management APP in the destination projection device 200 may further include a projection display APP configured to display the screen display content of the source projection device 100.

[0113] The framework layer provides the application programming interface (API) and programming framework for applications in the application layer.

[0114] As shown in FIG. 4, the framework layer may include a system base framework, a device connection management framework, a data management framework, etc. The base framework may include an event management module configured to deliver an event. The source projection device may further include a gesture detection module configured to detect a gesture. Additionally, the base framework may further include another base capability module (not shown), for example, a window management module configured to manage a window program. The device connection management framework is for managing a cross-device connection protocol. The data management framework is for managing data communication in a cross-device scenario (e.g., a projection scenario). For example, the audio and video data module is used by the source projection device 100 to transmit audio and video data to the destination projection device 200. The chart layer data module is used by the source projection device 100 to transmit chart layer data to the destination projection device 200. The reverse control module is used by the destination projection device 200 to transmit event data to the source projection device 100.

[0115] In this embodiment of the application, in order to improve the reliability of the reverse control operation, after a touch-slide action sliding from outside the projection window to inside the projection window is detected, the destination projection device 200 may perform event conversion or compensation on the touch-slide event generated by the touch-slide action whose starting position is within the hot zone to generate an edge slide event of the source projection device. The hot zone may be a target area formed by extending a preset distance outward from the edge of the projection window. Correspondingly, an event conversion compensation module and a hot zone management module may be added to the destination projection device 200. The hot zone management module may manage information such as the size and position of the hot zone. The event conversion compensation module may determine an event in the target area based on the hot zone information, and perform event conversion or compensation on the determined event. For a specific processing process, please refer to the embodiment of the method described later.

[0116] The event conversion compensation module may be located in the basic framework or in the device connection APP. The hot zone management module may be an independent module or may be integrated into another existing module. For example, if the window management module manages the window information of the projection window, the region information of the target region may be added to the window management module to implement the function of the hot zone management module. In this figure, an example in which the event conversion compensation module is located in the basic framework and the hot zone management module is an independent module is used for explanation.

[0117] The driver layer is for providing functions such as discovery, authentication, and connection of terminal devices, and may correspondingly include functional modules such as a device discovery module, a device authentication module, a device connection module, etc. The device connection APP may use the device discovery module to search for and discover other terminal devices, use the device authentication module to perform authentication between the terminal devices, and then use the device connection module to establish a connection between the terminal devices.

[0118] It should be understood that the framework layer and the driver layer only show some modules related to a projection scenario, and may further include other modules. For example, the framework layer may further include a notification management module, and the driver layer may further include a display driver, a sensor driver, etc. In this embodiment, the other modules specifically included in the framework layer and the driver layer are not particularly limited.

[0119] An example of a reverse control procedure for the source projection device 100 and the destination projection device 200 will now be described with reference to a projection scenario.

[0120] After the touch sensor 200K in the destination projection device 200 receives the touch action, a corresponding hardware interrupt is sent to the driver layer. The driver layer may encapsulate the touch action into a touch event (e.g., a touch down event, a touch move event, and a touch up event) and report the touch event to the framework layer. A series of touch events may be generated in the process of one touch-slide action of the user. After receiving each touch event, the event management module of the framework layer may determine a window corresponding to the touch event based on the coordinates of the touch event. When the window corresponding to the touch down event is a hot zone and the window corresponding to the touch up event is a projection window, the event conversion compensation module may be called to map the coordinates of each touch event to the corresponding coordinates in the screen area of ​​the source projection device 100. The event transformation and compensation module may obtain hot zone information from the hot zone management module, perform event transformation or compensation on the touch events obtained through coordinate mapping based on the hot zone information, and then return the touch events to the event management module. The event management module then delivers these touch events to the projection management APP, and the device connection APP in the projection management APP calls the reverse control module to transmit the touch events to the originating projection device 100.

[0121] After receiving the touch event, the reverse control module of the source projection device 100 may report the touch event to the projection management APP, and may use the projection management APP to call the gesture detection module, or may directly report the touch event to the gesture detection module. The gesture detection module may detect a gesture (e.g., an edge slide gesture) for the series of received touch events, and may report the gesture detection result to a related application in the application layer after the gesture is detected. The related application in the application layer may refresh the screen display content based on the detected gesture, and may call a chart layer data module in the data management framework to transmit the refreshed screen display content to the destination projection device 200. After the chart layer data module of the destination projection device 200 receives the refreshed screen display content, the projection display APP refreshes the display content of the projection window based on the screen display content received by the chart layer data module.

[0122] A device control method provided in an embodiment of this application will now be described.

[0123] When a user wants to trigger an edge slide gesture on a source projection device, the touch-slide action performed by the user on the destination projection device relative to the projection window may include the following two cases: In the first case, the starting position of the touch-slide action is located outside the projection window (e.g., the touch-slide action shown by the arrow 3 in FIG. 2), and in the second case, the starting position of the touch-slide action is located within the projection window (e.g., the touch-slide action shown by the arrow 4 in FIG. 2). Hereinafter, the device control process in these two cases will be described separately.

[0124] In the first case, the starting position of the touch-slide action is located outside the projection window.

[0125] An edge slide event that can trigger an edge slide gesture on the sending projection device should meet the completeness requirement: A complete edge slide event contains the following elements: one touch down event (short for Down event), multiple touch move events (short for Move events), and one touch up event (short for Up event).

[0126] When the starting position of the touch-slide action input by the user on the destination projection device is located outside the projection window, the corresponding down event is also located outside the projection window. To ensure that the edge slide event transmitted to the source projection device meets the completeness requirement, in this embodiment, the destination projection device may generate a touch-slide event in response to the touch-slide action input by the user by sliding from outside the projection window to inside the projection window, and then perform an event conversion or compensation process on the touch-slide event generated by the touch-slide action to generate a complete edge slide event starting from the edge of the projection window, and then send the complete edge slide to the source projection device to improve the success rate of the edge slide gesture of the source projection device being triggered by the user on the destination projection device.

[0127] Considering that the touch-and-slide action triggered by the user on the destination projection device and the end position of the touch-and-slide action for controlling the destination projection device may also enter the projection window, in this embodiment, projection The device may use a target area around the projection window as a detection hot zone (referred to as hot zone for short) and perform event conversion or correction processing on touch-slide events generated by touch-slide actions whose starting positions fall within the hot zone.

[0128] FIG. 5 is a schematic diagram of a target area according to an embodiment of this application. As shown in FIG. 5, the target area S is an area formed by extending a preset distance outward from the edge of the projection window, and is similar to the screen edge area A shown in FIG. 2. , the target area S may include multiple areas of four sub-areas on the top, bottom, left and right. The widths of the sub-areas may be the same, for example, 2 mm, or the widths of the sub-areas may be different. In this figure, an example in which the target area S includes four sub-areas with the same width is used for explanation.

[0129] As shown in FIG. 5, when the starting position of the touch-slide action is located outside the projection window, there may be two cases. In the first case, the starting position of the touch-slide action is located outside the target area S, as shown by the arrow 5 in FIG. 5, and in the second case, the starting position of the touch-slide action is located within the target area S, as shown by the arrow 6 in FIG. 5. For the two different cases, the destination projection device may execute different event handling processes. The event handling processes are described below.

[0130] FIG. 6 is a schematic diagram of an event processing process according to an embodiment of this application. As shown in FIG. 6, the destination projection device may generate a touch-slide event in response to a touch-slide operation performed by a user by sliding from outside the projection window to inside the projection window. Additionally, the destination projection device may determine whether a starting touch point of the touch-slide operation is in a target area. When the starting touch point of the touch-slide operation is located within the target area, the acquired touch-slide event may be converted into an edge slide event of the projection window. When the starting touch point of the touch-slide operation is located outside the target area, the touch-slide event may be determined as an event for the screen display content on the destination projection device.

[0131] Specifically, the destination projection device may generate a series of touch events based on the user's touch-slide action, and obtain an event sequence of corresponding touch-slide events based on the coordinates and time of each touch event. For a first touch event in the event sequence (specifically, a touch event corresponding to a starting touch point of the touch-slide action), it may be determined whether the coordinates of the touch event are located within the target area. If the coordinates of the touch event are located within the target area, the touch-slide action may be considered to be an edge-slide action with respect to the projection window. In this case, the touch-slide event may be converted into an edge-slide event of the source projection device. If the coordinates of the touch event are located outside the target area, the touch-slide action may be considered to be a slide event with respect to the screen display content on the destination projection device. In this case, a control function corresponding to the touch-slide event, such as scrolling the screen content or switching the interface, may be performed.

[0132] In an optional implementation, when generating an edge slide event, in this embodiment, the destination projection device may perform an event coordinate transformation on touch events (herein referred to as hot zone events) whose coordinates are located in the target area (i.e., hot zone) and are within the touch slide event, so that the hot zone events become screen edge events of the source projection device. The edge slide event is generated based on these screen edge events and the touch events that are located within the projection window and are within the touch slide event.

[0133] Specifically, before the event coordinate conversion is performed, coordinate mapping may first be performed for each touch event in the touch slide event. As described above, after the destination projection device generates a touch event, the destination projection device may map the coordinates of the touch event in the projection window to corresponding coordinates in the screen area of ​​the source projection device. In this embodiment, the destination projection device may also map the coordinates of each touch event in the touch slide event to corresponding coordinates in the screen area of ​​the source projection device based on the same coordinate mapping scheme. That is, the screen coordinates of the touch event on the destination projection device are mapped to the screen coordinates on the source projection device.

[0134] As shown in FIG. 7, the screen coordinate system of the projection window is xy, and the coordinate origin o is located in the upper left corner of the projection window. The screen coordinate system of the destination projection device is x'y', and the coordinate origin o' is located in the upper left corner of the screen of the destination projection device. Based on the coordinate of o in the x'y' coordinate system, the coordinate mapping relationship between the two coordinate systems may be obtained. If the coordinate of o in the x'y' coordinate system is (x0, y0), the coordinate mapping relationship between the two coordinate systems is as follows:

number

[0135] (x',y') represents the coordinates of any point on the screen of the destination projection device in the x'y' coordinate system (specifically, the screen coordinates on the destination projection device). (x,y) represents the corresponding coordinates of the point in the xy coordinate system (specifically, the window coordinates with respect to the projection window). When the screen display content of the source projection device is displayed in 1:1 mode in the projection window, (x,y) is the screen coordinate of the point on the source projection device. When the screen display content of the source projection device is scaled in the projection window, (x,y) may be scaled inversely to obtain the screen coordinate of the point on the source projection device. For example, if the projection window is 1.5 times larger than the screen of the source projection device, (x,y) may be reduced by 1.5 times (i.e., divided by 1.5) to obtain the screen coordinate of the point on the source projection device.

[0136] Alternatively, the mapping relationship between the screen coordinate system of the destination projection device and the screen coordinate system of the source projection window may be calculated every time after the projection window is enlarged or reduced. When the coordinate mapping is performed, the mapping relationship is directly used to perform the coordinate mapping to obtain the screen coordinates on the source projection device corresponding to any point on the destination projection device. Additionally, the above coordinate conversion process is only an example and does not limit this application. In this embodiment of this application, the conversion manner between the screen coordinates on the destination projection device and the screen coordinates on the source projection window is not particularly limited. For ease of explanation, an example in which the screen display content of the source projection device is displayed in the projection window in 1:1 mode is used for explanation.

[0137] After the coordinate mapping is performed, the touch events within the hot zone (i.e., hot zone events) are converted into screen edge events of the originating projection device, and as a result, a complete edge slide event is obtained and sent to the originating projection device.

[0138] For example, the coordinates of a touch event obtained through coordinate mapping are (x,y). Assume the screen width of the source projection device is W and the height is H. For different touch-slide actions, the event coordinate transformation process of the hot zone event is shown in the table below. [Table 1]

[0139] For example, the screen coordinate (x0,y0) of o in Figure 7 is (80,40), the width of each sub-region in the hot zone is 10, and the screen coordinate of the touch event corresponding to point P in the left inward slide event on the destination projection device is (75,140). After coordinate mapping, the screen coordinate corresponding to point P on the source projection device is (75-80,140-40), i.e., (-5,100). After event coordinate transformation, point P is changed to P'(0,100).

[0140] It should be understood that the above description is provided as an example of a touch-slide operation corresponding to a navigation bar function gesture. For another edge-slide gesture, there may be a case of sliding downward inward. In this case, when the event coordinate transformation is performed, the touch event (x, y) may be transformed into a screen edge event (x, 0). Additionally, the above process of generating a screen edge event is merely an example and does not limit the present application. When a screen edge event is generated, there is no strict time-sequential execution relationship between the mapping process corresponding to equation (1), the window coordinate scaling process, and the event coordinate transformation process. In a specific implementation, adjustments may be made as necessary as long as the related parameters are adaptively adjusted.

[0141] FIG. 8 is a schematic diagram of an event processing result according to an embodiment of this application. As shown in FIG. 8, ▽ represents a down event, ○ represents a move event, and △ represents an up event. As shown in FIG. 8(a), the starting touch point (down event) of the touch-slide operation is located within the target area S. As shown in FIG. 8(b), after the event coordinate transformation, the down event becomes a screen edge event of the projection window (i.e., the source projection device).

[0142] It will be understood that in FIG. 8, the touch point is enlarged for ease of explanation of the event coordinate transformation process, and only one touch event is included within the target area, whereas in a real application, the target area may include multiple touch events.

[0143] As shown in (a) and (b) of FIG. 8, when the starting touch point of the touch-slide action is within the target area, after the event coordinate transformation, the touch events in the projection window include a down event, a move event, and an up event. Specifically, the destination projection device may obtain a complete edge slide event based on the screen edge event obtained through the event coordinate transformation and the touch event corresponding to the touch-slide action in the projection window, and send the complete edge slide event to the source projection device.

[0144] In another optional implementation, in this embodiment, the destination projection The device may alternatively perform event transformation or compensation based on the touch event whose coordinates are located within the projection window and are within the touch slide event to generate a touch down event located at an edge of the screen of the originating projection device, and generate an edge slide event based on the touch down event and the touch event whose coordinates are located within the projection window and are within the touch slide event.

[0145] As shown by arrow 6 in Figure 5, the start position of the touch-slide action is located outside the projection window, and the projection window does not include the down event. projection To obtain a complete edge slide event, the device may generate a down event (herein referred to as a target down event) located at the screen edge of the originating projection device based on an initial touch event (a move event, here referred to as a target event) corresponding to the touch slide motion in the projection window, and obtain an edge slide event based on the target down event and the touch event corresponding to the touch slide motion in the projection window.

[0146] To improve the continuity between events in touch-slide events, the coordinates of the target-down event may be the same as the coordinates of the target event. The target event is located at the edge of the projection window and is a screen edge event of the originating projection device. Correspondingly, the target-down event is also a screen edge event of the originating projection device.

[0147] In a specific implementation, a target down event may be used as compensation by the event compensation process. For example, a target down event may be inserted before a target event. Specifically, the timestamp of the target down event is earlier than the timestamp of the target event. Alternatively, a target down event may be generated through event type conversion. Specifically, a target event may be converted directly into a target down event, i.e., the event type of the target event is changed from a move event to a down event.

[0148] It can be understood that when an edge slide event is generated, coordinate mapping needs to be performed for each touch event in the projection window, similar to the process of generating an edge slide event based on the hot zone event described above. For a specific coordinate mapping method, please refer to the related description above. The details will not be described again here. The coordinates being the same means that the coordinates of the target down event and the target event in the same coordinate system are the same. When an edge slide event is generated, an event type conversion or compensation may be performed on the target event first to obtain a target down event, and then coordinate mapping may be performed on each touch event in the projection window. Alternatively, coordinate mapping may be performed on each touch event in the projection window first, and then a target down event may be generated based on the target event.

[0149] FIG. 9 is a schematic diagram of another event processing result according to an embodiment of this application. In FIG. 9, the same touch event method as FIG. 8 is still used. ▽ represents a down event, ○ represents a move event, and △ represents an up event. As shown in FIG. 9(a), the starting touch point of the touch-slide action is located outside the projection window, and the first touch event located within the projection window is a move event. As shown in FIG. 9(b), after the event compensation process, one down event whose coordinates are the same as those of the move event is added. As shown in FIG. 9(c), after the event type conversion, the first move event within the projection window becomes a down event.

[0150] An example is used in which a user triggers a gesture to return to a home screen using a touch-slide action. When gesture navigation mode is enabled on the source projection device 100, as shown in FIG. AAs shown, a user inputs a touch-slide action (pointed to the black arrow in the figure) on the destination projection device 200, sliding from outside the projection window to inside the projection window. In the current relational reverse control solution, the destination projection device 200 feeds back touch events in the projection window to the source projection device 100 after coordinate mapping, but these events do not include a down event. Correspondingly, the source projection device 100 receives one incomplete edge slide event, and thus cannot trigger the edge slide gesture, i.e., the user's reverse control action fails. However, in this embodiment, the destination projection device 200 may use the target area as a hot zone, and perform event conversion or compensation processing on the event in the hot zone or projection window to generate an edge slide event that includes a target down event. Correspondingly, the source projection device 100 receives one complete edge slide event. Thus, figure As shown in 10B, the gesture to return to the home screen may be successfully triggered, and the action of returning to the home screen may be executed. When updating the screen display content, the source projection device 100 may transmit the updated screen display content to the destination projection device 200, and the destination projection device 200 updates the screen display content displayed in the projection window accordingly.

[0151] Considering that a user may slide to the projection window when performing a touch-slide operation on the screen display content of the destination projection device, in order to avoid collision with the touch-slide operation on the projection window, in this embodiment, the destination projection device may provide a projection window moving function. A user may move the projection window when it is necessary to perform a touch-slide operation on the screen display content near the projection window. For example, a user may move the projection window by dragging a window edge area of ​​the projection window. Additionally, the destination projection device may also provide a function to minimize the projection window. For example, as shown in (a) of FIG. 11, a function bar may be added to the projection window, and a minimize control 2001 is provided on the function bar. A user may tap the minimize control 2001 to minimize the projection window 115. As shown in (b) of FIG. 11, the destination projection device may display a minimize window 2002 corresponding to the projection window near the screen edge, and a user may resume displaying the projection window by tapping the minimize window 2002. Additionally, the function bar may further provide other controls, such as a close control 2003 configured to close the projection window and a scaling control 2004 used by the user to enlarge or reduce the projection window.

[0152] In order to help users better understand the above-mentioned operation area of ​​the projection reverse direction control, in this embodiment, the destination projection device may execute related function description. The function description may be provided in an APP (for example, the "Tips" APP provided by Huawei) to describe the device function.

[0153] FIG. 12 is a schematic diagram of another user interface according to an embodiment of this application. As shown in FIG. 12(a), the "Tips" APP is used as an example. After a user starts the "Tips" APP, the corresponding user interface 10 may include a search box 101, a feature recommendation bar 102, and a Tips feature bar 103. The search box 101 is used to search for features that the user wants to know about. The feature recommendation bar 102 may recommend some new features, popular features, etc. The Tips feature bar 103 may provide various category options of tips, such as a "New Features" option 1031, a "Get Familiar" option 1032, a "Video and Audio Entertainment" option 1033, an "AI Life" option 1034, and a "Health" option 1035 as shown in the figure.

[0154] The description of the projected reverse control may be located under one of the categories of the hint feature bar 103, for example, under a category corresponding to a "New Features" option 1031. As shown in Figures 12(a) and 12(b), the user may tap the "New Features" option 1031 to launch the corresponding new features interface 20 to view the description of the projected reverse control.

[0155] As shown in FIG. 12(b), the new function interface 20 may include a function demonstration bar 201 and a function description bar 202. In the function demonstration bar 201, the projection reverse control operation and its related edge-sensing area may be explained in a short video manner. For example, as shown in FIG. 12(b), the edge-sensing area (including the above-mentioned area B and area S) corresponding to the edge-slide gesture on the destination projection device may be shown, so that the user can more intuitively understand the reverse control function of the edge-slide gesture. In the function description bar 202, the reverse control function of the edge-slide gesture may be explained in a text manner, for example, an example of the text shown in FIG. 12(b) is "You can trigger the edge-slide gesture of the source projection device by sliding from the edge-sensing area in the projection window or the edge-sensing area outside the projection window into the projection window."

[0156] The "Tips" APP may also provide a card corresponding to the description of the projection reverse control in the function recommendation bar 103. The user may tap the card to view the description of the projection reverse control. Additionally, the user may alternatively search the search bar 102 for the description of the projection reverse control.

[0157] It will be understood that the above-described interface is only one example for illustrative purposes, and the elements in the interface may be adjusted as necessary, and the interfaces corresponding to the "Tips" APP on different types of terminal devices may be different.

[0158] In order to meet various demands of users and improve the user experience, in this embodiment, the terminal device may also provide an edge-sensing area setting function corresponding to the area S. Specifically, the function may be provided in one option of the system setting function. For example, as shown in (a) of FIG. 13, a projection reverse control option 301 may be added to the auxiliary function interface 30 corresponding to the "auxiliary function" option. The user taps the projection reverse control option 301 to enter the corresponding projection reverse control setting interface 40 (see (b) of FIG. 13) and set the edge-sensing area in the interface. In order to make it easier for the user to understand the setting function, the above-mentioned description of the projection reverse control may be provided in the projection reverse control setting interface 40. Specifically, the user may understand the function description of the projection reverse control function in the above-mentioned "Tips" APP, or may learn the function description of the projection reverse control function in the edge-sensing area setting function.

[0159] As shown in (b) of FIG. 13, the projection reverse control setting interface 40 may include a function description bar 401 and a setting bar 402. The contents displayed in the function description bar 401 are similar to those shown in (b) of FIG. 12. The projection reverse control function may be introduced in a simple picture and text manner. For example, an edge sensing area 1 corresponding to area B and an edge sensing area 2 corresponding to area S may be shown in the picture. The text may be "Sliding from edge sensing area 1 or edge sensing area 2 to the projection window will trigger an edge slide gesture of the source projection device". The setting bar 402 may provide an associated control for the user to set the width of the edge sensing area 2 (i.e., area S). The control may be a slide control as shown in the figure, and both ends of the control may display a slide range (an example of the range here is 0 mm to 15 mm). The control may alternatively include a single selection control corresponding to a plurality of pre-set area widths, or may be another type of control, such as a text control, which may be specifically set as needed. This is not particularly limited in this embodiment.

[0160] The device control process in these two cases will be described below.

[0161] In the second case, the starting position of the touch-slide action is located within the projection window.

[0162] When the starting position of the touch-slide action is within the projection window, the destination projection device may perform coordinate mapping on the touch event corresponding to the touch-slide action within the projection window to obtain a touch-slide event, and then send the touch-slide event to the source projection device.

[0163] As shown in FIG. 2, when a user performs a touch-slide action on a projection window on a destination projection device, a case as indicated by arrow 5 is likely to occur. Specifically, the starting position of the touch-slide action is located outside the window edge region. Correspondingly, the starting position of the touch-slide event that the source projection device receives from the destination projection device is located outside the screen edge region. In this way, the edge slide gesture is not triggered.

[0164] In order to improve the success rate of an edge slide gesture of a source projection device being triggered by a user on a destination projection device, in this embodiment, when processing a touch slide event sent from the destination projection device, the source projection device may use a larger hot zone (i.e., a screen edge area) to detect the edge slide gesture.

[0165] FIG. 14 is a schematic diagram of a screen edge region according to an embodiment of this application. As shown in FIG. 14, the source projection device may generate a screen edge region (herein referred to as a first region A1) when powered on. After the projection connection is established, a screen edge region (herein referred to as a second region A2) may be further generated. Both the first region A1 and the second region A2 are regions formed by extending inward from the screen edge of the source projection device, and the coverage of the second region A2 is larger than that of the first region A1. Similar to the first region A1, the second region A2 may include multiple regions of four sub-regions on the top, bottom, left and right. The width of each sub-region may be a multiple, for example, two times, of the width of the corresponding sub-region in the first region A1.

[0166] When processing the touch slide event, as shown in FIG. 15, the source projection device may determine whether the touch slide event is sent by the destination projection device after the touch slide event is detected. If the touch slide event is sent by the destination projection device, the second region may be used as the screen edge region, and an edge slide gesture is detected for the received touch slide event. If the touch slide event is not sent by the destination projection device, i.e., the touch slide event is triggered by a user on the source projection device, the first region may be used as the screen edge region, and an edge slide gesture is detected for the detected touch slide event.

[0167] In other words, for touch-slide events triggered by a user on the source projection device and touch-slide events sent by the destination projection device, the only difference is the extent of the screen edge area used when the source projection device detects an edge-slide gesture, otherwise the detection process is the same. For any touch-slide event, if an edge-slide gesture is detected, the source projection device may execute a control function corresponding to the detected edge-slide gesture.

[0168] In a specific implementation, a mark may be added to the touch-slide event sent by the destination projection device, and the source projection device may determine the source of the touch-slide event based on the mark.

[0169] The example where the user triggers a gesture to return to the home screen using a touch-slide action is still used. AAs shown, the window edge regions on the projection window and corresponding to the first region A1 and the second region A2 of the source projection device 200 are the third region B1 and the fourth region B2, respectively. A user inputs a touch-slide action (see the black arrow in this figure) on the destination projection device 200, sliding from the fourth region B2 outside the third region B1 into the projection window. In the current reverse control solution, after the destination projection device 200 feeds back a touch-slide event corresponding to the touch-slide action to the source projection device 200, the source projection device 200 cannot trigger an edge-slide gesture because the starting position of the touch-slide event is not within the first region A1, i.e., the user's reverse control action fails. However, in this embodiment, the source projection device 200 performs edge gesture detection for the touch-slide event based on the second region A2. The start position of the touch-slide action is located within the fourth region B2, and the start position of the touch-slide event corresponding to the touch-slide action is located within the second region A2. Therefore, the source projection device 200 successfully triggers the gesture to return to the home screen, figure 16B, a return to home screen operation may be performed. When updating the screen display content, the source projection device 200 may transmit the updated screen display content to the destination projection device 200, and the destination projection device 200 updates the screen display content displayed in the projection window accordingly.

[0170] Specifically, for the same edge slide gesture, a user may slide from a first area A1 on the source projection device as required to trigger the edge slide gesture, and on the destination projection device, a user may slide from a fourth area B2 corresponding to the second area A2 as required to trigger the edge slide gesture. In this way, a user can conveniently perform a reverse control operation, and the reliability of the reverse control operation is improved.

[0171] Similarly, in order to help the user to better understand the operation area of ​​the above-mentioned projection reverse direction control, in this embodiment, the source projection device may execute a related function description. An example in which the description of the projection reverse direction control is provided in the new function interface of the "Tips" APP is still used for description.

[0172] FIG. 17 is a schematic diagram of a new function interface 50 on a source projection device. The interface shown in FIG. 17 is similar to the interface shown in FIG. 12(b), except that the contents shown in the function demonstration bar and the function description bar are different. As shown in FIG. 17, the function demonstration bar 501 may show sensing areas corresponding to edge slide gestures on the source projection device and the destination projection device, respectively. An example of the function description text in the function description bar 502 may be "The edge slide gesture of the source projection device can be triggered by sliding inward from edge sensing area 1 of the source projection device or sliding inward from edge sensing area 2 in the projection window. The width of the projection sensing area 3, which corresponds to edge sensing area 2 on the source projection device, is greater than the width of edge sensing area 1." The edge sensing area 1 is the area A1 mentioned above, the edge sensing area 2 is the area B2 mentioned above, and the projection sensing area 3 is the area A2 mentioned above.

[0173] Similarly, the source projection device may also provide a projection sensing area setting function. The specific setting manner is similar to the setting manner of the edge sensing area in FIG. 13. FIG. 18 is a schematic diagram of a projection reverse control interface 60 on the source projection device. The interface shown in FIG. 18 is similar to the interface shown in FIG. 13(b), except that the contents shown in the function description bar and the setting bar are different. As shown in FIG. 18, the contents displayed in the function description bar 601 are similar to those shown in FIG. 17. The projection reverse control function may be simply described in the manner of pictures and texts. For example, an edge sensing area 1 corresponding to the area A1, an edge sensing area 2 corresponding to the area B2, and a projection sensing area 3 corresponding to the area A2 may be shown in the picture. The text may be "Slide inward from the edge sensing area 1 or slide inward from the edge sensing area 2 to trigger the edge slide gesture of the source projection device". The setting bar 602 may provide a related control for the user to set the width of the area B2. For the specific setting manner of the control, please refer to the related description in FIG. 13. The details will not be explained again here.

[0174] According to the device control method provided in this embodiment, the destination projection device may convert a touch slide event generated by a touch slide action performed by a user by sliding from outside the projection window to inside the projection window into an edge slide event of the source projection window. In this way, the source projection device may detect an edge slide gesture based on the complete edge slide event, which can improve the success rate of the edge slide gesture of the source projection device being triggered by a user on the destination projection device, thereby improving the reliability and user experience of the reverse control operation. Additionally, the source projection device uses the first region as a screen edge region to detect an edge slide gesture for the touch slide event triggered by a user on the source projection device, and uses a second region larger than the first region as a screen edge region to detect an edge slide gesture for the received touch slide event. In this way, the success rate of the edge slide gesture of the source projection device being triggered by a user on the destination projection device can be improved, thereby improving the reliability and user experience of the reverse control operation.

[0175] Based on the same inventive concept, in the implementation of the above-mentioned method, an embodiment of this application provides a device control device. The embodiment of the device corresponds to the embodiment of the above-mentioned method. For readability, the details in the embodiment of the above-mentioned method are not described one by one in the embodiment of this device. However, it should be clear that the device in this embodiment can correspondingly implement all the contents in the embodiment of the above-mentioned method.

[0176] 19 is a schematic diagram of the structure of a device control device according to an embodiment of this application. As shown in FIG. 19, the device provided in this embodiment may include a display module 310, an input module 320, a processing module 330, and a communication module 340.

[0177] The display module 310 is configured to support the mobile terminal in performing interface display operations in the above-mentioned embodiments and / or other processes of the techniques described in this specification. The display unit may be a touch screen, other hardware, or a combination of hardware and software.

[0178] The input module 320 is configured to receive a user's input, such as touch input, voice input, or gesture input, on a display interface of the mobile terminal. The input module is configured to assist the source projection device and / or the destination projection device in performing the process of receiving touch-slide actions input by a user in the above-mentioned embodiments and / or other processes of the techniques described in this specification. The input module may be a touch screen, other hardware, or a combination of hardware and software.

[0179] The processing module 330 is configured to support the mobile terminal in performing event processing operations of the source projection device and / or the destination projection device in the above-described embodiments and / or other processes of the techniques described in this specification.

[0180] The communications module 340 is configured to support the mobile terminal in performing operations related to the communication processes between the source and destination projection devices in the above-described embodiments and / or other processes of the techniques described in this specification.

[0181] The apparatus provided in this embodiment can implement the above-mentioned method embodiments, and the implementation principles and technical effects are similar and will not be described again here.

[0182] It will be clearly understood by those skilled in the art that the division of the above-mentioned functional units or modules is used as an example for illustration, for the purpose of convenient and concise description. In actual applications, the above-mentioned functions may be allocated to different functional units or modules for implementation based on requirements, i.e., the internal structure of the device is divided into different functional units or modules to implement all or part of the above-mentioned functions. The functional units or modules in the embodiments may be integrated into one processing unit, each unit may exist physically alone, or two or more units may be integrated into one unit. The integrated units may be implemented in the form of hardware or in the form of software functional units. Additionally, the specific names of the functional units or modules are merely provided to facilitate distinction from each other, and are not intended to limit the scope of protection of this application. For the specific work processes of the units or modules in the above-mentioned system, please refer to the corresponding processes in the above-mentioned method embodiments. Details will not be described again here.

[0183] An embodiment of the present application further provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the method in the above-mentioned method embodiments.

[0184] An embodiment of the present application further provides a computer program product, which, when run on a computer, enables a terminal device to perform the method in the above-mentioned method embodiments.

[0185] An embodiment of the present application further provides a chip system including a processor, the processor being coupled to a memory, and the processor executing a computer program stored in the memory to implement the method in the above-mentioned method embodiment. The chip system may be a single chip or a chip module including multiple chips.

[0186] All or part of the above-described embodiments may be implemented using software, hardware, firmware, or any combination thereof. When software is used to implement the above-described embodiments, all or part of the embodiments may be implemented in the form of a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the procedures or functions according to the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in or transmitted through a computer-readable storage medium. The computer instructions may be transmitted from a website, computer, server, or data center to another website, computer, server, or data center in a wired (e.g., coaxial cable, optical fiber, or digital subscriber line) or wireless (e.g., infrared, radio, or microwave) manner. The computer-readable storage medium may be any available medium accessible by a computer, or a data storage device integrating one or more available media, such as a server or data center. The usable media may be magnetic media (e.g., floppy disk, hard disk, or magnetic tape), optical media (e.g., DVD), or semiconductor media (e.g., Solid State Drive (SSD)).

[0187] Those skilled in the art may understand that all or part of the steps of the method in the embodiment may be implemented by a computer program that instructs relevant hardware. The program may be stored in a computer-readable storage medium. When the program is executed, the steps in the method embodiment may be included. The above-mentioned storage medium may include any medium that can store program code, such as a ROM, a random access memory RAM, a magnetic disk, or an optical disk.

[0188] In the above embodiments, the description of each embodiment has its own focus. For the parts that are not described in detail or recorded in the embodiments, please refer to the related descriptions in other embodiments.

[0189] In the embodiments provided in this application, it should be understood that the disclosed method / device and method may be implemented in other ways. For example, the described embodiment of the device / device is only an example. For example, the division into modules or units is only a logical functional division, and may be other divisions in actual implementation. For example, multiple units or components may be combined or integrated into another system, and some features may be ignored or not implemented. Additionally, the shown or discussed mutual couplings, direct couplings, or communication connections may be implemented through some interfaces. Indirect couplings or communication connections between devices or units may be implemented in electronic, mechanical, or other forms.

[0190] As used in this specification and the claims accompanying this application, the term "comprising" is to be understood as indicating the presence of stated features, integers, steps, operations, elements, and / or components, but not excluding the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or combinations thereof.

[0191] The term "and / or" as used in this specification and the claims accompanying this application is also to be understood to refer to and include any and all possible combinations of one or more of the associated listed items.

[0192] As used in this specification and the claims appended hereto, the term "if" may be interpreted as "when," "with," "response to determining," or "response to detecting," depending on the context. Similarly, depending on the context, the phrase "if it is determined that" or "if (a described condition or event) is detected" may be interpreted as "once it is determined that," "in response to determining," "once (a described condition or event) is detected," or "in response to detecting (a described condition or event)."

[0193] Additionally, in this specification and the claims appended hereto, the terms "first," "second," "third," etc. are intended to be distinguishing and descriptive only and are not to be understood as indications or suggestions of relative importance.

[0194] References to "one embodiment," "some embodiments," and the like described in this specification of this application indicate that one or more embodiments of this application include a particular feature, structure, or characteristic described with reference to the embodiment. Thus, the phrases "in one embodiment," "some embodiments," "some other embodiments," "other embodiments," and the like appearing in different places in this specification are not necessarily meant to refer to the same embodiment. Instead, the phrase means "one or more, but not all, embodiments," unless specifically emphasized otherwise. The terms "including," "having," and variations thereof all mean "including, but not limited to," unless specifically emphasized otherwise.

[0195] Finally, it should be noted that the above embodiments are not intended to limit this application, but merely to describe the technical solutions of this application. Although this application has been described in detail with reference to the above embodiments, it should be understood that those skilled in the art may still make modifications to the technical solutions recorded in the above embodiments, or replace some or all of its technical features with equivalents, without departing from the scope of the technical solutions of the embodiments of this application.

Claims

1. 1. A device control method applied to a system including a destination projection device and a source projection device, comprising: displaying a projection window of the source projection device by the destination projection device; generating, by the destination projection device, a first touch-slide event in response to a first touch-slide action performed by a user by sliding from outside the projection window to inside the projection window; converting the first touch-slide event into an edge slide event of the source projection device and sending the edge slide event to the source projection device when a starting touch point of the first touch-slide action is located within a target area, the target area being an area formed by extending outward from an edge of the projection window a preset distance; updating, by the source projection device, a screen display content based on the edge slide event.

2. The first touch-slide event includes a series of touch events generated by the first touch-slide motion, and converting the first touch-slide event into an edge slide event of the source projection device includes: converting touch events whose coordinates are located within the target area and that are within the first touch slide event into screen edge events of the source projection device; generating the edge slide event based on the screen edge event and a touch event whose coordinates are located within the projection window and are within the first touch slide event.

3. The first touch-slide event includes a series of touch events generated by the first touch-slide motion, and converting the first touch-slide event into an edge slide event of the source projection device includes: generating a touch down event based on a target event, the target event being an initial touch event generated by the first touch-slide action on the projection window, and the touch down event being a screen edge event of the originating projection device; generating the edge slide event based on the touch down event and a touch event located within the projection window and within the first touch slide event.

4. Generating a touch down event based on a target event inserting the touch down event before the target event, the coordinates of the touch down event being the same as the coordinates of the target event; or The method of claim 3 further comprising converting the target event into the touch down event.

5. 5. The method of claim 1, further comprising updating, by the destination projection device, a screen display content of the destination projection device in response to the first touch-slide event when the starting touch point of the first touch-slide action is located outside the target area.

6. in response to a second touch-slide action performed by the user on the source projection device, detecting, by the source projection device, an edge slide gesture for a second touch-slide event generated by the second touch-slide action using the first region as a screen edge region; generating, by the destination projection device, a third touch-slide event corresponding to the source projection device in response to a third touch-slide action performed by the user sliding inward from the projection window, and transmitting the third touch-slide event to the source projection device; After receiving the third touch-slide event, detecting an edge slide gesture for the third touch-slide event by the source projection device using a second region as a screen edge region, where the first region and the second region are both regions formed by extending inward from a screen edge of the source projection device, and a coverage of the second region is greater than a coverage of the first region; 6. The method of claim 1, further comprising: executing, by the source projection device, in response to the edge slide gesture, a control function corresponding to the edge slide gesture and updating the screen display content.

7. The method comprises: transmitting, by the source projection device, the screen display content to the destination projection device; The method of any one of claims 1 to 6, further comprising: after receiving the on-screen content, displaying, by the destination projection device, the on-screen content in the projection window.

8. 1. A device control method applied to a destination projection device, comprising: displaying a projection window of a source projection device; generating a touch-slide event in response to a touch-slide action performed by a user by sliding from outside the projection window to inside the projection window; and converting the touch slide event into an edge slide event of the source projection device when an initial touch point of the touch slide operation is located within a target area, the target area being an area formed by extending outward from an edge of the projection window a predetermined distance.

9. The touch slide event includes a series of touch events generated by the touch slide motion, and converting the touch slide event into an edge slide event of the source projection device includes: converting touch events whose coordinates are located within the target area and that are within the touch slide event into screen edge events for the source projection device; 10. The method of claim 8, further comprising: generating the edge slide event based on the screen edge event and a touch event whose coordinates are located within the projection window and are within the touch slide event.

10. The touch slide event includes a series of touch events generated by the touch slide motion, and converting the touch slide event into an edge slide event of the source projection device includes: generating a touch down event based on a target event, the target event being an initial touch event generated by the touch-slide action on the projection window, and the touch down event being a screen edge event of the originating projection device; 10. The method of claim 8, further comprising: generating the edge slide event based on the touch down event and a touch event located within the projection window and within the touch slide event.

11. Generating a touch down event based on a target event inserting the touch down event before the target event, the coordinates of the touch down event being the same as the coordinates of the target event; or The method of claim 10 further comprising converting the target event into the touch down event.

12. updating a screen display content of the destination projection device in response to the touch-slide event when the initiating touch point of the touch-slide action is located outside the target area; or The method according to any one of claims 8 to 11, further comprising receiving a screen display content corresponding to the touch slide event and transmitted by the source projection device, and displaying the screen display content within the projection window.

13. 1. A device control method applied to a source projection device, comprising: When a touch-slide action is received by a user on the source projection device, using a first region as a screen edge region, detecting an edge slide gesture for a touch-slide event generated by the touch-slide action; When a touch slide event transmitted by a destination projection device is received, using a second region as a screen edge region to detect an edge slide gesture for the touch slide event, wherein the first region and the second region are both regions formed by extending inward from a screen edge of the source projection device, and a coverage of the second region is greater than a coverage of the first region; and executing a control function corresponding to the edge slide gesture and updating a screen display content.

14. A terminal device comprising a memory and a processor, the memory being configured to store a computer program, and the processor being configured to execute the method according to any one of claims 8 to 13 when invoking the computer program.

15. A computer readable storage medium storing a computer program, the computer program implementing the method according to any one of claims 8 to 13 when executed by a processor.

Images