Equipment interaction method and electronic equipment
By obtaining the relative position and attitude data of the electronic device, determining the touch attitude, and realizing interaction between devices, solving the problem of NFC tag restrictions in the prior art, and achieving flexible interaction and functional expansion between devices.
Patent Information
- Application Number
- CN202411056035.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2020-08-05
- Publication Date
- 2024-11-12
AI Technical Summary
The existing cross-device interaction methods rely on NFC tags, limiting the interaction between devices and requiring devices to have NFC functions, so they cannot achieve flexible data transmission and functional expansion between all devices.
By obtaining the relative position and attitude data between electronic devices, determining the touch attitude, thereby realizing interaction between devices, including data transmission, display screen expansion, input function expansion and operation area expansion, using UWB or Bluetooth positioning chips and gyroscopic screws Sensors such as instruments and acceleration sensors obtain position and attitude data.
It realizes flexible interaction between devices, eliminates dependence on NFC functions, supports interaction between devices of different types and the same types, and improves interoperability and functional expansion capabilities between devices.
Smart Images

Figure CN118945629A_ABST
Abstract
Description
[0001] This application is a divisional application. The application number of the original application is 202010780044.6, and the original application date is August 5, 2020. The entire contents of the original application are incorporated into this application by reference. Technical Field
[0002] The present application relates to the field of terminal technology, and in particular to a device interaction method and an electronic device. Background Art
[0003] With the development of smart terminal technology, the functions of electronic devices are becoming more and more powerful, and there are more and more scenarios where electronic devices need to interact with each other. In the process of using electronic devices, it is often necessary to switch the data in one electronic device to another electronic device for display. Or, it is necessary to expand the functions of one electronic device to another electronic device.
[0004] At present, the way to achieve this cross-device interaction can be: set a near field communication (NFC) tag in the electronic device. When two or more electronic devices need to exchange data or extend functions, the corresponding service is pulled up through the NFC tag. For example, the NFC tag of the first electronic device touches the NFC tag of the second electronic device. Then, a trust relationship can be established between the first electronic device and the second electronic device. Afterwards, the first electronic device can transfer the first file to the second electronic device for display. Alternatively, some functions of the first electronic device can be extended to the second electronic device for implementation.
[0005] In the above method, electronic devices initiate data transmission or function expansion services by touching NFC tags. Although the implementation method is relatively convenient, it requires the electronic devices to have NFC functions. Summary of the invention
[0006] The embodiments of the present application provide a device interaction method and an electronic device to trigger interaction between electronic devices based on different touch gestures between the electronic devices.
[0007] In a first aspect, an embodiment of the present application provides a device interaction method, the method being applied to a first electronic device, comprising:
[0008] Acquiring a relative position of a second electronic device;
[0009] determining the approach direction of the second electronic device according to the relative position;
[0010] Acquire first posture data of the first electronic device and second posture data of the second electronic device;
[0011] determining a touch posture of the first electronic device and the second electronic device according to the approach direction, the first posture data, and the second posture data;
[0012] According to the touch gesture, the second electronic device is determined as an extension device of the first electronic device.
[0013] In an embodiment of the present application, when the second electronic device approaches the first electronic device, the first electronic device obtains the relative position of the second electronic device. The first electronic device determines the approach direction of the second electronic device based on the relative position. Further, the first electronic device determines the touch gesture of the first electronic device and the second electronic device in combination with the gesture data of the first electronic device and the second electronic device. Based on the touch gesture, the first electronic device can determine the second electronic device as an extended device. Thereby, operations such as data transmission, display screen expansion, input function expansion, and operation area expansion can be performed in the second electronic device. It can be seen that based on the embodiment of the present application, electronic devices can interact with each other based on different touch gestures between electronic devices.
[0014] In combination with the first aspect, in some implementations of the first aspect, obtaining the relative position of the second electronic device includes:
[0015] Detecting the location data of the first electronic device or the second electronic device through an ultra-wideband (UWB) positioning chip or a Bluetooth positioning chip;
[0016] The relative position of the second electronic device is determined according to the position data.
[0017] In conjunction with the first aspect, in some implementations of the first aspect, obtaining the first posture data of the first electronic device includes any one of the following methods or a combination thereof:
[0018] Detecting motion posture data of the first electronic device through a gyroscope sensor;
[0019] The acceleration data of the first electronic device is detected by the acceleration sensor.
[0020] In combination with the first aspect, in some implementations of the first aspect, obtaining second posture data of the second electronic device includes:
[0021] receiving the second posture data sent by the second electronic device;
[0022] The second posture data includes: motion posture data, acceleration data or horizontal and vertical screen state data of the second electronic device.
[0023] In combination with the first aspect, in some implementations of the first aspect, the first gesture data includes a first touch signal, and the second gesture data includes a second touch signal;
[0024] Determining the touch posture of the first electronic device and the second electronic device according to the approach direction, the first posture data, and the second posture data includes:
[0025] The edge or surface touched by the first electronic device and the second electronic device is determined according to the first touch signal and the second touch signal.
[0026] In combination with the first aspect, in some implementations of the first aspect, the first posture data includes first acceleration data;
[0027] Correspondingly, the landscape or portrait state of the first electronic device is determined according to the first acceleration data, or the speed at which the first electronic device moves toward the second electronic device is determined, or the tilt angle of the first electronic device is determined.
[0028] In combination with the first aspect, in some implementations of the first aspect, the second posture data includes second acceleration data or horizontal and vertical screen state data;
[0029] Correspondingly, the horizontal or vertical screen state of the second electronic device is determined according to the second acceleration data or the horizontal or vertical screen state data; or,
[0030] The speed at which the second electronic device moves toward the first electronic device or the tilt angle of the second electronic device is determined according to the second acceleration data.
[0031] In combination with the first aspect, in some implementations of the first aspect, the touch gestures of the first electronic device and the second electronic device include any one of the following:
[0032] The first long side of the first electronic device touches the second long side of the second electronic device;
[0033] The first long side of the first electronic device touches the first short side of the second electronic device;
[0034] The second short side of the first electronic device touches the third long side of the second electronic device;
[0035] The second short side of the first electronic device touches the third short side of the second electronic device;
[0036] The first surface of the first electronic device touches the second surface of the second electronic device;
[0037] A preset edge or surface of the second electronic device touches the accessory of the first electronic device.
[0038] In combination with the first aspect, in some implementations of the first aspect, the touch direction of the touch gesture includes:
[0039] The first electronic device and the second electronic device move toward each other and then touch each other.
[0040] In combination with the first aspect, in some implementations of the first aspect, determining the second electronic device as an extension device of the first electronic device according to the touch gesture includes:
[0041] If the first electronic device and the second electronic device are devices of the same type; then
[0042] When the first electronic device actively applies a touch to the second electronic device, the first electronic device is determined as a main device, and the second electronic device that passively receives the touch is determined as an extended device; or,
[0043] When the second electronic device actively applies a touch to the first electronic device, the first electronic device is determined as a main device; and the second electronic device that actively applies a touch is determined as an extended device.
[0044] In combination with the first aspect, in some implementations of the first aspect, the method further includes:
[0045] If the first electronic device and the second electronic device move towards each other and touch each other;
[0046] A first prompt message is displayed in the first electronic device; the first prompt message is used to prompt the user to determine the main device from the first device or the second device.
[0047] In combination with the first aspect, in some implementations of the first aspect, the method further includes:
[0048] If the first electronic device and the second electronic device are different types of devices; then
[0049] When the first electronic device is a preset type device, the first electronic device is determined as a main device, and the second electronic device is determined as an extended device.
[0050] In combination with the first aspect, in some implementations of the first aspect, after determining the second electronic device as an extension device of the first electronic device, the method further includes:
[0051] According to the display screen information of the first electronic device and the second electronic device, determining the display screens of the first electronic device and the second electronic device as a combined screen;
[0052] sending the first data in the first electronic device to the second electronic device based on the combined screen; or,
[0053] Display the first data on the combined screen; or,
[0054] The combined screen area is determined as the operation area.
[0055] In combination with the first aspect, in some implementations of the first aspect, sending the first data in the first electronic device to the second electronic device based on the combined screen includes:
[0056] Detecting a first gesture action acting on first data;
[0057] Displaying a moving track of the first data on the combined screen according to the first gesture action;
[0058] The first data is transmitted from the first electronic device to the second electronic device according to the first gesture action.
[0059] In combination with the first aspect, in some implementations of the first aspect, displaying the first data on the combined screen includes:
[0060] A second gesture action is detected; or a third gesture action is detected; fourth gesture action data sent by the second electronic device is received; a fourth gesture action is determined according to the fourth gesture action data; if the combination of the third gesture action and the fourth gesture action is a set action; then,
[0061] Determine to display the first data on the combined screen.
[0062] In combination with the first aspect, in some implementations of the first aspect, the method further includes:
[0063] According to the second gesture action or the combined action, a thumbnail of the first data is displayed on a display screen of the first electronic device or the combined screen.
[0064] In combination with the first aspect, in some implementations of the first aspect, the first data is a user interface displayed in the first electronic device;
[0065] Displaying the first data on the combined screen includes:
[0066] dividing the user interface into a first sub-interface and a second sub-interface;
[0067] Displaying the first sub-interface on a display screen of the first electronic device;
[0068] The data of the second sub-interface is sent to the second electronic device, so that the second sub-interface is displayed in the second electronic device.
[0069] In combination with the first aspect, in some implementations of the first aspect, a first user interface is displayed in the first electronic device;
[0070] Displaying the first data on the combined screen includes:
[0071] detecting a first operation on a first control in the first user interface;
[0072] In response to the first operation, instruction information for displaying a second control or displaying a first sub-interface is sent to the second electronic device.
[0073] In combination with the first aspect, in some implementations of the first aspect, determining the combined screen area as the operation area includes:
[0074] Detecting a first input instruction, and determining that the first input instruction acts on an area where the display screen of the second electronic device is located;
[0075] According to the first input instruction, sending a second input instruction to the second electronic device, so that the second electronic device responds according to the second input instruction;
[0076] Receive a response result generated by the second electronic device according to the second input instruction.
[0077] In a second aspect, an embodiment of the present application provides an electronic device, wherein the device is a first electronic device, including:
[0078] A microphone; one or more processors; a memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and the one or more computer programs include instructions, which, when executed by the electronic device, cause the electronic device to perform the following steps:
[0079] Acquiring a relative position of a second electronic device;
[0080] determining the approach direction of the second electronic device according to the relative position;
[0081] Acquire first posture data of the first electronic device and second posture data of the second electronic device;
[0082] determining a touch posture of the first electronic device and the second electronic device according to the approach direction, the first posture data, and the second posture data;
[0083] According to the touch gesture, the second electronic device is determined as an extension device of the first electronic device.
[0084] In conjunction with the second aspect, in some implementations of the second aspect, when the instruction is executed by the device, the device specifically performs the following steps:
[0085] Detecting the location data of the first electronic device or the second electronic device by using a UWB positioning chip or a Bluetooth positioning chip;
[0086] The relative position of the second electronic device is determined according to the position data.
[0087] In conjunction with the second aspect, in some implementations of the second aspect, when the instruction is executed by the device, the device specifically performs the following steps:
[0088] Detecting motion posture data of the first electronic device through a gyroscope sensor;
[0089] The acceleration data of the first electronic device is detected by the acceleration sensor.
[0090] In conjunction with the second aspect, in some implementations of the second aspect, when the instruction is executed by the device, the device specifically performs the following steps:
[0091] receiving the second posture data sent by the second electronic device;
[0092] The second posture data includes: motion posture data, acceleration data or horizontal and vertical screen state data of the second electronic device.
[0093] In combination with the second aspect, in some implementations of the second aspect, the first gesture data includes a first touch signal, and the second gesture data includes a second touch signal;
[0094] When the instruction is executed by the device, the device specifically performs the following steps:
[0095] The edge or surface touched by the first electronic device and the second electronic device is determined according to the first touch signal and the second touch signal.
[0096] In conjunction with the second aspect, in some implementations of the second aspect, the first posture data includes first acceleration data;
[0097] Accordingly, when the instruction is executed by the device, the device specifically performs the following steps:
[0098] The horizontal or vertical screen state of the first electronic device is determined according to the first acceleration data, or the speed at which the first electronic device moves toward the second electronic device is determined, or the tilt angle of the first electronic device is determined.
[0099] In conjunction with the second aspect, in some implementations of the second aspect, the second posture data includes second acceleration data or horizontal and vertical screen state data;
[0100] Accordingly, when the instruction is executed by the device, the device specifically performs the following steps:
[0101] determining the horizontal or vertical screen state of the second electronic device according to the second acceleration data or the horizontal or vertical screen state data; or,
[0102] The speed at which the second electronic device moves toward the first electronic device or the tilt angle of the second electronic device is determined according to the second acceleration data.
[0103] In conjunction with the second aspect, in some implementations of the second aspect, the touch gestures of the first electronic device and the second electronic device include any one of the following:
[0104] The first long side of the first electronic device touches the second long side of the second electronic device;
[0105] The first long side of the first electronic device touches the first short side of the second electronic device;
[0106] The second short side of the first electronic device touches the third long side of the second electronic device;
[0107] The second short side of the first electronic device touches the third short side of the second electronic device;
[0108] The first surface of the first electronic device touches the second surface of the second electronic device;
[0109] A preset edge or surface of the second electronic device touches the accessory of the first electronic device.
[0110] In combination with the second aspect, in some implementations of the second aspect, the touch direction of the touch gesture includes:
[0111] The first electronic device and the second electronic device move toward each other and then touch each other.
[0112] In conjunction with the second aspect, in some implementations of the second aspect, when the instruction is executed by the device, the device specifically performs the following steps:
[0113] If the first electronic device and the second electronic device are devices of the same type; then
[0114] When the first electronic device actively applies a touch to the second electronic device, the first electronic device is determined as a main device, and the second electronic device that passively receives the touch is determined as an extended device; or,
[0115] When the second electronic device actively applies a touch to the first electronic device, the first electronic device is determined as a main device; and the second electronic device that actively applies a touch is determined as an extended device.
[0116] In conjunction with the second aspect, in some implementations of the second aspect, when the instruction is executed by the device, the device specifically performs the following steps:
[0117] If the first electronic device and the second electronic device move towards each other and touch each other;
[0118] A first prompt message is displayed in the first electronic device; the first prompt message is used to prompt the user to determine the main device from the first device or the second device.
[0119] In conjunction with the second aspect, in some implementations of the second aspect, when the instruction is executed by the device, the device specifically performs the following steps:
[0120] If the first electronic device and the second electronic device are different types of devices; then
[0121] When the first electronic device is a preset type device, the first electronic device is determined as a main device, and the second electronic device is determined as an extended device.
[0122] In conjunction with the second aspect, in some implementations of the second aspect, when the instruction is executed by the device, the device specifically performs the following steps:
[0123] According to the display screen information of the first electronic device and the second electronic device, determining the display screens of the first electronic device and the second electronic device as a combined screen;
[0124] sending the first data in the first electronic device to the second electronic device based on the combined screen; or,
[0125] Display the first data on the combined screen; or,
[0126] The combined screen area is determined as the operation area.
[0127] In conjunction with the second aspect, in some implementations of the second aspect, when the instruction is executed by the device, the device specifically performs the following steps:
[0128] Detecting a first gesture action acting on first data;
[0129] Displaying a moving track of the first data on the combined screen according to the first gesture action;
[0130] The first data is transmitted from the first electronic device to the second electronic device according to the first gesture action.
[0131] In conjunction with the second aspect, in some implementations of the second aspect, when the instruction is executed by the device, the device specifically performs the following steps:
[0132] A second gesture action is detected; or a third gesture action is detected; fourth gesture action data sent by the second electronic device is received; a fourth gesture action is determined according to the fourth gesture action data; if the combination of the third gesture action and the fourth gesture action is a set action; then,
[0133] Determine to display the first data on the combined screen.
[0134] In conjunction with the second aspect, in some implementations of the second aspect, when the instruction is executed by the device, the device specifically performs the following steps:
[0135] According to the second gesture action or the combined action, a thumbnail of the first data is displayed on a display screen of the first electronic device or the combined screen.
[0136] In conjunction with the second aspect, in some implementations of the second aspect, the first data is a user interface displayed in the first electronic device;
[0137] When the instruction is executed by the device, the device specifically performs the following steps:
[0138] dividing the user interface into a first sub-interface and a second sub-interface;
[0139] Displaying the first sub-interface on a display screen of the first electronic device;
[0140] The data of the second sub-interface is sent to the second electronic device, so that the second sub-interface is displayed in the second electronic device.
[0141] In combination with the second aspect, in some implementations of the second aspect, a first user interface is displayed in the first electronic device;
[0142] When the instruction is executed by the device, the device specifically performs the following steps:
[0143] detecting a first operation on a first control in the first user interface;
[0144] In response to the first operation, instruction information for displaying a second control or displaying a first sub-interface is sent to the second electronic device.
[0145] In conjunction with the second aspect, in some implementations of the second aspect, when the instruction is executed by the device, the device specifically performs the following steps:
[0146] Detecting a first input instruction, and determining that the first input instruction acts on an area where the display screen of the second electronic device is located;
[0147] According to the first input instruction, sending a second input instruction to the second electronic device, so that the second electronic device responds according to the second input instruction;
[0148] Receive a response result generated by the second electronic device according to the second input instruction.
[0149] In a third aspect, an embodiment of the present application provides an electronic device, comprising a storage medium and a central processing unit, wherein the storage medium may be a non-volatile storage medium, wherein a computer executable program is stored in the storage medium, and the central processing unit is connected to the non-volatile storage medium and executes the computer executable program to implement the method in the first aspect or any possible implementation manner of the first aspect.
[0150] In a fourth aspect, the present technical solution provides a chip, comprising a processor and a data interface, wherein the processor reads instructions stored in a memory through the data interface to execute the method in the first aspect or any possible implementation of the first aspect.
[0151] Optionally, as an implementation method, the chip may also include a memory, in which instructions are stored, and the processor is used to execute the instructions stored in the memory. When the instructions are executed, the processor is used to execute the method in the first aspect or any possible implementation method of the first aspect.
[0152] In a fifth aspect, the present technical solution provides a computer-readable storage medium, wherein the computer-readable medium stores a program code for execution by a device, wherein the program code includes instructions for executing the method in the first aspect or any possible implementation of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS
[0153] Figure 1 It is a structural diagram of a device interaction system according to an embodiment of the present application;
[0154] Figure 2A-2D is a schematic diagram of the relative positions and postures between devices in an embodiment of the present application;
[0155] Figure 3A-3B It is a structural schematic diagram of an electronic device according to an embodiment of the present application;
[0156] Figure 4A-4G This is a schematic diagram of a device interaction in a scenario of an embodiment of the present application;
[0157] Figure 5A-5E This is a schematic diagram of device interaction under scenario 2 of an embodiment of the present application;
[0158] Figure 6A-6D This is a schematic diagram of device interaction under scenario three of the embodiment of the present application;
[0159] Figure 6E-6F This is a schematic diagram of device interaction under scenario 4 of the embodiment of the present application;
[0160] Figure 7A-7G This is a schematic diagram of device interaction under scenario five of the embodiment of the present application;
[0161] Figure 8A-B This is a schematic diagram of device interaction under scenario 6 of an embodiment of the present application;
[0162] Fig. 9 It is a flow chart of the device interaction method of the embodiment of the present application. DETAILED DESCRIPTION
[0163] The technical solution in this application will be described below in conjunction with the accompanying drawings.
[0164] The device interaction method provided in the embodiment of the present application can be applied to Figure 1 In the system shown. Figure 1As shown, the system includes multiple electronic devices. The multiple electronic devices may be mobile phones, tablet computers, computers, wearable devices, vehicle-mounted devices, smart home devices, augmented reality (AR) / virtual reality (VR) devices, etc. The multiple electronic devices may be interconnected through a communication network. Exemplarily, the communication network may be a wired network or a wireless network. For example, the communication network may be a local area network (LAN) or a wide area network (WAN). The communication network may be implemented using any known network communication protocol, and the communication network protocol may be various wired or wireless communication protocols. Exemplarily, Figure 1 The various electronic devices in the system shown can establish a Wi-Fi connection through the Wi-Fi protocol. Alternatively, Figure 1 Each electronic device in the system shown can be connected to one or more servers after logging into the same account. Figure 1 The electronic devices in the system shown can establish a connection through near field communication technologies such as Bluetooth (BT), wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), and near field communication (NFC). After the connection is established, data can be transmitted between the electronic devices through technologies such as BT, Wi-Fi, and NFC. It should be noted that the technology used to establish a connection between electronic devices and the technology used for data transmission can be the same or different.
[0165] Exemplarily, the communication system includes a first electronic device and a second electronic device. The first electronic device displays a user interface of a first application. The second electronic device displays a user interface of a second application. For example, the first electronic device displays a user interface of a gallery application. The second electronic device displays a chat interface of an instant messaging application. The first electronic device can send image data in the gallery to the second electronic device. The second electronic device inputs the received image data into the chat interface for display.
[0166] There are many ways to trigger the first electronic device to send image data to the second electronic device. For example, when the first electronic device reads the NFC tag information of the second electronic device, the first electronic device sends the image data to the second electronic device. Different from the data transmission between devices triggered by NFC tags, the embodiment of the present application provides a method for determining the interactive service between devices based on the relative position and posture between the devices. The interactive service between devices determined according to the relative position and posture between the devices can be: cross-device data transmission, display screen expansion, function expansion, operation area expansion, etc.
[0167] In an embodiment of the present application, a method for determining the relative position and posture between devices may be: the first electronic device discovers the second electronic device; the first electronic device determines the relative position and posture between the first electronic device and the second electronic device based on the approach direction of the first electronic device and the second electronic device and the posture data of the first electronic device and the second electronic device.
[0168] Among them, the first electronic device discovers the second electronic device, which can be: the first electronic device measures the distance of the electronic devices in the surrounding space. The first electronic device uses the electronic devices within the first distance range as discovery devices. Alternatively, the first electronic device obtains electronic devices connected to the same network. For example, the first electronic device obtains various electronic devices connected to the same Wi-Fi network. The first electronic device uses various electronic devices connected to the Wi-Fi network as discovery devices. Alternatively, the first electronic device turns on the Bluetooth function, and the first electronic device uses various Bluetooth-enabled electronic devices as discovery devices. Alternatively, the first electronic device determines the device close to the first electronic device through a proximity light sensor, etc., and determines the device close to the first electronic device as a discovery device. After the first electronic device determines each discovery device, it can determine the second electronic device from the discovery devices based on the user's selection operation or the relative distance between the devices.
[0169] In the embodiment of the present application, the manner in which the first electronic device obtains the relative position of the first electronic device and the second electronic device may include: the first electronic device and the second electronic device determine the distance and orientation between the first electronic device and the second electronic device through wireless positioning technology. The wireless positioning may be positioning technology such as GPS, UWB, wifi, and Bluetooth.
[0170] In some embodiments, the first electronic device sends a first positioning signal to the second electronic device. After receiving the first positioning signal, the second electronic device sends a second positioning signal to the second electronic device. The first electronic device determines the distance and orientation of the second electronic device based on the first positioning signal and the second positioning signal. After the first electronic device determines the distance and orientation of the first electronic device, it can determine the approach direction of the second electronic device.
[0171] In other embodiments, an auxiliary device capable of transmitting / receiving positioning signals is provided in the space where the first electronic device and the second electronic device are located. The first electronic device and the second electronic device determine their own absolute position information through the auxiliary device, wherein the absolute position information may be three-degree-of-freedom position information or six-degree-of-freedom information, etc. The first electronic device determines the distance and orientation of the second electronic device based on the absolute position information of the first electronic device and the second electronic device. The first electronic device can determine the approach direction of the second electronic device based on the distance and orientation of the second electronic device.
[0172] Furthermore, in addition to obtaining the distance and orientation between the first electronic device and the second electronic device, the first electronic device can also obtain the acceleration, tilt angle, collision signal or horizontal and vertical screen status and other posture data of the first electronic device and the second electronic device. On the basis of determining the approach direction of the second electronic device, the first electronic device can further determine the relative position posture between the first electronic device and the second electronic device according to the posture data of the first electronic device and the second electronic device. Optionally, the relative position posture can also be referred to as the approach posture or touch posture of the first electronic device and the second electronic device.
[0173] In some embodiments, the relative position and posture between the devices may be: the distance between the predetermined side walls or predetermined surfaces of the two devices is less than a threshold, or the predetermined side walls or predetermined surfaces of the two devices collide at a certain angle, etc. The relative position and posture between the devices will be described below with reference to specific examples.
[0174] In some embodiments, Figure 2A-2D As shown, the first electronic device is a mobile phone 1, and the second electronic device is a mobile phone 2. Since a mobile phone is generally a rectangular parallelepiped, for the convenience of description, in the embodiment of the present application, the long side wall of the mobile phone is referred to as the long side, and the short side wall of the mobile phone is referred to as the short side. Then,
[0175] like Figure 2AAs shown, the relative position posture between mobile phone 1 and mobile phone 2 can be: the distance between the long side of mobile phone 1 and the long side of mobile phone 2 is less than the first threshold. Specifically, it can be further divided into: the long side of mobile phone 1 moves close to the long side of mobile phone 2, so that the distance between the long side of mobile phone 1 and the long side of mobile phone 2 is less than the first threshold. Or, the long side of mobile phone 2 moves close to the long side of mobile phone 1, so that the distance between the long side of mobile phone 1 and the long side of mobile phone 2 is less than the first threshold. Or, the long side of mobile phone 1 and the long side of mobile phone 2 move close to each other, so that the distance between the long side of mobile phone 1 and the long side of mobile phone 2 is less than the first threshold. More specifically, mobile phone 1 and mobile phone 2 both contain two long sides. When determining the relative position posture, mobile phone 1 and mobile phone 2 can further distinguish the long sides. For example, mobile phone 1 contains the first long side and the second long side. Mobile phone 2 contains the third long side and the fourth long side. Then the posture where the distance between the long sides of mobile phone 1 and mobile phone 2 is less than the first threshold can be further divided into: the distance between the first long side of mobile phone 1 and the third long side of mobile phone 2 is less than the first threshold; the distance between the first long side of mobile phone 1 and the fourth long side of mobile phone 2 is less than the first threshold; the distance between the second long side of mobile phone 1 and the third long side of mobile phone 2 is less than the first threshold; the distance between the second long side of mobile phone 1 and the fourth long side of mobile phone 2 is less than the first threshold. Of course, when distinguishing the long sides to determine the posture between mobile phone 1 and mobile phone 2, the moving approach direction of the device can also be considered, which will not be expanded here one by one.
[0176] Still Figure 2A For example, the relative position posture between mobile phone 1 and mobile phone 2 can be: the long side of mobile phone 1 collides with the long side of mobile phone 2 in a first posture. Specifically, the long side of mobile phone 1 moves close to the long side of mobile phone 2, and the long side of mobile phone 1 collides with the long side of mobile phone 2. Alternatively, the long side of mobile phone 2 moves close to the long side of mobile phone 1, and the long side of mobile phone 2 collides with the long side of mobile phone 1. Alternatively, the long side of mobile phone 1 and the long side of mobile phone 2 move close to each other, and the long side of mobile phone 1 collides with the long side of mobile phone 2. More specifically, when determining the collision posture, mobile phone 1 and mobile phone 2 can also distinguish between the long side and the short side. The distinction method can be found in Figure 2A The instructions in the text are not given here one by one.
[0177] like Figure 2BAs shown, the relative position posture between mobile phone 1 and mobile phone 2 can be: the distance between the long side of mobile phone 1 and the short side of mobile phone 2 is less than the second threshold. Specifically, it can be further divided into: the long side of mobile phone 1 moves close to the short side of mobile phone 2, so that the distance between the long side of mobile phone 1 and the short side of mobile phone 2 is less than the second threshold. Alternatively, the short side of mobile phone 2 moves close to the long side of mobile phone 1, so that the distance between the long side of mobile phone 1 and the short side of mobile phone 2 is less than the second threshold. Alternatively, the long side of mobile phone 1 and the short side of mobile phone 2 move close to each other, so that the distance between the long side of mobile phone 1 and the short side of mobile phone 2 is less than the second threshold. Of course, when determining the relative position posture, mobile phone 1 and mobile phone 2 can distinguish between the long side and the short side. The method of distinguishing can be seen in Figure 2A The instructions in the text are not given here one by one.
[0178] Still Figure 2B For example, the relative position posture between mobile phone 1 and mobile phone 2 can be: the long side of mobile phone 1 collides with the short side of mobile phone 2 in a second posture. Specifically, it can be further divided into: the long side of mobile phone 1 moves close to the short side of mobile phone 2, and the long side of mobile phone 1 collides with the short side of mobile phone 2. Or, the short side of mobile phone 2 moves close to the long side of mobile phone 1, and the short side of mobile phone 2 collides with the long side of mobile phone 1. Or, the long side of mobile phone 1 and the short side of mobile phone 2 move close to each other, and the long side of mobile phone 1 collides with the short side of mobile phone 2. Of course, when determining the relative position posture of mobile phone 1 and mobile phone 2, the long side and the short side can be distinguished. The method of distinguishing can be seen in Figure 2A The instructions in the text are not given here one by one.
[0179] like Figure 2C As shown, the relative position posture between mobile phone 1 and mobile phone 2 can be: the distance between the short side of mobile phone 1 and the short side of mobile phone 2 is less than the third threshold. Specifically, it can be further divided into: the short side of mobile phone 1 moves close to the short side of mobile phone 2, so that the distance between the short side of mobile phone 1 and the short side of mobile phone 2 is less than the third threshold. Alternatively, the short side of mobile phone 2 moves close to the short side of mobile phone 1, so that the distance between the short side of mobile phone 1 and the short side of mobile phone 2 is less than the third threshold. Alternatively, the short side of mobile phone 1 and the short side of mobile phone 2 move close to each other, so that the distance between the short side of mobile phone 1 and the short side of mobile phone 2 is less than the third threshold. Of course, when determining the relative position posture of mobile phone 1 and mobile phone 2, the short sides can be distinguished. The method of distinguishing can be seen in Figure 2A The instructions in the text are not given here one by one.
[0180] Still Figure 2CFor example, the relative position posture between mobile phone 1 and mobile phone 2 can be: the short side of mobile phone 1 collides with the short side of mobile phone 2 in a third posture. Specifically, it can be subdivided into: the short side of mobile phone 1 moves close to the short side of mobile phone 2, and the short side of mobile phone 1 collides with the short side of mobile phone 2. Or, the short side of mobile phone 2 moves close to the short side of mobile phone 1, and the short side of mobile phone 2 collides with the short side of mobile phone 1. Or, the short side of mobile phone 1 and the short side of mobile phone 2 move close to each other, and the short side of mobile phone 1 collides with the short side of mobile phone 2. Of course, when determining the relative position posture of mobile phone 1 and mobile phone 2, the short sides can be distinguished. The method of distinguishing can be seen in Figure 2A The instructions in the text are not given here one by one.
[0181] like Figure 2D As shown, the relative position posture between mobile phone 1 and mobile phone 2 can be: one surface of mobile phone 1 is stacked with one surface of mobile phone 2. For example, the display screen of mobile phone 1 is in contact with the display screen of mobile phone 2 and they are stacked at a certain tilt angle. For another example, the display screen of mobile phone 1 is in contact with the back of mobile phone 2 and they are stacked at a certain requested angle. Of course, when subdividing the posture, the moving direction and tilt angle of mobile phone 1 and mobile phone 2 can also be distinguished. Different stacking postures can be defined according to the different contact surfaces of the mobile phones when stacking, the different moving directions of the mobile phones, and the different tilt angles of the mobile phones, etc., and examples will not be given one by one here.
[0182] In the present embodiment, when the preset position and posture between the first electronic device and the second electronic device are satisfied, the first electronic device and the second electronic device can determine the interactive service between the devices according to the current position and posture. That is, the relative position and posture between the devices are associated with the interactive service. Different position and posture between the devices can trigger different interactive services between the devices. Different interactive services support different interactive modes and interactive contents. For example, when the first electronic device and the second electronic device are in the first relative position and posture, the first electronic device can send documents, pictures, etc. displayed on the display screen to the second electronic device. For another example, when the first electronic device and the second electronic device are in the second relative position and posture, the first electronic device determines the display screen of the second electronic device as an extended screen. The first electronic device displays a part of the user interface displayed on the display screen on its own display screen, and another part on the second electronic device. For another example, when the first electronic device and the second electronic device are in the third relative position and posture, the first electronic device can determine the second electronic device as a function expansion device. For example, the first electronic device determines to display an input control (such as a keyboard interface) in the second electronic device. When the user enters information in the input control of the second electronic device, the second electronic device sends the input information to the first electronic device. The first electronic device responds based on the input information. That is, under the third relative position and posture, the second electronic device can be extended to the input device of the first electronic device. For another example, when the first electronic device and the second electronic device are in a fourth relative position posture, the first electronic device can use the display area of the second electronic device as an operation extension area. When the input device of the first electronic device acts on the operation extension area, the first electronic device sends an input instruction to the second electronic device. The second electronic device responds according to the input instruction. That is, in the fourth relative position posture, the first electronic device can operate and control the first electronic device.
[0183] In some embodiments, when determining the interactive service to be initiated between devices based on the relative position and posture between the devices, constraints such as device type, application content displayed on the device display, and user gestures may also be added. For example, the first electronic device and the second electronic device are in a first relative position and posture. If the first electronic device and the second electronic device are devices of the same type, the first electronic device and the second electronic device may interact in a first interactive service manner. If the first electronic device and the second electronic device are devices of different types, the first electronic device and the second electronic device may interact in a second interactive service manner.
[0184] In a specific example, the first electronic device and the second electronic device are devices of the same type. When the long side of the first electronic device collides with the long side of the second electronic device, the first electronic device is determined to be the main device, and the second electronic device is determined to be the extension device of the first electronic device. Alternatively, the first electronic device is determined to be the extension device, and the second electronic device is determined to be the main device. Then, the main device can send the first data to the extension device. Alternatively, the main device uses the display screen of the extension device as the extension screen, and the main device displays part of the first data in the main device and displays another part of the first data in the extension device. Alternatively, the main device implements some functions of the main device on the extension device. For example, the extension device can be used as an input device of the main device, etc. Among them, which interactive service is specifically performed between the main device and the extension device can also be determined according to the interface content displayed in the main device and the extension device, the user's gesture action, etc. For example, the main device displays the user interface of the first application, and the extension device displays the main interface of the desktop application. When the main device obtains the first gesture action of the user on the main device and / or the extension device, the main device sends the first data to the extension device. When the second gesture action is detected, the main device uses the extension device as an extended display screen to jointly display the first data. When the second application is displayed in the device, the extension device can implement some functions of the main device.
[0185] In another specific example, the first electronic device and the second electronic device are electronic devices of different types. When the long side of the first electronic device collides with the long side of the second electronic device, the electronic device of the preset type can be determined as the main device. For example, the first electronic device is a mobile phone, and the second electronic device is a laptop computer. When the long side of the mobile phone collides with any side of the laptop computer display screen, the laptop computer is determined to be the main device, and the mobile phone is determined to be the extended device. At this time, the laptop computer can determine the spatial area where the mobile phone display screen is located as the virtual operating space. The operating space of the laptop computer input device can be extended to the virtual operating space. That is, the mobile phone can be controlled by the laptop computer input device.
[0186] For the electronic devices involved in the embodiments of the present application, Figure 3AAs shown, the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.
[0187] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.
[0188] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.
[0189] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.
[0190] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.
[0191] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuits sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.
[0192] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present invention is only a schematic illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.
[0193] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.
[0194] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared technology (IR), ultra-wideband (UWB), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the electromagnetic wave signal and filters it, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.
[0195] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, UWB and / or IR technology, etc. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS). In some embodiments, the electronic device may be positioned by using technologies such as GPS, UWB, IR, BT, and Wi-Fi. In an embodiment of the present application, the relative position and posture between electronic devices may be obtained based on wireless positioning technologies such as GPS, UWB, IR, BT, and Wi-Fi.
[0196] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.
[0197] The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.
[0198] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function, such as storing music, video and other files in the external memory card.
[0199] The internal memory 121 can be used to store computer executable program codes, which include instructions. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the electronic device 100 by running instructions stored in the internal memory 121, and / or instructions stored in a memory provided in the processor.
[0200] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.
[0201] The pressure sensor 180A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be set on the display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. A capacitive pressure sensor can be a parallel plate including at least two conductive materials. When a force acts on the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 100 determines the intensity of the pressure based on the change in capacitance. When a touch operation acts on the display screen 194, the electronic device 100 detects the intensity of the touch operation based on the pressure sensor 180A. The electronic device 100 can also calculate the position of the touch based on the detection signal of the pressure sensor 180A. In an embodiment of the present application, it can be determined whether other electronic devices are stacked on the electronic device based on the pressure signal detected by the pressure sensor 180A.
[0202] The gyro sensor 180B can be used to determine the motion posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (i.e., x, y, and z axes) can be determined by the gyro sensor 180B. The gyro sensor 180B can be used for anti-shake shooting. For example, when the shutter is pressed, the gyro sensor 180B detects the angle of the shaking of the electronic device 100, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the electronic device 100 through reverse movement to achieve anti-shake. The gyro sensor 180B can also be used for navigation and somatosensory game scenes. In an embodiment of the present application, the relative position posture of the first electronic device and the second electronic device can be determined based on the motion posture of the electronic device of the gyro sensor 180B.
[0203] The acceleration sensor 180E can detect the magnitude of the acceleration of the electronic device 100 in all directions (generally three axes). When the electronic device 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of the electronic device, and is applied to applications such as horizontal and vertical screen switching and pedometers. In the embodiment of the present application, the direction of movement of the electronic device and the horizontal and vertical screen state of the electronic device can be determined based on the acceleration data detected by the acceleration sensor 180E. In addition, the relative position and posture of the first electronic device and the second electronic device can be determined.
[0204] The distance sensor 180F is used to measure the distance. The electronic device 100 can measure the distance by infrared or laser. In some embodiments, when shooting a scene, the electronic device 100 can use the distance sensor 180F to measure the distance to achieve fast focus. In an embodiment of the present application, the clustering between the first electronic device and the second electronic device can be measured according to the distance sensor 180F.
[0205] The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The electronic device 100 emits infrared light outward through the light emitting diode. The electronic device 100 uses a photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 100. When insufficient reflected light is detected, the electronic device 100 can determine that there is no object near the electronic device 100. The electronic device 100 can use the proximity light sensor 180G to discover other electronic devices. After the electronic device 100 determines that the second electronic device is close to the electronic device 100 through the proximity light sensor 180G, the electronic device 100 can combine the wireless positioning technology to determine the relative position posture between the electronic device 100 and the second electronic device.
[0206] The ambient light sensor 180L is used to sense the brightness of the ambient light. The electronic device 100 can adaptively adjust the brightness of the display screen 194 according to the perceived brightness of the ambient light. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the electronic device 100 is in a pocket to prevent accidental touch. In an embodiment of the present application, the ambient light sensor 180L can cooperate with the proximity light sensor 180G to detect whether there is an electronic device approaching the electronic device 100.
[0207] The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. In the embodiment of the present invention, the Android system of the layered architecture is taken as an example to exemplify the software structure of the electronic device 100. Figure 3B 1 is a software structure diagram of the electronic device 100 according to an embodiment of the present application. Figure 3B As shown, the layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom, namely, the application layer, the application framework layer, the Android runtime (Android runtime) and the system library, and the kernel layer.
[0208] The application layer can include a series of application packages.
[0209] like Figure 3B As shown, the application package may include camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message and other applications.
[0210] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.
[0211] like Figure 3B As shown, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, etc. In an embodiment of the present application, the application framework layer may include an interactive service module. The interactive service module may be used to determine the relative position and posture between other electronic devices; determine the interactive service corresponding to the relative position and posture; provide data to the application of the application layer based on the determined interactive service or provide the data of the application layer to other electronic devices; or control other electronic devices to implement certain functions, etc.
[0212] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.
[0213] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.
[0214] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.
[0215] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.
[0216] Android Runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system.
[0217] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.
[0218] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.
[0219] The kernel layer is a layer between hardware and software. The kernel layer includes at least a display driver, a camera driver, an audio driver, a sensor driver, a UWB driver, a Bluetooth driver, a GPS driver, a Wi-Fi driver, etc. In an embodiment of the present application, the positioning data of the electronic device can be detected by a UWB driver, a Bluetooth driver, a GPS driver, a Wi-Fi driver, etc. The acceleration data and the device proximity data of the electronic device can be detected by the sensor driver. The UWB driver, the Bluetooth driver, the GPS driver, the Wi-Fi driver and the sensor driver send the detected data to the interactive service module to determine the relative position and posture between the devices through the interactive service module. Then determine the interactive service according to the relative position and posture. The interactive service module performs cross-device interaction with the second electronic device based on the determined interactive service.
[0220] Based on the structure of the interactive system and electronic device provided above, the device interaction method of the embodiment of the present application will be described in detail below in combination with specific scenarios.
[0221] Scenario 1: The user interface of the gallery application is displayed on device 1. Optionally, at least one picture in the gallery is selected. Figure 4A As shown, pictures 31 and 32 in the gallery are in a selected state. Of course, in some embodiments, all pictures in the gallery may be in an unselected state. The chat interface of the instant messaging application is displayed in device 2. Specifically, the chat interface of user A is displayed in device 2. In this scenario, the interaction between device 1 and device 2 may be:
[0222] In some embodiments, Figure 4A As shown, when the long side of device 2 touches the long side of device 1, an interactive service is triggered between device 1 and device 2. Optionally, the interactive service may be that the display screens of device 1 and device 2 can be used as a combined screen. The data displayed in the combined screen can be moved from one area to another. When the data displayed in the combined screen expands the area of device 1 and moves to the area of device 2, the data expansion device 1 can be transmitted to device 2.
[0223] like Figure 4A As shown, device 1 and device 2 determine that the long side of device 2 collides with the long side of device 1 may be: device 1 discovers device 2; device 1 and device 2 detect each other's position and posture to determine that the long side of device 2 collides with the long side of device 1. Among them, device 1 discovers device 2 may be: device 1 discovers device 2 from a device connected to the same communication network, for example, device 1 discovers device 2 from a device connected to the same Wi-Fi network or logged into the same account. In some embodiments, device 1 discovers device 2 from a device with a specific short-range communication function enabled, such as device 1 discovers device 2 from a device that turns on a Bluetooth signal or supports Wi-Fi direct connection and other technologies; or, device 1 discovers device 2 from a device within a certain distance range through UWB or Bluetooth ranging. Here are just some of the possible ways for device 1 to discover device 2, and other possible ways will not be given examples one by one.
[0224] In some embodiments, the process in which device 1 and device 2 detect that the long side of device 2 collides with the long side of device 1 may be: device 1 detects the positioning data of device 1 or device 2 through a Bluetooth driver or a UWB driver. The positioning data may be absolute positioning data; or it may be relative positioning data. The Bluetooth driver or the UWB driver sends the positioning data to the interactive service module of device 1. The interactive service module determines the position of device 2, the distance between device 2 and device 1, etc. according to the positioning data. That is, through the positioning data, device 1 can determine from which direction device 2 is moving. Optionally, the interactive service module can also calculate the moving speed of device 2 based on the positioning data. Of course, in this process, device 2 can also detect device 1. Optionally, device 1 and device 2 can exchange the detected positioning data.
[0225] In some embodiments, in addition to the above-mentioned positioning data, device 1 and device 2 may also exchange: acceleration data, tilt angle, moving speed, or posture data such as horizontal or vertical screen state. Optionally, when device 1 and device 2 collide, device 1 and device 2 detect a collision signal. Then, device 1 and device 2 may further exchange information such as the collision position and the collision position. Device 1 may determine that the long side of device 2 has collided with the long side of device 1 based on one or more of the positioning data, acceleration data, tilt angle, moving speed, horizontal or vertical screen state of device 1, and one or more of the positioning data, acceleration data, tilt angle, moving speed, horizontal or vertical screen state of device 2.
[0226] In some embodiments, device 1 detects that its own acceleration is approximately equal to zero. That is, device 1 is in a stationary state. The device detects that its own acceleration is not zero. That is, device 2 is in motion. During the movement of device 2, device 1 continuously detects the relative distance between device 1 and device 2. When the relative distance between device 1 and device 2 is less than a first threshold, device 1 and device 2 detect a collision signal within a preset time. Device 1 sends a first confirmation message of detecting a collision signal to device 2. And device 2 sends a second confirmation message of detecting a collision signal to device 1. At this time, device 1 confirms that a collision has occurred between device 1 and device 2, and the moving direction of device 2 can be determined. Afterwards, device 1 detects that it is in a vertical screen state. Device 2 sends acceleration data of device 2 to device 1. Device 1 can calculate that device 2 is in a vertical screen state based on the acceleration data of device 2. Device 1 can determine that the long side of device 2 collides with the long side of device 1 based on its own vertical screen and stationary posture information, the moving direction of device 2 moving toward device 1, and the fact that device 2 is in a vertical screen state. Among them, device 1 and device 2 can detect pressure signals through pressure sensors arranged around the body. Determine whether the electronic device has applied a collision or been collided through the pressure signal; and the location where the collision was applied or collided. Optionally, device 1 and device 2 can also collect audio signals through microphones. Determine whether the electronic device has applied a collision or been collided through the audio signal; and the location where the collision was applied or collided. In some embodiments, device 1 can calculate whether a collision occurs between device 1 and device 2 based on the distance between device 1 and device 2, the moving speed, the approach direction, the size of the device, etc. And determine the collision location and other data based on the calculation results.
[0227] In other embodiments, device 2 may send its portrait state, moving direction, and collision position to device 1. Device 1 determines that the long side of device 2 has collided with the long side of device 1 based on information such as its own stationary state, portrait state, and collision position, and in combination with the information sent by device 2. Of course, there are many ways for device 1 to determine that the long side of device 2 has collided with the long side of device 1, which will not be described in detail here.
[0228] In some embodiments, when the long side of device 2 touches the long side of device 1, the device that is collided can be used as the main device for subsequent interaction, and the device that applies the collision can be used as an extension device. Among them, the main device can control the extension device to perform certain operations. Or, the main device can implement certain functions in the extension device. In other embodiments, the device that is collided can also be determined as an extension device, and the device that applies the collision can be determined as the main device. This is not limited in this embodiment. Of course, in some embodiments, the determination of the main device and the extension device between device 1 and device 2 may not be performed.
[0229] Optionally, after the long side of device 2 collides with the long side of device 1, the types of device 1 and device 2 can be further determined. If device 1 and device 2 are of the same device type, the device that is collided can be used as the main device for subsequent interactions, and the device that causes the collision can be used as the extended device. If device 1 and device 2 are not of the same device type, a certain type of device can be pre-set as the main device. Figure 4A As shown, in this embodiment, device 1 is determined as the main device for subsequent interaction, and device 2 is determined as the extended device.
[0230] In some embodiments, when device 1 is determined as the main device and device 2 is determined as the extended device, device 1 provides the first data to device 2 by default. Device 1 may display a prompt message to prompt the user to select the first data provided to device 2 in extended device 1. Or, Figure 4A As shown, when the pictures 31 and 32 are already selected in the device 1, the device 1 automatically sends the data of the pictures 31 and 32 to the device 2. Alternatively, the device 1 sends the data of the pictures 31 and 32 to the device 2 after detecting the user's predetermined gesture action.
[0231] In some embodiments, after device 1 is determined to be a main device and device 2 is determined to be an extended device, device 1 may continue to detect the interface content displayed on the display screen. If device 1 displays the first user interface, device 1 may be determined to provide the first data to device 2. If the first user interface is displayed in device 2, device 1 may control device 2 to provide the first data to device 1.
[0232] In some embodiments, after device 1 is determined to be the main device and device 2 is determined to be the extended device, device 1 may continue to detect whether there is data in a selected state in the display screen. If there is first data in a selected state in device 1, device 1 may determine to provide the first data to device 2. If there is first data in a selected state in device 2, device 1 may control device 2 to provide the first data to device 1.
[0233] In some embodiments, after device 1 determines that the long side of device 2 collides with the long side of device 1, the display screens of device 1 and device 2 become a combined screen. Device 1 may determine the size of the display screen of device 2 according to the type of device 2. Optionally, device 1 and device 2 exchange their respective display screen information. Device 1 determines the display screen extension area of device 1 according to the display screen information of device 2. Similarly, device 2 determines the display screen extension area of device 2 according to the display screen information of device 1.
[0234] like Figure 4BAs shown, device 1 detects a user's pressing action on picture 31. If device 1 detects that the duration of the pressing action is greater than a second preset value, device 1 determines that a long press action is detected. Optionally, device 1 can send information about the long press action to device 2. Device 2 determines that a data selection action has occurred in the display area of device 1 based on the information about the long press action. Figure 4C As shown, in response to the detected long press action, device 1 generates a thumbnail 301 of picture 31 according to the information of picture 31. Among them, device 1 can generate thumbnail 301 according to the name, size, type and other information of picture 31. Based on the long press action, device 1 detects a drag action. Device 1 displays thumbnail 301 according to the coordinates of the drag action. That is, thumbnail 301 can move with the movement of the coordinates of the drag action. Optionally, device 1 can send the information of the drag action and thumbnail 301 to device 2. Device 2 can determine the action position of the drag action in device 1 according to the coordinates of the drag action and the size of thumbnail 301, or can determine whether thumbnail 301 can be fully displayed in device 1. If thumbnail 301 cannot be fully displayed in device 1, device 2 can also determine the part of the thumbnail displayed by device 2 according to the coordinates of the drag action, the size of thumbnail 301, and the size of the display screen of device 2.
[0235] In some embodiments, when the user drags the thumbnail 301, the device 1 further detects whether the thumbnail 301 can be fully displayed in the device 1. If the thumbnail 301 cannot be fully displayed in the device 1, a part of the thumbnail 301 can be displayed in the device 1, and the remaining part is displayed in the device 2. Optionally, the device 1 can determine whether the thumbnail 301 can be fully displayed in the display screen of the device 1 according to the coordinates of the drag action, the size of the thumbnail 301, and the position of the drag action relative to the thumbnail 301. And when the thumbnail 301 cannot be fully displayed in the device 1, the device 1 determines the part displayed in the device 1 and the part displayed in the device 2 according to the coordinates of the drag action, the size of the thumbnail 301, and the position of the drag action relative to the thumbnail 301. Optionally, the device 1 can also only determine the part displayed in the device 1. The device 1 sends the drag action information to the device 2, so that the device 2 determines the thumbnail part displayed in the device 2.
[0236] like Figure 4DAs shown, when the user's dragging action acts on the edge of device 1 close to device 2, device 1 determines to display area 3011 of thumbnail 301 on device 1. Device 2 determines to display area 3012 of thumbnail 301 on device 2 according to the dragging action coordinates sent by device 1. In another possible manner, after device 1 determines to display area 3011 of thumbnail 301 on device 1, device 1 sends information of area 3012 and display position information of area 3012 on device 2 to device 2, so that device 2 displays area 3012. Similarly, the dragging action moves to device 2. Device 2 determines whether thumbnail 301 can be fully displayed in device 2 according to the coordinates of the dragging action, the size of thumbnail 301 and the position of the dragging action relative to thumbnail 301. And when device 2 cannot fully display thumbnail 301, device 2 determines the part of thumbnail 301 displayed in device 2 and the part of thumbnail 301 displayed in device 1.
[0237] like Figure 4E As shown, in device 2, thumbnail 301 moves along with the dragging action. Figure 4F As shown, when device 2 detects the release action of the drag action of thumbnail 301, device 2 inputs the data of picture 31 into input box 302 and displays it as a message in the message display area of user A's chat interface.
[0238] Among them, device 2 expands device 1 to obtain the data of picture 31. Specifically, device 1 can send the data of picture 31 to device 2 after detecting a long press action on picture 31. Alternatively, device 1 can send the data of picture 31 to device 2 after detecting a drag action on thumbnail 301. Alternatively, device 2 can also send a request to device 1 to obtain the data of picture 31 after detecting a release action. Device 1 sends the data of picture 31 to device 2 according to the request.
[0239] In another optional embodiment, device 1 sets a border area in a display area close to device 2. Figure 4G As shown, when device 1 detects that the coordinates of the dragging action are located in the boundary area, thumbnail 301 is displayed in device 2, and thumbnail 301 is no longer displayed in device 1. Figure 4E .like Figure 4E As shown, the thumbnail 301 in the device 2 can continue to move along with the movement of the dragging action.
[0240] In this embodiment, the process of transferring the picture 32 in the device 1 to the device 2 can refer to the process of processing the picture 31, and will not be repeated. Of course, the user can also drag the picture 31 and the picture 32 at the same time by dragging, so as to drag the picture 31 and the picture 32 to the instant messaging application of the device 2 at the same time. Among them, the instant messaging application can determine the order of displaying the picture 31 and the picture 32 in the chat interface according to the order of receiving the picture 31 and the picture 32 data.
[0241] In an embodiment of the present application, in addition to transmitting image data between device 1 and device 2, application documents, folders, partial text / pictures / charts in electronic documents, configuration data of electronic devices, etc. can also be transmitted. For the specific process, please refer to the present scenario flow.
[0242] Scenario 2: Figure 5A As shown, a chess game is running in device 1. The display screen of device 1 displays the game interface 41. The main interface of the desktop application is displayed in device 2. In this scenario, device 2 can be used as an extended display screen of device 1. That is, the display screens of device 1 and device 2 serve as a combined screen to jointly display the game interface 41. The processing process includes:
[0243] like Figure 5A As shown, the long side of device 1 collides with the long side of device 2. The way in which device 1 detects that the long side of device 1 collides with the long side of device 2 can be seen in scenario 1. For example, device 1 determines that device 2 is located on one side of the long side of device 1 based on UWB or Bluetooth positioning. Device 1 detects its own acceleration data, and determines its own moving direction and portrait state based on the acceleration data. Device 1 determines that device 1 moves close to device 2 and the long side of device 1 touches the long side of device 2 based on its own moving direction, the direction of device 2 and the information that device 2 is in portrait state, and the information that device 2 is in portrait state.
[0244] In some embodiments, after the long side of device 1 collides with the long side of device 2, as shown in FIG. Figure 5D As shown, device 1 displays a part of the current game interface on device 1, and another part on device 2. For example, device 1 displays the black chess interface of the current game interface on device 1, and the white chess interface on device 2.
[0245] In some embodiments, before the display screen of device 1 is expanded to display on the display screen of device 2, the process further includes: device 1 obtains auxiliary information. The auxiliary information includes information such as device type and device account. Among them, it can be determined whether device 1 and device 2 are devices of the same type based on the device type information. It can be determined whether device 1 and device 2 are devices of the same user based on the device account.
[0246] In the embodiment of the present application, different interactive service modes can be determined according to the types of device 1 and device 2. For example, when device 1 and device 2 are devices of the same type, a first type of interactive service is performed between device 1 and device 2. In the first type of interactive service, device 1 can be determined as a main device and device 2 can be determined as an extended device. When device 1 and device 2 are devices of different types, a second type of interactive service is performed between device 1 and device 2, in which a device of a predetermined type can be determined as a main device in the second type of interactive service. In this scenario, device 1 and device 2 are electronic devices of the same type.
[0247] like Figure 5B As shown, when the device accounts of device 1 and device 2 are different, a prompt message indicating whether to become an extended device of device 1 is displayed in device 2. After device 2 detects a click action on the confirmation control, device 2 determines to become an extended device of device 1. Device 2 sends confirmation information to device 1 to become an extended device of device 1.
[0248] In some embodiments, device 1 displays the game interface combination on the display screens of device 1 and device 2, and further detects a first gesture action. Figure 5C As shown, the first gesture action is a pinching action that acts on device 1 and device 2 respectively. In response to the detected first gesture action, device 1 generates a thumbnail 401 of interface 41. Device 1 displays thumbnail 401 on the display screens of the first device and the second device in combination according to the coordinates of the first gesture action. The manner in which device 1 and device 2 display thumbnail 401 in combination can be referred to in Figure 4D A method of displaying thumbnails 301 is combined.
[0249] Furthermore, device 1 also obtains display screen information of device 2. The display screen information of device 2 includes the size of the display screen. Optionally, the display screen information of device 2 is sent by device 2 to device 1. Alternatively, device 1 determines the display screen information of device 2 according to the device type of device 2.
[0250] After device 1 obtains the display screen information of device 2, device 1 determines the display screen of device 2 as the extended screen of device 1. Figure 5D As shown, device 1 determines to display the first part 42 of game interface 41 on the display screen of device 1 and the second part 43 of game interface 41 on device 2 according to the sizes of the display screens of device 1 and device 2. The first part 42 and the second part 43 are combined into a complete game interface.
[0251] When an operation acting on the second part 43 is detected in device 2, device 2 sends an instruction corresponding to the operation to device 1. Device 1 determines to generate a response message according to the instruction. Device 1 updates the game interface according to the response message. Device 1 sends the updated second part to device 2 according to the updated game interface.
[0252] Based on the method of the embodiment of the present application, in addition to combining and displaying game interfaces on multiple electronic devices, pictures, electronic documents, videos, etc. can also be combined and displayed.
[0253] like Figure 5E As shown, if device 1 and device 2 are of the same type, but the display screen size of device 1 is larger than the display screen size of device 2, device 1 can set the edge 44 and edge 45 of the display screen of device 1 as the black frame display area, and the other areas outside the black frame area and the display screen of device 2 are combined as the display area of the game interface.
[0254] Scenario 3: Fig. 6A As shown, the display screen of device 1 displays the A video playback interface of the video application. The display screen of device 2 displays the main interface of the desktop application. The A video playback interface includes an introduction control, a comment control, and a bullet screen control. When the user clicks the bullet screen control, the A video playback interface displays a keyboard control. Based on Fig. 6A In the scenario shown, device 1 can use device 2 as an input device, including:
[0255] like Fig. 6A As shown, device 2 moves close to device 1 and the long side of device 2 touches the long side of device 1. Device 1 is determined to be the main device, and device 2 is determined to be the extended device. In some embodiments, device 2 may display a prompt message indicating whether to become the extended device of device 1.
[0256] In some embodiments, after device 1 is determined to be the master device, device 1 detects that a predetermined application is displayed on the current display screen or a predetermined interface is displayed. At this time, device 1 determines to display a control associated with the predetermined application or the predetermined interface on device 2. Fig. 6A As shown, the A video playback interface of the video application is displayed in device 1. When the user triggers the bullet screen control, the keyboard control is displayed in device 1. That is, when device 1 determines that the keyboard control is displayed in the current application or the current interface, device 1 determines to extend the function of the keyboard control to device 2.
[0257] Specifically, device 1 sends instruction information for displaying keyboard controls to device 2. Device 2 displays keyboard controls according to the instruction information. Figure 6BAs shown, the keyboard control can be displayed in full screen in device 2. Optionally, the keyboard control is displayed in horizontal screen state in device 2. When the user inputs in the keyboard control of device 2. Device 2 sends the information input through the keyboard control to device 1. Device 1 displays the input information as bullet screen in the A video playback interface. Of course, when the introduction control and comment control in the interface of device 1 are triggered, device 1 can also expand the corresponding sub-interface or control to device 2.
[0258] In another possible design, Figure 6C and Fig.6D As shown, when the long side of device 2 touches the short side of device 1 and a predetermined application or a predetermined interface is displayed in device 1, device 1 sends an instruction message for displaying a keyboard control to device 2. The keyboard control is displayed in a horizontal screen state in device 2. Alternatively, a predetermined application or a predetermined interface is displayed in device 1, and when the user clicks on the bullet screen control, device 1 sends an instruction message for displaying a keyboard control to device 2. The keyboard control is displayed in a horizontal screen state in device 2.
[0259] Scenario 4: Fig. 6E As shown, the user interface of the gallery application is displayed in device 1. The main interface of the desktop application is displayed in device 2. When device 1 and device 2 move toward each other in an inclined manner and parts of the surfaces of device 1 and device 2 overlap, device 1 determines that device 1 is the main device and device 2 is the extended device based on the contents in the display screens of device 1 and device 2.
[0260] like Fig. 6E As shown, when two of the pictures in device 1 are selected, device 1 sends the data of the two pictures to device 2. Device 2 displays a prompt message asking whether to receive the two pictures from device 1. When device 2 receives a click operation from the user on the confirmation control, device 2 receives the data of the two pictures sent by device 1.
[0261] like Fig. 6F As shown, the user interface of the gallery application is displayed in device 1. The user interface of the gallery application is displayed in device 2. Two pictures in device 1 are selected. One picture in device 2 is selected. According to the relative position and posture between device 1 and device 2 and the content displayed in the interface, data is exchanged between device 1 and device 2. Optionally, device 1 can send the two selected pictures to device 2; device 2 can also send the one selected picture to device 1. In some embodiments, Fig. 6F As shown, device 1 displays a prompt message of whether to receive 1 picture from device 2. Device 2 displays a prompt message of whether to receive 2 pictures from device 1. After the user triggers the confirmation control in device 1 and / or device 2, device 1 and / or device 2 receives the picture data from the other party.
[0262] Scene 5: Fig. 7A As shown, device 1 and device 2 are different types of electronic devices. For example, device 1 is a mobile phone and device 2 is a computer. Device 1 displays the first main interface of the mobile phone desktop application. Device 2 displays the second main interface of the computer desktop application. When the long side of device 1 collides with the short side of device 2, a prompt message may pop up on device 1 and device 2 asking whether to establish an association relationship. Fig. 7A As shown, a first pop-up box "Connecting to device 2 allows data to be shared between devices" may be displayed in device 1. Device 1 detects a click operation on the confirmation control in the first pop-up box. Figure 7B As shown, a second pop-up box "Please confirm the connection on device 2" may be displayed in device 1. At the same time, a third pop-up box "Click OK to authorize device 1 to connect to this computer and establish a trust relationship" is displayed in device 2. The third pop-up box may also include a prompt message "You can automatically connect and use related functions next time." After device 2 detects a click operation on the confirmation control within a preset time (such as 15s), device 2 sends the confirmation information to device 1. After a trust relationship is established between device 1 and device 2, interactive services such as data transmission, screen sharing, and function expansion can be performed between device 1 and the device.
[0263] In a specific example, if Figure 7C As shown, according to the device type, device 1 and device 2 determine device 2 as the main device and device 1 as the extended device. Device 2 obtains the spatial position information occupied by the display screen of device 1. Device 2 determines the spatial position occupied by the display screen of device 1 as the virtual operation space. Device 2 extends the operation space of its own input device to the virtual operation space. That is, device 2 can operate and control device 1 through the input device.
[0264] like Figure 7C As shown, device 2 detects the coordinates and instructions of the mouse pointer. When device 2 detects that the coordinates of the mouse pointer exceed the display screen area of device 2, device 2 determines whether the coordinates of the mouse pointer are located in the virtual operation space area. If the coordinates of the mouse pointer of device 2 are located in the virtual operation space area, device 2 sends the coordinates of the mouse pointer and the mouse pointer icon to device 1. The mouse pointer icon is displayed in device 1. Optionally, device 2 can also send indication information containing the coordinates of the mouse pointer to device 1, so that device 1 generates and displays the mouse pointer icon according to the indication information. When device 2 detects a first instruction to the mouse, such as moving the mouse or clicking the mouse, device 2 converts the first instruction into a second instruction in the virtual space area. Device 2 sends the second instruction to device 1. Device 1 moves the position of the mouse pointer and / or operates the data in device 1 according to the second instruction. For example, the operation of selecting picture 31.
[0265] The user can also drag the image 31 by mouse. Fig.7D As shown, the device 1 displays the thumbnail 301 according to the dragging action. Fig. 7E As shown, when the mouse pointer drags the thumbnail 301 to the device 2 and is released, the device 2 displays the icon of the picture 31 or displays the picture 31 on the desktop according to the data of the picture 31. Among them, the device 1 sends the data of the picture 31 to the device 2. For example, when the device 1 detects that the picture 31 is in the selected state, the picture 31 is sent to the device 2.
[0266] In some embodiments, Figure 7F As shown, when device 1 is superimposed on the display screen area of the device at an inclined angle, device 1 sends the selected picture 31 and picture 32 data to device 2. Figure 7G As shown, device 2 receives the data of picture 31 and picture 32 sent by device 1, and displays the icons of picture 31 and picture 32 on the desktop.
[0267] Scene 6: Fig. 8A As shown, the tablet computer 51 has an accessory 52. The accessory 52 may be a leather case with a keyboard, a support frame, etc. In addition to colliding with the edge or surface of the tablet computer 51 to trigger the interactive service between devices, the mobile phone 53 may also be in a certain relative position and posture with the accessory 52 and trigger the interactive service based on the relative position and posture with the accessory 52. Fig. 8A As shown, when the long side of the mobile phone 53 touches the long side of the accessory 52, the first interactive service is triggered between the mobile phone 53 and the tablet computer 51. Figure 8B As shown, when the long side of the mobile phone 53 touches the short side of the accessory 52 , the second interactive service is triggered between the mobile phone 53 and the tablet computer 51 .
[0268] Based on the above described embodiments, the method steps for interaction between devices in the embodiments of the present application are as follows: Fig. 9 As shown, it may include:
[0269] 101. A first electronic device obtains a relative position of a second electronic device.
[0270] In some embodiments, the first electronic device can detect the location data of the first electronic device or the second electronic device through a UWB positioning chip or a Bluetooth positioning chip; determine the relative position of the second electronic device based on the location data. Optionally, the detected location data contains the relative position of the first electronic device and the second electronic device. The first electronic device can directly determine the relative position based on the data. Optionally, the first electronic device can calculate the relative position based on intermediate data such as the distance between the first electronic device and the second electronic device, the moving speed of the first electronic device or the second electronic device, and the tilt angle contained in the location data.
[0271] 102. The first electronic device determines an approach direction of the second electronic device according to the relative position.
[0272] In some embodiments, the first electronic device may determine the approach direction of the second electronic device according to the relative position determined in 101. Based on the approach direction, it may be determined from which direction the second electronic device is approaching the first electronic device. In some embodiments, the approach direction of the second electronic device may include the following situations: the first electronic device remains stationary and the second electronic device moves toward the first electronic device; the second electronic device remains stationary and the first electronic device moves toward the second electronic device; the first electronic device and the second electronic device move toward each other.
[0273] In some embodiments, the approach direction may include: the second electronic device moves from the front of the screen of the first electronic device to the first electronic device; the second electronic device moves from one side of the first electronic device to the first electronic device. Other possible approach directions are not listed here one by one.
[0274] 103. The first electronic device obtains first posture data of the first electronic device and second posture data of the second electronic device.
[0275] In some embodiments, the first posture data includes any one of the following methods or a combination thereof: detecting motion posture data of the first electronic device through a gyroscope sensor; detecting acceleration data of the first electronic device through an acceleration sensor. According to the acceleration data, the horizontal or vertical screen state of the first electronic device or the speed at which the first electronic device moves toward the second electronic device or the tilt angle of the first electronic device can be determined.
[0276] In some embodiments, the first electronic device receives the second posture data sent by the second electronic device; the second posture data includes: motion posture data, acceleration data or horizontal and vertical screen state data of the second electronic device.
[0277] Correspondingly, based on the acceleration data in the second posture data or the horizontal or vertical screen state data, the horizontal or vertical screen state of the second electronic device can be determined; or, based on the acceleration data in the second posture data, the speed at which the second electronic device moves toward the first electronic device or the tilt angle of the second electronic device can be determined.
[0278] In some embodiments, the first gesture data includes a first touch signal, and the second gesture data includes a second touch signal. Accordingly, the edge or surface touched by the first electronic device and the second electronic device can be determined according to the first touch signal and the second touch signal.
[0279] In some embodiments, the first electronic device may acquire the first touch signal by collecting a pressure signal through a pressure sensor disposed on the periphery of the first electronic device, and determining a collision position of the first electronic device according to the pressure signal, such as a collision on a side or surface.
[0280] In some embodiments, the first electronic device can calculate whether a collision occurs between the first electronic device and the second electronic device based on the distance, moving speed, approach direction, device size, etc. between the first electronic device and the second electronic device, and determine data such as the collision location based on the calculation result.
[0281] In some embodiments, the first electronic device can collect sound through a microphone, and determine whether a collision occurs between the first electronic device and the second electronic device based on the sound signal, and can determine the collision location based on the size of the sound signals collected by multiple microphones.
[0282] 104. The first electronic device determines a touch posture of the first electronic device and the second electronic device according to the approach direction, the first posture data, and the second posture data.
[0283] In some embodiments, the collision gesture may refer to a gesture when the distance between the first electronic device and the second electronic device is less than a first threshold, or may refer to a gesture when the first electronic device and the second electronic device are in contact.
[0284] In some embodiments, the gesture between the first electronic device and the second electronic device may include the movement direction of the first electronic device and the second electronic device, such as the first electronic device moving toward the second electronic device and then touching; the second electronic device moving toward the first electronic device and then touching; or the first electronic device and the second electronic device moving toward each other and then touching.
[0285] In some embodiments, the touch gestures of the first electronic device and the second electronic device may be determined based on the approach direction determined by the various gesture data acquired in 103 , the first gesture data, and the second gesture data.
[0286] The touching gestures of the first electronic device and the second electronic device include any one of the following:
[0287] The first long side of the first electronic device touches the second long side of the second electronic device;
[0288] The first long side of the first electronic device touches the first short side of the second electronic device;
[0289] The second short side of the first electronic device touches the third long side of the second electronic device;
[0290] The second short side of the first electronic device touches the third short side of the second electronic device;
[0291] The first surface of the first electronic device touches the second surface of the second electronic device;
[0292] A preset edge or surface of the second electronic device touches the accessory of the first electronic device.
[0293] Specifically, the touch gestures of the first electronic device and the second electronic device can be referred to in Figure 2A-2D , Figure 4A , Figure 5A , 6A , Fig. 7A , 8A and 8B.
[0294] 105. The first electronic device determines the second electronic device as an extension device of the first electronic device according to the touch gesture.
[0295] In some embodiments, Figure 5A-5E As shown, if the first electronic device and the second electronic device are devices of the same type; the device that actively applies touch in the first electronic device and the second electronic device can be determined as the main device, and the device that passively receives touch can be determined as the extended device. That is, when the touch gesture includes the first electronic device actively touching the second electronic device, the first electronic device is the main device and the second electronic device is the extended device. In the embodiment of the present application, whether the two electronic devices are of the same type can be set according to actual needs. For example, the set types include mobile phone type, computer type, and home appliance type. When the first electronic device is a mobile phone type and the second electronic device is a mobile phone type, the types are the same. When the first electronic device is a mobile phone type and the second electronic device is a computer type, the types are different. In other embodiments, it can also be divided into mobile phone 1 type, mobile phone 2 type, computer 1 type, and computer 2 type. If the first electronic device is a mobile phone 1 type and the second electronic device is a mobile phone 2 type. It can be considered that the first electronic device and the second electronic device are of different types. The above is only an example, and the specific division method can be set according to actual needs.
[0296] In some embodiments, Figure 4A-4G as well as Figure 6A-6DAs shown, the device that actively applies touch in the first electronic device and the second electronic device can be determined as an extended device, and the device that passively receives touch can be determined as a main device. That is, when the touch gesture includes the first electronic device passively receiving the touch of the second electronic device, the first electronic device is the main device and the second electronic device is the extended device.
[0297] In some embodiments, if the first electronic device and the second electronic device move toward each other and touch each other, Figure 6E-6F As shown, the first prompt information can be displayed in the first electronic device and / or the second electronic device; the first prompt information is used to prompt the user to determine the main device from the first device or the second device.
[0298] In some embodiments, if the first electronic device and the second electronic device are different types of devices; Figures 7A-7G As shown, the device that meets the first device type in the first electronic device and the second electronic device is determined as the main device, and the other device is determined as the extended device. For example, devices of device types such as tablet computers, computers, laptops, and televisions are defaulted to the main device. Mobile phones or wearable devices are defaulted to extended devices. The specific setting method can be set according to actual needs.
[0299] In some embodiments, after determining the second electronic device as an extension device of the first electronic device, the display screens of the first electronic device and the second electronic device may be determined as a combined screen according to the display screen information of the first electronic device and the second electronic device; alternatively, the main device may determine the display screens of the first electronic device and the second electronic device as a combined screen according to the display screen information of the first electronic device and the second electronic device. In one example, the first electronic device is the main device.
[0300] In some embodiments, after the first electronic device determines the display screens of the first electronic device and the second electronic device as a combined screen based on the display screen information of the first electronic device and the second electronic device; the first electronic device can send the first data in the first electronic device to the second electronic device based on the combined screen; or, display the first data on the combined screen; or, determine the combined screen area as an operation area.
[0301] In some embodiments, data can be transmitted between the first electronic device and the second electronic device. Figure 4A-4G As shown, after the first electronic device detects the first gesture action acting on the first data, it can display the movement track of the first data in the combined screen according to the first gesture action; and can transmit the first data from the first electronic device to the second electronic device according to the first gesture action. Figure 6E-6F Shown and Figure 7F-7GAs shown, according to the prompt information, the main device can transmit data to the expansion device. Optionally, the main device can also receive data sent by the expansion device.
[0302] In some embodiments, the first electronic device may display the first data in the combined screen. Optionally, the first data may be a picture, an electronic document, a user interface of an application, etc. Figure 5A-5E As shown, the first electronic device detects a second gesture action; or, detects a third gesture action; receives fourth gesture action data sent by the second electronic device; determines a fourth gesture action based on the fourth gesture action data; if the combined action of the third gesture action and the fourth gesture action is a set action; then, the first electronic device determines to display the first data on the combined screen.
[0303] In some embodiments, Figure 5C As shown, according to the second gesture action or the combined action, a thumbnail of the first data is displayed on the display screen of the first electronic device or the combined screen.
[0304] In some embodiments, Figure 5A-5E As shown, when the first data is a user interface displayed in a first electronic device, displaying the first data on the combined screen may include: dividing the user interface into a first sub-interface and a second sub-interface; displaying the first sub-interface on a display screen of the first electronic device; and sending data of the second sub-interface to the second electronic device to display the second sub-interface in the second electronic device.
[0305] In some embodiments, a first user interface is displayed in a first electronic device; and the first data is displayed on the combined screen, including: detecting a first operation on a first control in the first user interface; and in response to the first operation, sending an instruction message to the second electronic device to display a second control or to display the first sub-interface. The second control is a control associated with the first control. For example, a keyboard control will pop up after clicking the bullet screen button. Figures 6A-6D As shown, the comment sub-interface, keyboard controls, etc. in the video playback interface can be displayed in the second electronic device.
[0306] In some embodiments, Figure 7A-7E The first electronic device determines the combined screen area as the operation area, including: detecting a first input instruction, and determining that the first input instruction acts on the area where the display screen of the second electronic device is located; sending a second input instruction to the second electronic device according to the first input instruction; so that the second electronic device responds according to the second input instruction; receiving the response result generated by the second electronic device according to the second input instruction. Figure 7A-7GAs shown, the expansion device can be used as an operating area expansion device of the main device. The expansion device can be operated by the main device.
[0307] like Figures 8A-8B As shown, the second electronic device can also touch the accessory of the first electronic device. When the touching gesture meets certain conditions, the first electronic device and the second electronic device can perform the following operations. Figures 4A-7G The interaction between.
[0308] The above is only a specific implementation of the present application. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. The protection scope of the present application should be based on the protection scope of the claims.
Claims
1. A device interaction method, It is characterized in that The method is applied to a first electronic device, comprising: Acquiring a relative position of a second electronic device; determining the approach direction of the second electronic device according to the relative position; Acquire first posture data of the first electronic device and second posture data of the second electronic device; The second electronic device is determined as an extension device of the first electronic device according to the approach direction, the first posture data and the second posture data.
2. The method according to claim 1, It is characterized in that Acquiring the relative position of the second electronic device includes: Detecting the location data of the first electronic device or the second electronic device by using an ultra-wideband UWB positioning chip or a Bluetooth positioning chip; The relative position of the second electronic device is determined according to the position data.
3. The method according to claim 1 or 2, It is characterized in that Acquiring the first posture data of the first electronic device includes any one of the following methods or a combination thereof: Detecting motion posture data of the first electronic device through a gyroscope sensor; The acceleration data of the first electronic device is detected by the acceleration sensor.
4. The method according to claim 3, It is characterized in that Acquiring second posture data of the second electronic device includes: receiving the second posture data sent by the second electronic device; The second posture data includes: motion posture data, acceleration data or horizontal and vertical screen state data of the second electronic device.
5. The method according to any one of claims 1 to 4, It is characterized in that Determining the second electronic device as an extension device of the first electronic device according to the approach direction, the first posture data, and the second posture data includes: determining a touch posture of the first electronic device and the second electronic device according to the approach direction, the first posture data, and the second posture data; According to the touch gesture, the second electronic device is determined as an extension device of the first electronic device.
6. The method according to claim 5, It is characterized in that The first gesture data includes a first touch signal, and the second gesture data includes a second touch signal; Determining the touch posture of the first electronic device and the second electronic device according to the approach direction, the first posture data, and the second posture data includes: The edge or surface touched by the first electronic device and the second electronic device is determined according to the first touch signal and the second touch signal.
7. The method according to any one of claims 1 to 6, It is characterized in that The first posture data includes first acceleration data; Correspondingly, the horizontal or vertical screen state of the first electronic device is determined according to the first acceleration data, or the speed at which the first electronic device moves toward the second electronic device is determined, or the tilt angle of the first electronic device is determined.
8. The method according to any one of claims 1 to 7, It is characterized in that The second posture data includes second acceleration data or horizontal and vertical screen state data; Correspondingly, determining the horizontal or vertical screen state of the second electronic device according to the second acceleration data or the horizontal or vertical screen state data; or, The speed at which the second electronic device moves toward the first electronic device or the tilt angle of the second electronic device is determined according to the second acceleration data.
9. The method according to any one of claims 1 to 8, It is characterized in that The touching gestures of the first electronic device and the second electronic device include any one of the following: The first long side of the first electronic device touches the second long side of the second electronic device; The first long side of the first electronic device touches the first short side of the second electronic device; The second short side of the first electronic device touches the third long side of the second electronic device; The second short side of the first electronic device touches the third short side of the second electronic device; The first surface of the first electronic device touches the second surface of the second electronic device; A preset edge or surface of the second electronic device touches the accessory of the first electronic device.
10. The method according to any one of claims 1 to 9, It is characterized in that According to the touch gesture, determining the second electronic device as an extension device of the first electronic device includes: If the first electronic device and the second electronic device are devices of the same type; then When the first electronic device actively applies a touch to the second electronic device, the first electronic device is determined as a main device, and the second electronic device that passively receives the touch is determined as an extended device; or, When the second electronic device actively applies a touch to the first electronic device, the first electronic device is determined as a main device; and the second electronic device that actively applies a touch is determined as an extended device.
11. The method according to claim 10, It is characterized in that The method further comprises: If the first electronic device and the second electronic device move towards each other and touch each other; A first prompt message is displayed in the first electronic device; the first prompt message is used to prompt the user to determine the main device from the first device or the second device.
12. The method according to claim 10, It is characterized in that The method further comprises: If the first electronic device and the second electronic device are different types of devices; then When the first electronic device is a preset type device, the first electronic device is determined as a main device, and the second electronic device is determined as an extended device.
13. The method according to any one of claims 1 to 12, It is characterized in that After determining the second electronic device as an extension device of the first electronic device, the method further includes: According to the display screen information of the first electronic device and the second electronic device, determining the display screens of the first electronic device and the second electronic device as a combined screen; sending the first data in the first electronic device to the second electronic device based on the combined screen; or, Display the first data on the combined screen; or, The combined screen area is determined as the operation area.
14. The method according to claim 13, It is characterized in that Sending the first data in the first electronic device to the second electronic device based on the combined screen includes: Detecting a first gesture action acting on first data; Displaying a moving track of the first data on the combined screen according to the first gesture action; The first data is transmitted from the first electronic device to the second electronic device according to the first gesture action.
15. The method according to claim 13, It is characterized in that Displaying the first data on the combined screen includes: A second gesture action is detected; or a third gesture action is detected; fourth gesture action data sent by the second electronic device is received; a fourth gesture action is determined according to the fourth gesture action data; if the combination of the third gesture action and the fourth gesture action is a set action; then, Determine to display the first data on the combined screen.
16. The method according to claim 15, It is characterized in that The method further comprises: According to the second gesture action or the combined action, a thumbnail of the first data is displayed on a display screen of the first electronic device or the combined screen.
17. The method according to claim 13, 15 or 16, It is characterized in that The first data is a user interface displayed in the first electronic device; Displaying the first data on the combined screen includes: dividing the user interface into a first sub-interface and a second sub-interface; Displaying the first sub-interface on a display screen of the first electronic device; The data of the second sub-interface is sent to the second electronic device, so that the second sub-interface is displayed in the second electronic device.
18. The method according to claim 13, It is characterized in that Displaying a first user interface in the first electronic device; Displaying the first data on the combined screen includes: detecting a first operation on a first control in the first user interface; In response to the first operation, instruction information for displaying a second control or displaying a first sub-interface is sent to the second electronic device.
19. The method according to claim 13, It is characterized in that Determining the combined screen area as the operation area includes: Detecting a first input instruction, and determining that the first input instruction acts on an area where the display screen of the second electronic device is located; According to the first input instruction, sending a second input instruction to the second electronic device, so that the second electronic device responds according to the second input instruction; Receive a response result generated by the second electronic device according to the second input instruction.
20. An electronic device, It is characterized in that The device is a first electronic device, comprising: A microphone; one or more processors; a memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and the one or more computer programs include instructions, which, when executed by the electronic device, cause the electronic device to perform the following steps: Acquiring a relative position of a second electronic device; determining the approach direction of the second electronic device according to the relative position; Acquire first posture data of the first electronic device and second posture data of the second electronic device; The second electronic device is determined as an extension device of the first electronic device according to the approach direction, the first posture data and the second posture data.
21. The device according to claim 1, It is characterized in that When the instruction is executed by the device, the device specifically performs the following steps: Detecting the location data of the first electronic device or the second electronic device by using a UWB positioning chip or a Bluetooth positioning chip; The relative position of the second electronic device is determined according to the position data.
22. The device according to claim 20 or 21, It is characterized in that When the instruction is executed by the device, the device specifically performs the following steps: Detecting motion posture data of the first electronic device through a gyroscope sensor; The acceleration data of the first electronic device is detected by the acceleration sensor.
23. The device according to claim 22, It is characterized in that When the instruction is executed by the device, the device specifically performs the following steps: receiving the second posture data sent by the second electronic device; The second posture data includes: motion posture data, acceleration data or horizontal and vertical screen state data of the second electronic device.
24. The device according to any one of claims 20 to 23, It is characterized in that Determining the second electronic device as an extension device of the first electronic device according to the approach direction, the first posture data, and the second posture data includes: determining a touch posture of the first electronic device and the second electronic device according to the approach direction, the first posture data, and the second posture data; According to the touch gesture, the second electronic device is determined as an extension device of the first electronic device.
25. The device according to claim 24, It is characterized in that The first gesture data includes a first touch signal, and the second gesture data includes a second touch signal; When the instruction is executed by the device, the device specifically performs the following steps: The edge or surface touched by the first electronic device and the second electronic device is determined according to the first touch signal and the second touch signal.
26. The apparatus according to any one of claims 20 to 25, It is characterized in that The first posture data includes first acceleration data; Accordingly, when the instruction is executed by the device, the device specifically performs the following steps: The horizontal or vertical screen state of the first electronic device is determined according to the first acceleration data, or the speed at which the first electronic device moves toward the second electronic device is determined, or the tilt angle of the first electronic device is determined.
27. Apparatus according to any one of claims 20 to 26, It is characterized in that The second posture data includes second acceleration data or horizontal and vertical screen state data; Accordingly, when the instruction is executed by the device, the device specifically performs the following steps: determining the horizontal or vertical screen state of the second electronic device according to the second acceleration data or the horizontal or vertical screen state data; or, The speed at which the second electronic device moves toward the first electronic device or the tilt angle of the second electronic device is determined according to the second acceleration data.
28. Apparatus according to any one of claims 20 to 27, It is characterized in that The touching gestures of the first electronic device and the second electronic device include any one of the following: The first long side of the first electronic device touches the second long side of the second electronic device; The first long side of the first electronic device touches the first short side of the second electronic device; The second short side of the first electronic device touches the third long side of the second electronic device; The second short side of the first electronic device touches the third short side of the second electronic device; The first surface of the first electronic device touches the second surface of the second electronic device; A preset edge or surface of the second electronic device touches the accessory of the first electronic device.
29. The apparatus according to any one of claims 20 to 28, It is characterized in that When the instruction is executed by the device, the device specifically performs the following steps: If the first electronic device and the second electronic device are devices of the same type; then When the first electronic device actively applies a touch to the second electronic device, the first electronic device is determined as a main device, and the second electronic device that passively receives the touch is determined as an extended device; or, When the second electronic device actively applies a touch to the first electronic device, the first electronic device is determined as a main device; and the second electronic device that actively applies a touch is determined as an extended device.
30. The apparatus according to claim 29, It is characterized in that When the instruction is executed by the device, the device specifically performs the following steps: If the first electronic device and the second electronic device move towards each other and touch each other; When the first electronic device is a preset type device, the first electronic device is determined as a main device, and the second electronic device is determined as an extended device.
31. The apparatus according to claim 29, It is characterized in that When the instruction is executed by the device, the device specifically performs the following steps: If the first electronic device and the second electronic device are different types of devices; then A device that complies with the first device type among the first electronic device and the second electronic device is determined as a main device, and the other device is determined as an extended device.
32. Apparatus according to any one of claims 20 to 31, It is characterized in that When the instruction is executed by the device, the device specifically performs the following steps: According to the display screen information of the first electronic device and the second electronic device, determining the display screens of the first electronic device and the second electronic device as a combined screen; sending the first data in the first electronic device to the second electronic device based on the combined screen; or, Displaying the first data on the combined screen; or, The combined screen area is determined as the operation area.
33. The apparatus according to claim 32, It is characterized in that When the instruction is executed by the device, the device specifically performs the following steps: Detecting a first gesture action acting on first data; Displaying a moving track of the first data on the combined screen according to the first gesture action; The first data is transmitted from the first electronic device to the second electronic device according to the first gesture action.
34. The apparatus according to claim 32, It is characterized in that When the instruction is executed by the device, the device specifically performs the following steps: A second gesture action is detected; or a third gesture action is detected; fourth gesture action data sent by the second electronic device is received; a fourth gesture action is determined according to the fourth gesture action data; if the combination of the third gesture action and the fourth gesture action is a set action; then, Determine to display the first data on the combined screen.
35. The apparatus according to claim 34, It is characterized in that When the instruction is executed by the device, the device specifically performs the following steps: According to the second gesture action or the combined action, a thumbnail of the first data is displayed on a display screen of the first electronic device or the combined screen.
36. Apparatus according to claim 32, 34 or 35, It is characterized in that The first data is a user interface displayed in the first electronic device; When the instruction is executed by the device, the device specifically performs the following steps: dividing the user interface into a first sub-interface and a second sub-interface; Displaying the first sub-interface on a display screen of the first electronic device; The data of the second sub-interface is sent to the second electronic device, so that the second sub-interface is displayed in the second electronic device.
37. The apparatus according to claim 32, It is characterized in that Displaying a first user interface in the first electronic device; When the instruction is executed by the device, the device specifically performs the following steps: detecting a first operation on a first control in the first user interface; In response to the first operation, instruction information for displaying a second control or displaying a first sub-interface is sent to the second electronic device.
38. The apparatus according to claim 32, It is characterized in that When the instruction is executed by the device, the device specifically performs the following steps: Detecting a first input instruction, and determining that the first input instruction acts on an area where the display screen of the second electronic device is located; sending a second input instruction to the second electronic device according to the first input instruction; so that the second electronic device responds according to the second input instruction; Receive a response result generated by the second electronic device according to the second input instruction.
39. A computer storage medium, It is characterized in that It comprises computer instructions, and when the computer instructions are executed on an electronic device, the electronic device executes the device interaction method as claimed in any one of claims 1 to 19.
