Interaction method and system, and related apparatus

Abstract

Embodiments of the present application provide an interaction method and system, and a related apparatus. In the interaction method provided in the present application, when a first device approaches a second device, target content to be transmitted can be determined between the first device and the second device. The first device may transmit the target content to the second device, or the second device may transmit the target content to the first device. In this way, two electronic devices can implement proximity-triggered data transmission without requiring an NFC chip.

Description

An interaction method, system and related device

[0001] This application claims priority to Chinese Patent Application No. 202510241780.7, filed on February 28, 2025, entitled "An Interactive Method, System and Related Device", filed on March 18, 2025, entitled "An Interactive Method, System and Related Device", filed on June 13, 2025, entitled "An Interactive Method, System and Related Device", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of terminal and communication technology, and in particular to an interaction method, system and related device. Background Technology

[0003] With the continuous development of internet technology, more and more electronic devices can interact with each other (e.g., data transfer, screen mirroring, etc.). For example, files on a mobile phone can be transferred to another mobile phone, tablet, computer, or other electronic device. Generally, when electronic device A interacts with electronic device B, the user needs to operate on electronic device A to establish a connection with electronic device B. Then, the user finds the data to be sent on electronic device A and sends it to electronic device B. This process is relatively cumbersome for the user.

[0004] Therefore, how to make electronic devices interact more conveniently is an urgent problem to be solved. Summary of the Invention

[0005] This application provides an interaction method, system, and related apparatus, through which electronic devices can interact more conveniently.

[0006] Firstly, this application provides an interaction method applicable to an interactive system. The interactive system may include a first device and a second device, the first device and the second device establishing a first communication connection. The first device includes a screen, and the screen includes a touch sensor. The method may include: the first device acquiring a first signal through the touch sensor, and determining a target area on the screen corresponding to the second device based on the first signal; the target area being an area on the screen where a capacitance change occurs when the second device approaches the first device; the first device receiving first indication information sent by the second device, the first indication information indicating a change in the motion state of the second device; and the first device sending first content to the second device based on the target area.

[0007] The target area includes the area on the screen of the first device that the second device maps onto when the second device is hovered, or the contact area between the screens of the second device and the first device.

[0008] The change in motion state includes: the motion state changing to a stationary state, and the duration of the second device being stationary is greater than the first preset duration.

[0009] Using the interaction method provided in the first aspect, when the first device detects the target area of ​​the second device on the screen, and when the second device detects a change in the movement state of the second device, the second device can obtain the first content present in the target area of ​​the second device on the screen of the first device. In this way, interaction between the first and second devices can be achieved without much user intervention, and the second device can easily obtain the first content from the first device. This first aspect can achieve the effect of "absorbing" the first content from the first device when the second device approaches it, thus improving the user experience.

[0010] In one possible implementation, after the first device receives the first instruction information sent by the second device, the method may further include: the first device receiving the second content sent by the second device.

[0011] In this way, by bringing the second device close to the first device, the first device can acquire the second content from the second device without much user intervention. In other words, the user only needs to bring the second device close to the first device to achieve the effect of the first device "sucking" the second content from the second device. This simplifies user operations and improves the user experience.

[0012] In one possible implementation, the first device sends first content to the second device based on the target area, which may include: the first device determining that there is third content at the center point or centroid of the corresponding target area on the screen, and sending the first content to the second device.

[0013] The existence of a third content at the center point or centroid of the target area can include: the third content is located at the center point or centroid of the target area, or the distance between the center point or centroid and the third content is less than a distance threshold of 2.

[0014] In this way, the first device can determine whether to send the first content to the second device by whether the third content exists in the target area.

[0015] Alternatively, in another possible implementation, the first device sends the first content to the second device based on the target area, which may include: the first device determining that there is no third content in the target area of ​​the screen, and then sending the first content to the second device.

[0016] In this way, the first device does not need to calculate the center point or centroid of the target area. It can directly determine whether there is third content in the target area. If there is third content, the first device sends the first content to the second device.

[0017] In one possible implementation, the third content includes an application icon, a video file icon, an image file icon, a document file icon, a spreadsheet file icon, or any one of a video, image, document, or spreadsheet; the first content includes an application identifier, an application installation package, or any one of a video, image, document, or spreadsheet.

[0018] In one possible implementation, if the third content is an application icon, the first content is the application identifier or the application's installation package; or, if the third content is a video file icon or a video, the first content is a video; or, if the third content is an image file icon or an image, the first content is an image; or, if the third content is a document file icon or a document, the first content is a document; or, if the third content is a table file icon or a table, the first content is a table.

[0019] In this way, the first device can determine to send the first content based on the third content.

[0020] In one possible implementation, determining the target area of ​​the second device on the screen based on the first signal may specifically include: the first device generating a grayscale image from the first signal and determining the target feature area in the grayscale image, wherein the target feature area is the area in the grayscale image where the grayscale value of a pixel is higher than a grayscale threshold; and the first device determining the target feature area as the target area of ​​the second device on the screen.

[0021] In this way, the first device can accurately determine the target area of ​​the second device on the screen of the first device.

[0022] In one possible implementation, determining the target feature region as the target region corresponding to the second device on the screen may include: if the target feature region satisfies a first condition and / or a second condition, the first device determines the target feature region as the hovering region corresponding to the second device on the screen; wherein, the first condition includes the target feature region having a first shape; the second condition includes the difference between the size of the target feature region and the top size of the second device being less than a first threshold. The size of the target feature region may include the length, width, and height of the target feature region, or the area of ​​the target feature region. The size of the shorter side may include the length and width of the top of the second device, or the area of ​​the top; or, the length, width, or the area of ​​the bottom of the second device.

[0023] In this way, content transmission between the first and second devices can only be triggered if the target feature area meets the first condition and / or the second condition. For example, the first device sends first content to the second device, or the second device sends second content to the first device. This avoids the possibility of the first device mistakenly triggering interaction with the second device due to changes in the capacitance of the first device's screen caused by other objects approaching the first device.

[0024] In one possible implementation, before the first device receives the second content sent by the second device, the method may further include: the first device sending a first instruction to the second device, the first instruction being used to instruct the second device to send the second content to the first device; the second device receiving and responding to the first instruction, sending the second content to the second device.

[0025] In this way, the second device can know when to send the second content based on the first instruction.

[0026] In one possible implementation, the first device sends a first instruction to the second device, which may specifically include: the first device determining that there is no content at the center point or centroid of the corresponding target area on the screen, or that there is no content within a range where the distance from the center point or centroid is less than a preset distance, and then sending the first instruction to the second device.

[0027] In this way, the first device will send the first instruction to the second device only when there is no content in the target area.

[0028] Alternatively, in another possible implementation, the first device sends a first instruction to the second device, which may specifically include: the first device determining that there is no content in the corresponding target area on the screen, and sending the first instruction to the second device.

[0029] In this way, the first device does not need to calculate the center point or centroid of the target area; it can directly check whether there is content within the target area. Only when there is no content in the target area will the first device send the first instruction to the second device.

[0030] In one possible implementation, before the first device sends the first content to the second device based on the target area, the method may further include: the first device establishing a second communication connection with the second device, wherein the transmission rate of the second communication connection is higher than the transmission rate of the first communication connection;

[0031] The first device sends first content to the second device based on the target area, which may include: the first device sending the first content to the second device through a second communication connection based on the target area.

[0032] In this way, by transmitting the first content through the second communication connection, the first device can transmit the first content to the second device more quickly.

[0033] In one possible implementation, the second content includes any one of the user interface, images, videos, tables, documents, and animations currently displayed on the second device.

[0034] Secondly, an interaction method is provided, which can be applied to a first device, the first device including a touch sensor. The method includes: the first device acquiring a first signal through the touch sensor, and determining a target area on the screen corresponding to the second device based on the first signal, the target area being an area on the screen where capacitance changes when the second device approaches the first device; the first device establishing a first communication connection with the second device; the first device receiving first indication information sent by the second device, the first indication information being used to indicate a change in the motion state of the second device; and the first device sending first content to the second device based on the target area.

[0035] The target area includes the area on the screen of the first device that the second device maps onto when the second device is hovered, or the contact area between the screens of the second device and the first device.

[0036] The change in motion state includes: the motion state changing to a stationary state, and the duration of the second device being stationary is greater than the first preset duration.

[0037] Using the interaction method provided in the first aspect, when the first device detects the target area of ​​the second device on the screen, and when the second device detects a change in the movement state of the second device, the second device can obtain the first content present in the target area of ​​the second device on the screen of the first device. In this way, interaction between the first and second devices can be achieved without much user intervention, and the second device can easily obtain the first content from the first device. This first aspect can achieve the effect of "absorbing" the first content from the first device when the second device approaches it, thus improving the user experience.

[0038] In one possible implementation, after the first device receives the first instruction information sent by the second device, the method may further include: the first device receiving the second content sent by the second device.

[0039] In this way, by bringing the second device close to the first device, the first device can acquire the second content from the second device without much user intervention. In other words, the user only needs to bring the second device close to the first device to achieve the effect of the first device "sucking" the second content from the second device. This simplifies user operations and improves the user experience.

[0040] In one possible implementation, the first device sends first content to the second device based on the target area, which may include: the first device determining that there is third content at the center point or centroid of the corresponding target area on the screen, and sending the first content to the second device.

[0041] The existence of a third content at the center point or centroid of the target area can include: the third content is located at the center point or centroid of the target area, or the distance between the center point or centroid and the third content is less than a distance threshold of 2.

[0042] In this way, the first device can determine whether to send the first content to the second device by whether the third content exists in the target area.

[0043] Alternatively, in another possible implementation, the first device sends the first content to the second device based on the target area, which may include: the first device determining that there is no third content in the target area of ​​the screen, and then sending the first content to the second device.

[0044] In this way, the first device does not need to calculate the center point or centroid of the target area. It can directly determine whether there is third content in the target area. If there is third content, the first device sends the first content to the second device.

[0045] In one possible implementation, the third content includes an application icon, a video file icon, an image file icon, a document file icon, a spreadsheet file icon, or any one of a video, image, document, or spreadsheet; the first content includes an application identifier, an application installation package, or any one of a video, image, document, or spreadsheet.

[0046] In one possible implementation, if the third content is an application icon, the first content is the application identifier or the application's installation package; or, if the third content is a video file icon or a video, the first content is a video; or, if the third content is an image file icon or an image, the first content is an image; or, if the third content is a document file icon or a document, the first content is a document; or, if the third content is a table file icon or a table, the first content is a table.

[0047] In this way, the first device can determine to send the first content based on the third content.

[0048] In one possible implementation, determining the target area of ​​the second device on the screen based on the first signal may specifically include: the first device generating a grayscale image from the first signal and determining the target feature area in the grayscale image, wherein the target feature area is the area in the grayscale image where the grayscale value of a pixel is higher than a grayscale threshold; and the first device determining the target feature area as the target area of ​​the second device on the screen.

[0049] In this way, the first device can accurately determine the target area of ​​the second device on the screen of the first device.

[0050] In one possible implementation, determining the target feature region as the target region corresponding to the second device on the screen may include: if the target feature region satisfies a first condition and / or a second condition, the first device determines the target feature region as the hovering region corresponding to the second device on the screen; wherein, the first condition includes the target feature region having a first shape; the second condition includes the difference between the size of the target feature region and the top size of the second device being less than a first threshold. The size of the target feature region may include the length, width, and height of the target feature region, or the area of ​​the target feature region. The size of the shorter side may include the length and width of the top of the second device, or the area of ​​the top; or, the length, width, or the area of ​​the bottom of the second device.

[0051] In this way, content transmission between the first and second devices can only be triggered if the target feature area meets the first condition and / or the second condition. For example, the first device sends first content to the second device, or the second device sends second content to the first device. This avoids the possibility of the first device mistakenly triggering interaction with the second device due to changes in the capacitance of the first device's screen caused by other objects approaching the first device.

[0052] In one possible implementation, before the first device receives the second content sent by the second device, the method may further include: the first device sending a first instruction to the second device, the first instruction being used to instruct the second device to send the second content to the first device.

[0053] In this way, the second device can know when to send the second content based on the first instruction.

[0054] In one possible implementation, the first device sends a first instruction to the second device, which may specifically include: the first device determining that there is no content at the center point or centroid of the corresponding target area on the screen, or that there is no content within a range where the distance from the center point or centroid is less than a preset distance, and then sending the first instruction to the second device.

[0055] In this way, the first device will send the first instruction to the second device only when there is no content in the target area.

[0056] Alternatively, in another possible implementation, the first device sends a first instruction to the second device, which may specifically include: the first device determining that there is no content in the corresponding target area on the screen, and sending the first instruction to the second device.

[0057] In this way, the first device does not need to calculate the center point or centroid of the target area; it can directly check whether there is content within the target area. Only when there is no content in the target area will the first device send the first instruction to the second device.

[0058] In one possible implementation, before the first device sends the first content to the second device based on the target area, the method may further include: the first device establishing a second communication connection with the second device, wherein the transmission rate of the second communication connection is higher than the transmission rate of the first communication connection;

[0059] The first device sends first content to the second device based on the target area, which may include: the first device sending the first content to the second device through a second communication connection based on the target area.

[0060] In this way, by transmitting the first content through the second communication connection, the first device can transmit the first content to the second device more quickly.

[0061] In one possible implementation, the second content includes any one of the user interface, images, videos, tables, documents, and animations currently displayed on the second device.

[0062] Thirdly, an interaction method is provided, which can be applied to a second device. The method may include: sending first indication information to a first device, the first indication information being used to indicate a change in the motion state of the second device, and the first device and the second device establishing a first communication connection; receiving first content sent by the first device, the first content being content determined by a target area of ​​the screen of the first device, the target area being an area in the screen of the first device where a capacitance change occurs when the second device approaches the first device.

[0063] Using the method provided by the third party, the second device can acquire the first content sent by the first device when it is close to the first device. This allows the second device to "absorb" content from the first device without requiring much user intervention, thus improving the user experience.

[0064] In one possible implementation, after sending the first instruction information to the first device, the method may further include: sending the second content to the first device.

[0065] In this way, by bringing the second device close to the first device, the first device can acquire the second content from the second device without much user intervention. In other words, the user only needs to bring the second device close to the first device to achieve the effect of the first device "sucking" the second content from the second device. This simplifies user operations and improves the user experience.

[0066] In one possible implementation, before sending the second content to the first device, the method may further include: receiving a first instruction sent by the first device, the first instruction being used to instruct the second device to send the second content to the first device.

[0067] In this way, the second device can know when to send the second content based on the first instruction.

[0068] In one possible implementation, if an application icon is present in the target area, the first content is the application's identifier or the application's installation package; or, if a video file icon or video is present in the target area, the first content is the video; or, if an image file icon or image is present in the target area, the first content is the image; or, if a document file icon or document is present in the target area, the first content is the document; or, if a table file icon or table is present in the target area, the first content is the table.

[0069] Content present in the target area (e.g., application icons, video file icons, etc.) may include: the content being above the center point or centroid of the target area, or the content being within the target area, or the content being less than a preset distance from the center point or centroid of the target area.

[0070] In this way, the first content can be determined by the content in the target area.

[0071] In one possible implementation, the target area includes the area on the screen of the first device that the second device maps onto when the second device is hovered, or the contact area between the screen of the second device and the screen of the first device; the change in motion state includes: the motion state changing to a stationary state, and the duration of the second device being stationary is greater than a first preset duration; the second content includes any one of the user interface, image, video, table, document, and animation currently displayed by the second device.

[0072] Fourthly, an interaction method is provided, which can be applied to an interaction system including a first device and a second device. The first device and the second device establish a first communication connection. The interaction method may include: when the second device hovers over or touches the screen of the first device, the first device sends first content in a target area to the second device. The target area is the area mapped onto the screen of the first device when the second device hovers over or touches the screen of the first device, or the contact area between the screen of the second device and the screen of the first device.

[0073] Alternatively, the second device may send second content to the first device, the second content being the content currently displayed on the second device.

[0074] The first content may include any one of the following: file, image, folder, video, audio, window, control, or application interface; the second content may include any one of the following: file, image, folder, video, audio, window, control, or application interface.

[0075] The interaction method provided in the fourth aspect allows users to interact with the first and second devices simply by hovering or lightly touching the second device to the screen of the first device, without much user intervention. The second device can easily access the first content on the first device. Alternatively, when the second device approaches the first device, it can "suck" away the first content from the first device, thus improving the user experience.

[0076] In one possible implementation, sending the first content within the target area to the second device may specifically include: when the first content is displayed in the target area of ​​the first device and the desktop is displayed on the second device, the first device determines to send the first content within the target area to the second device.

[0077] In this way, the first device can determine the interaction content (such as the first content) and the interaction direction (i.e., the first device sends the first content to the second device) based on the content currently displayed on the two devices.

[0078] In one possible implementation, the second device sending the second content to the first device may specifically include: when the desktop is displayed in the target area of ​​the first device and the second device displays the second content, the first device determines that the second device will send the second content to the first device.

[0079] In this way, the first device can determine the interaction content (such as the second content) and the interaction direction (i.e., the second device sends the second content to the first device) based on the content currently displayed on the two devices.

[0080] In one possible implementation, the second device sends second content to the first device, which may specifically include: when the desktop is displayed in the target area of ​​the first device and the second device is also displaying its desktop, the second device sends the desktop data of the second device to the first device.

[0081] The method may further include: the first device receiving desktop data from the second device and displaying the desktop content of the second device.

[0082] In this way, users can project the desktop of the second device onto the first device by hovering the second device close to or lightly touching the screen of the first device.

[0083] In one possible implementation, the method may further include: when the first device displays first content and the second device displays second content, the second device displays a first pop-up window, the first pop-up window including a first control and a second control; in response to a user's operation on the first control, the second device sends the second content to the first device; or...

[0084] In response to the user's operation on the second control, the second device sends a first instruction to the first device, the first instruction instructing the first device to send first content to the second device; the first device receives and responds to the first instruction, and sends the first content to the second device.

[0085] In this way, when both devices are displaying content, a pop-up can be provided for the user to choose the content and direction of interaction.

[0086] In one possible implementation, before the second device is hovered over or tapped against the screen of the first device, the method may further include: the first device displaying the application interface of a first application, and the first content within the target area including controls in the application interface of the first application; and the second device displaying the desktop.

[0087] After the first device sends the first content within the target area to the second device, the method may further include: the second device receiving the first content and displaying the first content.

[0088] In this way, the first device can send a portion of the application interface (e.g., controls) to the second device.

[0089] In one possible implementation, before the second device is hovered over or tapped against the screen of the first device, the method may further include: the first device displaying an application interface of a second application, the application interface of which displays a first document, which includes any one of a file, a spreadsheet, or an email; and the second device displaying the second content.

[0090] After the second device sends the second content to the first device, the method may further include: the first device receiving the second content and inserting the second content into the first document.

[0091] In this way, users can precisely insert the second content displayed on the second device into the document displayed on the first device by hovering the second device close to or lightly touching the screen of the first device.

[0092] In one possible implementation, when the first device displays the first content and the second device displays the desktop, the method may further include: when the first device displays the first content and the third content, the second device displays the desktop, and the first device determines that the first content is within the target area, the first device determines to send the first content within the target area to the second device.

[0093] In this way, when the first device displays multiple contents, the first device can determine the interactive content based on the target area of ​​the user's second device hovering near or lightly touching the screen of the first device.

[0094] In one possible implementation, the method may further include: a first device receiving a first operation, the first operation being used to select first content in the first device; and the first device sending the first content to the second device when the second device hovers over or taps against the screen of the first device.

[0095] In this way, the user can first select interactive content on the first device, and then the user can send the selected interactive content to the second device by hovering the second device close to or tapping it on the screen of the first device.

[0096] In one possible implementation, the method may further include: a second device receiving a second operation for selecting second content in the second device; and the second device sending the second content to the first device when the second device hovers over or touches the screen of the first device.

[0097] In this way, the user can first select interactive content on the second device, and then the user can send the selected interactive content to the first device by hovering the second device close to or lightly touching the screen of the first device.

[0098] In one possible implementation, when the second device hovers over or touches the screen of the first device, the method may include: a touch sensor in the first device acquiring a first signal, and determining a target area on the screen of the first device where the second device hovers over or touches the screen based on the first signal, the target area being an area on the screen of the first device where a capacitance change occurs when the second device approaches the first device; the first device receiving first indication information sent by the second device, the first indication information being used to indicate a change in the motion state of the second device.

[0099] A change in motion state may include: the motion state changing to a stationary state, and the duration of the second device being stationary is longer than a first preset duration.

[0100] In this way, the first device can detect when the second device is hovering or lightly touching the screen of the first device.

[0101] In one possible implementation, before the touch sensor in the first device acquires the first signal, the method may further include: the second device receiving a trigger operation, the trigger operation including a user holding the second device and drawing a circle, or a user tapping the back cover of the second device; in response to the trigger operation, the second device sending a second instruction to the first device, the second instruction being used to instruct the second device to begin acquiring the first signal through the touch sensor.

[0102] In this way, the touch sensor of the first device only begins to collect the first signal after the second device receives a trigger operation from the user explicitly requesting touchscreen detection. That is, the touch sensor in the first device does not need to collect the first signal before the second device receives the user's trigger operation. This saves power consumption in the first device.

[0103] Fifthly, an electronic device is provided, comprising: one or more processors and a memory, a screen including a touch sensor; the screen and the memory are coupled to the one or more processors, the memory being used to store computer program code including computer instructions, and the one or more processors invoking the computer instructions to cause the electronic device to perform any possible implementation of the method performed by the first device in the first to fourth aspects.

[0104] A sixth aspect provides an electronic device comprising: one or more processors and a memory coupled to the one or more processors, the memory being used to store computer program code, the computer program code including computer instructions, wherein the one or more processors invoke the computer instructions to cause the electronic device to perform any possible implementation of a method performed by a second device in the first to fourth aspects.

[0105] A seventh aspect provides a computer-readable storage medium, characterized in that it includes computer instructions that, when executed on an electronic device, cause the electronic device to perform any one of the methods that can be implemented by the first or second device in the first to fourth aspects.

[0106] Eighthly, a computer program product is provided, characterized in that it includes computer instructions that, when executed on an electronic device, cause the electronic device to perform any one of the methods that can be implemented by the first or second device in the first to fourth aspects. Attached Figure Description

[0107] Figure 1 is a schematic diagram of the system 10 provided in an embodiment of this application;

[0108] Figure 2 is a schematic diagram of a scene in which the mobile phone 100 moves toward the screen of the computer 200 according to an embodiment of this application;

[0109] Figure 3 is a schematic diagram of the structure of the screen 201 of the computer 200 provided in the embodiment of this application;

[0110] Figure 4 is a schematic diagram of the electrode distribution of the touch sensor provided in an embodiment of this application;

[0111] Figure 5 is a set of grayscale image schematic diagrams provided in the embodiments of this application;

[0112] Figure 6 is a flowchart illustrating an interaction method provided in an embodiment of this application;

[0113] Figure 7 is a schematic diagram of a scenario where the hovering area of ​​the mobile phone 100 on the screen of the computer 200 is an application icon, according to an embodiment of this application.

[0114] Figure 8 is a schematic diagram of a scenario where the hovering area of ​​the mobile phone 100 on the screen of the computer 200 is an image, according to an embodiment of this application.

[0115] Figure 9 is a schematic diagram of a scenario in which the hovering area of ​​the mobile phone 100 on the screen of the computer 200 has no content, according to an embodiment of this application.

[0116] Figure 10 is a schematic diagram of another scenario where the hovering area of ​​the mobile phone 100 on the screen of the computer 200 is an application icon, according to an embodiment of this application.

[0117] Figure 11A is a hardware structure diagram of the first device provided in an embodiment of this application;

[0118] Figure 11B is a hardware and software architecture diagram of the first device provided in an embodiment of this application;

[0119] Figure 12A is a hardware structure diagram of the second device provided in an embodiment of this application;

[0120] Figure 12B is a software structure block diagram of the second device provided in an embodiment of this application;

[0121] Figure 13 is a schematic diagram of different interaction scenarios provided in the embodiments of this application;

[0122] Figure 14 is a schematic diagram of an interaction scenario between a mobile phone 100 and a computer 200 provided in an embodiment of this application;

[0123] Figure 15 is a schematic diagram of another interaction scenario between a mobile phone 100 and a computer 200 provided in an embodiment of this application;

[0124] Figure 16 is a schematic diagram of another interaction scenario between a mobile phone 100 and a computer 200 provided in an embodiment of this application;

[0125] Figure 17 is a schematic diagram of another interaction scenario between a mobile phone 100 and a computer 200 provided in an embodiment of this application;

[0126] Figure 18 is a schematic diagram of an interaction scenario between a mobile phone 100 and a computer 200, and between a mobile phone 100 and a computer 300, provided in an embodiment of this application.

[0127] Figure 19 is a schematic diagram of another interaction scenario between a mobile phone 100 and a computer 200 provided in an embodiment of this application;

[0128] Figure 20 is a schematic diagram of another interaction scenario between a mobile phone 100 and a computer 200 provided in an embodiment of this application;

[0129] Figure 21 is a schematic diagram of a user operating a mobile phone 100 according to an embodiment of this application;

[0130] Figure 22 is a schematic diagram of a user interface of a computer 200 provided in an embodiment of this application;

[0131] Figure 23 is a schematic diagram of another user interface of the computer 200 provided in an embodiment of this application;

[0132] Figure 24 is a schematic diagram of another user interface of the computer 200 provided in an embodiment of this application;

[0133] Figure 25 is a schematic diagram of another user interface of the computer 200 provided in an embodiment of this application;

[0134] Figure 26 is a schematic diagram of another user interface of the computer 200 provided in an embodiment of this application;

[0135] Figure 27 is a software structure block diagram of the mobile phone 100 and computer 200 provided in the embodiments of this application. Detailed Implementation

[0136] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0137] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. The terms “first” and “second” are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as “first” or “second” may explicitly or implicitly include one or more of that feature. “First” and “second,” etc., are used to distinguish different objects, not to describe a particular order of objects. For example, a first object and a second object are used to distinguish different objects, not to describe a particular order of objects.

[0138] In the description of the embodiments in this application, unless otherwise stated, "multiple" means two or more. For example, multiple processing units refer to two or more processing units; multiple systems refer to two or more systems.

[0139] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or related scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0140] The term "and / or" in this application is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A existing alone, A and B existing simultaneously, and B existing alone.

[0141] The term "user interface (UI)" used in the following embodiments of this application refers to the medium interface through which an application or operating system interacts and exchanges information with the user. It realizes the conversion between the internal form of information and the form that the user can accept. The user interface is source code written in a specific computer language such as Java or Extensible Markup Language (XML). The interface source code is parsed and rendered on the electronic device, ultimately presenting content that the user can recognize. A common form of user interface is the graphical user interface (GUI), which refers to a user interface related to computer operation displayed graphically. It can be visible interface elements such as text, icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, and widgets displayed on the screen of an electronic device.

[0142] Currently, in some embodiments, both electronic device A and electronic device B may have near-field communication (NFC). Electronic device A and electronic device B can establish a communication connection through their respective NFC chips. For example, the NFC chip of electronic device B may store the Bluetooth address of electronic device B. When a user touches electronic device A with electronic device B, electronic device A can read the Bluetooth address of electronic device B through its NFC chip, and then electronic device A and electronic device B can establish a communication connection. Afterwards, electronic device A and electronic device B can interact (e.g., multi-screen collaboration, file transfer, file sharing, etc.).

[0143] The NFC chip in electronic device A and the NFC chip in electronic device B enable them to quickly establish a communication connection. However, both electronic devices A and B need to have NFC chips to communicate and interact. NFC chips increase the cost of electronic products.

[0144] This application provides an interaction method, system, and related apparatus. In the interaction method provided by this application, two electronic devices do not need to have NFC chips to achieve "tap-to-transfer" data. That is, when the first device approaches the second device, the first device can transmit the target content to the second device, or the second device can transmit the target content to the first device.

[0145] The following first describes the interactive system provided in the embodiments of this application. In these embodiments, the interactive system may include a first device and a second device. The first device detects the second device approaching the first device via its screen and can determine the position, shape, height, etc., of the second device on the screen of the first device. This determines the target position of the second device on the screen of the first device. Based on this target position, the first device performs a first operation, or the second device performs a second operation.

[0146] The first operation includes, but is not limited to, the first device sending content (e.g., images, documents, files, tables, etc.) at the target location to the second device and obtaining the content currently displayed by the second device. The second operation includes, but is not limited to, the second device sending the currently displayed content to the first device and the second device obtaining the content at the target location of the first device.

[0147] The first device may include, but is not limited to, electronic devices with touch panels (TP), such as mobile phones, tablets, and computers. The second device may include, but is not limited to, electronic devices such as mobile phones, tablets, watches, and fitness trackers.

[0148] For example, the interactive system provided in this application embodiment may be system 10 shown in FIG1. ​​The first device may include computer 200 shown in FIG1, and the second device may include mobile phone 100 shown in FIG1.

[0149] Figure 1 illustrates, exemplarily, a system 10 provided in an embodiment of this application. As shown in Figure 1, system 10 may include a mobile phone 100 and a computer 200. The mobile phone 100 and the computer 200 may establish a communication connection 1.

[0150] First, as the user holds the mobile phone 100 and approaches the screen of the computer 200, the screen of the computer 200 can detect the proximity of the mobile phone to the screen, the target position of the mobile phone 100 on the screen, and the shape (including but not limited to area size, length, width, etc.) of the area on the screen where the capacitance of the mobile phone 100 changes. Whether the mobile phone 100 is in contact with the screen of the computer 200 or not, the shape of the area (sensing area) where the screen capacitance changes can vary. For example, when the top (or bottom) of the mobile phone 100 is close to the screen of the computer 200, the shape of the area where the capacitance changes can be elongated; or, when the mobile phone 100 makes point contact with the screen of the computer 200, the shape can be a dot or a circle; or, when the mobile phone 100 makes surface contact with the screen, the shape can be a rectangle or other planar shape.

[0151] As the mobile phone 100 moves closer to the screen of the computer 200, the computer 200 can determine that a device is approaching the screen based on changes in the capacitance value displayed on the screen. The computer 200 can also obtain device information from the mobile phone 100 based on the communication connection between the computer 200 and the mobile phone 100, thereby confirming that the device approaching the screen is the mobile phone 100.

[0152] Furthermore, the computer 200 can also determine the target area on the screen that generates the capacitance change, as well as the shape of the target area, based on the change in the capacitance value on the screen. Then, the computer 200 can also determine the target position of the mobile phone 100 on the screen of the computer 200 based on the target area.

[0153] Optionally, the screen of computer 200 can also detect the height of mobile phone 100 from the screen of computer 200. In this embodiment, the height of mobile phone 100 from the screen of computer 200 may include, but is not limited to, the distance between the top of mobile phone 100 and the screen of computer 200, or the distance between the center of mobile phone 100 and the screen of computer 200.

[0154] For example, taking the phone hovering over or touching the screen (or lightly touching the computer screen) as an example, as shown in Figure 2, the phone 100 moves towards the screen 201 of the computer 200. When the phone 100 approaches the screen 201 of the computer 200 and hovers above or touches (or lightly touches) the screen 201 of the computer 200, the screen 201 of the computer 200 can detect the target position of the phone 100 on the screen of the computer 200, as well as the height of the phone 100 from the screen of the computer 200.

[0155] Then, the mobile phone 100 can detect changes in its own motion state, such as from movement to stillness. After that, the mobile phone 100 can send instruction 1 to the computer 200, which can be used to instruct the mobile phone 100 to stop moving.

[0156] Next, after receiving instruction 1 from mobile phone 100, computer 200 can determine the target content based on the target position of mobile phone 100 on the screen of computer 200, and send the target content to mobile phone 100.

[0157] In some examples, the target content is the content displayed at a target location on the screen of computer 200, such as files, documents, tables, pictures, etc.

[0158] Alternatively, when the content displayed at the target location on the screen of computer 200 is an application icon, the target content can also be an application installation package or an application identifier (e.g., the application name).

[0159] Alternatively, in some other examples, when there is no content at the target location on the screen of computer 200, mobile phone 100 can send the content displayed on its screen to computer 200. That is, mobile phone 100 can project the content it will display onto computer 200.

[0160] When the computer 200 determines the target position of the mobile phone 100 on the screen of the computer 200, and the motion state of the mobile phone 100 changes (e.g., from motion to stillness), the specific interaction between the computer 200 and the mobile phone 100 can be found in the description in Figure 6, which will not be elaborated here.

[0161] The following explanation will use a computer 200 as the first device and a mobile phone 100 as the second device as an example.

[0162] The following details how the computer 200 screen determines the target position of the mobile phone 100 on the computer 200 screen. For example, the computer 200 can first determine the target area corresponding to the mobile phone 100 on its screen. This target area may include the area on the computer 200 screen that the mobile phone 100 is mapped onto when it is hovered near the screen, or the contact area between the mobile phone 100 and the computer 200 screen. After determining the target area, the computer 200 can determine the target position of the mobile phone 100 on the computer 200 screen based on this target area. For example, the target position of the mobile phone 100 on the computer 200 screen can be the center point or centroid of the target area.

[0163] Figures 3-5 exemplarily illustrate how computer 200 determines the target area corresponding to mobile phone 100 on the screen of computer 200.

[0164] Figure 3 illustrates a schematic diagram of the structure of the screen 201 of the computer 200. In this embodiment, the screen 201 may be referred to as a touch screen, touch panel, etc. As shown in Figure 3, the screen 201 may include a protective glass, a touch panel sensor (TP sensor) 202, and a display screen. Wherein:

[0165] The protective glass is generally located on the outermost layer of the screen 201 and is used to protect the internal display screen and touch sensor 202 from physical damage (such as scratches, impacts, etc.). The protective glass can usually be made of high-strength glass material, but this application embodiment does not limit the specific material to which the protective glass is made.

[0166] Touch sensor 202 is the input layer of screen 201 and can be located between the protective glass and the display screen. Touch sensor 202 can be used to detect the user's finger touch or stylus operation and convert these operations into electrical signals. Touch sensor 202 can then transmit these electrical signals to the processor of computer 200.

[0167] In some examples, the touch sensor 202 can also detect objects approaching it. Furthermore, the touch sensor 202 can also detect the shape of objects approaching it.

[0168] In one possible implementation, the touch sensor 202 may include an X-axis electrode layer and a Y-axis electrode layer. The X-axis electrode layer and the Y-axis electrode layer are distributed crosswise on the touch sensor 202, forming an M*N array distribution as shown in Figure 4. The computer 200 detects the change in capacitance value at each coordinate point by scanning along the X and Y axes at a specific detection frequency, and can determine the target position of the approaching object on the screen 201 (e.g., the hovering area of ​​the mobile phone 100 shown in Figure 4) based on the change in capacitance value at each coordinate point.

[0169] In one implementation, when an object (e.g., mobile phone 100) approaches and hovers above the touch sensor 202, the object absorbs a portion of the excitation signal emitted by the TX transmitting electrode, thereby weakening the excitation signal received by the receiving electrode RX. When scanning and detecting the mutual capacitance at each coordinate point on the touch sensor 202, the computer 200 can calculate the object hovering area based on the capacitance change at each coordinate point. In another implementation, when an object (e.g., mobile phone 100) approaches and hovers above the touch sensor 202, the object's capacitance is superimposed on the touch sensor's capacitance (the self-capacitance formed by TX and ground, or the self-capacitance formed by RX and ground), causing a change in the screen capacitance. When scanning and detecting the self-capacitance of each electrode at each coordinate point on the touch sensor 202, the computer 200 can calculate the object hovering area based on the capacitance change of each electrode at each coordinate point.

[0170] The display screen can be used to display images, videos, etc. The display screen can include a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc.

[0171] In some examples, computer 200 calculates the target area of ​​mobile phone 100 based on the capacitance change at each coordinate point on touch sensor 202. This may include: computer 200 converting the touch signal acquired by touch sensor 202 (e.g., the capacitance change at each coordinate point on touch sensor 202) into a two-dimensional matrix, where each value in the matrix represents the signal strength at that location. Then, computer 200 generates a grayscale image from the two-dimensional matrix. The higher the signal strength in the two-dimensional matrix, the larger the pixel value of the corresponding pixel in the grayscale image. Finally, computer 200 can determine the target area of ​​mobile phone 100 based on the grayscale image. This target area is the area on the screen of computer 200 where a capacitance change occurs when mobile phone 100 is brought close to computer 200. This target area includes the area on the screen of computer 200 where mobile phone 100 is mapped when mobile phone 100 is hovered (e.g., the top or bottom of mobile phone 100 mapped onto the screen of computer 200), or the contact area between mobile phone 100 and the screen of computer 200.

[0172] For example, the grayscale image generated by computer 200 can be grayscale image 501 as shown in Figure 5(a). Figure 5(a) exemplarily shows grayscale image 501 and grayscale scale 5010 corresponding to grayscale image 501. The grayscale scale 5010 is used for the range of grayscale values ​​of pixels in the grayscale image from 0 to 80. Among them, the grayscale value of the darkest pixel in grayscale image 501 is 0, and the grayscale value of the lightest pixel is 80.

[0173] Understandably, when an object approaches and hovers above the screen 201 of the computer 200, the capacitance change in the corresponding hovering area of ​​the touch sensor 202 will be greater than the capacitance change in other areas of the touch sensor 202 where no object is nearby or where there is no touch. When the touch signal collected by the touch sensor 202 is converted into a grayscale image, the area with the larger grayscale value in the grayscale image is the hovering area of ​​the phone 100. However, due to some interference (e.g., ripple interference generated by the power supply circuit of the touch sensor 202), the capacitance value of some areas where no object is nearby and no touch operation occurs will also increase significantly. This results in the corresponding grayscale image having not only a larger grayscale value corresponding to the hovering area, but also other areas with larger grayscale values ​​(which can also be referred to as noise in the grayscale image).

[0174] In one possible implementation, the computer 200 can denoise the raw data collected by the touch sensor 202, or denoise the grayscale image generated from the raw data collected by the touch sensor 202. Taking the computer 200 denoising a grayscale image 501 as an example, the computer 200 can perform image processing on the grayscale image 501 (e.g., temporal denoising (Kalman filtering, adaptive filtering, wavelet thresholding), frequency domain filtering, image morphological processing (dilation, erosion), etc.) to reduce noise in the grayscale image 501, resulting in a grayscale image with observable local target features. This application embodiment does not limit the specific process of image processing performed by the computer 200.

[0175] For example, after processing the grayscale image 501, the computer 200 can obtain the grayscale image 502 shown in Figure 5(b). The grayscale value of each pixel in the grayscale image 502 can be determined based on the grayscale scale 5010. The grayscale image 502 may include a target feature region 5021. The target feature region 5021 is the region in the grayscale image 502 where the grayscale value of a pixel is higher than a grayscale threshold. The computer 200 can determine that the target feature region 5021 is the target region corresponding to the mobile phone 100 on the screen 201 of the computer 200.

[0176] In some examples, the grayscale threshold may be configured in the system of computer 200. This application does not limit the specific size of the grayscale threshold in its embodiments.

[0177] In one possible implementation, computer 200 can detect the distance between mobile phone 100 and the screen of computer 200, and then perform image processing on grayscale image 501 based on the distance between mobile phone 100 and the screen of computer 200. For example, without exceeding a distance threshold detectable by the touch sensor on the screen of computer 200, the greater the distance between mobile phone 100 and the screen of computer 200, the smaller the touch signal value detected by the touch sensor (e.g., the smaller the capacitance value in the touch sensor). Conversely, the smaller the distance between mobile phone 100 and the screen of computer 200, the larger the touch signal value detected by the touch sensor (e.g., the larger the capacitance value in the touch sensor). When computer 200 performs filtering processing on the grayscale image generated by the signal detected by the touch sensor (this filtering processing is one of the image processing methods described above, such as Kalman filtering or adaptive filtering mentioned above), computer 200 can determine the filtering threshold based on the distance between mobile phone 100 and the screen of computer 200. The greater the distance between the mobile phone 100 and the screen of the computer 200, the smaller the filtering threshold that the computer 200 can determine. Conversely, the smaller the distance between the mobile phone 100 and the screen of the computer 200, the larger the filtering threshold that the computer 200 can determine.

[0178] In this way, when the mobile phone 100 is hovered at different distances from the computer 200, the computer 200 can use different filtering thresholds to filter the grayscale image generated by the signal detected by the touch sensor, enabling the computer 200 to accurately determine the target area corresponding to the mobile phone 100 on the computer 200 screen. For example, when the mobile phone 100 is wearing a phone case, and the user lightly touches the mobile phone 100 to the computer 200 screen, the actual distance between the mobile phone 100 and the computer 200 screen is greater than 0. That is, there is a distance between the mobile phone 100 and the computer 200 screen, and this distance is the thickness of the phone case. In this case, the computer 200 can still accurately detect the target area corresponding to the mobile phone 100 on the computer 200 screen.

[0179] Optionally, in some possible implementations, the computer 200 can also determine whether the detected target feature region 5021 meets the first condition. If the first condition is met, the computer 200 determines that the target feature region 5021 is the target region corresponding to the mobile phone 100 on the screen 201 of the computer 200. Then, the computer 200 determines the coordinates of the center position or centroid position of the target feature region 5021 and uses the coordinates of the center position or centroid position of the target feature region 5021 as the target position corresponding to the mobile phone 100 on the screen of the computer 200. If the first condition is not met, the computer 200 can continue to periodically scan the screen to obtain the capacitance value or capacitance change value in the touch sensor.

[0180] For example, the first condition may be that the shape of the target feature region 5021 is a first shape (e.g., a rectangle, a strip, a circle, a dot, or other planar shapes, etc.). Alternatively, the first condition may also be that the grayscale value of each pixel in the target feature region 5021 is greater than a first threshold. This application embodiment does not specifically limit the first condition. The first shape may also be any shape configured by the computer 200 system (e.g., a rectangle, a square, a triangle, etc.), and this application embodiment does not limit this. The first threshold may be configured by the computer 200 system, and this application embodiment does not limit the specific value of the first threshold.

[0181] Furthermore, in one possible implementation, the computer 200 can determine the similarity value between the shape of the target feature region 5021 and the first shape. If the similarity value is higher than the similarity threshold, the computer 200 can determine that the shape of the target feature region 5021 is the first shape.

[0182] It is understood that the similarity threshold can be configured by the computer 200 system. For example, the similarity threshold can be 95%, 98%, etc. This application embodiment does not limit the specific value of the similarity threshold.

[0183] Optionally, in some possible implementations, the computer 200 can also determine whether the detected target feature region 5021 meets the second condition. If the second condition is met, the computer 200 determines that the target feature region 5021 is the target region corresponding to the mobile phone 100 on the screen 201 of the computer 200. Then, the computer 200 determines the coordinates of the center position or centroid position of the target feature region 5021 and uses these coordinates as the target position of the mobile phone 100 on the screen of the computer 200. If the second condition is not met, the computer 200 can continue to periodically scan the screen to obtain the capacitance value or capacitance change value in the touch sensor.

[0184] For example, the second condition may be that the difference between the size of the target feature region 5021 and the top size of the mobile phone 100 is less than a second threshold. The size of the target feature region 5021 may include the length and / or width of the target feature region 5021, or the area of ​​the target feature region 5021. The top size of the mobile phone 100 may include the length and width of the top of the mobile phone 100, or the area of ​​the top of the mobile phone 100. The computer 200 obtains the top size of the mobile phone 100, and then the computer 200 can compare the size of the target feature region 5021 with the top size of the mobile phone 100. For example, the computer 200 can calculate the difference 1 between the length of the target feature region 5021 and the length of the top of the mobile phone 100, and calculate the difference 2 between the width of the target feature region 5021 and the width of the top of the mobile phone 100. Then, the computer 200 can determine whether the difference 1 and the difference 2 are less than the second threshold. For example, computer 200 can calculate the difference 3 between the area of ​​the target feature region 5021 and the area of ​​the top of mobile phone 100. Then, computer 200 can determine whether the difference 3 is less than a second threshold.

[0185] It is understood that the second threshold can be configured by the system of computer 200. This application embodiment does not limit the specific value of the second threshold.

[0186] In one possible implementation, computer 200 determines that the target feature region 5021 satisfies either the first condition or the second condition. If satisfied, computer 200 determines that the target feature region 5021 is the target region corresponding to mobile phone 100 on computer screen 201. Then, computer 200 determines the coordinates of the center or centroid of the target feature region 5021 and uses these coordinates as the target position of mobile phone 100 on computer screen 200. Optionally, in another possible implementation, computer 200 determines the coordinates of the center or centroid of the target feature region 5021 only after both the first and second conditions are satisfied.

[0187] In one possible implementation, after the computer 200 determines the coordinates of the center or centroid of the target feature region 5021, it can convert these coordinates into screen coordinates. The computer 200 uses these screen coordinates as the target position of the mobile phone 100 on the computer 200 screen. Then, it determines whether there is corresponding first content at these screen coordinates. If there is, the first content is sent to the mobile phone 100; otherwise, it retrieves the second content displayed by the mobile phone 100.

[0188] For details, please refer to the descriptions of Figures 6-10 below; they will not be repeated here.

[0189] Based on the above description of how system 10 and computer 200 determine the target area corresponding to mobile phone 100 on the screen of computer 200, the following describes an interaction method provided by an embodiment of this application with reference to the accompanying drawings.

[0190] Figure 6 illustrates a flowchart of an interaction method provided in an embodiment of this application. As shown in Figure 6, an interaction method provided in an embodiment of this application may include the following steps:

[0191] S601. The first device establishes a first connection with the second device.

[0192] The first device may establish a first connection with the second device. For example, the first connection may be a wireless communication connection, and the first device and the second device may establish a wireless communication connection through wireless communication technology.

[0193] Wireless communication technology refers to the methods and measures taken to wirelessly transmit data from one electronic device to another according to established communication rules. For example, in the embodiments of this application, wireless communication technology may include, but is not limited to: Bluetooth (BT) technology (e.g., ordinary BT technology, Bluetooth Low Energy (BLE) technology), Wireless Fidelity (WiFi) technology, ZigBee technology, Ultra Wideband (UWB) technology, Near Field Communication (NFC) technology, or StarFlash technology, etc. In addition, wireless communication technology may also include wireless communication technologies evolved from the above-mentioned communication technologies, as well as wireless communication technologies with the same or similar functions and capable of substituting for each other, etc., which are not limited in this application.

[0194] Each wireless communication technology has a corresponding wireless communication protocol. For example, BT technology corresponds to the BT protocol, WiFi technology corresponds to the WiFi protocol, and StarScan technology corresponds to the StarScan protocol. When two communication devices communicate using a wireless communication technology, both parties use the wireless communication protocol corresponding to that technology to process the transmitted data.

[0195] Since current wireless communication technologies are generally limited to short distances, they can also be called short-range wireless communication technology or short-range communication technology.

[0196] In some feasible examples, the first and second devices can connect to an access point via short-range wireless communication technology, thereby accessing the same wireless local area network (WLAN). In this way, the first and second devices can establish a wireless communication connection through the WLAN. For example, the first and second devices can first connect to an access point (i.e., a WiFi router) using WiFi technology, and then the access point can connect the smartphone and laptop to the WLAN.

[0197] For example, the first and second devices can access the same Bluetooth beacon using Bluetooth technology, thus establishing a wireless communication connection between them.

[0198] The embodiments of this application do not limit the specific connection method of the first connection, nor the specific process of establishing the first connection between the first device and the second device.

[0199] In this application embodiment, the first device may not be limited to electronic devices with a touch panel (TP), such as mobile phones, tablets, and computers. The second device may include, but is not limited to, electronic devices such as mobile phones, tablets, watches, and wristbands. For example, the first device may be the computer 200 mentioned above. The second device may be the mobile phone 100 mentioned above.

[0200] In one possible implementation, after the first device and the second device establish a first connection, the second device can send its device information to the first device. The device information includes, but is not limited to, one or more of the following: the device name, device dimensions, and device model. The device dimensions may include the dimensions of the shorter side (length, width, or area of ​​the shorter side).

[0201] In some examples, embodiments of this application can distinguish between long and short sides by the lengths of the four cross-sections (top, bottom, left, and right) of the casing of the second device. That is, the short sides can be the two shorter cross-sections among the four cross-sections of the casing of the second device. For example, as shown in FIG1, the casing of the mobile phone 100 can include four cross-sections (top, bottom, left, and right), where the top cross-section can also be called the top or uppermost point. The bottom cross-section can also be called the bottom or lowermost point. The left and right cross-sections can be called the left and right sides of the exterior of the mobile phone 100, respectively. Based on the lengths of the four cross-sections (top, bottom, left, and right) that the casing of the mobile phone 100 can include, the shorter cross-sections (top and bottom) can be called the short sides, and the longer cross-sections (left and right) can be called the long sides.

[0202] Alternatively, in some other examples, the short side and long side mentioned in the embodiments of this application do not refer to the cross-section, but rather to one side of the cross-section. For example, in the embodiments of this application, one side of the top or bottom of the second device that is close to the display screen can be called the short side, and one side of the left or right side of the second device that is close to the display screen can be called the long side.

[0203] The following description uses the example of a cross-section of the top or bottom of the second device as its short side. That is, the short side of the second device can be either the top or the bottom of the second device. In some examples, the dimensions of the short side of the second device can include the length and width of the top of the second device, or the area of ​​the top. In other examples, the dimensions of the short side of the second device can include the length and width of the bottom of the second device, or the area of ​​the bottom.

[0204] Alternatively, in another possible implementation, the first device may also pre-store the dimensions of the short side of the second device. These dimensions are not sent to the first device by the second device after the connection is established.

[0205] In some examples, the first device may be configured with a fixed short side dimension of an electronic device within the system. This fixed short side dimension may be the average of the short side dimensions of commercially available electronic devices, or it may be the average of the short side dimensions of a second device of a different model manufactured by the same manufacturer as the first device. This application does not limit this specific aspect.

[0206] Optionally, the first device can also update the pre-saved short side size of the second device based on the size of the short side of the newly released second device.

[0207] In other examples, the first device pre-stores the dimensions of the short side of the second device, including the dimensions of the short side corresponding to different device models. The first device can determine the dimensions of the short side of the second device based on its device model. After the first device establishes a connection with the second device, the second device can send its device model to the first device. Alternatively, when the first device detects the second device, it can determine the device model based on user interaction. For example, the user can directly input the device model of the second device in the first device, or the first device can display an option box showing different device models. The user can then select the device model of the second device from this option box.

[0208] The following explanation uses the dimensions of the short side of the second device (length and width of the short side, or area of ​​the short side) as an example to illustrate the dimensions of the top of the second device (length and width of the top, or area of ​​the top).

[0209] S602. The first device detects the second device and determines the target position of the second device on the screen of the first device.

[0210] In one possible implementation, the first device may include a screen, which may include a touch sensor. For example, the structure of the screen can be seen in Figure 3 above, and will not be described again here. The touch sensor can acquire the capacitance value of its capacitor. When the second device approaches the screen of the first device, the first device can detect the approach of the second device through the capacitance value acquired by the touch sensor and determine the target position of the second device on the screen of the first device.

[0211] Understandably, when the distance between the second device and the first device is less than a distance threshold of 1, the capacitance value of the touch sensor in the area of ​​the second device closer to the screen of the first device will increase. Therefore, the second device can determine its corresponding target position on the screen of the first device based on the capacitance value in the touch sensor. This distance threshold of 1 is the critical distance at which the touch sensor in the first device can detect a touch signal. That is, when the distance between the second device and the first device exceeds this distance threshold of 1, the capacitance value of the touch sensor in the first device will not change due to the first device.

[0212] In this embodiment of the application, the top of the second device being close to the screen of the first device may include: the top of the second device being in contact with the screen of the first device (for example, the distance between the top of the second device and the screen of the first device is 0), or the top of the second device not being in contact with the screen of the first device and hovering above the screen (for example, the distance between the top of the second device and the screen of the first device is distance 1, which is less than a distance threshold 1).

[0213] In this embodiment, the area where the capacitance of the screen of the first device changes due to the contact (or light contact) of the top of the second device with the screen of the first device is referred to as the contact area. Similarly, the area where the capacitance of the screen of the first device changes due to the proximity of the second device when the top of the second device is not in contact with the screen of the first device is referred to as the hovering area.

[0214] First, the first device can determine the hovering or contact area of ​​the second device (this hovering or contact area is the target area mentioned above) by measuring the capacitance value in the touch sensor. Then, the first device can determine the center point or centroid within this hovering or contact area. This target location is the center point or centroid of the hovering area. For details on how the first device determines the hovering or contact area of ​​the second device on the screen of the first device based on the capacitance value collected by the touch sensor, please refer to the description in Figure 5 above; it will not be repeated here.

[0215] Optionally, in one possible implementation, when the first device collects the capacitance value via the touch sensor, it can initially use a lower sampling rate or scanning frequency to obtain the capacitance value. When it determines that a device is near the screen of the first device based on the capacitance value collected by the touch sensor, the first device then uses a higher sampling rate or scanning frequency to obtain the capacitance value again. Finally, based on the capacitance value obtained at the higher sampling rate or scanning frequency, the target position of the second device on the screen of the first device is determined.

[0216] In this way, using a lower sampling rate or scanning frequency when no nearby device is detected saves power. Once a nearby device is detected, using a higher sampling rate or scanning frequency allows for more accurate acquisition of capacitance changes in the device's hovering area. This enables more accurate localization of the device's hovering or contact area, and the target location can then be determined based on this hovering or contact area.

[0217] S603. The second device detects a change from a moving state to a stationary state.

[0218] The user can hold the second device, move it toward the first device, and hover or lightly touch the screen surface of the first device. The top of the second device is close to the screen of the first device.

[0219] The second device can detect when it changes from a moving state to a stationary state. For example, the second device may include a motion sensor (e.g., an accelerometer, a gyroscope, etc.), which can collect data such as changes in acceleration, center of gravity, direction of motion, and angular velocity. The second device can determine the change from a moving state to a stationary state based on these changes in acceleration, angular velocity, etc.

[0220] S604. The second device sends a first message to the first device, the first message instructing the second device to change from a moving state to a stationary state.

[0221] The second device can send a first message to the first device, which can be used to instruct the second device to change from a moving state to a stationary state.

[0222] Optionally, the second device sends the first information to the first device only when the second device changes from a moving state to a stationary state, and the duration of the stationary state is greater than the first duration, that is, when the second device is in a hovering state. In this way, it can avoid accidentally triggering the first device to execute the following step S606a or the following step S606b.

[0223] In this embodiment of the application, steps S603 and S604 can be executed before step S602, after step S602, or simultaneously with step S602. This embodiment of the application does not limit this.

[0224] S605. The first device establishes a second connection with the second device.

[0225] When the first device detects the target area of ​​the second device on the screen of the first device, and the second device determines that it has changed from a moving state to a stationary state, the first device can also establish a second connection with the second device.

[0226] In some examples, the transmission rate and power consumption of the second connection can be higher than that of the first connection. That is, the second device and the first device can quickly transmit data and interact in real time through the second connection. For example, the second connection could be HarmonyLink (HML), or Wi-Fi Direct, etc.

[0227] In some examples, the first connection can be called a weak connection, and the second connection can be called a strong connection.

[0228] Optionally, the first device and the second device may not establish a second connection and can interact directly through the first connection. That is, step S605 is an optional step.

[0229] Subsequently, the first device and the second device can interact through either the second connection or the first connection. In this embodiment, there are two scenarios for the first device and the second device to interact.

[0230] Scenario 1: The target location on the screen of the first device has content, and the first device executes step S606a below. Scenario 2: The target location on the screen of the first device has no content, and the first device executes steps S606b-S607 below.

[0231] S606a. Based on the target location, the first device sends the first content to the second device.

[0232] First, the first device can determine whether there is content at the target location. For example, when the target location is within the area where content 1 is located on the screen, or when the distance between the target location and the area where content 1 is located on the screen is less than a distance threshold 2, the first device can determine that the target location has content, and that content is content 1. Content 1 may include, but is not limited to, folders, tables, documents, images, application icons, videos, audio, animations, etc.

[0233] If content 1 is present at the target location, the first device can send the first content to the second device.

[0234] In some examples, if content 1 is an app icon, the first content sent from the first device to the second device can be the app's identifier (e.g., the app's name or ID) or the app's installation package. For example, as shown in FIG. 7, the screen of computer 200 can display a user interface 700. This user interface 700 can include icons 701 for the computer application, 702 for document 1, 703 for the recycle bin application, 704 for the video application, 705 for picture 1, and 706 for table 1. This user interface 710 can be referred to as the desktop or main interface of computer 200. Mobile phone 100 can display the user interface 710, which can include icons for multiple applications (e.g., icons for email applications, gallery applications, and music applications, etc.). This user interface 710 can be referred to as the desktop or main interface of mobile phone 100. When a user wants mobile phone 100 to access the video application on computer 200, the user can hold mobile phone 100 and move it towards computer 200, then hover it close to the screen of computer 200. The target location of mobile phone 100 on computer 200 can be location 1 as shown in Figure 7. An icon 704 for a video application is located at location 1. When mobile phone 100 changes from a moving state to a stationary state, hovering above or touching (or lightly touching) the screen of computer 200, and computer 200 determines that the icon 704 of the video application is within the hovering or touching area of ​​mobile phone 100, or the distance between the icon 704 and the hovering or touching area of ​​mobile phone 100 is less than a threshold 2, computer 200 can send the installation package of the video application to mobile phone 100, or send the identifier of the video application to mobile phone 100. After receiving the installation package, mobile phone 100 can install the application. Alternatively, after receiving the identifier of the video application, mobile phone 100 can download the installation package of the video application according to the identifier. Mobile phone 100 can then install the video application. For example, when the mobile phone 100 installs the video application, the mobile phone 100 can display the user interface 720 shown in FIG7, which may include the icon 721 of the video application.

[0235] Optionally, in some other examples, if content 1 is an image file, the first device can send the image to the second device. After receiving the image, the second device can display it. Alternatively, after receiving the image, the second device can save it in its gallery. For example, as shown in Figure 8, the computer 200 can display a user interface 700. The user interface 700 can be described in Figure 7 above and will not be repeated here. The mobile phone 100 can display a user interface 710, which can also be described in Figure 7 above and will not be repeated here. When a user wants the mobile phone 100 to access image 1 from the computer 200, the user can hold the mobile phone 100 and move it towards the computer 200, then hover or touch (or, as described by tapping) it on the screen of the computer 200. The target position of the mobile phone 100 on the computer 200 can be position 2 as shown in Figure 8. The icon 705 for image 1 is located at position 2. When the mobile phone 100 changes from a moving state to a stationary state, and the computer 200 determines that the hovering or contact area of ​​the mobile phone 100 is position 2, the computer 200 can send image 1 to the mobile phone 100. After receiving image 1, the mobile phone 100 can display image 1. As shown in the user interface 720 displayed by the mobile phone 100 in Figure 8, the mobile phone 100 can display image 1 in the user interface 720.

[0236] Optionally, after receiving the image 1, the mobile phone 100 can also save the image 1 to its gallery.

[0237] Alternatively, in some examples, if a user wants their mobile phone 100 to access an open image (Image 1) displayed on computer 200, the user can move their mobile phone 100 towards computer 200 and then hover or touch (or tap) the area where the open image 1 is located. When the mobile phone 100 comes to a standstill, and computer 200 determines that the hovering or touching area of ​​the mobile phone 100 is the area where the open image 1 is located, computer 200 can send image 1 to mobile phone 100. Mobile phone 100 can then display image 1 on its current screen or save it to its gallery. When the user opens the gallery on mobile phone 100, a thumbnail of image 1 can be displayed in the gallery.

[0238] In other words, users simply need to hover the second device over the content they want to access on the first device, and the second device can then retrieve the desired content from the first device. This achieves an interactive effect where the second device "sucks" content away from the first device. During the interaction, users only need to move the second device and hover it near the screen of the first device; the operation is simple and enhances the user experience.

[0239] In one possible implementation, if the distance between the target location and multiple contents displayed on the screen of the first device is less than a distance threshold of 2, then the first device can select the content with the smallest distance from the target location as the target content, that is, the content sent to the second device.

[0240] Optionally, in some feasible examples, the content of the target area in the first device (e.g., the icon 704 of the video application shown in FIG. 7, and the icon 705 of picture 1 shown in FIG. 8) is highlighted, for example, or the color is darkened, etc. This application embodiment does not limit the method of highlighting the content of the target area.

[0241] S606b. Based on the target location, the first device sends a first instruction to the second device.

[0242] The first device can determine that there is no content at the target location. Specifically, the first device can determine that there is no content at the target location when the target location is not within an area containing content, or when the distance between the target location and the area containing the content displayed on the screen of the first device is greater than a distance threshold of 2.

[0243] If there is no content at the target location, the first device can send a first instruction to the second device. This first instruction can instruct the second device to send second content. The second content can be the content currently displayed on the second device or its user interface. The content currently displayed on the second device includes, but is not limited to, folders, tables, documents, pictures, application icons, videos, audio, animations, etc.

[0244] S607. Based on the first instruction, the second device sends the second content to the first device.

[0245] Based on the first instruction, the second device can send the second content to the first device.

[0246] In some examples, the second content can be an image displayed on a second device. The second device can send the image to the first device. After receiving the image, the first device can display it. Alternatively, after receiving the image, the first device can save it without displaying it. Or, the first device can receive the image, display it, and save it simultaneously.

[0247] For example, as shown in Figure 9, computer 200 currently displays a user interface 700, which can be referred to in the description of Figure 7 and will not be repeated here. Mobile phone 100 displays an image display interface 900. This image display interface 900 can display image 2. When the user wants image 2 displayed on mobile phone 100 to be quickly sent to computer 200, the user can move mobile phone 100 toward the screen of computer 200 and hover it over an area of ​​the screen that has no content. When mobile phone 100 changes from a moving state to a hovering state, and computer 200 determines that the target position of mobile phone 100 on the screen of computer 200 (i.e., position 3 in Figure 9) has no content, computer 200 can send a command to mobile phone 100, which can instruct mobile phone 100 to send the displayed image 2 to computer 200. Based on the command from computer 200, mobile phone 100 can send image 2 to computer 200. Computer 200 can then display image 2. For example, as shown in the user interface 910 of computer 200 in FIG9, computer 200 can display an icon 911 for image 2. When the user clicks the icon 911, computer 200 can display the content of image 2. Alternatively, computer 200 can directly display the content of image 2 in the user interface 910.

[0248] In some examples, the second content can be a user interface displayed on a second device. The second device can send the displayed user interface to the first device. After receiving the user interface, the first device can display it. That is, it achieves the effect of projecting the user interface from the second device onto the first device.

[0249] For example, as shown in Figure 10, computer 200 currently displays a user interface 700, which can be referred to in the description of Figure 7 and will not be repeated here. Mobile phone 100 displays a user interface 710, which can also be referred to in the description of Figure 7 and will not be repeated here. When a user wants the user interface displayed on mobile phone 100 to be quickly projected onto computer 200, the user can move mobile phone 100 toward the screen of computer 200 and hover it over an area of ​​the screen that is empty. When mobile phone 100 changes from a moving state to a hovering state, and computer 200 determines that the hovering area of ​​mobile phone 100 is empty, computer 200 can send a command to mobile phone 100, which can instruct mobile phone 100 to project the displayed user interface 710 onto computer 200. Based on the command from computer 200, mobile phone 100 can project user interface 710 onto computer 200. Computer 200 can then display user interface 710. For example, as shown in FIG10, the user interface 1000 of the computer 200 displays a window 1001, the content of which is the same as the content of the user interface 710 of the mobile phone 100.

[0250] In other words, users only need to hover the second device over an area of ​​the first device that has no content, and the second device can project its user interface onto the first device. Alternatively, the first device can retrieve the content displayed on the second device. This achieves the effect of the second device quickly projecting its user interface onto the first device, or it can achieve the interactive effect of the first device "sucking" content from the second device. During the interaction, users only need to move the second device and hover it near the screen of the first device; the operation is simple and can improve the user experience.

[0251] In this embodiment of the application, the interaction process between the first device and the second device is not limited to scenarios 1 and 2 described above. It also includes the following specific scenarios:

[0252] Scenario 3: The first device can display an open document, PowerPoint presentation, chat interface, or memo page. The second device can display any of the following content: images, videos, tables, or animations. When the second device is brought close to the first device, the images, videos, tables, animations, etc., displayed on the second device can be precisely inserted into the document, PowerPoint presentation, chat interface, or memo page displayed on the first device. The insertion position is the corresponding target position of the second device on the screen of the first device.

[0253] For example, computer 200 may display an open document, and mobile phone 100 may display an image. The user can bring mobile phone 100 close to the target area in the open document displayed on computer 200 where the image needs to be inserted. Mobile phone 100 can hover over the target area. Then, mobile phone 100 can send the image to computer 200. Computer 200 can then insert the image into the target area. In this way, the user only needs to bring mobile phone 100 close to the screen of computer 200 to accurately insert images into the document on computer 200.

[0254] Scenario 4: The first device can display an image editing interface, video editing interface, drawing interface, etc., including both a content display area and a control window. This control window can include one or more controls for editing or manipulating the content displayed in the content display area. The second device can display the desktop or any other content. When the second device is near the first device, the first device can send the control window to the second device. The user can then use this control window on the second device to control, edit, or draw content within the content display area.

[0255] Optionally, after the first device sends the control window to the second device, the control window may still be displayed on the first device.

[0256] For example, computer 200 displays a video editing interface, which includes a control window 1. This control window 1 includes multiple controls for editing the video displayed in the video editing interface. When the top of mobile phone 100 is close to the control window 1 displayed on the screen of computer 200, computer 200 can send the control window 1 to mobile phone 100. Mobile phone 100 can then display the control window 1. The user can operate within the control window 1 displayed on mobile phone 100; for example, clicking the "Add Music" control and selecting the music to add. In response to the user's operation, mobile phone 100 can instruct computer 200 to add music. Computer 200 can then add music to the currently displayed video.

[0257] Scenario 5: Both the first and second devices can display any content (both devices display content). When the second device approaches the first device, either the second device or the first device can display an option box or a transition area. The user selects content to be transmitted from the first device to the second device, or vice versa, through this option box or transition area.

[0258] For example, if there is content at position 1 on the computer 200 screen, and the mobile phone 100 also displays one or more of the following: images, videos, documents, spreadsheets, and animations, then the mobile phone 100 can display an options box. This options box can be used to allow the user to make an active selection. The user can choose to send the content at position 1 on the computer 200 screen to the mobile phone 100, or the user can choose to send the content displayed on the mobile phone 100 to the computer 200.

[0259] For example, if mobile phone 100 is hovering near position 1 on the screen, and computer 200's screen displays a desktop, but position 1 on the desktop is empty, and multiple pieces of content are displayed in split-screen mode on mobile phone 100, an option box can also be displayed on mobile phone 100. The user can use this option box to select the content that mobile phone 100 will send to computer 200. For instance, this option box may include identifiers of the multiple pieces of content displayed in split-screen mode on mobile phone 100. The user can select the identifier of the content they want to send to computer 200, and in response to this user action, mobile phone 100 can send the corresponding content to computer 200.

[0260] For example, computer 200 may display an open folder containing icons of multiple files. Mobile phone 100 may display a gallery containing thumbnails of multiple photos. When mobile phone 100 is brought close to computer 200, an icon for file 1 appears on the corresponding target location on the computer 200 screen. Mobile phone 100 may display a scrolling area containing the icon for file 1. When the user drags the icon for file 1 out of this scrolling area, computer 200 can send file 1 to mobile phone 100. The user can also drag thumbnails displayed on mobile phone 100 into this scrolling area; in response to this user action, mobile phone 100 can send the photo corresponding to the dragged thumbnail to computer 200.

[0261] In this embodiment, a user can bring a second device close to a first device and then transfer a file (document, image, video, app) corresponding to an icon on the first device, an already opened file (image), or part or all of an application interface from the first device to the second device. Alternatively, the second device can precisely insert images, videos, animations, tables, etc., displayed on the second device into an open file, an open memo page, an open PowerPoint presentation, or an open chat interface displayed on the first device. In this embodiment, when the second device is close to the first device, the first device can determine whether it is sending content to the second device or vice versa. Alternatively, the user's operation can determine whether the first device is sending content to the second device or vice versa.

[0262] Furthermore, in one possible implementation, when the first device determines whether it is sending content to the second device or vice versa, the first device can determine this based on the target location of the second device on the first device and / or the currently displayed interface content on the first device. For example, in scenarios 1 and 2 above, if the first device displays a desktop and the second device displays content at the target location corresponding to the first device, then the first device can send the first content to the second device. If the first device displays a desktop and the second device does not display content at the target location corresponding to the first device, then the second device can send the second content to the first device. As another example, if the first device displays an open file and the second device displays images, videos, animations, tables, etc., the second device can send the images, videos, animations, tables, etc. displayed on the second device to the first device. The first device can then insert the received images, videos, animations, tables, etc., into the open file.

[0263] It is also understood that, in the embodiments of this application, the method by which the first device and the second device determine whether the first device sends content to the second device or the second device sends content to the first device may be changed as the system is upgraded. This application does not limit this aspect.

[0264] In other examples, the first device can also determine whether it is the first device sending the first content to the second device or the second device sending the second content to the first device based on whether the edge of the second device's screen closest to the first device is the longer edge or the shorter edge. For example, if the edge of the second device's screen closest to the first device is the longer edge, then the second device sends the second content to the first device. If the edge of the second device's screen closest to the first device is the shorter edge, then the first device sends the first content to the second device.

[0265] The interaction method provided in this application allows users to simply hover the second device over the content they want to access on the first device, enabling the second device to retrieve that content. This achieves an interactive effect where the second device "sucks" content from the first device. Alternatively, users can hover the second device over an area of ​​the first device where there is no content, allowing the second device to project its user interface onto the first device. Or, the first device can retrieve the content displayed on the second device. This allows the second device to quickly project its user interface onto the first device, or to "suck" content from the second device. During the interaction, users only need to move the second device and hover it near the screen of the first device; the operation is simple and improves the user experience.

[0266] Optionally, in one possible example, the first device only needs to determine the target area corresponding to the second device within the first device, without needing to determine the target location based on that target area. When the first device determines that there is first content within the target area, it can send the first content within the target area to the second device. When the first device determines that there is no content within the target area, it can instruct the second device to send the second content from the second device to the first device.

[0267] Furthermore, in one possible implementation, when the first device determines that multiple pieces of content exist within the target area, the first device can send all of the multiple pieces of content to the second device. Alternatively, the first device can select a first piece of content from the multiple pieces of content and send it to the second device. The first piece of content is the content selected by the user among the multiple pieces of content, or the content that occupies the largest area of ​​the target area among the multiple pieces of content, or the content located at the center point or centroid of the target area. This application does not limit this.

[0268] Figure 11A is a hardware structure diagram of the first device provided in an embodiment of this application. As shown in Figure 11A, the first device may include a touch sensor 1101, a display screen 1102, a touch chip 1103, a graphics processing unit (GPU) 1104, an application processor 1105, and a communication module 1106, etc.

[0269] It is understood that the structure illustrated in this embodiment does not constitute a specific limitation on the first device. In other embodiments, the first device may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0270] The touch sensor 1101 can be used to detect touch operations or capacitive signals generated when another device approaches the touch sensor 1101, and upload the capacitive signals to the touch chip 1103. The touch sensor 1101 can be the touch sensor 202 shown in Figure 3. For details, please refer to the description of the touch sensor 202 above, which will not be repeated here.

[0271] Touch sensor 1101 can be set on display screen 1102. Touch sensor 1101 and display screen 1102 together form a touch screen, also known as a "touch screen".

[0272] The display screen 1102 may be the display screen of the screen 201 shown in Figure 3. Please refer to the description of the display screen in the screen 201 above, which will not be repeated here.

[0273] The touch chip 1103 can determine the capacitance signals that meet the touch conditions based on all capacitance signals on the capacitance matrix detected by the touch sensor 1101. For example, the touch condition can be that the capacitance signal strength is greater than a strength threshold of 1. In some embodiments of this application, the touch chip 1103 reports the capacitance signals that meet the touch conditions and the coordinates of the capacitance signals to the application processor 1105. Based on the coordinates of the aforementioned capacitance signals, the application processor 1105 can invoke a device identification method to determine another device, such as a second device, that is close to the display screen 1102 of the first device, and its corresponding hovering area on the display screen 1102.

[0274] Application processor 1105 can also send first content to the second device based on the determined hovering area, or instruct the second device to send second content to the first device. The first and second content are described above and will not be repeated here.

[0275] The first device can realize display functions through GPU 1104, display screen 1102, and application processor 1105. GPU 1104 is a microprocessor for image processing, connected to display screen 1102 and application processor 1105. GPU 1104 is used to perform mathematical and geometric calculations for graphics rendering.

[0276] The first device can establish a communication connection with another device (e.g., the second device) through the communication module 1106. Furthermore, the first device can also send content to and receive content sent by the other device through the communication module 1106.

[0277] Optionally, a microprocessor may be present in the touch sensor 1101. This microprocessor can be used to determine the target area on the display screen 1102 of another device, such as a second device, that is close to the first device, based on all the capacitance signals on the capacitance matrix detected by the touch sensor 1101.

[0278] Optionally, the microprocessor can work with the application processor 1105 to determine another device, such as a second device, near the display screen 1102 of the first device, within a corresponding hovering area on the display screen 1102. For example, the microprocessor can generate a grayscale image by generating all capacitance signals on the capacitance matrix detected by the touch sensor 1101. This grayscale image is then sent to the application processor 1105, which can process the grayscale image and, based on it, determine the target area on the display screen 1102 for another device, such as a second device, near the display screen 1102 of the first device.

[0279] In this embodiment, the software system of the first device may adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This embodiment uses the layered architecture Android system as an example to illustrate the software structure of the first device.

[0280] Referring to Figure 11B, which exemplarily illustrates the hardware and software architecture of the first device provided in this embodiment of the application, the first device can determine the hovering area of ​​the second device on the screen of the first device by detecting the capacitance value through a touch sensor, and determine the content of the interaction between the first device and the second device based on the hovering area. With simple operations, the user can enable the first device to "absorb" content from the second device, or the second device to "absorb" content displayed on the first device.

[0281] As shown in Figure 11B, this hardware and software architecture diagram includes a hardware layer and a software framework for the first device. The hardware layer may include a touch sensor. This touch sensor can be used to detect the capacitance value of the capacitor array within the touch sensor. For details, please refer to the descriptions of touch sensor 202 or touch sensor 1101 above; they will not be repeated here.

[0282] The software framework of the first device can be divided into several layers using a layered architecture, each with a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments, the Android system can be divided from top to bottom into an application framework layer, a hardware abstraction layer (HAL) layer, and a kernel layer. Wherein:

[0283] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0284] The kernel layer is the layer between hardware and software. It can contain display drivers, sensor drivers, touch chip drivers, and may also include camera drivers, audio drivers, and so on. The HAL layer and the kernel layer can respond to functions called by the application framework layer and perform corresponding operations.

[0285] In some embodiments of this application, the application framework layer may include a screen position determination module. The HAL layer may include a touch layer process and a device identification module. The kernel layer may include a touch chip driver. The touch sensor may periodically collect the capacitance value of the capacitor array in the touch sensor and send the collected capacitance value to the touch chip driver in the kernel layer. The touch chip driver may send the received capacitance value to the touch screen process in the HAL layer. The touch screen process may call the device identification module in the HAL layer. The device identification module may generate a grayscale image based on the capacitance value and determine the target feature region in the grayscale image. The target feature region is the hovering area of ​​another device close to the screen of the first device on the screen of the first device. For details about the target feature region, please refer to the description of the target feature region 5021 in Figure 5 above, which will not be repeated here. The device identification module may determine the coordinates of the center point or centroid of the target feature region. Then, the device identification module may upload the coordinates of the center point or centroid to the screen position determination module in the application framework layer. The screen position determination module can convert the center point or centroid into screen coordinates, and then determine whether there is content on the hovering area of ​​the second device's screen on the first device based on these screen coordinates. If there is content, the first device sends the first content to the second device; if there is no content, the first device instructs the second device to send the second content to the first device.

[0286] Figure 12A is a hardware structure diagram of the second device provided in an embodiment of this application. As shown in Figure 12A, the second device may include a processor 1201, a display screen 1202, a communication module 1203, and a sensor module 1204, etc. The sensor module 1204 may include an accelerometer 1204A, a gyroscope 1204B, and a distance sensor 1204C, etc.

[0287] It is understood that the structure illustrated in this embodiment does not constitute a specific limitation on the second device. In other embodiments, the second device may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0288] Processor 1201 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. The different processing units may be independent devices or integrated into one or more processors.

[0289] The controller can serve as the nerve center and command center of an electronic device. Based on the instruction opcode and timing signals, the controller generates operation control signals to control the fetching and execution of instructions.

[0290] The processor 1201 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 1201 is a cache memory. This memory can store instructions or data that the processor 1201 has just used or is recurring. If the processor 1201 needs to use the instruction or data again, it can directly retrieve it from the memory. This avoids repeated accesses, reduces the waiting time of the processor 1201, and thus improves the efficiency of the system.

[0291] In some embodiments of this application, the processor 1201 may include one or more interfaces. These interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0292] The I2C interface is a bidirectional synchronous serial bus that includes a serial data line (SDA) and a serial clock line (SCL).

[0293] The I2S interface can be used for audio communication. In some embodiments, the processor 1201 may include multiple I2S buses.

[0294] The PCM interface can also be used for audio communication, sampling, quantizing, and encoding analog signals.

[0295] The UART interface is a general-purpose serial data bus used for asynchronous communication. This bus can be a bidirectional communication bus. It converts the data to be transmitted between serial and parallel communication.

[0296] The MIPI interface can be used to connect the processor 1201 and the display screen 1202. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), etc. In some embodiments, the processor 1201 and the display screen 1202 communicate via the DSI interface to realize the display function of the electronic device.

[0297] The GPIO interface can be configured via software. The GPIO interface can be configured as either control signals or data signals.

[0298] A USB interface is an interface that conforms to the USB standard specification, specifically including Mini USB, Micro USB, and USB Type-C interfaces. A USB interface can be used to connect a charger to charge electronic devices, and also for transferring data between electronic devices and peripherals. It can also be used to connect headphones for audio playback. This interface can also be used to connect other electronic devices, such as AR devices.

[0299] It is understood that the interface connection relationships between the modules illustrated in this embodiment are merely illustrative and do not constitute a limitation on the structure of the electronic device. In other embodiments of this application, the electronic device may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0300] The display screen 1202 is used for the user interface of the second device, displaying images, videos, etc.

[0301] The second device can establish a communication connection with other devices (e.g., the first device) through the communication module 1203. In some examples, the second device can also receive first content sent by the first device through the communication module 1203. In other examples, the second device can also send second content to the first device through the communication module 1203.

[0302] The second device can determine whether it is in a hovering state using sensor module 1204. The accelerometer (A) 1204A measures the magnitude of acceleration in various directions (typically three axes). When the electronic device is stationary, it can detect the magnitude and direction of gravity. It can also be used to identify the attitude of the electronic device. The gyroscope (G) 1204B can be used to determine the motion attitude of the electronic device. The distance sensor 1204C is used to measure distance.

[0303] In some possible implementations, the second device can determine its motion state using data collected by the accelerometer 1204A and the gyroscope 1204B. Specifically, the second device can determine its acceleration using data collected by the accelerometer 1204A and its angular velocity using data collected by the gyroscope 1204B. The second device can determine its motion state (e.g., from motion to rest) by observing changes in acceleration and angular velocity.

[0304] Optionally, the second device can also determine the current attitude of the second device (e.g., the tilt angle of the second device, the orientation of the top of the second device, etc.) through the accelerometer sensor 1204A and the gyroscope sensor 1204B.

[0305] The second device can perform its functions and provide services to users through its running operating system. For example, the second device may, but is not limited to, running an operating system... Or other operating systems.

[0306] Figure 12B is a software structure block diagram of the second device according to an embodiment of this application.

[0307] A layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the system is divided into four layers, from top to bottom: the application layer, the application framework layer, the runtime and system libraries, and the kernel layer.

[0308] The application layer may include a series of application packages. These application packages may be developed by the manufacturer of the second device, the vendor of the operating system for the second device, or by a third-party application vendor; this application does not limit the scope of these developments.

[0309] As shown in Figure 12B, the application package may include applications (also referred to as apps) such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and SMS.

[0310] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0311] As shown in Figure 12B, the application framework layer may include a window manager, content provider, view system, phone manager, resource manager, notification manager, etc.

[0312] The window manager is used to manage windowed applications. It can retrieve screen size, determine the presence of a status bar, lock the screen, and capture screenshots, among other things.

[0313] Content providers store and retrieve data, making that data accessible to applications. This data may include videos, images, audio, made and received phone calls, browsing history and bookmarks, phone books, etc.

[0314] A view system includes visual controls, such as controls for displaying text and controls for displaying images. View systems can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text notification icon could include views for displaying text and views for displaying images.

[0315] The phone manager is used to provide communication functions for the second device. For example, it manages call status (including connection and hang-up).

[0316] The file explorer provides applications with various resources, such as localized strings, icons, images, layout files, video files, and more.

[0317] The notification manager allows applications to display notifications in the status bar. These can be used to convey informational messages and can disappear automatically after a short pause, requiring no user interaction. For example, the notification manager can be used to notify users of download completion or message alerts. The notification manager can also display notifications as icons or scrolling text in the top status bar, such as notifications from background applications, or as dialog-style notifications on the screen. Examples include displaying text messages in the status bar, emitting sounds, vibrating electronic devices, and flashing indicator lights.

[0318] The runtime consists of the core libraries and the virtual machine. The runtime is responsible for system scheduling and management.

[0319] The core library consists of two parts: one part is the functionalities that the programming language (e.g., Java) needs to call, and the other part is the system's core library.

[0320] The application layer and application framework layer run in a virtual machine. The virtual machine executes the programming files (e.g., .jave files) of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0321] System libraries can include multiple functional modules. For example: surface manager, media libraries, 3D graphics processing libraries (e.g., OpenGL ES), 2D graphics engines (e.g., SGL), etc.

[0322] The Surface Manager is used to manage the display subsystem and provides the fusion of two-dimensional (2D) and three-dimensional (3D) layers for multiple applications.

[0323] The media library supports playback and recording of various common audio and video formats, as well as still image files. It supports multiple audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG.

[0324] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0325] A 2D graphics engine is a graphics engine for 2D drawing.

[0326] The kernel layer is the layer between hardware and software. The kernel layer includes at least the display driver, camera driver, audio driver, sensor driver, and virtual card driver.

[0327] In the embodiments of this application, the interaction scenarios between the first device and the second device are not limited to scenarios 1 to 5 described above. More interaction scenarios between the first device and the second device will be described below.

[0328] In this embodiment, the first device and the second device can perform detection separately to identify when the second device moves or hovers near or touches the screen of the first device. When a communication connection is established between the first device and the second device (e.g., the second connection mentioned above), the first device or the second device can determine the interaction content between the first device and the second device based on the content currently displayed on the first device's screen (e.g., the content displayed in the target area where the second device hovers near or touches the screen of the first device) and the content currently displayed on the second device's screen.

[0329] The following description uses a computer 200 as the first device and a mobile phone 100 as the second device. The first device in this application includes, but is not limited to, electronic devices with touch panels (TP), such as computers, tablets, and mobile phones. The second device may include, but is not limited to, electronic devices such as mobile phones, tablets, watches, and wristbands. It is understood that the interaction scenarios between the computer 200 and the mobile phone 100 described in this application are also applicable to different electronic devices such as computers and watches, tablets and mobile phones.

[0330] Based on the content displayed on the current screen of computer 200 and the current screen of mobile phone 100, this application embodiment divides the interaction scenarios between mobile phone 100 and computer 200 into multiple categories.

[0331] Figure 13 illustrates different interactive scenarios. For example, as shown in Figure 13, the horizontal axis represents different display contents of the mobile phone 100 (e.g., from left to right, the horizontal axis represents the mobile phone displaying the desktop, the mobile application not supporting interaction, the mobile application (supporting interaction) and mobile components, etc.). The vertical axis represents different display contents of the computer 200 (e.g., from bottom to top, the vertical axis represents the computer displaying the desktop, displaying a container, the container containing items (the items support interaction), the container not supporting interaction or components, etc.).

[0332] The following section will introduce the relevant terms, such as component, container, mobile application supports interaction, mobile application does not support interaction, items within a container support interaction, and container does not support interaction.

[0333] 1. Component: A component is a basic module in a user interface that implements interactive functions. It includes visual elements such as buttons, forms, navigation menus, file icons, files, and application icons.

[0334] 2. Container: A container is capable of holding, organizing, and managing other objects (data, components, elements, or services). Containers can also be used to handle data acquisition, state management, and business logic processing for the other objects they contain. In some examples, the other objects contained within a container can also be referred to as items within the container. For instance, an application can be considered a container in a broad sense, and the various functional modules or components displayed within the application can all be considered items within the container. Containers can be nested; that is, one container can be nested within another.

[0335] 3. Mobile application supports interaction: This means that when the mobile phone 100 is hovered over or tapped on the screen of the computer 200, the application interface or controls displayed on the mobile phone 100 can be sent to the computer 200, or the content displayed on the computer 200 can be inserted into or saved to the application interface displayed on the mobile phone 100.

[0336] 4. Mobile application does not support interaction: This means that when the mobile phone 100 is hovered over or tapped on the screen of the computer 200, the application interface or controls displayed on the mobile phone 100 cannot send data to the computer 200, or the application interface displayed on the mobile phone 100 cannot receive content sent by the computer 200.

[0337] 5. Items within the container support interaction: This means that when the mobile phone 100 is hovered over or tapped on the screen of the computer 200, the items contained in the container displayed by the computer 200 (such as controls, files, etc. in the application interface or window) can be sent to the mobile phone 100.

[0338] 6. Container does not support interaction: This means that when the mobile phone 100 is hovered over or tapped on the screen of the computer 200, the container displayed in the computer 200 cannot send messages to the mobile phone 100, or the container cannot receive content sent by the mobile phone 100.

[0339] In some feasible examples, whether the applications, containers, and items within the containers of mobile phone 100 or computer 200 support interaction can be configured by the application or container developers, or can be set by the users of mobile phone 100 or computer 200. This application embodiment does not limit this aspect.

[0340] As shown in Figure 13, multiple interaction scenarios between mobile phone 100 and computer 200, and the corresponding interaction content for each scenario, include:

[0341] ①. Interaction Scenario 1: Mobile phone 100 displays a component (e.g., an open file or a selected file icon) on its current screen, and computer 200 also displays a component (e.g., an open file or a selected file icon) on its current screen. When mobile phone 100 hovers near or lightly touches the screen of computer 200, a pop-up window can be displayed on mobile phone 100. This pop-up window can prompt the user to select the file transfer direction on mobile phone 100. That is, the user can choose on mobile phone 100 whether to transfer the open file (or selected file) on mobile phone 100 to computer 200, or to transfer the open file (or selected file) on computer 200 to mobile phone 100.

[0342] Understandably, in interaction scenario 1, mobile phone 100 and computer 200 are not limited to displaying files; they can also display other content (e.g., tables, images, audio, video, etc.). That is, when both mobile phone 100 and computer 200 display content (content other than the desktop, such as documents, tables, images, audio, video, etc.), when mobile phone 100 hovers near or lightly touches the screen of computer 200, a pop-up window can appear on mobile phone 100. The user can choose in this pop-up window whether mobile phone 100 sends interactive content to computer 200 or computer 200 sends interactive content to mobile phone 100.

[0343] For example, as shown in FIG14, the screen of mobile phone 100 displays an image display interface 900, which includes image 2. The screen of computer 200 displays a user interface 1400, which includes table 1. When mobile phone 100 hovers near or touches the screen of computer 200, user interface 1410 can be displayed on mobile phone 100. User interface 1410 may include pop-up windows 1411 and 1412. Pop-up window 1411 may include a table 1 identifier 14111, control 14112, and control 14113. The user can click control 14112, and in response to the user's operation, computer 200 can send table 1 to mobile phone 100. The user can click control 14113 to choose whether mobile phone 100 should not receive table 1. In some examples, mobile phone 100's decision to not receive table 1 may include two scenarios. Scenario 1: Computer 200 sends Form 1 to mobile phone 100, but mobile phone 100 refuses to receive Form 1 in response to the user clicking control 14113. Scenario 2: Computer 200 does not send Form 1 to mobile phone 100.

[0344] The pop-up window 1412 may include an identifier 14121 for image 2, controls 14122 and 14123. The user can click on control 14122, and in response to this user action, mobile phone 100 can send image 2 to computer 200. The user can click on control 14123 to choose not to send control 14123 to computer 200.

[0345] In some examples, a user can choose to send image 2 from mobile phone 100 to computer 200, or choose to send form 1 from computer 200 to mobile phone 100. That is, the user can choose to click controls 14112, 14123, or 14113 and 14122. In other examples, a user can choose to send image 2 from mobile phone 100 to computer 200, and simultaneously choose to send form 1 from computer 200 to mobile phone 100. That is, the user can choose to click controls 14112 and 14122.

[0346] ②. Interaction Scenario 2: The mobile phone 100 has the application corresponding to the component installed, and the content displayed on the current screen of the computer 200 includes the component (the component is an open file or a selected file identifier). The file on the computer 200 is transferred to the mobile phone 100.

[0347] In other words, the screen of mobile phone 100 can display an application interface, while the screen of computer 200 can display components. The content of these components is not limited to files; it can also include other content (such as spreadsheets, images, audio, video, etc.). For example, mobile phone 100 can display a document application, while the screen of computer 200 can display an open or selected file. When mobile phone 100 hovers near or taps against the screen of computer 200, computer 200 can send the open or selected file to mobile phone 100. Mobile phone 100 can then open the file sent by computer 200 using the document application.

[0348] ③. Interaction Scenario 3: The computer 200 currently displays a container on its screen, and the container contains items. The computer 200 transmits the items in the container to the mobile phone 100.

[0349] In other words, the computer 200 currently displays a container containing an item. When the mobile phone 100 hovers near or lightly touches the screen of the computer 200, the computer 200 can send the item from the container to the mobile phone 100. The mobile phone 100 can then display the item.

[0350] For example, computer 200 can display an application interface for a drawing application, which includes one or more functional areas. The mobile phone 100 also has this drawing application installed. When the mobile phone 100 hovers near or touches the screen of computer 200 (e.g., hovering near or touching the interface of the drawing application or a functional area within the drawing application's interface on the screen of computer 200), the mobile phone 100 can display one or more functional areas of the drawing application from computer 200.

[0351] For example, as shown in Figure 15, mobile phone 100 displays a desktop, while computer 200 displays a user interface 1500. This user interface 1500 includes an application interface for a drawing application, which may include a color palette area. When mobile phone 100 hovers near or touches the screen of computer 200, mobile phone 100 can display a user interface 1510, which includes a color palette area 1511. In some examples, the drawing application is installed on mobile phone 100, and mobile phone 100 can render the color palette area 1511 locally. In other examples, the drawing application is not installed on mobile phone 100, and computer 200 can render the color palette area 1511 and send the rendered color palette area 1511 to mobile phone 100. In this way, the user can switch brush colors on mobile phone 100 and draw on computer 200.

[0352] In one possible implementation, when the mobile phone 100 displays the color palette function area, the computer 200 may not display the color palette function area. In this way, more area can be used to display the user's drawing on the computer 200.

[0353] Alternatively, when the mobile phone 100 displays the color palette function area, the computer 200 may still display the color palette function area. This application embodiment does not limit this.

[0354] For example, computer 200 displays a video conferencing interface that includes participant information. When mobile phone 100 hovers near or lightly touches the screen of computer 200, computer 200 can send the participant information from the video conferencing interface to mobile phone 100. Mobile phone 100 can then display this participant information. This allows the user to view the meeting content on computer 200 and the participant information on mobile phone 100. In some examples, mobile phone 100 may have video conferencing software installed. Computer 200 can send the participant information to mobile phone 100, and mobile phone 100 can render and display the participant information locally using the video conferencing software. In other examples, mobile phone 100 may not have the video conferencing software installed. Computer 200 can render the participant information and send the rendered information to mobile phone 100. Mobile phone 100 can then display the rendered participant information sent by computer 200.

[0355] In one possible implementation, while the mobile phone 100 displays the rendered participant information, the computer 200 may not display that participant information. This prevents the participant information from obscuring the main meeting content (e.g., documents, images, etc. shared during the meeting).

[0356] Alternatively, when mobile phone 100 displays the rendered attendee information, computer 200 can also display the attendee information. This application embodiment does not limit this to any particular method.

[0357] For example, computer 200 displays a game interface, which may include a game map. When mobile phone 100 hovers near or touches the screen of computer 200, computer 200 can send the game map from the game interface to mobile phone 100. Mobile phone 100 can then display the game map. In some examples, mobile phone 100 may have the game application installed. After receiving the instruction from computer 200 to display the game map, mobile phone 100 can render the game map locally using the game application and then display it. In other examples, mobile phone 100 may not have the game application installed. Computer 200 can render the game map and then send the rendered game map to mobile phone 100. Mobile phone 100 receives the rendered game map and displays it.

[0358] In one possible implementation, when the mobile phone 100 displays the game map, the computer 200 may not display the game map. This way, the content of the game interface displayed on the computer 200 will no longer be obscured by the game map.

[0359] Alternatively, when the mobile phone 100 displays the game map, the computer 200 may still display the game map. This application embodiment does not limit this.

[0360] ④. Interaction Scenario 4: A component is displayed on the current screen of mobile phone 100 (e.g., an open file or a selected file identifier), and a container is displayed on the current screen of computer 200. Mobile phone 100 transfers the file within the component to computer 200, and computer 200 can process the file within the container.

[0361] In other words, the content currently displayed on the screen of mobile phone 100 (not limited to files, but also including tables, pictures, audio, video, etc.) is displayed in a container on computer 200. This container includes, but is not limited to, the desktop and application interfaces (e.g., office software interfaces, email sending interfaces, document reading interfaces, etc.). When mobile phone 100 hovers near or lightly touches the screen of computer 200, mobile phone 100 can send the displayed content to computer 200. Computer 200 can then process the content within the displayed container. For example, if the content sent by mobile phone 100 is a file, and the container displayed by computer 200 is a file management interface, then computer 200 can add the file to that file management interface. Similarly, if the container displayed by computer 200 is an email sending interface, then computer 200 can add the file to the attachments of the email sending interface.

[0362] ⑤. Interaction Scenario 5: Mobile phone 100 is currently displaying the application interface, computer 200 is displaying the desktop, and mobile phone 100 is transferring the application interface to computer 200.

[0363] In other words, the phone 100 screen currently displays an application interface, while the computer 200 screen currently displays the desktop. When the phone 100 hovers near or lightly touches the screen of the computer 200, the phone 100 can send the application interface to the computer 200. The computer 200 can then display the application interface on its desktop.

[0364] ⑥. Interaction Scenario 6: The mobile phone 100 displays a component on its current screen (for example, the component is an open file or a selected file identifier), the computer 200 displays the desktop, the mobile phone 100 sends the component (for example, the open file) to the computer 200, and the computer 200 can add the file to the desktop.

[0365] In other words, the content currently displayed on the screen of mobile phone 100 (not limited to files, but also including tables, pictures, audio, video, etc.) is displayed as a desktop on computer 200. When mobile phone 100 hovers near or lightly touches the screen of computer 200, mobile phone 100 can send the displayed content to computer 200. Computer 200 can then add that content to its desktop.

[0366] ⑦. Interaction Scenario 7: The mobile phone 100 screen displays a desktop, and the computer 200 screen displays a component (for example, an open file or a selected file identifier). The computer 200 can transfer the file to the mobile phone 100.

[0367] In other words, mobile phone 100 is currently displaying its desktop, while computer 200 is displaying content. The content displayed on computer 200 may include, but is not limited to, files, pictures, videos, audio, application interfaces, etc. When mobile phone 100 hovers near or lightly touches the screen of computer 200, computer 200 can transmit the content it is displaying to mobile phone 100.

[0368] ⑧. Interaction Scenario 8: The desktop is displayed on the screen of mobile phone 100, and the container is displayed on the screen of computer 200. Computer 200 transfers the items in the container to mobile phone 100.

[0369] In other words, the screen of mobile phone 100 displays a desktop, while the screen of computer 200 displays a container. This container can be the application interface of an application installed on computer 200, which may include one or more functional areas. When mobile phone 100 hovers near or touches the screen of computer 200, computer 200 can send one or more functional areas of the application interface, or send the application interface itself, to mobile phone 100.

[0370] In some examples, the content displayed on computer 200 may be the application interface of a third-party application. When mobile phone 100 does not support the third-party application, but the user wants to view the application interface of the third-party application on mobile phone 100, the user can hover mobile phone 100 close to or tap it against the screen of computer 200. When mobile phone 100 hovers close to or taps it against the screen of computer 200, computer 200 transmits the application interface of the third-party application to mobile phone 100. Mobile phone 100 can then display the application interface of the third-party application. For example, as shown in Figure 16, mobile phone 100 displays a user interface 710, which can be referred to as the desktop of mobile phone 100. Computer 200 displays a user interface 1600, which includes the application interface of the third-party application. When mobile phone 100 hovers close to or taps it against the screen of computer 200, computer 200 transmits the application interface of the third-party application to mobile phone 100. After receiving the application interface of the third-party application, the mobile phone 100 can display the user interface 1610, which may include the application interface 1611 of the third-party application.

[0371] In this way, when mobile phone 100 does not support the third-party application and cannot open its interface, the user can have computer 200, which supports the application, open it first. Then, by hovering or lightly touching the screen of computer 200, the computer 200 can send the displayed interface of the third-party application to mobile phone 100. This allows mobile phone 100 to display the interface of a third-party application that it originally did not support.

[0372] For example, as shown in FIG17, mobile phone 100 displays user interface 710, which can be referred to as the desktop of mobile phone 100. Computer 200 displays user interface 1700, which can include application interface 1701 of a music performance application. The application interface 1701 can include a performance content display area 1702 and a performance function area 1703. The performance content display area 1702 can be used for the user to play notes. The performance function area 1703 can be used to provide user-operable piano keys. Displaying the application interface 1701 can be referred to as a container, and the performance function area 1703 can be referred to as an item within the container.

[0373] When the mobile phone 100 hovers near or lightly touches the screen of the computer 200, the computer 200 can send the application interface of the music performance application to the mobile phone 100. After receiving the application interface of the music performance application, the mobile phone 100 can display the user interface 1710. The user interface 1710 may include a performance content display area 1712 and a performance function area 1713.

[0374] In this way, mobile phone 100 can act as a clone of computer 200, displaying the same application interface as the current interface of computer 200. Users can play simultaneously on both mobile phone 100 and computer 200.

[0375] In other examples, when the mobile phone 100 hovers near or touches the screen of the computer 200, the computer 200 can send the performance function area 1703 to the mobile phone 100. After the mobile phone 100 receives the performance function area 1703, it can display the performance function area 1703 (not shown in the figure).

[0376] ⑨. Interactive Scenario 9: Mobile phone 100 displays the desktop, computer 200 displays the desktop, and the desktop of mobile phone 100 can be projected onto computer 200.

[0377] In other words, both mobile phone 100 and computer 200 display their desktops. When mobile phone 100 hovers near or lightly touches the screen of computer 200, the desktop of mobile phone 100 can be projected onto computer 200. For details, please refer to the description of Figure 10 above, which will not be repeated here.

[0378] In other interactive scenarios, when a user wants computer 300 to obtain interactive content (e.g., files, images, audio, video, spreadsheets, etc.) from computer 200, the user can first hover mobile phone 100 near or lightly touch the screen of computer 200 to allow mobile phone 100 to obtain the interactive content from computer 200. Then, the user can further hover mobile phone 100 near or lightly touch the screen of computer 300 to allow mobile phone 100 to send the obtained interactive content to computer 300.

[0379] For example, as shown in FIG18, mobile phone 100 can display user interface 710. Computer 200 can display user interface 700, which may include image 1. When mobile phone 100 hovers near or touches the screen of computer 200, computer 200 can send image 1 to mobile phone 100. Mobile phone 100 can display user interface 720, which may include image 1. For details regarding user interface 700, user interface 710, and user interface 720, please refer to the description in FIG8 above; they will not be repeated here. For the specific process by which mobile phone 100 obtains image 1 from computer 200 when mobile phone 100 hovers near or touches the screen of computer 200, please refer to the description in FIG8 above; they will not be repeated here.

[0380] When the mobile phone 100 hovers near or lightly touches the screen of the computer 300, the mobile phone 100 can send image 1 to the computer 300. After receiving image 1, the computer 300 can display a user interface 1800, which can include image 1.

[0381] In some other scenarios, when a user wants to send interactive content from mobile phone 100 to computer 200, the user first selects the interactive content on mobile phone 100, and then hovers or taps mobile phone 100 against the screen of computer 200. In response to the user's action, mobile phone 100 can send the selected interactive content to computer 200. For example, as shown in Figure 19, mobile phone 100 displays user interface 1900, which may include multiple images, including image 1901. The user can select image 1901 in user interface 1900. Computer 200 can display user interface 700. For details on user interface 700, please refer to the description of Figure 7 above; it will not be repeated here. When mobile phone 100 hovers or taps the screen of computer 200, mobile phone 100 can send image 1901 to computer 200. After receiving image 1901, computer 200 can display image 1901 in user interface 700.

[0382] In other scenarios, when a user wants to send interactive content from computer 200 to mobile phone 100, the user first selects the interactive content on computer 200, and then hovers or taps mobile phone 100 against the screen of computer 200. In response to the user's action, computer 200 can send the selected interactive content to mobile phone 100. For example, as shown in Figure 20, computer 200 can display user interface 700, which may include image 1. For details about user interface 700, please refer to the description of Figure 7 above; it will not be repeated here. Mobile phone 100 displays user interface 710, which can also be described in the description of Figure 7 above; it will not be repeated here. The user can select image 1 in the user interface 700 displayed on computer 200. When mobile phone 100 hovers or taps the screen of computer 200, computer 200 can send image 1 to mobile phone 100. After receiving image 1, mobile phone 100 can display user interface 720, which may include image 1. For details about the user interface 720, please refer to the description of Figure 7 above, which will not be repeated here.

[0383] In some other scenarios, after the mobile phone 100 interacts with the computer 200, the user can input an operation into the mobile phone 100 to trigger the computer 200 to enable touchscreen detection. For ease of description, the operation used to trigger the computer 200 to enable touchscreen detection will be referred to as the trigger operation below. In response to this trigger operation, the mobile phone 100 can send a second instruction to the computer 200. This second instruction is used to instruct the computer 200 to enable touchscreen detection. After receiving the second instruction, the computer 200 enables touchscreen detection, that is, the computer 200 can activate the touch sensor in the computer 200 and begin executing the above step S602.

[0384] For example, the triggering operation includes, but is not limited to, the user tapping the back cover of the phone 100, or the user holding the phone 100 in a circle (as shown in Figure 21). In some examples, the triggering operation can be customized by the user. This application embodiment does not limit the specific type of the triggering operation.

[0385] Alternatively, users can enable touchscreen detection in the computer's settings interface to achieve the above function.

[0386] In some scenarios, when mobile phone 100 interacts with computer 200, the top of mobile phone 100 needs to hover close to or lightly touch a fixed area on the screen of computer 200 in order for mobile phone 100 to interact with computer 200. As shown in Figure 22, there is a fixed area on the display screen of computer 200. This fixed area can be defined by the user or configured by the system of computer 200, and this embodiment of the application does not limit it.

[0387] In some examples, the fixed area can be determined according to the displayed content. Users can define interactive areas for some application interfaces on computer 200. Some application interfaces are defined as non-interactive areas. When an interactive application interface is displayed on computer 200, and the top of mobile phone 100 needs to hover close to or lightly touch the area of ​​the interactive application interface on the screen of computer 200 (e.g., the fixed area shown in Figure 22), mobile phone 100 can transfer content from mobile phone 100 to the interactive application interface area on computer 200, or mobile phone 100 can retrieve the content displayed in the interactive application interface area.

[0388] In some feasible examples, the fixed area is a physical area on the screen of computer 200. When mobile phone 100 hovers near or touches this fixed area, the capacitance value of the fixed area changes due to the proximity of mobile phone 100, and the change is greater than a preset threshold. When mobile phone 100 hovers near or touches other areas on the screen of computer 200 other than the fixed area, the capacitance value of other areas does not change, or the capacitance value of other areas changes, but the change is less than the preset threshold. When mobile phone 100 hovers near or touches this fixed area, mobile phone 100 can transmit content from mobile phone 100 to the application interface area of ​​interactive application on computer 200, or mobile phone 100 can obtain the content displayed in the application interface area of ​​interactive application. When mobile phone 100 hovers near or touches other areas on the screen of computer 200 other than the fixed area, mobile phone 100 and computer 200 do not perform subsequent interaction steps. That is, mobile phone 100 does not send interactive content to computer 200, and computer 200 does not send interactive content to mobile phone 100.

[0389] In other scenarios, when mobile phone 100 hovers near or lightly touches the screen of computer 200, windows, application interfaces, files, folders, desktop, etc. displayed on computer 200 can all be sent to mobile phone 100 as interactive content. That is, everything with boundaries displayed on computer 200 can be used as interactive content.

[0390] In some examples, computer 200 may display multiple windows. When mobile phone 100 hovers near or touches the screen of computer 200, computer 200 first determines the target position of mobile phone 100 on the screen of computer 200. Then, computer 200 can send the window at the target position or the window closest to the target position to mobile phone 100.

[0391] Optionally, in some examples, computer 200 may display multiple windows, some of which overlap. When mobile phone 100 hovers near or taps against the screen of computer 200, computer 200 can send the topmost window to mobile phone 100. For example, as shown in Figure 23, computer 200 may display window A and window B, with window B partially overlapping window A. When mobile phone 100 hovers near or taps against the screen of computer 200, computer 200 can send window B to mobile phone 100.

[0392] In some examples, computer 200 may display multiple controls. When mobile phone 100 hovers near or touches the screen of computer 200, computer 200 first determines the target position of mobile phone 100 on the screen of computer 200. Then, computer 200 can send the control at the target position or the control closest to the target position to mobile phone 100.

[0393] In some other examples, computer 200 may display multiple controls, some of which overlap. When mobile phone 100 hovers near or touches the screen of computer 200, computer 200 can send the topmost control to mobile phone 100. For example, as shown in Figure 24, computer 200 may display a window that includes control 1 and control 2, with control 2 partially overlapping control 1. When mobile phone 100 hovers near or touches the screen of computer 200, computer 200 can send control 2 to mobile phone 100.

[0394] In some examples, computer 200 may display multiple folders. For instance, as shown in Figure 25, computer 200 may display a window that includes folder 1 and folder 2. The user can select a folder on computer 200. As shown in Figure 26, the user can select folder 2 on computer 200. When mobile phone 100 hovers near or taps the screen of computer 200, computer 200 can send folder 2 to mobile phone 100.

[0395] Figure 27 is a software structure block diagram of the mobile phone 100 and computer 200 provided in this embodiment of the application. As shown in Figure 27, the software framework of the computer 200 can be divided into several layers by a layered architecture, each layer having a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments, the software system can be divided into an application layer, a system service layer, a hardware abstraction layer, and a kernel layer. Wherein:

[0396] The application layer may include a series of application packages. These application packages may be developed by the manufacturer of the second device, the vendor of the operating system for the second device, or by a third-party application vendor; this application does not limit the scope of these developments.

[0397] As shown in Figure 27, an application package can include applications (also known as applications) such as system settings, gallery, and file management.

[0398] In this embodiment of the application, the system settings may further include a setting item that can be used to enable the TP detection function of the computer 200.

[0399] The system service layer may include a collaborative framework and a multimodel sensor data platform (MSDP).

[0400] The computer 200 can sense surrounding devices through this collaborative framework. The computer 200 can send broadcast signals (e.g., Bluetooth broadcast) to the collaborative framework of the mobile phone 100, or receive broadcast signals sent by the mobile phone 100 through this collaborative framework. The collaborative framework of the computer 200 can perform distance measurement based on the sent or received broadcast signals to determine the distance between the computer 200 and the mobile phone 100. When the collaborative framework senses the presence of a device in the vicinity, and the distance to that device is less than or equal to a distance threshold of 1, the collaborative framework can determine that a screen interaction event has occurred.

[0401] A multi-model sensor data platform can provide an interface for sensing between multiple devices, shield the differences between different underlying sensors, and perform data fusion on data collected from multiple sensors.

[0402] Optionally, the software system of computer 200 may also include an input system and an interactive interface. This input system and interactive interface may be contained within either the system service layer or the hardware abstraction layer.

[0403] The hardware abstraction layer can include the touchscreen process. The kernel layer can include the touch chip driver.

[0404] In this embodiment, when the user enables the TP detection function of the computer 200 through settings, the collaborative framework can inform the touchscreen process that the computer 200 has enabled the TP detection function. When the collaborative framework detects a device in the vicinity, it can also inform the touchscreen process. The touchscreen process can then enable the detection state. The touch sensor of the computer 200 can periodically collect the capacitance value of the capacitor array in the touch sensor and send the collected capacitance value to the touch chip driver in the kernel layer. The touch chip driver can send the coordinates of the area with capacitance change to the collaborative framework. The collaborative framework can determine whether there is content displayed in the target area where the mobile phone 100 is hovering or lightly touching the screen of the computer 200 based on the coordinates of the area with capacitance change. The touch chip driver can then send the received capacitance value to the touchscreen process. The touchscreen process can then call the touchscreen sensing algorithm to determine the target area where the capacitance value has changed.

[0405] Computer 200 can also use this collaborative framework to determine whether mobile phone 100 is currently displaying content, and the type of content being displayed.

[0406] The interaction interface can be used to obtain information from the collaboration framework regarding whether a current setting is enabled and whether there are any screen interaction events, and then send status indication information to the touchscreen process. When it is determined that the current setting is enabled but there are no screen interaction events, the interaction interface can send status indication information 1 to the touchscreen process, which instructs the touchscreen process to enter the pre-detection state. When it is determined that the current setting is enabled and there are screen interaction events, the interaction interface can send status indication information 2 to the touchscreen process, which instructs the touchscreen process to enter the formal detection state. The frame rate in the pre-detection state is lower than the frame rate in the formal detection state.

[0407] As shown in Figure 27, the software framework of mobile phone 100 can be divided into several layers using a layered architecture, each with a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments, the software system can be divided into an application layer, a system service layer, a hardware abstraction layer, and a kernel layer. Wherein:

[0408] The application layer may include a series of application packages. These application packages may be developed by the manufacturer of the second device, the vendor of the operating system for the second device, or by a third-party application vendor; this application does not limit the scope of these developments.

[0409] As shown in Figure 27, an application package can include applications such as gallery and file management (also known as applications).

[0410] The system service layer may include a collaborative framework and a multi-model sensor data platform.

[0411] Mobile phone 100 can establish a communication connection with computer 200 through a collaborative framework, and send the motion status of mobile phone 100 (e.g., from motion to a state of being stationary).

[0412] A multi-model sensor data platform can provide an interface for sensing between multiple devices, shield the differences between different underlying sensors, and perform data fusion on data collected from multiple sensors.

[0413] The hardware abstraction layer may include a sensor hub. The sensor hub can determine the motion state of the mobile phone 100 based on data collected by sensors. For example, the sensor hub can determine the motion state and attitude of the mobile phone 100 based on acceleration and angular velocity data collected by an inertial measurement unit (IMU). For instance, the sensor hub can determine whether the mobile phone 100 is currently hovering or touching another device based on data collected by the IMU.

[0414] The kernel layer can include a sensor driver. This sensor driver can receive acceleration and angular velocity data acquired by the IMU and send these data to the sensor hub.

[0415] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0416] As used in the above embodiments, depending on the context, the term "when..." can be interpreted as meaning "if...", "after...", "in response to determining...", or "in response to detecting...". Similarly, depending on the context, the phrase "when determining..." or "if (the stated condition or event) is interpreted as meaning "if determining...", "in response to determining...", "when (the stated condition or event) is detected", or "in response to detecting (the stated condition or event)".

[0417] The various illustrative logic units and circuits described in the embodiments of this application can be implemented or operate the described functions using a general-purpose processor, digital information processor, application-specific integrated circuit (ASIC), field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof. The general-purpose processor can be a microprocessor; alternatively, it can also be any conventional processor, controller, microcontroller, or state machine. The processor can also be implemented using a combination of computing devices, such as a digital information processor and a microprocessor, multiple microprocessors, one or more microprocessors combined with a digital information processor core, or any other similar configuration.

[0418] The steps of the methods or algorithms described in the embodiments of this application can be directly embedded in hardware, a software unit executed by a processor, or a combination of both. The software unit can be stored in RAM, flash memory, ROM, EPROM, EEPROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium in the art. Exemplarily, the storage medium can be connected to the processor so that the processor can read information from and write information to the storage medium. Optionally, the storage medium can also be integrated into the processor. The processor and the storage medium can be disposed in an ASIC, which can be disposed in a first device or a second device. Optionally, the processor and the storage medium can also be disposed in different components of the first device or the second device.

[0419] It should be understood that in the various embodiments of this application, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0420] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive), etc.

[0421] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

Claims

1. An interaction method, characterized in that, Applied to an interactive system, the interactive system including a first device and a second device, the first device and the second device establishing a first communication connection, the first device including a screen, the screen including a touch sensor, the method including: The first device acquires a first signal through the touch sensor and determines the target area of ​​the second device on the screen based on the first signal. The target area is the area on the screen where the capacitance changes when the second device approaches the first device. The first device receives a first indication message sent by the second device, the first indication message being used to indicate a change in the motion state of the second device; The first device sends first content to the second device based on the target area.

2. The method according to claim 1, characterized in that, After the first device receives the first indication information sent by the second device, the method further includes: The first device receives the second content sent by the second device.

3. The method according to claim 2, characterized in that, The target area includes the area on the screen of the first device that the second device maps onto when the second device is hovered, or the contact area between the screen of the second device and the screen of the first device.

4. The method according to claim 3, characterized in that, The change in motion state includes: the motion state changing to a stationary state, and the duration of the second device being in the stationary state being greater than a first preset duration.

5. The method according to claim 4, characterized in that, The first device sends first content to the second device based on the target area, including: The first device determines that there is third content at the center point or centroid of the target area on the screen, and sends the first content to the second device.

6. The method according to claim 4, characterized in that, The first device sends first content to the second device based on the target area, including: The first device determines that there is third content in the target area on the screen, and sends the first content to the second device.

7. The method according to any one of claims 5 or 6, characterized in that, The third content includes any one of the following: application icon, video file icon, image file icon, document file icon, spreadsheet file icon, video, image, document, and spreadsheet. The first content includes any one of the following: application identifier, application installation package, video, image, document, and table.

8. The method according to claim 7, characterized in that, When the third content is the icon of the application, the first content is the identifier of the application or the installation package of the application; or, If the third content is the icon of the video file or the video itself, then the first content is the video; or, If the third content is the icon of the image file or the image itself, then the first content is the image; or, If the third content is the icon of the document file or the document itself, then the first content is the document; or, In the case where the third content is the icon of the table file or the table itself, the first content is the table.

9. The method according to claim 8, characterized in that, The step of determining the target area of ​​the second device on the screen based on the first signal specifically includes: The first device generates a grayscale image based on the first signal and determines the target feature region in the grayscale image, wherein the target feature region is the region in the grayscale image where the grayscale value of a pixel is higher than a grayscale threshold; The first device determines the target feature region as the target region corresponding to the second device on the screen.

10. The method according to claim 9, characterized in that, Determining the target feature region as the target region corresponding to the second device on the screen includes: If the target feature region satisfies a first condition and / or a second condition, the first device determines the target feature region as the corresponding target region of the second device on the screen; wherein... The first condition includes that the shape of the target feature region is the first shape; The second condition includes that the difference between the size of the target feature region and the size of the short side of the second device is less than a first threshold.

11. The method according to claim 2, characterized in that, Before the first device receives the second content sent by the second device, the method further includes: The first device sends a first instruction to the second device, the first instruction being used to instruct the second device to send the second content to the first device; The second device receives and responds to the first instruction, and sends the second content to the first device.

12. The method according to claim 11, characterized in that, The first device sends a first instruction to the second device, specifically including: The first device determines that there is no content at the center point or centroid of the target area, or that there is no content within a range less than a preset distance from the center point or centroid, and then sends the first instruction to the second device.

13. The method according to claim 11, characterized in that, The first device sends a first instruction to the second device, specifically including: The first device determines that there is no content in the target area and sends a first instruction to the second device.

14. The method according to any one of claims 8-13, characterized in that, Before the first device sends the first content to the second device based on the target area, the method further includes: The first device establishes a second communication connection with the second device, and the transmission rate of the second communication connection is higher than that of the first communication connection; The first device sends first content to the second device based on the target area, including: The first device sends the first content to the second device via the second communication connection based on the target area.

15. The method according to claim 14, characterized in that, The second content includes any one of the user interface, images, videos, tables, documents, and animations currently displayed on the second device.

16. An interaction method applied to a first device, the first device including a screen, the screen including a touch sensor, the method comprising: The first device acquires a first signal through the touch sensor and determines the target area of ​​the second device on the screen based on the first signal. The target area is the area on the screen where the capacitance changes when the second device approaches the first device. The first device and the second device establish a first communication connection. The first device receives a first indication message sent by the second device, the first indication message being used to indicate a change in the motion state of the second device; The first device sends first content to the second device based on the target area.

17. The method according to claim 16, characterized in that, After the first device receives the first indication information sent by the second device, the method further includes: The first device receives the second content sent by the second device.

18. The method according to claim 17, characterized in that, The target area includes the area on the screen of the first device that the second device maps onto when the second device is hovered, or the contact area between the screen of the second device and the screen of the first device.

19. The method according to claim 18, characterized in that, The change in motion state includes: the motion state changing to a stationary state, and the duration of the second device being in the stationary state being greater than a first preset duration.

20. The method according to claim 19, characterized in that, The first device sends first content to the second device based on the target area, including: The first device determines that there is third content at the center point or centroid of the target area on the screen, and sends the first content to the second device.

21. The method according to claim 20, characterized in that, The third content includes any one of the following: application icon, video file icon, image file icon, document file icon, spreadsheet file icon, video, image, document, and spreadsheet. The first content includes any one of the following: application identifier, application installation package, video, image, document, and table.

22. The method according to claim 21, characterized in that, When the third content is the icon of the application, the first content is the identifier of the application or the installation package of the application; or, If the third content is the icon of the video file or the video itself, then the first content is the video; or, If the third content is the icon of the image file or the image itself, then the first content is the image; or, If the third content is the icon of the document file or the document itself, then the first content is the document; or, In the case where the third content is the icon of the table file or the table itself, the first content is the table.

23. The method according to any one of claims 16-22, characterized in that, The step of determining the target area of ​​the second device on the screen based on the first signal specifically includes: The first device generates a grayscale image from the first signal and determines the target feature region in the grayscale image, wherein the target feature region is the region in the grayscale image where the grayscale value of a pixel is higher than a grayscale threshold; The first device determines the target feature region as the target region corresponding to the second device on the screen.

24. The method according to claim 23, characterized in that, Determining the target feature region as the target region corresponding to the second device on the screen includes: If the target feature region satisfies a first condition and / or a second condition, the first device determines the target feature region as the corresponding target region of the second device on the screen; wherein... The first condition includes that the shape of the target feature region is the first shape; The second condition includes that the difference between the size of the target feature region and the size of the short side of the second device is less than a first threshold.

25. The method according to claim 17, characterized in that, Before the first device receives the second content sent by the second device, the method further includes: The first device sends a first instruction to the second device, the first instruction being used to instruct the second device to send the second content to the first device.

26. The method according to claim 25, characterized in that, The first device sends a first instruction to the second device, specifically including: The first device determines that there is no content at the center point or centroid of the target area, and sends the first instruction to the second device.

27. The method according to any one of claims 24-26, characterized in that, Before the first device sends the first content to the second device based on the target area, the method further includes: The first device establishes a second communication connection with the second device, and the transmission rate of the second communication connection is higher than that of the first communication connection; The first device sends first content to the second device based on the target area, including: The first device sends the first content to the second device via the second communication connection based on the target area.

28. The method according to claim 27, characterized in that, The second content includes any one of the user interface, images, videos, tables, documents, and animations currently displayed on the second device.

29. An interaction method, characterized in that, The method, using a second device, includes: Send a first indication message to the first device, the first indication message being used to indicate a change in the motion state of the second device, and the first device and the second device establishing a first communication connection; The device receives first content sent by the first device, the first content being content determined by a target area of ​​the screen of the first device, the target area being the area on the screen of the first device where capacitance changes occur when the second device approaches the first device.

30. The method according to claim 29, characterized in that, After sending the first instruction information to the first device, the method further includes: Send the second content to the first device.

31. The method according to claim 30, characterized in that, Before sending the second content to the first device, the method further includes: The device receives a first instruction sent by the first device, the first instruction being used to instruct the second device to send second content to the first device.

32. The method according to claim 31, characterized in that, If an application icon exists in the target area, the first content is the application's identifier or the application's installation package; or, If a video file icon or the video exists in the target area, then the first content is the video; or, If an icon of an image file or the image itself exists in the target area, then the first content is the image; or, If the target area contains a document file icon or the document itself, then the first content is the document. or, If the target area contains an icon of a table file or the table itself, the first content is the table.

33. The method according to any one of claims 29-32, characterized in that, The target area includes the area on the screen of the first device that the second device maps onto when the second device is hovered, or the contact area between the screen of the second device and the screen of the first device. The change in motion state includes: the motion state changing to a stationary state, and the duration of the second device being in the stationary state being greater than a first preset duration; The second content includes any one of the user interface, images, videos, tables, documents, and animations currently displayed on the second device.

34. An interaction method, characterized in that, The method is applied to an interactive system, which includes a first device and a second device, wherein the first device and the second device establish a first communication connection, and the method includes: When the second device hovers over or touches the screen of the first device, the first device sends the first content within the target area to the second device. The target area is the area mapped onto the screen of the first device when the second device hovers over or touches the screen of the first device, or the contact area between the screen of the second device and the screen of the first device. Alternatively, when the second device hovers over or touches the screen of the first device, the second device sends second content to the first device, the second content being the content currently displayed on the second device.

35. The method according to claim 34, characterized in that, The first device sending the first content within the target area to the second device specifically includes: If the first content is displayed in the target area of ​​the first device and the desktop is displayed on the second device, the first device determines to send the first content in the target area to the second device.

36. The method according to claim 34, characterized in that, The second device sends second content to the first device, specifically including: If the desktop is displayed in the target area of ​​the first device and the second device displays the second content, the first device determines that the second device will send the second content to the first device.

37. The method according to claim 34, characterized in that, The second content sent by the second device to the first device specifically includes: When the desktop is displayed in the target area of ​​the first device and the desktop is displayed on the second device, the second device sends the desktop data of the second device to the first device; The method further includes: The first device receives desktop data from the second device and displays the desktop content of the second device.

38. The method according to claim 34, characterized in that, The method further includes: When the first device displays the first content and the second device displays the second content, the second device displays a first pop-up window, which includes a first control and a second control. In response to a user's action on the first control, the second device sends second content to the first device; or... In response to a user's operation on the second control, the second device sends a first instruction to the first device, the first instruction being used to instruct the first device to send the first content to the second device; The first device receives and responds to the first instruction, and sends the first content to the second device.

39. The method according to claim 34 or 35, characterized in that, Before the second device is hovered over or lightly touched to the screen of the first device, the method further includes: The first device displays the application interface of the first application, and the first content within the target area includes the controls in the application interface of the first application; The second device displays the desktop; After the first device sends the first content within the target area to the second device, the method further includes: The second device receives the first content and displays the first content.

40. The method according to claim 34 or 38, characterized in that, Before the second device is hovered over or lightly touched to the screen of the first device, the method further includes: The first device displays the application interface of the second application, and the application interface of the second application displays a first document, which includes any one of a file, a spreadsheet, or an email. The second device displays the second content; After the second device sends the second content to the first device, the method further includes: The first device receives the second content and inserts the second content into the first document.

41. The method according to claim 35, characterized in that, When the first device displays the first content and the second device displays the desktop, the method further includes: When the first device displays first content and third content, the second device displays the desktop, and the first device determines that the first content is within the target area, the first device sends the first content within the target area to the second device.

42. The method according to claim 34 or 35, characterized in that, The method further includes: The first device receives a first operation, which is used to select the first content in the first device; When the second device hovers over or touches the screen of the first device, the first device sends the first content to the second device.

43. The method according to claim 34 or 36, characterized in that, The method further includes: The second device receives a second operation, the second operation being used to select the second content in the second device; When the second device hovers over or touches the screen of the first device, the second device sends the second content to the first device.

44. The method according to any one of claims 34-43, characterized in that, When the second device is hovered over or lightly touched to the screen of the first device, the method includes: The touch sensor in the first device collects a first signal and determines, based on the first signal, whether the second device is hovering over or lightly touching the target area on the screen of the first device. The target area is the area on the screen where capacitance changes when the second device approaches the first device. The first device receives a first indication message sent by the second device, the first indication message being used to indicate a change in the motion state of the second device.

45. The method according to claim 44, characterized in that, Before the touch sensor in the first device acquires the first signal, the method further includes: The second device receives a trigger operation, the trigger operation including the user holding the second device and drawing a circle, or the user tapping the back cover of the second device; In response to the triggering operation, the second device sends a second instruction to the first device, the second instruction being used to instruct the second device to begin acquiring the first signal through the touch sensor.

46. ​​The method according to any one of claims 34-43, characterized in that, The first content includes any one of the following: file, image, folder, video, audio, window, control, and application interface; the second content includes any one of the following: file, image, folder, video, audio, window, control, and application interface.

47. An electronic device, characterized in that, The electronic device includes: one or more processors and a memory, a screen including a touch sensor; the screen and the memory are coupled to the one or more processors, the memory being used to store computer program code including computer instructions, and the one or more processors invoking the computer instructions to cause the electronic device to perform the method as described in any one of claims 1-46.

48. An electronic device, characterized in that, The electronic device includes: one or more processors and a memory, the memory being coupled to the one or more processors, the memory being used to store computer program code, the computer program code including computer instructions, the one or more processors invoking the computer instructions to cause the electronic device to perform the method performed by any one of the second devices as claimed in claims 1-46.

49. A computer-readable storage medium, characterized in that, Includes computer instructions that, when executed on an electronic device, cause the electronic device to perform the method as described in any one of claims 1-46 by the first or second device.

50. A computer program product, characterized in that, Includes computer instructions that, when executed on an electronic device, cause the electronic device to perform the method as described in any one of claims 1-46 by the first or second device.

Images