Information exchange system and apparatus, communication device, and computer-readable storage medium

Abstract

The present application relates to the technical field of communications. Disclosed are an information exchange system and apparatus, a communication device, and a computer-readable storage medium. In the system, a device can recognize a gesture of a user with respect to the device by means of a received wireless signal, when the recognized gesture is a first gesture, information in the device is sent to an associated device, and when the recognized gesture is a second gesture, information in the associated device is acquired, so as to complete information exchange between different devices, so that the user implements the first gesture and the second gesture respectively with respect to the associated first device and second device, i.e., the first device and the second device can be triggered to perform information exchange, and during information exchange, the user does not need to perform operations on the devices, thereby simplifying the information exchange process, thus improving information exchange efficiency.

Description

Information interaction systems, devices, communication equipment and computer-readable storage media

[0001] This application claims priority to Chinese Patent Application No. 202411581505.1, filed on November 6, 2024, entitled "Information Interaction System, Apparatus, Communication Equipment and Computer-Readable Storage Medium", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of communication technology, and in particular to an information interaction system, device, communication equipment, and computer-readable storage medium. Background Technology

[0003] With the development and popularization of computer technology, the types and numbers of electronic devices are increasing. Simultaneously, the number of electronic devices owned by individuals is also constantly growing. This means that more efficient information interaction methods are needed in scenarios such as home, office, and smart cockpits, making the need for information interaction between different devices extremely urgent. Therefore, how to improve the efficiency of information interaction between different devices, build efficient and unique inter-device interaction methods, and enhance the user experience has become a pressing issue that needs to be addressed.

[0004] Transferring information between different devices can mean transferring information from one device to another. Currently, this is primarily done via touchscreens. Taking file transfer as an example, a user operates on the touchscreen of device A to select a file to share. Then, the user performs another operation on the touchscreen of device A, selecting device B to receive the file. Afterward, the user sends the selected file on the touchscreen of device A, triggering device A to send the file to device B, which then receives the file sent by device A.

[0005] However, in the process of transmitting information between different devices based on touch screens, users need to perform multiple operations on the device's touch screen, which makes the information interaction process between different devices cumbersome and inefficient. Summary of the Invention

[0006] This application provides an information interaction system, apparatus, communication device, and computer-readable storage medium, which can improve the efficiency of information interaction between different devices. The technical solution is as follows:

[0007] In a first aspect, an information interaction system is provided, which includes at least two communication devices that can interact with each other. The communication device that sends information during the information interaction is referred to as the first device, and the communication device that receives information during the information interaction is referred to as the second device. The wireless signal received by the first device is referred to as the first wireless signal, and the wireless signal received by the second device is referred to as the second wireless signal. The first device and the second device are associated devices with each other.

[0008] The first device is configured to send information to the second device based on a received first wireless signal. For example, the first device is configured to: identify a gesture of a first user toward the first device based on the first wireless signal; and if the gesture toward the first device is a first gesture, send first information to the second device. Wherein, the first gesture indicates sending information to the associated device, and the first information is information from the first device that participates in cross-device information interaction.

[0009] The second device is used to obtain information from the first device based on a received second wireless signal. For example, the second device is used to: identify a gesture of a second user towards the second device based on the second wireless signal; and if the gesture towards the second device is a second gesture, obtain first information. The second user and the first user can be the same user or different users, and the second gesture indicates the acquisition of information from the associated device.

[0010] In the aforementioned system, the device can recognize user gestures directed at it via received wireless signals. If the recognized gesture is a first gesture, the device sends information from the associated device; if the recognized gesture is a second gesture, the device retrieves information from the associated device, thus completing information interaction between different devices. By having the user perform the first gesture and the second gesture on the associated first and second devices respectively, information interaction between the first and second devices can be triggered. During the information interaction process, the user does not need to operate the device, simplifying the information interaction process and thereby improving information interaction efficiency.

[0011] In one possible implementation, the first device is further configured to: acquire multiple first signal state information, perform temporal feature analysis on the multiple first signal state information to obtain first temporal features; and identify a first user's gesture toward the first device based on the first temporal features. The first temporal features indicate the changes in the first signal state information over time. Similarly, the second device identifies a second user's gesture toward the second device based on a second wireless signal, which will not be elaborated further here.

[0012] Based on the above possible implementation methods, and based on the changes in signal state information corresponding to the wireless signal received by the first device, gestures directed at the first device can be accurately identified, thereby improving the accuracy of gesture recognition based on wireless signals, so as to enable information interaction based on accurate gesture recognition results.

[0013] In one possible implementation, the first wireless signal is a WiFi signal, and the first signal state information includes the channel state information corresponding to the first wireless signal. Based on this, the first device is further configured to: perform multi-dimensional feature analysis on the changes in channel state information among multiple first signal state information based on first timing features, to obtain first frequency domain features, first time domain features, and first spatial domain features; and identify the gestures of the first user towards the first device based on the first frequency domain features, first time domain features, and first spatial domain features. Wherein, the first frequency domain features represent the changes in channel state information among multiple first signal state information in the frequency domain, the first time domain features represent the changes in channel state information among multiple first signal state information in the time domain, and the first spatial domain features represent the changes in channel state information among multiple first signal state information in the spatial domain.

[0014] Based on the above possible implementation methods, the influence of the user's gesture wireless signal on multiple dimensions such as frequency domain, time domain and spatial domain is fully considered, so that the recognized gestures can be more accurate, thereby further improving the accuracy of gesture recognition based on wireless signals, so as to carry out information interaction based on accurate gesture recognition results.

[0015] In one possible implementation, the first gesture corresponds to a first target frequency domain feature, a first target time domain feature, and a first target spatial domain feature. The first target frequency domain feature represents the change in channel state information corresponding to the wireless signal in the frequency domain under the influence of the first gesture; the first target time domain feature represents the change in channel state information corresponding to the wireless signal in the time domain under the influence of the first gesture; and the first target spatial domain feature represents the change in channel state information corresponding to the wireless signal in the spatial domain under the influence of the first gesture. Based on this, the first device is further configured to recognize the first gesture as a gesture of the first user towards the first device if at least one of the first, second, and third conditions is met. The first condition is that the first frequency domain feature matches the first target frequency domain feature; the second condition is that the first time domain feature matches the first target time domain feature; and the third condition is that the first spatial domain feature matches the first target spatial domain feature.

[0016] Based on the above possible implementation methods, by matching the multi-dimensional features corresponding to the first wireless signal with the multi-dimensional features corresponding to the first gesture, it is possible to accurately identify whether the user's gesture toward the first device is the first gesture, thereby further improving the accuracy of gesture recognition based on wireless signals, so as to carry out information interaction based on accurate gesture recognition results.

[0017] In one possible implementation, the first device is further configured to: determine a first distance based on a first temporal feature; and if the first distance is less than or equal to a first threshold, perform a step of identifying a first user's gesture toward the first device based on the first temporal feature. Here, the first distance is the distance between the spatial location that triggered the change in the first signal state information and the first device.

[0018] Based on the above possible implementation methods, wireless signals that are not affected by gestures directed at the first device can be filtered out, avoiding gesture recognition of these wireless signals and saving computing resources of the first device.

[0019] In one possible implementation, the first device is further configured to determine first information based on the content displayed by the first device in the currently displayed interface, and the first information is associated with the content.

[0020] Based on the above possible implementation methods, there is no need for users to select the information to be sent, which simplifies the information interaction process across devices and improves the efficiency of information interaction across devices.

[0021] In one possible implementation, the first device actively sends first information to the second device. For example, the first device is further configured to broadcast an information sending request if the gesture directed at the first device is a first gesture; the second device is further configured to receive the information sending request, and if the gesture directed at the second device is a second gesture, obtain the first information from the information sending request. The information sending request instructs that information be sent to the associated device.

[0022] Based on the above possible implementation methods, the first device actively requests the second device to push the first information, which can avoid the second device broadcasting the information acquisition request and also avoid irrelevant devices receiving and processing the information acquisition request.

[0023] In one possible implementation, the second device confirms with the first device whether to perform cross-device information interaction, and performs cross-device information interaction with the first device based on the confirmation result. For example, the second device is further configured to: obtain a first gesture distance for each of the multiple third devices from information sending requests; if the first gesture distance for the first device is the smallest among the multiple third devices and the gesture for the second device is a second gesture, send an information interaction confirmation request to the first device; if an information interaction confirmation response is received from the first device, obtain first information from the information sending request of the first device. Here, the first gesture distance is the distance between the device and the first gesture for the device, the information interaction confirmation request is used to confirm whether to perform cross-device information interaction, and the information interaction confirmation response indicates confirmation of cross-device information interaction.

[0024] Based on the above possible implementation methods, the second device confirms with the device with the smallest first gesture distance among multiple third devices whether to participate in cross-information interaction through an information interaction request. Only after the device with the smallest first gesture distance confirms will the second device obtain the information of that device from the information sending request, so as to ensure the correctness of information interaction.

[0025] In one possible implementation, the second device actively requests first information from the first device. For example, the second device is further configured to broadcast an information acquisition request if the gesture directed at the second device is a second gesture; the first device is further configured to receive the information acquisition request, and if the gesture directed at the first device is a first gesture, send the first information to the second device based on the information acquisition request; the second device is further configured to receive the first information. The information acquisition request indicates a request to acquire information from an associated device.

[0026] Based on the above possible implementation methods, the second device actively requests the first device to obtain the first information, which can prevent the first device from broadcasting the first information and also prevent irrelevant devices from receiving and processing the first information.

[0027] In one possible implementation, the first device confirms with the second device whether to perform cross-device information interaction, and performs cross-device information interaction with the second device based on the confirmation result. For example, the first device is further configured to: obtain the second gesture distance of each of the multiple fourth devices from information acquisition requests; if the second gesture distance of the second device is the smallest among the multiple fourth devices and the gesture for the first device is the first gesture, send an information interaction confirmation request to the second device; if an information interaction confirmation response is received from the second device, send first information to the first device. Here, the second gesture distance is the distance between the device and the second gesture for the device, the information interaction confirmation request is used to confirm whether to perform cross-device information interaction, and the information interaction confirmation response indicates confirmation of cross-device information interaction.

[0028] Based on the above possible implementation methods, the first device sends a first message to the device with the smallest second gesture distance among multiple fourth devices through an information interaction request, to confirm whether it will participate in cross-information interaction. Only after the device with the smallest second gesture distance confirms will the first message be sent to that device, so as to ensure the correctness of information interaction.

[0029] In one possible implementation, the system further includes a fifth device, through which the first device sends first information to the second device. For example, the first device is further configured to send an information sending request to the fifth device if the gesture directed at the first device is a first gesture; the second device is further configured to send an information retrieval request to the fifth device if the gesture directed at the second device is a second gesture; the fifth device is configured to determine the association between the second device and the first device based on the information sending request and the information retrieval request, and then send the first information from the information sending request to the second device. The information sending request indicates sending information to the associated device, and the information retrieval request indicates requesting to retrieve information from the associated device.

[0030] Based on the above possible implementation methods, the fifth device assists in completing the cross-device information exchange between the first device and the second device, avoiding the first device's broadcast information sending request and the second device's broadcast information receiving request, thereby avoiding irrelevant devices from receiving and processing information sending requests and information receiving requests.

[0031] In one possible implementation, the fifth device may receive information transmission requests from multiple third devices, all of which are associated devices of the second device, including the first device. In this scenario, the fifth device selects the first information from the multiple third devices and sends it to the second device. For example, the fifth device is further configured to: obtain a first gesture distance from each of the multiple third devices' information transmission requests; if the first gesture distance of the first device is the smallest among the multiple third devices, send the first information from the first device's information transmission request to the second device. Here, the first gesture distance is the distance between the device and the first gesture directed at the device.

[0032] Based on the above possible implementation methods, the fifth device can accurately determine the associated device that is interacting with the second device across devices from among the multiple associated devices of the second device, and send the determined device's information on participating in the cross-device information interaction to the second device to ensure the accuracy of the information interaction.

[0033] In one possible implementation, the fifth device may receive information acquisition requests from multiple fourth devices, all of which are associated devices of the first device, including the second device. The fifth device can select from the multiple fourth devices to send first information to the second device. For example, the fifth device is further configured to: obtain a second gesture distance from each of the multiple fourth devices based on their information acquisition requests; and if the second gesture distance of the second device is the smallest among the multiple fourth devices, send the first information to the second device. Here, the second gesture distance is the distance between the device and the second gesture directed at the device.

[0034] Based on the above possible implementation methods, the fifth device can accurately determine the associated device that performs cross-device information interaction with the first device from among the multiple associated devices of the first device, and send the first information for determining the first device to the determined device, so as to ensure the accuracy of information interaction.

[0035] Secondly, an information interaction device is provided, which is applied to a first device in the aforementioned information interaction system, and is used to implement various functions of the first device in the aforementioned information interaction system. For example, the device includes: a gesture recognition unit, used to recognize a gesture of a first user toward the first device based on a first wireless signal, wherein the first wireless signal is a wireless signal received by the first device; and a sending unit, used to send first information to a second device in the information interaction system if the gesture toward the first device is a first gesture, wherein the first gesture indicates sending information to an associated device, the second device being an associated device of the first device, and the first information being information in the first device participating in cross-device information interaction.

[0036] Thirdly, an information interaction device is provided, which is applied to a second device in the aforementioned information interaction system, and is used to implement various functions of the second device in the aforementioned information interaction system. For example, the device includes: a gesture recognition unit, used to recognize a gesture of a second user toward the second device based on a second wireless signal, wherein the second wireless signal is a wireless signal received by the second device; and a first acquisition unit, used to acquire first information of a first device in the information interaction system if the gesture toward the second device is a second gesture, wherein the second gesture indicates the acquisition of information from an associated device, the first device being an associated device of the second device, and the first information being information from the first device participating in cross-device information interaction.

[0037] Fourthly, a communication device is provided, the communication device including a processor coupled to a memory, the memory storing at least one piece of program code, the processor executing the program code to perform the following steps: based on a first wireless signal, recognizing a gesture of a first user toward the communication device, the first wireless signal being a wireless signal received by the communication device; if the gesture toward the communication device is a first gesture, sending first information to a second device in an information interaction system in which the communication device is located, the first gesture indicating sending information to an associated device, the second device being an associated device of the communication device, the first information being information in the communication device participating in cross-device information interaction.

[0038] Fifthly, a communication device is provided, the communication device including a processor coupled to a memory, the memory storing at least one piece of program code, the processor executing the program code to perform the following steps: based on a second wireless signal, recognizing a gesture of a second user toward the communication device, the second wireless signal being a wireless signal received by the communication device; if the gesture toward the second device is a second gesture, acquiring first information of a first device in an information interaction system in which the communication device is located, the second gesture indicating the acquisition of information in an associated device, the first device being an associated device of the second device, the first information being information in the first device participating in cross-device information interaction.

[0039] A sixth aspect provides a computer-readable storage medium storing at least one piece of program code that is read by a processor to enable a communication device to perform the functions of a first device in a system provided by the first aspect above or any alternative method of the first aspect above.

[0040] A seventh aspect provides a computer-readable storage medium storing at least one piece of program code that is read by a processor to enable a communication device to perform the functions of a second device in a system provided as described in the first aspect or any alternative method of the first aspect.

[0041] Eighthly, a computer program product or computer program is provided, the computer program product or computer program including program code stored in a computer-readable storage medium, a processor reading the program code from the computer-readable storage medium, the processor executing the program code, causing a communication device to perform the functions of the first device in the system provided by the first aspect or various optional implementations of the first aspect.

[0042] Ninthly, a computer program product or computer program is provided, the computer program product or computer program including program code stored in a computer-readable storage medium, a processor reading the program code from the computer-readable storage medium, the processor executing the program code, causing a communication device to perform the functions of the second device in the system provided by the first aspect or various optional implementations of the first aspect.

[0043] Based on the implementation methods provided in the above aspects, this application can be further combined to provide more implementation methods. Attached Figure Description

[0044] Figure 1 is a schematic diagram of the application environment of an information interaction system provided in an embodiment of this application;

[0045] Figure 2 is a schematic diagram of the gesture release process provided in an embodiment of this application;

[0046] Figure 3 is a schematic diagram of cross-device information interaction based on a gesture recognition module provided in an embodiment of this application;

[0047] Figure 4 is a flowchart illustrating an information interaction method provided in an embodiment of this application;

[0048] Figure 5 is a schematic diagram of a cross-device information interaction process provided in an embodiment of this application;

[0049] Figure 6 is a schematic diagram of cross-device information interaction based on release gestures and grasping gestures provided in an embodiment of this application;

[0050] Figure 7 is a schematic diagram of a gesture recognition process based on WiFi signals provided in an embodiment of this application;

[0051] Figure 8 is a schematic diagram of the data flow corresponding to a data transfer method 1 provided in an embodiment of this application;

[0052] Figure 9 is a schematic diagram of the data flow corresponding to a data transfer method 2 provided in an embodiment of this application;

[0053] Figure 10 is a schematic diagram of the data flow corresponding to a data transfer method 3 provided in an embodiment of this application;

[0054] Figure 11 is a schematic diagram of the structure of an information interaction device provided in an embodiment of this application;

[0055] Figure 12 is a schematic diagram of another information interaction device provided in an embodiment of this application;

[0056] Figure 13 is a schematic diagram of the structure of a communication device provided in an embodiment of this application. Detailed Implementation

[0057] To facilitate understanding of the specific implementation methods of this application, some terms involved in the specific implementation methods are introduced as follows.

[0058] Wireless local area networks (WLANs) are network systems that use wireless communication technology to enable wireless connections and data transmission between devices within a local area network. The coverage range of a WLAN typically ranges from tens of meters to several kilometers, depending on the equipment used and the environment.

[0059] Wireless communication technology is a communication method that uses the characteristic that electromagnetic wave signals can propagate in free space to exchange information, such as WiFi communication technology or Bluetooth communication technology.

[0060] Wireless signal: refers to a signal transmitted through wireless communication technology, such as WiFi signal or Bluetooth signal.

[0061] Multiple-input multiple-output (MIMO) is a multi-antenna wireless communication system model that can use multiple antennas at the transmitting end to independently transmit signals, while at the receiving end multiple antennas to receive and recover the original information.

[0062] Path: refers to the channel through which a wireless signal propagates. In wireless communication, in addition to the direct path and the ground reflection path, electromagnetic waves also encounter scattering paths caused by various obstacles during propagation.

[0063] A link is a communication link consisting of a pair of transmit and receive antennas in a MIMO system. A communication link transmits electromagnetic waves through multiple paths.

[0064] Channel state information (CSI) is used to describe the attributes of a communication link in a wireless communication system. CSI characterizes attenuation factors such as signal scattering, environmental attenuation, distance attenuation, channel attenuation, and frequency response along the communication link. The CSI can differ between different links (i.e., antenna pairs). The presence or movement of a human body causes changes in the CSI received by the wireless signal receiver. These changes can be used by the receiver to infer the presence of limb movements by people in the vicinity. Changes in CSI include variations in amplitude and / or phase.

[0065] Gesture: refers to the posture of the hand. Unlike traditional gestures for touch screens, the gestures in this application are air gestures, which do not touch the device or touch screen.

[0066] Gesture execution: refers to the action of a user making a certain gesture.

[0067] Associated devices of a communication device: These are communication devices that are associated with the given communication device. The association between multiple communication devices can be manifested as follows: multiple communication devices share the same association information, which indicates that they are associated. This association information can be a user account shared by all the communication devices. For example, a user can own multiple communication devices, and the user can bind their user account to these devices to indicate that they belong to that user. Thus, the user account allows the association of these multiple communication devices. Alternatively, the association information can be shared pairing information, indicating that the multiple communication devices have completed pairing. For example, multiple communication devices can pair via Bluetooth. After successful pairing, pairing information is generated, which is shared by all the communication devices, allowing them to be associated.

[0068] This application provides an information interaction method that enables information interaction between different devices based on gesture recognition using wireless signals. This method is applied to an information interaction system, and the application environment of this system is described below with reference to Figure 1.

[0069] Figure 1 is a schematic diagram of the application environment of an information interaction system provided in an embodiment of this application. As shown in Figure 1, the application environment of the information interaction system can be an indoor environment, such as an indoor environment in a home, company, or some public places. A WLAN is deployed in the indoor environment. The WLAN includes one or more wireless devices 101. Here, the number of wireless devices 101 in the WLAN is not limited in this embodiment of the application.

[0070] To facilitate understanding of the information interaction method provided in this application, a wireless device 101 is shown in Figure 1. The wireless device 101 shown is any wireless device 101 in a WLAN. The wireless device 101 is used to provide wireless signals to the communication device 102. Taking WiFi signals as an example, the wireless device can be a wireless access point (AP), router, or switch, etc. Here, the device type of the wireless device 101 is not limited in this embodiment of the application; the wireless device only needs to be able to provide WiFi signals.

[0071] The information interaction system includes multiple communication devices 102, which can perform cross-device information interaction. The communication devices 102 are terminals used by users, such as mobile phones, tablets, music players, laptops, desktop computers, or wearable electronic devices, and can also be Internet of Things devices such as refrigerators, washing machines, microwave ovens, and rice cookers. Here, this application embodiment does not limit the device type of the communication devices 102.

[0072] Any communication device 102 can be connected to wireless device 101 via a wireless network card. Wireless device 101 can connect to one or more communication devices 102. In Figure 1, two communication devices 102 are connected to wireless device 101. The number of communication devices connected to wireless device 101 is not limited to two; it can be one or more.

[0073] Wireless device 101 is the transmitter of wireless signals, and communication device 102 connected to wireless device 101 is the receiver of wireless signals. Information is transmitted between communication device 102 and wireless device 101 via WiFi signals. Taking two communication devices 102 shown in Figure 1, device A and device B, as an example, in the scenario where device A sends information to device B, assuming that information 103 in device A is the information to be sent, the wireless network card in device A sends at least one data packet to wireless device 101. This data packet carries information 103. Wireless device 101 converts the received data packet into a WiFi signal and sends the WiFi signal to device B. The wireless network card in device B converts the WiFi signal sent by wireless device 101 into a data packet and parses information 103 from the converted data packet. This achieves the transmission of information 103 between device A and device B, i.e., information interaction regarding information 103 is realized between device A and device B. Information 103 can be any content in device A. Through the above information transmission method, device A shares its local information with device B. The aforementioned wireless network card supports the WiFi protocol, enabling it to send and receive WiFi signals and convert WiFi signals into data signals.

[0074] WiFi signals are electromagnetic waves. When wireless device 101 sends a WiFi signal to device B, it is essentially sending an electromagnetic wave to device B. If both wireless device 101 and device B support MIMO technology, multiple links will exist between them. Each link transmits electromagnetic waves through multiple paths. In other words, the WiFi signal sent by wireless device 101 can reach device B via different paths. For example, the WiFi signal can be transmitted to device B via a direct path between wireless device 101 and device B, or it can be transmitted to device B via a reflected path. Alternatively, after wireless device 101 sends out a WiFi signal, the signal may be reflected by an obstacle to device B. This obstacle can be any object in the environment (such as indoor facilities or walls) or even a person in the environment.

[0075] Figure 1 illustrates an example where device B, device A, and wireless device 101 are located in the same indoor environment. In other embodiments, device A and device B are located in different indoor environments, for example, device A and device B are located in different rooms, and wireless device 101 can be located in the room where device A or device B is located, or wireless device 101 can be located in a different room from device A and device B.

[0076] Figure 1 illustrates an example where device A and device B are connected to the same wireless device 101. In other embodiments, device A and device B are connected to different wireless devices 101. For example, a WLAN includes wireless device 1 and wireless device 2. Device A is connected to wireless device 1, and device B is connected to wireless device 2. In a scenario where device A sends information to device B, a data packet carrying information 103 is sent to wireless device 1. Wireless device 1 forwards the data packet to wireless device 2, and wireless device 2 converts the data packet into a WiFi signal and sends the WiFi signal to device B. In other embodiments, wireless device 1 and wireless device 2 are located in different WLANs. For example, wireless device 1 is located in WLAN 1, and wireless device 2 is located in WLAN 2. WLAN 1 and WLAN 2 are connected via a communication network. In a scenario where device A sends information to device B, a data packet carrying information 103 is sent to wireless device 1. Wireless device 1 forwards the data packet to the communication network, and the communication network forwards the data packet to wireless device 2. Wireless device 2 converts the data packet into a WiFi signal and sends the WiFi signal to device B.

[0077] The above description uses the example of information exchange between different communication devices 102 via wireless signals transmitted by wireless device 101. In other embodiments, the communication device 102 in the information exchange system supports transmitting wireless signals, and different communication devices 102 in the information exchange system exchange information via wireless signals. Taking Bluetooth signals as an example, in a scenario where device A sends information to device B, a Bluetooth connection is established between device A and device B. The Bluetooth module in device A sends a Bluetooth signal to device B to carry information 103. The Bluetooth module in device B receives the Bluetooth signal sent by device A and parses information 103 from the received Bluetooth signal. The Bluetooth module supports the Bluetooth protocol and can send and receive Bluetooth signals and convert between Bluetooth signals and data signals. When the wireless signal is a Bluetooth signal, the distance between the different communication devices 102 exchanging information is within the maximum transmission distance of the Bluetooth signal.

[0078] According to any of the methods described above for transmitting information 103 from device A to device B, device B can also send information from device B back to device A, which will not be elaborated further here.

[0079] The above example assumes that the communication device 102 performing information interaction in the information interaction system is located in an indoor environment. In other embodiments, the communication device 102 performing information interaction in the information interaction system may also be located in other environments outside the indoor environment. Taking an outdoor environment as an example, assuming the wireless signal is a WiFi signal, a wireless device 101 is deployed in an outdoor environment, and a communication device 102 connected to the wireless device 101 interacts with another communication device 102 through the wireless device 101. Alternatively, assuming the wireless signal is a Bluetooth signal, different communication devices 102 interact with each other through Bluetooth signals in an outdoor environment. Taking a cabin environment as another example, assuming the wireless signal is a WiFi signal, a wireless device 101 is deployed in the cabin environment of a vehicle (such as a car, airplane, ship, etc.). Any communication device 102 connected to the wireless device 101 in the cabin environment can interact with another communication device 102. The other communication device 102 can connect to the wireless device 101 or to a wireless device outside the cabin environment. Alternatively, assuming the wireless signal is a WiFi signal, different communication devices 102 in the cockpit environment can interact via Bluetooth. Here, this embodiment does not limit the environment in which the communication devices 102 interact, as long as they can interact via wireless signals. When the wireless signal is Bluetooth, the communication device 102 may or may not be connected to the wireless device 101. Therefore, when the wireless signal is Bluetooth, the environment in which the communication device 102 interacts may or may not contain the wireless device 101.

[0080] The above description uses an example of two communication devices 102 interacting in an information interaction system. One communication device 102 is the information sender, and the other is the information receiver; for example, device A is the information sender, and device B is the information receiver. In other embodiments, the information interaction system may have more than two communication devices 102 interacting; for example, one communication device 102 is the information sender, and at least two communication devices 102 are information receivers. Similar to devices A and B, the multiple communication devices 102 interacting can be in the same environment or in different environments.

[0081] In some embodiments, the information interaction system is a logical interaction system, and multiple communication devices 102 that perform cross-device information interaction form the information interaction system. In some embodiments, the information interaction system is a physical interaction system. For example, the information interaction system includes an intermediate device (such as the fifth device mentioned below), which provides information forwarding services for the communication devices 102 that perform cross-device information interaction. The multiple communication devices 102 that perform cross-device information interaction access the intermediate device, so that the information interaction system includes the multiple communication devices 102. The multiple communication devices 102 use the intermediate device to transmit information participating in cross-device information interaction.

[0082] During wireless communication, wireless signals destined for communication device 102 exist in the surrounding environment. Taking WiFi as an example, if wireless device 101 connected to device A sends a WiFi signal to device A, then WiFi signals destined for device A exist in the surrounding environment. Similarly, taking Bluetooth as an example, after device A and device B establish a Bluetooth connection, device B sends a Bluetooth signal to device A, then Bluetooth signals destined for device A exist in the surrounding environment.

[0083] For any communication device 102, a user may perform various physical actions in the surrounding environment, such as making one or more gestures. The surrounding environment of the communication device 102 can refer to the environment within a preset distance range from the communication device 102. This preset distance can be 3 meters, less than 3 meters, or greater than 3 meters. Here, this application embodiment does not limit the preset distance. On the path of the wireless signal transmitted to this communication device 102, each of the user's physical actions can affect the wireless signal. Compared with the state of the wireless signal before it was transmitted to the user, the user's physical actions will cause the amplitude and / or phase of the wireless signal to fluctuate, thereby causing the state of the wireless signal to change when it reaches the communication device 102. Different physical actions of the user in the surrounding environment have different degrees of influence on the state of the wireless signal. Correspondingly, different gestures of the user in the surrounding environment have different degrees of influence on the state of the wireless signal when it reaches the communication device 102, resulting in different changes in the state of the wireless signal. For example, the fluctuation range of the signal state information value used to reflect the state of the wireless signal is different.

[0084] In view of this, for scenarios involving information interaction between different communication devices (i.e., cross-device information interaction scenarios), this application designs a set of combined gestures, which are used to trigger information interaction between different communication devices. The combined gestures include two gestures: a first gesture and a second gesture, with the first gesture corresponding to the second gesture.

[0085] The first gesture indicates sending information to the associated device, triggering the sending end to perform an information sending operation. This first gesture can also be called a sending gesture, and the information sending operation refers to the operation of sending information across devices. The second gesture indicates retrieving information from the associated device, triggering the receiving end to perform an information retrieval operation. This second gesture can also be called a receiving gesture, and the information retrieval operation refers to the operation of retrieving information across devices.

[0086] The first gesture and the second gesture are gestures with different implementation processes. For example, the first gesture and the second gesture are a set of gestures that are reversed in sequence. For instance, the first gesture is a grasping gesture, and the second gesture is a releasing gesture; the implementation processes of the grasping and releasing gestures are shown in Figure 2. Another example is that the first gesture is an upward swipe gesture (i.e., an up swipe gesture), and the second gesture is a downward swipe gesture (i.e., a down swipe gesture); yet another example is that the first gesture is a rightward swipe gesture (i.e., a rightward swipe gesture), and the second gesture is a leftward swipe gesture (i.e., a leftward swipe gesture). In other embodiments, the first gesture and the second gesture are not gestures with different implementation processes; they are only gestures with different postures. Here, the embodiments of this application do not limit the first gesture and the second gesture.

[0087] In cross-device information interaction scenarios, users can perform a first gesture or a second gesture in the environment surrounding the communication device 102 to affect the state of the wireless signal when it reaches the communication device 102, causing a change in the state of the wireless signal. Based on the change in the wireless signal, the communication device 102 can recognize the user's gesture towards the communication device 102 and perform information sending or information receiving operations based on the recognized gesture (first gesture or second gesture), thereby realizing cross-device information interaction based on gesture recognition of wireless signals.

[0088] In cross-device information interaction scenarios, for ease of description, the communication device 102 that sends information is referred to as the first device (i.e., the information sender), and the communication device 102 that receives information is referred to as the second device (i.e., the information receiver). The user who uses the first device for information interaction is referred to as the first user, and the user who uses the second device for information interaction is referred to as the second user. The first user and the second user may be the same user or different users. For example, in the scenario where device A sends information to device B, device A is the first device, the user using device A is the first user, and device B is the second user.

[0089] In a cross-device information interaction scenario, a first user performs a first gesture on a first device, for example, by performing a gesture in the environment surrounding the first device. The first device, based on the signal state information corresponding to the received wireless signal, recognizes the user's gesture (i.e., performs gesture recognition). Upon recognizing the first gesture, the first device sends information from the first device to a second device, thus performing an information transmission operation. Similarly, a second user performs a second gesture on a second device, for example, by performing a gesture in the environment surrounding the second device. The second device, based on the signal state information corresponding to the received wireless signal, recognizes the user's gesture (i.e., performs gesture recognition). Upon recognizing the second gesture, the second device retrieves information from the first device, thus performing an information retrieval operation.

[0090] The gesture recognition performed in the first and second devices can be implemented by a module within the devices. Taking WiFi as an example, assuming the first device is device A in Figure 1 and the second device is device B in Figure 1, as shown in Figure 3, both device A and device B include upper-layer applications and WLAN hardware modules. The upper-layer applications include an information transmission tool (not shown in the figure) and a gesture recognition module. The WLAN hardware module is used to obtain WiFi signal status information (such as CSI) based on the received WiFi signal. The gesture recognition module is used to determine the changes in WiFi signal status information based on the WiFi signal status information obtained by the WLAN hardware module, and perform gesture recognition based on the changes in WiFi signal status information. The information transmission tool is used to perform information sending or information receiving operations based on the recognition results of the gesture recognition module.

[0091] The WLAN hardware module can be a wireless network card supporting the WiFi protocol. If the wireless signal is Bluetooth, the WLAN hardware module can be replaced with a Bluetooth module; the difference is that the Bluetooth module acquires the signal status information of the Bluetooth signal. The gesture recognition module is a software module implemented in software. In other embodiments, the gesture recognition module can also be implemented in hardware or a combination of hardware and software. The information transmission tool is a software module implemented in software. In other embodiments, the information transmission tool can also be implemented in hardware or a combination of hardware and software. This application does not limit the implementation method of the gesture recognition module and the information transmission tool.

[0092] For any of the communication devices described above, the communication device receives wireless signals sent by other devices during wireless communication. Taking WiFi as an example, the other device is the wireless device that the communication device connects to. Taking Bluetooth as an example, the other device is another communication device that the communication device connects to via Bluetooth.

[0093] The wireless signal received by the communication device can carry at least one data packet. Each data packet is carried by a portion of the wireless signal. Based on this portion of the wireless signal, signal status information can be obtained. The signal status information indicates the state of this portion of the wireless signal when it reaches the communication device. The signal status information corresponds to the data packet and this portion of the wireless signal.

[0094] When the wireless signal is a WiFi signal, the signal status information includes the signal strength of this portion of the wireless signal and / or the Channel State Information (CSI) corresponding to this portion of the wireless signal. The CSI includes the number of antennas and the number of subcarriers. The number of antennas refers to the number of transmit / receive antenna pairs between the wireless device transmitting this portion of the wireless signal and the communication device receiving this portion of the wireless signal. The number of subcarriers refers to the number of subcarriers occupied by the corresponding data packets in this portion of the wireless signal. When the wireless signal is a Bluetooth signal, the signal status information includes the signal strength of this portion of the wireless signal.

[0095] Below, with reference to the information interaction system and signal status information described above, and in conjunction with Figure 4, the flow of the information interaction method provided in this application will be introduced. This method is applied to an information interaction system, which includes a first device and a second device. The method includes the following steps.

[0096] 401. The first device identifies the gesture of the first user toward the first device based on the first wireless signal, wherein the first wireless signal is the wireless signal received by the first device.

[0097] The definitions of the first device and the first user have been described above and will not be repeated here. During wireless communication, the first device receives wireless signals sent by other devices. The first wireless signal is a portion of the received wireless signals used to carry a data packet.

[0098] The first device can identify the first user's gestures on the first device based on the state when the first wireless signal arrives at the first device. For example, steps 411 to 413 below.

[0099] Step 411: The first device acquires multiple first signal status information, which indicates the status of the first wireless signal used to carry a data packet when it arrives at the first device.

[0100] The first signal status information refers to the signal status information of the first wireless signal. Each piece of first signal status information corresponds to a first wireless signal and the data packet carried by that wireless signal.

[0101] During wireless communication, the first device receives a first wireless signal sent by another device (referred to as the first wireless device). The first wireless signal is a WiFi signal. Taking the first signal status information, including the CSI corresponding to the first wireless signal, as an example, the process of acquiring multiple first signal status information can be as follows: During the wireless communication between the first device and the first wireless device, the first wireless device continuously sends wireless signals to the first device, and the first device continuously receives wireless signals. Based on the received wireless signals, multiple CSIs corresponding to the wireless signals are collected. For example, the first device parses each data packet carried in the wireless signal. For a first wireless signal used to carry any data packet, based on the number of subcarriers occupied by the data packet and the number of antennas occupied by the first wireless signal between the first and first wireless devices, the CSI corresponding to the data packet is obtained. The time when the CSI is obtained is taken as the acquisition time corresponding to that CSI. The first device, according to the order of acquisition time, assembles the multiple acquired CSIs into a CSI time series. This CSI time series includes multiple CSIs and the acquisition time corresponding to each CSI. These multiple CSIs are also multiple first signal status information, and this CSI time series is the signal status time series corresponding to the wireless signal.

[0102] Taking the signal strength corresponding to the first wireless signal as an example, the process of acquiring multiple first signal state information can be as follows: During the wireless communication between the first device and other devices, the other devices continuously send wireless signals to the first device, and the first device continuously receives wireless signals. Based on the received wireless signals, multiple signal strengths corresponding to the wireless signals are collected. For example, the first device parses each data packet carried in the wireless signal, detects the signal strength of the first wireless signal used to carry any data packet when it arrives at the first device, and uses the time of detecting the signal strength as the acquisition time corresponding to that signal strength. The first device, according to the order of acquisition time, assembles the detected multiple signal strengths into a signal strength time series. The signal strength time series includes multiple signal strengths and the acquisition time corresponding to each signal strength. These multiple signal strengths are also multiple first signal state information, and the signal strength time series is the signal state time series corresponding to the wireless signal. In this case, the wireless signal is a WiFi signal or a Bluetooth signal. If the wireless signal is a WiFi signal, the other device is the first wireless device. If the wireless signal is a Bluetooth signal, the other device is the communication device (such as the second device) that establishes a Bluetooth connection with the first device.

[0103] The above description uses the example of a first device acquiring first signal status information in real time. In other embodiments, a first user performs an information interaction operation on the first device, and the first device responds to this information interaction operation by executing step 411. The information interaction operation indicates cross-device information interaction; for example, the information interaction operation is an operation to select information to be sent. The user performs this operation on the first device, and the first device responds to this operation by executing step 411. Another example is an information interaction operation to open an information transmission tool on the first device. The user performs this operation on the information transmission tool on the first device, and the first device responds to this operation by executing step 411.

[0104] Optionally, step 411 can be performed by the wireless communication module in the first device. The wireless communication module is a module in the communication device used for wireless communication, which is used to receive and parse wireless signals. If the wireless signal is a WiFi signal, the wireless communication module can be the WLAN hardware module shown in Figure 3. If the wireless signal is a Bluetooth signal, the wireless communication module can be a Bluetooth module.

[0105] The wireless communication module can be a built-in module of the communication device, such as those found in typical smart devices (like smartphones). This application utilizes the built-in wireless communication module of the communication device to collect signal status information of the wireless signal, enabling subsequent gesture recognition based on the collected signal status and information interaction based on the recognition results. This allows for efficient information transmission between different communication devices by leveraging the wireless communication channel on top of existing communication functions. Furthermore, deploying gesture recognition functionality on top of existing communication capabilities using the wireless communication channel eliminates the need for additional gesture recognition sensors in the communication device, thus freeing it from hardware constraints and reducing costs. For example, in Figure 3, the WLAN hardware module is the built-in wireless communication module, while the gesture recognition module is a newly added gesture recognition function module. In other embodiments, the wireless communication module is not a built-in module of the communication device; users can also plug in the wireless communication module to enable the communication device to interact using the information interaction method provided in this application.

[0106] In a scenario where a first device sends information to a second device, a first user performs a first gesture on the first device, for example, the first user performs a first gesture in the surrounding environment of the first device. This first gesture on the first device can affect the state of the first wireless signal when it reaches the first device, causing changes in multiple first signal state information collected during the gesture's execution. This allows the first device to subsequently perform gesture recognition based on the multiple first signal state information, identifying the first user's first gesture on the first device, and thus triggering the first device to send information from the first device to the second device.

[0107] Taking Figure 5 as an example, assuming the first device is device A and the first gesture is gesture A, in step 1 on the device A side, the first user performs gesture A in the surrounding environment of device A. Taking gesture A as a grasping gesture as an example, as shown in Figure 6, the first user performs a grasping gesture in front of device A to trigger device A to send information 103 from device A to device B.

[0108] In some embodiments, after performing step 411, the first device may further preprocess the plurality of first signal state information. Alternatively, preprocessing of the plurality of first signal state information may not be performed. Exemplarily, the preprocessing procedure includes at least one of steps A1 and A2 described below.

[0109] Step A1: The first device filters multiple first signal state information through a low-pass filter to obtain multiple filtered first signal state information.

[0110] Filtering can remove noise contained in the first signal state information. This noise can be changes in the signal state information caused by factors other than gestures. Thus, through step A1, the noise in the first signal state information can be prevented from affecting the gesture recognition result. Subsequent gesture recognition based on multiple filtered first signal state information can improve the accuracy of gesture recognition based on wireless signals.

[0111] Step A2: The first device will divide the signal state time series into multiple subsequences according to the first duration. The signal state time series includes multiple first signal state information and the acquisition time corresponding to each first signal state information. Each subsequence includes each first signal state information acquired within a first duration and the acquisition time corresponding to each first signal state information.

[0112] The first duration can be 3 seconds or 2 seconds, and the first duration can be set according to the specific implementation scenario. Here, the embodiment of this application does not limit the value of the first duration.

[0113] For example, taking a signal state time series acquired within 9 seconds as an example, assuming the first duration is 3 seconds, the first signal state information and its sampling time acquired from the 0th second to the 3rd second are divided into subsequence 1, the first signal state information and its sampling time acquired from the 3rd second to the 6th second are divided into subsequence 2, and the first signal state information and its sampling time acquired from the 6th second to the 9th second are divided into subsequence 3.

[0114] When the first signal state information includes CSIs, since the number of data packets carried by the wireless signal received by the first device in different time periods may be the same or different, and each CSI in the signal state time series corresponds to one data packet, the number of CSIs in multiple sub-sequences may be the same or different. The first device can also perform alignment operations on the CSIs in multiple sub-sequences to make the number of CSIs in multiple sub-sequences the same. For example, for any sub-sequence, if the number of CSIs in the sub-sequence is less than the target number, the first device inserts a reference CSI and the acquisition time corresponding to the reference CSI into the sub-sequence to perform interpolation processing on the sub-sequence so that the number of CSIs in the sub-sequence is equal to the target number; if the number of CSIs in the sub-sequence is greater than the target number, the first device deletes at least one CSI and the acquisition time corresponding to the at least one CSI from the sub-sequence to perform extraction processing on the sub-sequence so that the number of CSIs in the sub-sequence is equal to the target number; if the number of CSIs in the sub-sequence is equal to the target number, no interpolation or extraction processing is performed on the sub-sequence. The target number is the desired number of CSIs in the subsequence, and the reference CSI is the preset CSI.

[0115] The first device can perform step A1 or step A2 on the signal state time series, and the result of step A1 or step A2 is the preprocessed signal state time series. Alternatively, the first device can first perform step A1 on the signal state time series, and then perform step A2 on the filtered signal state time series to obtain multiple subsequences, which are the preprocessed signal state time series. Or, the first device can first perform step A2 on the signal state time series to obtain multiple subsequences, and then perform step A1 on each of these subsequences to obtain multiple filtered subsequences, which are the preprocessed signal state time series.

[0116] After obtaining the preprocessed signal state time series, the first device performs step 402 on the first signal state information in the preprocessed signal state time series. In some embodiments, the first device does not preprocess the signal state time series, but performs step 402 on the first signal state information in the signal state time series.

[0117] The above preprocessing process can be performed by the wireless communication module in the first device, or by the gesture recognition module in the first device.

[0118] After acquiring multiple first signal state information, the first device identifies the first user's gestures towards the first device based on these first signal state information. These multiple first signal state information can be unprocessed first signal state information or first signal state information from a preprocessed signal state time series. The first user's gestures towards the first device refer to gestures performed by the first user in the environment surrounding the first device.

[0119] During the process of collecting these multiple first signal state information, one or more users may have performed gestures in the surrounding environment of the first device, or no user may have performed gestures. The first device identifies the user's gestures towards the first device based on the multiple first signal state information to determine whether a user has performed a gesture towards the first device and, if so, what kind of gesture was performed. Based on this, the first device identifies the first user's gesture towards the first device based on the multiple first signal state information and can obtain an identification result. This identification result can be any one of a first identification result, a second identification result, or a third identification result. The first identification result indicates that the user's gesture towards the device is a first gesture, meaning that the user has performed a first gesture towards the device; the second identification result indicates that the user's gesture towards the device is a second gesture, meaning that the user has performed a second gesture towards the device; the third identification result indicates no action, meaning that the user has not performed either the first or second gesture towards the device.

[0120] A gesture to the first device can cause a change in the signal state information corresponding to the wireless signal. Based on this, the first device can identify the gesture of the first user to the first device based on the changes in multiple first signal state information. For example, the changes in the first signal state information can be the changes in the first signal state information over time. The process of identifying the gesture of the first user to the first device is as described in steps 412 and 413 below.

[0121] Step 412: The first device performs timing feature analysis on multiple first signal state information to obtain first timing features, which indicate the changes of the first signal state information over time.

[0122] The first temporal feature is the temporal feature corresponding to a sub-segment of the signal state time sequence. The sub-segment includes each first signal state information collected within a second duration and the corresponding collection time for each first signal state information. Compared to a sub-sequence, a sub-segment is a signal state time sequence with a smaller granularity in sampling duration; acquiring a sub-segment is equivalent to acquiring a smaller-granularity signal state time sequence. The second duration is less than or equal to the first duration, and the second duration is the maximum duration required for the user to perform the first or second gesture, such as 1 second or 0.5 seconds. The second duration can be set according to the first or second gesture; however, this embodiment does not limit the second duration.

[0123] Before extracting time-series features from multiple first signal state information, the first device divides the signal state time series into multiple segments based on the second duration. For example, if the signal state time series has not been divided into multiple subsequences, and the acquisition duration of the signal state time series is greater than the second duration, the first device divides the signal state time series into multiple segments according to the second duration (this process is the same as step A2 above); if the signal state time series has been divided into multiple subsequences, if the second duration is equal to the first duration, each subsequence is taken as a segment of the signal state time series; if the second duration is less than the second duration, for each subsequence, the first device divides the subsequence segment into multiple segments according to the second duration (this process is the same as step A2 above). Of course, if the acquisition duration of the signal state time series is equal to the second duration, the signal state time series is taken as a sub-segment. If the acquisition duration of the signal state time series is less than the second duration, a reference SCI and the acquisition time corresponding to each reference SCI are inserted into the signal state time series to increase the acquisition duration of the signal state time series to the second duration. The signal state time series after inserting the reference SCI is taken as a sub-segment. Therefore, it can be seen that the first device can obtain at least one sub-segment based on the signal state time series.

[0124] After obtaining at least one sub-segment, for each sub-segment, the first device performs timing feature extraction on the sub-segment to obtain the timing feature corresponding to the sub-segment. The timing feature indicates the change of the first signal state information in the sub-segment over time, and the timing feature is a first timing feature. The method of timing feature extraction is not limited in this application embodiment.

[0125] Taking Figure 7 as an example, the WLAN hardware module in the first device outputs a signal state time series to the gesture recognition module in the first device. The signal state time series includes multiple SCI time series 71 based on the WiFi signal, and each SCI time series 71 is a subsequence of the signal state time series. The gesture recognition module performs time series feature analysis on each input SCI time series 71. For example, for each input SCI time series 71, with d as the window sliding step size and the second duration as the window size, the window slides on each SCI time series 71, and after each slide, the area covered by the window on the SCI time series 71 is divided into a sub-segment 72, thus realizing the segmentation of the SCI time series. The gesture recognition module performs time series feature analysis (i.e., time series feature extraction) on each sub-segment 72 to obtain the time series features corresponding to each sub-segment 72.

[0126] In other embodiments, for each sub-segment, before performing time-series feature extraction on the sub-segment, the first device may further perform smoothing processing on the sub-segment to obtain a smoothed sub-segment. Then, time-series feature extraction is performed on the smoothed sub-segment to obtain the first time-series feature corresponding to the sub-segment. The smoothing process may involve filtering multiple first signal state information in the sub-segment using a low-pass filter to obtain a filtered sub-segment, thereby removing noise contained in the first signal state information of the sub-segment.

[0127] After obtaining the first timing feature corresponding to the sub-segment, the first device performs the following step 413 based on the first timing feature.

[0128] In other embodiments, other communication devices may exist around the first device. Gestures from other users targeting these other communication devices may also cause changes in the first signal state information. Furthermore, user gestures targeting communication devices are generally close to the device. Therefore, gestures within a first threshold distance from the communication device can be considered as user gestures targeting that device. Given that physical obstacles may also cause changes in the wireless signal state information, the first threshold can be set relatively small to avoid misidentifying changes in signal state information caused by obstacles around the communication device as changes caused by gestures targeting that device, thus preventing the misidentification of obstacles as gestures. The first threshold can be 0.5 meters, 0.3 meters, or 0.2 meters. The first threshold can be set according to the actual application scenario; this embodiment does not limit the first threshold.

[0129] After obtaining the first temporal feature corresponding to the sub-segment, the first device identifies the first user's gesture on the first device based on the first threshold and the first temporal feature.

[0130] For example, the first device determines a first distance based on a first timing feature. This first distance corresponds to the sub-segment and is the distance between the spatial location that causes a change in the first signal state information of the sub-segment and the first device. Given that users typically perform gestures on the display screen of a communication device, in some embodiments, the first distance may also be the distance between the spatial location that causes a change in the first signal state information of the sub-segment and the display screen of the first device.

[0131] The determination of the first distance is as follows: For example, each inflection point of the CSI in the first time-series feature is a point where the CSI changes. For each inflection point of the CSI in the first time-series feature, the first device uses a power distribution delay algorithm to calculate at least one inflection point distance. Each inflection point distance corresponds to one inflection point, and each inflection point distance is the distance between the spatial location of the wireless signal at the inflection point corresponding to the CSI of the wireless signal and the display screen of the first device. If an inflection point distance is calculated, this inflection point distance is used as the first distance. If multiple inflection point distances are calculated, the minimum distance among these multiple inflection point distances is used as the first distance to ensure that the gesture recognized for the first device based on the first distance is the gesture closest to the first device.

[0132] In other embodiments, the first device inputs a first temporal feature into a distance recognition model, the distance recognition model outputs a distance corresponding to the first temporal feature based on the first temporal feature, and the first device uses the distance output by the distance recognition model as the first distance.

[0133] The distance recognition model can be an artificial intelligence (AI) model, including machine learning models, deep learning models, or reinforcement learning models, such as neural network models that support machine learning, deep learning, or reinforcement learning. This application embodiment does not limit the structure and type of the distance recognition model. The distance recognition model is trained using multiple samples and their corresponding target distances. A sample is the temporal feature corresponding to a wireless signal affected by an obstacle. This obstacle can be a first gesture, a second gesture, or a physical obstacle. The target gesture corresponding to this sample is the distance between the obstacle and the communication device receiving the wireless signal. The AI ​​model is trained using the target distance corresponding to each sample as the desired output, enabling the trained AI model to learn the ability to output the target distance corresponding to the input sample. The trained AI model is thus the distance recognition model.

[0134] After determining the first distance through any of the above methods, if the first distance is less than or equal to the first threshold, it indicates that the first wireless signal corresponding to the sub-segment is suspected to be affected by the gesture directed at the first device. Then, the first device performs the following step 413 based on the first timing feature to identify the gesture directed at the first device, thereby avoiding misidentification of gestures directed at other communication devices as gestures directed at the first device.

[0135] If the first distance is greater than the first threshold, it means that the first wireless signal corresponding to that segment is not affected by the gesture directed at the first device, and the first device does not perform step 413 below. This allows the wireless signals that are not affected by the gesture directed at the first device to be filtered out, avoiding gesture recognition of these wireless signals and saving the computing resources of the first device.

[0136] Step 413: The first device identifies the gesture of the first user on the first device based on the first temporal feature.

[0137] In the environment surrounding the first device, one or more users may have performed gestures, or no user may have performed gestures. The first device identifies whether a user has performed a gesture on the first device based on changes in the first signal state information. If a gesture has been performed, the first device further identifies the specific type of gesture performed by the user on the first device based on changes in the first signal state information.

[0138] When the first wireless signal is a WiFi signal, the gestures of the first user toward the first device will affect the CSI of the first wireless signal from multiple dimensions such as antenna, time and frequency. Antenna, time and frequency are reflected in the spatial domain, time domain and frequency domain respectively. The first device can perform multi-dimensional feature analysis on the changes of CSI in multiple first signal state information in multiple dimensions such as spatial domain, time domain and frequency domain to perform gesture recognition, as in steps B1 and B2 below.

[0139] Step B1: Based on the first timing feature, the first device performs multi-dimensional feature analysis on the changes of CSI in multiple first signal state information to obtain the first frequency domain feature, the first time domain feature, and the first spatial domain feature. The first frequency domain feature represents the changes of CSI in multiple first signal state information in the frequency domain, the first time domain feature represents the changes of CSI in multiple first signal state information in the time domain, and the first spatial domain feature represents the changes of CSI in multiple first signal state information in the spatial domain.

[0140] Multi-dimensional feature analysis refers to analyzing the changes in CSI across multiple dimensions, including the frequency domain, time domain, and spatial domain. Each first-time-series feature corresponds to a sub-segment. Step B1 is executed for each sub-segment's corresponding first-time-series feature. Changes in CSI in the time domain refer to the changes in CSI amplitude and phase over time; changes in CSI in the frequency domain refer to the changes in CSI amplitude and phase over the number of subcarriers; and changes in CSI in the time domain refer to the changes in CSI amplitude and phase over the number of antennas.

[0141] For each segment corresponding to the first time-series feature, the first device performs multi-dimensional encoding and embedding processing on the first time-series feature to change the expression form of the first time-series feature. Based on the time-series feature after encoding and embedding processing, multi-dimensional feature analysis is performed on the change of CSI in the segment to obtain the first frequency domain feature, the first time domain feature and the first spatial domain feature.

[0142] For the first time-series feature after encoding and embedding, the first device performs convolution calculation on the first time-series feature in the antenna dimension (i.e., spatial dimension) to obtain the first spatial domain feature corresponding to the sub-segment, performs convolution calculation on the first time-series feature in the subcarrier dimension (i.e., frequency domain dimension) to obtain the first frequency domain feature corresponding to the sub-segment, and performs convolution calculation on the first time-series feature in the time dimension (i.e., time domain dimension) to obtain the first time domain feature corresponding to the sub-segment.

[0143] Taking the temporal features corresponding to a segment 72 in Figure 7 as an example, the gesture recognition module in the first device encodes and embeds these temporal features using multi-dimensional features to obtain processed temporal features, and then performs gesture recognition based on these processed temporal features. For example, in the antenna dimension, the gesture recognition module performs multiple convolution calculations on the processed temporal features to obtain the spatial domain features (i.e., the first spatial domain features) corresponding to segment 72; in the frequency domain dimension, it performs multiple convolution operations on the spatial domain features to obtain the frequency domain features (i.e., the first frequency domain features) corresponding to segment 72; and in the time domain dimension, it performs multiple convolution operations on the frequency domain features to obtain the time domain features (i.e., the first time domain features) corresponding to segment 72. Each convolution calculation includes operations such as convolution with the corresponding features using a convolution kernel, activation of the convolution result using a rectified linear unit (ReLU), and pooling of the activation result. Convolution calculations can be implemented using convolutional neural networks.

[0144] By performing multi-dimensional convolution calculations on the first temporal feature, the first temporal feature is mapped to a high-dimensional space, obtaining the multi-dimensional features (including temporal domain features, frequency domain features, and spatial domain features) corresponding to the segment. This captures the more complex temporal dependencies and spatial features of the CSI, enabling comprehensive and detailed extraction of the complex features of the CSI. Subsequent gesture recognition based on these complex CSI features further improves the accuracy of gesture recognition based on wireless signals. In other embodiments, when performing convolution calculations on the first temporal feature in the time dimension, the first device may also introduce a self-attention mechanism to accurately capture the features of CSI changing over time, further improving the accuracy of subsequent gesture recognition based on wireless signals.

[0145] Step B2: The first device identifies the gesture of the first user on the first device based on the first frequency domain feature, the first time domain feature and the first spatial domain feature.

[0146] In this embodiment, the first gesture corresponds to the first target frequency domain feature, the first target time domain feature, and the first target spatial domain feature; the second gesture corresponds to the first target frequency domain feature, the second target time domain feature, and the second target spatial domain feature. For any one of the first and second target frequency domain features, the target frequency domain feature refers to the frequency domain feature of the CSI corresponding to the wireless signal under the influence of the corresponding gesture, representing the change of the CSI corresponding to the wireless signal in the frequency domain under the influence of the corresponding gesture. For any one of the first and second target time domain features, the target time domain feature refers to the frequency domain feature of the CSI corresponding to the wireless signal under the influence of the corresponding gesture, representing the change of the CSI corresponding to the wireless signal in the time domain under the influence of the corresponding gesture. For any one of the first and second target spatial domain features, the target time domain feature refers to the spatial domain feature of the CSI corresponding to the wireless signal under the influence of the corresponding gesture, representing the change of the CSI corresponding to the wireless signal in the spatial domain under the influence of the corresponding gesture.

[0147] In the aforementioned application environment, any communication device stores: a correspondence between a first gesture and the frequency domain features, time domain features, and spatial domain features of a first target (referred to as the first correspondence), and a correspondence between a second gesture and the frequency domain features, time domain features, and spatial domain features of a second target (referred to as the second correspondence). This allows the communication device to identify the user's gestures towards the device during information interaction based on these two correspondences and the frequency domain features, time domain features, and spatial domain features corresponding to the received wireless signal. For example, both the first device and the second device store these two correspondences.

[0148] Next, we will introduce how the first device recognizes the first user's gestures on the first device based on these two correspondences, the first frequency domain feature, the first time domain feature, and the first spatial domain feature.

[0149] For any given segment, the first frequency domain feature, the first time domain feature, and the first spatial domain feature are matched. The first device matches the first frequency domain feature with the first target frequency domain feature and the second target frequency domain feature, respectively, to obtain the frequency domain matching result for that segment. For example, the similarity between the first frequency domain feature and the first target frequency domain feature is calculated, and the similarity between the first frequency domain feature and the second target frequency domain feature is also calculated. For any target frequency domain feature among the first and second target frequency domain features, the similarity between the target frequency domain feature and the first frequency domain feature indicates the degree of similarity between them. If the similarity between any target frequency domain feature and the first frequency domain feature is greater than or equal to a similarity threshold, it indicates that the first frequency domain feature and the target frequency domain feature are relatively similar, and thus the target frequency domain feature matches the first frequency domain feature. If the similarity between the target frequency domain feature and the first frequency domain feature is less than the similarity threshold, it indicates that the first frequency domain feature and the target frequency domain feature are not very similar, and thus the target frequency domain feature does not match the first frequency domain feature.

[0150] Following a similar matching method, the first device matches the first time-domain feature with the first target time-domain frequency-domain feature and the second target time-domain feature respectively to obtain the time-domain matching result corresponding to the segment, and matches the first spatial-domain feature with the first target spatial-domain feature and the second target spatial-domain feature respectively to obtain the spatial-domain matching result corresponding to the segment.

[0151] Based on the frequency domain matching result, time domain matching result, and spatial domain matching result corresponding to the sub-segment, the first device identifies the gesture of the first user towards the first device, and obtains the recognition result for the sub-segment. For example, for the first frequency domain feature, first time domain feature, and first spatial domain feature corresponding to the sub-segment, if at least one of the first, second, and third conditions is met, the first device recognizes the first gesture as a gesture towards the first device, and the recognition result is the first recognition result. The first condition is that the first frequency domain feature matches the first target frequency domain feature; the second condition is that the first time domain feature matches the first target time domain feature; and the third condition is that the first spatial domain feature matches the first target spatial domain feature. If at least one of the following conditions is met: the first frequency domain feature matches the second target frequency domain feature; the first time domain feature matches the second target time domain feature; and the first spatial domain feature matches the second target spatial feature, the second gesture is recognized as a gesture towards the first device, and the recognition result is the second recognition result. If the first frequency domain feature does not match the first target frequency domain feature or the second target frequency domain feature, the first time domain feature does not match the first target time domain feature or the second target time domain feature, and the first spatial domain feature does not match the first target spatial domain feature or the second target spatial domain feature, the first device obtains the third recognition result.

[0152] Taking Figure 7 as an example, the gesture recognition module recognizes the user's gesture toward device A based on the spatial, frequency, and temporal features corresponding to sub-segment 72, and obtains the recognition result to complete the gesture recognition for the sub-segment. The recognition result may indicate that the user's gesture toward the device is a grasping gesture (i.e., the first gesture), or it may indicate that the user's gesture toward the device is a release gesture (i.e., the second gesture), or it may indicate no action.

[0153] In a scenario where the first device sends information to the second device, in order to trigger the first device to send information, the first user performs a first gesture on the first device. For the sub-segment affected by the first gesture, when recognizing the first user's gesture on the first device according to the first frequency domain feature, the first time domain feature and the first spatial domain feature corresponding to the sub-segment, the gesture on the first device is recognized as the first gesture.

[0154] Taking Figure 5 as an example, assuming the first device is device A and the first gesture is gesture A, in step 2 on the device A side, device A collects the CSI sequence of the received WiFi signal, and performs cross-device gesture recognition based on the CSI sequence of the WiFi signal (such as the process shown in steps 412 and 413 above) to identify the gesture of the first user towards device A.

[0155] In other embodiments, for any sub-segment corresponding to a first frequency domain feature, a first time domain feature, and a first spatial domain feature, the first device first determines whether the amplitude of the first frequency domain feature is greater than or equal to a first amplitude, whether the amplitude of the first time domain feature is greater than or equal to a second amplitude, and whether the amplitude of the first spatial domain feature is greater than or equal to a third amplitude. If at least one of the following conditions is met: the amplitude of the first frequency domain feature is greater than or equal to the first amplitude, the amplitude of the first time domain feature is greater than or equal to the second amplitude, and the amplitude of the first spatial domain feature is greater than or equal to the third amplitude, then it indicates that the sub-segment is suspected to be affected by a gesture, and the first device uses this sub-segment as a candidate sub-segment and performs step B2 for the candidate sub-segment. If the amplitude of the first frequency domain feature is less than the first amplitude, the amplitude of the first time domain feature is less than the second amplitude, and the amplitude of the first spatial domain feature is less than the third amplitude, then the sub-segment is not affected by a gesture, and the first device does not perform step B2 for this sub-segment.

[0156] According to the above determination method, the first device can determine which segments are candidate segments from multiple segments, and perform gesture recognition on the candidate segments instead of on the non-candidate segments, thereby saving the computing resources of the first device.

[0157] Steps 412 and 413 above, based on the changes in signal state information corresponding to the wireless signal received by the first device, can accurately identify the gesture directed at the first device, thus improving the accuracy of gesture recognition based on wireless signals. Steps B1 and B2 above fully consider the influence of the user's gesture wireless signal in multiple dimensions such as the frequency domain, time domain, and spatial domain, making the identified gesture more accurate, thereby further improving the accuracy of gesture recognition based on wireless signals. The multi-dimensional features corresponding to the first gesture (such as the first target frequency domain features, the first target time domain features, and the first target spatial domain features) can comprehensively and meticulously reflect the influence of the first gesture on the CSI corresponding to the wireless signal. Therefore, the multi-dimensional features corresponding to the first gesture are deep and complex features of the CSI mined for the first gesture. Through the multi-dimensional features corresponding to the first gesture, the action semantics of the first gesture can be accurately represented. Furthermore, by matching the multi-dimensional features corresponding to the first wireless signal (such as the first frequency domain features, the first time domain features, and the first spatial domain features) with the multi-dimensional features corresponding to the first gesture, it is possible to accurately identify whether the user's gesture directed at the first device is the first gesture, thereby further improving the accuracy of gesture recognition based on wireless signals. Any of the above methods for improving the accuracy of gesture recognition based on wireless signals enables the first device to perform information interaction based on accurate gesture recognition results.

[0158] After recognizing the gesture of the first user towards the first device, if the gesture towards the first device is a first gesture and a first distance is determined based on the first timing feature, the first device can use the first distance as the first gesture distance of the first device, which is the distance between the device and the first gesture towards the device. If the first distance is the distance between the spatial location that caused the change in the first signal state information and the display screen of the first device, then the first gesture distance is also the distance between the display screen of the device and the first gesture towards the device.

[0159] In other embodiments, the processes shown in steps 412 and 413 above are implemented by a gesture recognition model. The gesture recognition model can be an AI model, including a machine learning model, a deep learning model, or a reinforcement learning model, such as a neural network model that supports machine learning, deep learning, or reinforcement learning. Here, the embodiments of this application do not limit the structure and type of the gesture recognition model.

[0160] The gesture recognition model is used to recognize user gestures towards a communication device based on subsequences of signal state sequences corresponding to wireless signals. The specific process of gesture recognition is described in steps 412 and 413 above, and will not be repeated here. The gesture recognition model is trained based on multiple first samples and multiple second samples. Each first sample includes a subsequence affected by a first gesture towards the communication device, and each second sample includes a subsequence affected by a second gesture towards the communication device. Multiple first samples correspond to a first recognition result, and multiple second samples correspond to a second recognition result. During training, the training device uses each first sample or each second sample as an input sample and the recognition result corresponding to the input sample as the desired model output to train the AI ​​model multiple times. In each training process, multiple first samples and multiple samples are input into the AI ​​model. Based on the error between the recognition result output by the AI ​​model and the recognition result corresponding to the input sample, the model parameters of the AI ​​model are updated, and the AI ​​model with updated model parameters enters the next training session until the error between the recognition result output by the AI ​​model and the recognition result corresponding to the input sample is less than or equal to a first error threshold, at which point the AI ​​model training is complete. After multiple training iterations, the AI ​​model learns the ability to recognize user gestures towards a communication device based on changes in signal state information within a subsequence. This allows the trained AI model to identify user gestures towards the communication device based on changes in signal state information within the subsequence; the trained AI recognition model is thus a gesture recognition model. In other embodiments, multiple first samples and multiple second samples also correspond to gesture distances. The gesture distance corresponding to the first sample is the distance between the communication device (or its display screen) and the first gesture towards the communication device, and the gesture distance corresponding to the second sample is the distance between the communication device (or its display screen) and the second gesture towards the communication device. During training, the AI ​​model is trained multiple times using the recognition results and corresponding distances of the input samples as the desired model output. In each training iteration, the AI ​​model also performs gesture distance recognition for each input sample and outputs the gesture distance recognized for each input sample. When updating the AI ​​model's parameters, the model parameters are also updated based on the error between the gesture distance corresponding to each input sample and the gesture distance recognized by the AI ​​model for each input sample. After multiple training iterations, the AI ​​model training is complete when the error between the recognition result output by the AI ​​model and the recognition result corresponding to the input sample is less than or equal to a first error threshold, and the error between the gesture distance corresponding to the input sample and the gesture distance recognized by the AI ​​model for the input sample is less than or equal to a second error threshold. This enables the trained AI model (i.e., the gesture recognition model) to also learn the ability to recognize gesture distances based on changes in signal state information within subsequences.

[0161] The gesture recognition model can be deployed in the communication device, and the gesture recognition model is used to perform gesture recognition based on multiple signal state information. Taking the communication device as the first device as an example, the first device inputs a subsequence of the acquired signal state time series into the gesture recognition model. The gesture recognition model recognizes the gesture of the first user towards the first device based on the subsequence (i.e., steps 412 and 413), and outputs the recognition result corresponding to each sub-segment of the subsequence. For the recognition result corresponding to each sub-segment, it is determined whether to execute the following step 402. If the gesture recognition model can also learn the ability to recognize the gesture distance based on the changes in signal state information in the subsequence, when the first device uses the gesture recognition model to recognize the gesture towards the first device, the gesture recognition model also performs gesture distance recognition based on the subsequence, and also outputs the gesture distance corresponding to each sub-segment of the subsequence. The first device uses the gesture distance corresponding to each sub-segment as the first gesture distance.

[0162] After recognizing the gesture of the first user toward the first device, if the gesture toward the first device is the first gesture, the first device performs step 402 below; if the gesture toward the first device is not the first gesture, step 402 below is not performed.

[0163] 402. If the gesture for the first device is a first gesture, the first device sends first information to the second device. The first gesture indicates that information is sent to the associated device. The second device is the associated device of the first device. The first information is the information in the first device that participates in cross-device information interaction.

[0164] The first gesture and the second gesture correspond to each other, and the combined gesture of the first gesture and the second gesture is used to trigger information interaction between different communication devices.

[0165] In a scenario where a first device sends information to a second device, a first gesture is used to trigger the first device to send information from its own memory. For example, after the first device recognizes a gesture directed at it, if the gesture is a first gesture and corresponds to a second gesture, the first device sends first information to the second device, thus enabling information interaction with the second device triggered by a combination of gestures. For instance, the first device stores a correspondence between the first and second gestures (referred to as a gesture correspondence), indicating that the combined gesture, the first gesture, and the second gesture correspond. After recognizing a gesture directed at it, the first device queries the gesture correspondence for the recognized gesture. If the recognized gesture is the first gesture, it sends first information to the second device.

[0166] The first information can be any content in the first device, such as text information, video, audio, files, images, or other types of information. This application embodiment does not limit the type of information for the first information. The first device has at least one associated device, and the second device can be any of the associated devices of the first device.

[0167] If the gesture for the first device is a first gesture, the first device determines first information from the information in the first device and sends the first information to the second device. The first device determines the first information using either method 1 or method 2 described below.

[0168] Method 1: The first device determines the first information based on the content displayed on the interface currently displayed on the first device, and the first information is associated with the content.

[0169] For example, if the gesture on the first device is a first gesture, the first device takes a screenshot of the currently displayed interface to obtain a target image, which includes the content displayed on the currently displayed interface. The first device identifies the content in the target image and determines first information based on the identified content. Specifically, the first device determines the first information based on the service to which the identified content belongs. For example, if the first device plays a video on the currently displayed interface, and the identified service belongs to the video playback service, then the video to which the content belongs is taken as the first information. As another example, if the first device displays images from an album on the currently displayed interface, and the identified service belongs to the image display service, then the image to which the content belongs is taken as the first information. As another example, if the first device displays a file list on the currently displayed interface, and the identified service belongs to the file display service, then each file involved in the content (i.e., the file list) is taken as the first information. As yet another example, if the first device displays the content of a document on the currently displayed interface, and the identified service belongs to the document display service, then the document to which the content belongs is taken as the first information.

[0170] In the above-mentioned determination method 1, no user operation is required, which simplifies the information interaction process across devices and improves the efficiency of information interaction across devices.

[0171] Method 2: The first user first selects the information to be sent from the information in the first device, and then performs a first gesture on the first device. Subsequently, if the gesture of the first user on the first device is recognized as the first gesture, the first device determines the information to be sent as the first information.

[0172] In the above-mentioned determination method 1, the accuracy of the information sent during the information interaction process is improved by allowing the user to select the information to be sent.

[0173] In some embodiments, a first gesture corresponds to an information sending command, and a second gesture corresponds to an information retrieval command. The information sending command indicates information to be sent to an associated device, and the information retrieval command indicates information to be retrieved from the associated device. Any communication device in the above application environment stores a correspondence between the first gesture and the information sending command (referred to as a third correspondence) and a correspondence between the second gesture and the information retrieval command (referred to as a fourth correspondence), so that in cross-device information interaction scenarios, the communication device can interact with the associated device by executing the command corresponding to the recognized gesture.

[0174] Taking the communication device as the first device as an example, after recognizing the first user's gesture towards the first device through step 401, the first device determines the instruction corresponding to the recognized gesture based on the third and fourth correspondence relationships. For example, if the gesture towards the first device is a first gesture, the first device determines that the gesture towards the first device corresponds to an information sending instruction based on the third correspondence relationship, and sends the first information to the second device by executing the information sending instruction. For example, the first device determines the first information from the information in the first device by executing the information sending instruction, and sends the first information to the second device.

[0175] Taking Figure 5 as an example, in step 2 on device A, the gesture recognition module of device A identifies the user's gesture towards device A as gesture A. In step 3 on device A, the gesture recognition module obtains the information sending command corresponding to gesture A through command discrimination. In step 7, the information sending command is executed to send information (i.e., the first information) to device B, thereby realizing cross-device data traffic and completing information interaction between devices. For example, based on the information sending command, the gesture recognition module calls the screenshot function of device A to take a screenshot of the current screen page of device A. Based on the screenshot result, it determines the information to be sent and sends an information sending command to the information transmission tool in device A, indicating the information to be sent in the information sending command so that the information transmission tool can send the corresponding information to device B based on the indication of the information sending command.

[0176] The processes shown in steps 401 to 402 above are the method steps applied to the first device in the information interaction method provided in this application.

[0177] 403. The second device identifies the gestures of the second user toward the second device based on the second wireless signal, wherein the second wireless signal is the wireless signal received by the second device.

[0178] The second device is an associated device of the first device. The definitions of the second device and the second user have been described above and will not be repeated here. During wireless communication, the second device receives wireless signals sent by other devices. The second wireless signal is a portion of the received wireless signals and is used to carry a data packet.

[0179] The second device can identify the second user's gestures on the second device based on the state when the second wireless signal arrives at the second device. For example, steps 431 to 433 below.

[0180] Step 431: The second device acquires multiple second signal status information, which indicate the status of the second wireless signal used to carry a data packet when it reaches the second device.

[0181] In this step 431, the same principle applies as step 411 above. The difference is that in this step 431, the multiple second signal states acquired by the second device correspond to the second wireless signal and the data packet carried by the second wireless signal. The second wireless signal is the wireless signal received by the second device. Here, this application embodiment will not elaborate on this step 431.

[0182] Step 431 can be performed by the wireless communication module in the second device. The wireless communication module is a module in the communication device used for wireless communication, which is used to receive and parse wireless signals. When the wireless signal is a WiFi signal, the wireless communication module can be the WLAN hardware module shown in Figure 3. When the wireless signal is a Bluetooth signal, the wireless communication module can be a Bluetooth module.

[0183] In a scenario where the first device sends information to the second device, the second device acts as the information receiver. The second user performs a second gesture on the second device, for example, in the surrounding environment. This second gesture affects the state of the second wireless signal upon reaching the second device, causing changes in multiple second signal state information collected during the gesture's execution. This allows the second device to perform gesture recognition based on the multiple second signal state information, identifying the second user's gesture and triggering the second device to retrieve information from the first device.

[0184] Taking Figure 5 as an example, assuming the second device is device B and the second gesture is gesture B, in step 4 on the device B side, the second user performs gesture B in the surrounding environment of device B. Taking gesture B as a grasping gesture as an example, as shown in Figure 6, the second user performs a grasping gesture in front of device B, triggering device B to obtain information 103 from device A.

[0185] In some embodiments, after performing step 431, the second device may further preprocess the multiple second signal state information, the process being similar to the process by which the first device preprocesses the multiple first signal state information. Of course, the second device may also choose not to preprocess the multiple first signal state information.

[0186] After acquiring multiple second signal state information, the second device identifies the second user's gestures towards the second device based on these second signal state information. These multiple second signal state information can be either unprocessed or preprocessed. The second user's gestures towards the second device refer to gestures performed by the second user in the environment surrounding the second device.

[0187] During the process of collecting these multiple second signal state information, one or more users may have performed gestures in the surrounding environment of the second device, or no user may have performed gestures. The second device identifies the user's gestures towards the second device based on the multiple second signal state information, in order to identify whether a user has performed a gesture towards the second device and, if so, what kind of gesture has been performed. Based on this, the second device identifies the second user's gestures towards the second device based on the multiple second signal state information, and can obtain an identification result, which is any one of the first identification result, the second identification result, or the third identification result.

[0188] Gestures on the second device can cause changes in the signal state information corresponding to the wireless signal. Based on this, the second device can recognize the second user's gestures on the second device based on changes in multiple second signal state information. For example, the changes in the second signal state information can be changes in the second signal state information over time. The process of recognizing the second user's gestures on the second device is as described in steps 432 and 433 below.

[0189] Step 432: The second device performs timing feature analysis on multiple second signal state information to obtain second timing features, which indicate the changes of the second signal state information over time.

[0190] The second temporal feature is the temporal feature corresponding to a sub-segment of the signal state time sequence including the second signal state information. This sub-segment includes each piece of second signal state information collected within a second duration and the collection time corresponding to each piece of second signal state information. This step 432 is similar to step 412 above, and therefore, this embodiment of the application will not elaborate on step 432 here.

[0191] After obtaining the second timing feature, the second device performs the following step 433 based on the second timing feature.

[0192] In other embodiments, after obtaining the second timing feature, the first device identifies the second user's gesture towards the second device based on the first threshold and the second timing feature. For example, the second device determines a second distance based on the second timing feature. This second distance is the distance between the spatial location that caused the change in the second signal state information and the second device (or the display screen of the second device). The method for determining the second distance can refer to the method for determining the first distance, and will not be repeated here. If the second distance is less than or equal to the first threshold, it indicates that the second wireless signal corresponding to the second timing feature is suspected to be affected by the gesture towards the second device. The second device then performs step 433 based on the second timing feature to identify the gesture towards the second device, thereby avoiding misidentification of gestures towards other communication devices as gestures towards the second device. If the second distance is greater than the first threshold, it indicates that the second wireless signal corresponding to that segment is not affected by the gesture towards the second device, and the second device does not perform step 433. This allows filtering out wireless signals not affected by the gesture towards the second device, avoiding gesture recognition of these wireless signals, and saving the computing resources of the second device.

[0193] Step 433: The second device identifies the gesture of the second user on the second device based on the second temporal features.

[0194] In the surrounding environment of the second device, one or more users may have performed gestures, or no user may have performed gestures. The second device can identify whether a user has performed a gesture on the second device based on changes in the second signal state information. If a gesture has been performed, the second device can further identify which specific gesture the user performed on the second device based on changes in the second signal state information.

[0195] When the second wireless signal is a WiFi signal, the gestures of the second user toward the second device will affect the CSI of the second wireless signal from multiple dimensions such as antenna, time and frequency. Antenna, time and frequency are reflected in the spatial domain, time domain and frequency domain respectively. The second device can perform multi-dimensional feature analysis on the changes of CSI in multiple second signal state information in multiple dimensions such as spatial domain, time domain and frequency domain to perform gesture recognition, as in steps C1 and C2 below.

[0196] Step C1: Based on the second timing features, the second device performs multi-dimensional feature analysis on the changes of CSI in multiple second signal state information to obtain second frequency domain features, second time domain features, and second spatial domain features. The second frequency domain features represent the changes of CSI in multiple second signal state information in the frequency domain, the second time domain features represent the changes of CSI in multiple second signal state information in the time domain, and the second spatial domain features represent the changes of CSI in multiple second signal state information in the spatial domain.

[0197] Step C1 is similar to step B1 above, and will not be described again in this embodiment of the application.

[0198] Step C2: The second device identifies the second user's gestures on the second device based on the second frequency domain features, the second time domain features, and the second spatial domain features.

[0199] Step C2 is similar to step B2 above.

[0200] Taking any sub-segment containing multiple second signal state information as an example, for the second frequency domain feature, second time domain feature, and second spatial domain feature corresponding to this sub-segment, if at least one of the fourth, fifth, and sixth conditions is met, the second device recognizes the second gesture as a gesture directed at the second device. In this case, the recognition result is the second recognition result. Specifically, the fourth condition is a match between the second frequency domain feature and the second target frequency domain feature; the fifth condition is a match between the second time domain feature and the second target time domain feature; and the sixth condition is a match between the second spatial domain feature and the second target spatial feature. The specific matching process can be found in the relevant description in step B2 above and will not be repeated here.

[0201] If at least one of the following conditions is met: the second frequency domain feature matches the first target frequency domain feature; the second time domain feature matches the first target time domain feature; or the second spatial domain feature matches the first target spatial domain feature, the second device recognizes the first gesture as a gesture directed at the second device, and the recognition result is the first recognition result. If the second frequency domain feature does not match either the first or second target frequency domain feature, and the second time domain feature does not match either the first or second target time domain feature, and the second spatial domain feature does not match either the first or second target spatial domain feature, the second device obtains a third recognition result.

[0202] In a scenario where the first device sends information to the second device, in order to trigger the second device to obtain information from the first device, the second user performs a second gesture on the second device. For the sub-segment affected by the second gesture, when recognizing the second user's gesture on the second device according to the second frequency domain feature, second time domain feature and second spatial domain feature corresponding to the sub-segment, the gesture on the second device is recognized as the second gesture.

[0203] Taking Figure 5 as an example, assuming the second device is device B and the second gesture is gesture B, in step 5 on the device B side, device B collects the CSI sequence of the received WiFi signal, and performs cross-device gesture recognition based on the CSI sequence of the WiFi signal (such as the process shown in steps 432 and 433 above) to identify the gesture of the second user towards device B.

[0204] After recognizing the second user's gesture towards the second device, if the gesture towards the second device is a second gesture and a second distance is determined based on the second timing feature, the second device can use the second distance as the second gesture distance of the second device. The second gesture distance is the distance between the device and the second gesture towards the device. If the second distance is the distance between the spatial location that triggered the change in the second signal state information and the display screen of the second device, then the second gesture distance is also the distance between the display screen of the device and the second gesture towards the device.

[0205] In other embodiments, the processes shown in steps 432 and 433 above are implemented using a gesture recognition model. For example, the second device inputs a subsequence of the signal state time series obtained in step 431 into the gesture recognition model. The gesture recognition model recognizes the second user's gestures toward the second device based on the subsequence (i.e., steps 432 and 433), and outputs the recognition results corresponding to each segment of the subsequence. For each segment's recognition result, it determines whether to execute step 404 below. If the gesture recognition model can also learn the ability to recognize gesture distance based on changes in signal state information in the subsequence, when the second device uses the gesture recognition model to recognize gestures toward the second device, the gesture recognition model also performs gesture distance recognition based on the subsequence and outputs the gesture distances corresponding to each segment of the subsequence. The second device uses the gesture distances corresponding to each segment as the second gesture distance.

[0206] The second device identifies the second user's gesture towards the second device by performing step 403. If the gesture towards the second device is a second gesture, then step 404 is performed. If the gesture towards the second device is not a second gesture, then step 404 is not performed.

[0207] Step 403 may be executed before step 401, or at any time during the process from step 401 to step 402, or after step 402. In this embodiment, the execution order of step 403 is not limited.

[0208] 404. If the gesture for the second device is a second gesture, the second device obtains the first information, and the second gesture indicates that information in the associated device is obtained.

[0209] Among them, the second device has at least one associated device, and the first device is any one of the associated devices of the second device.

[0210] In a scenario where the first device sends information to the second device, the second gesture is used to trigger the second device to retrieve information from the first device. For example, after the second device recognizes a gesture directed at it, if the gesture is a second gesture and corresponds to the first gesture, the second device retrieves the first information from the first device to achieve information interaction with the first device triggered by combining gestures. For instance, the second device stores a gesture correspondence between the first and second gestures. After recognizing a gesture directed at it, the second device queries the gesture correspondence for the recognized gesture. If the recognized gesture is the second gesture, it retrieves the first information from the first device.

[0211] In other embodiments, the first gesture corresponds to an information sending command, the second gesture corresponds to an information retrieval command, and the second device stores a third correspondence and a fourth correspondence. After recognizing the second user's gesture towards the second device through step 403 above, the second device determines the command corresponding to the recognized gesture based on the third and fourth correspondences. For example, if the gesture towards the second device is the second gesture, the second device determines, based on the fourth correspondence, that the recognized gesture corresponds to an information retrieval command, and retrieves the first information from the first device by executing the information retrieval command.

[0212] Taking Figure 5 as an example, in step 2 on the device B side, the gesture recognition module of device B identifies the user's gesture towards device B as gesture B. In step 6 on the device B side, the gesture recognition module obtains the information retrieval instruction corresponding to gesture B through instruction discrimination. In step 7, by executing the information retrieval instruction, information from device A is obtained to realize cross-device data traffic and complete information interaction between devices.

[0213] In some embodiments, after obtaining the first information, the second device may display the first information on the currently displayed interface to indicate to the second user that the first information has been obtained. In other embodiments, after obtaining the first information, the second device does not perform the step of displaying the first information on the currently displayed interface, but instead displays a prompt message on the currently displayed interface to indicate to the second user that the first information has been obtained.

[0214] In steps 402 and 404, the first information is sent from the first device to the second device and the first device receives the first information, thereby enabling the flow of the first information between the first device and the second device. When executing steps 402 and 404, the cross-device data flow between the first device and the second device can be completed using any of the following data flow methods 1 to 3 for the first information.

[0215] Data transfer method 1: The second device actively requests the first device to obtain the first information in the first device.

[0216] As shown in Figure 8, data transfer method 1 includes steps D1 to D4.

[0217] Step D1: If the gesture for the second device is the second gesture, the second device broadcasts an information acquisition request, which indicates a request to acquire information from the associated device.

[0218] In this embodiment, the second device and its associated device (such as the first device) have the same association information, and the information acquisition request includes the association information and the address information of the second device. In other embodiments, the information acquisition request of any communication device (such as the second device) also includes a second gesture distance of the communication device.

[0219] Assuming a gesture from the second user towards the second device is identified in step 403, if the gesture towards the second device is a second gesture, the second device generates an information retrieval request based on the association information and the address information of the second device, or, based on the association information, the distance of the second gesture, and the address information. After generating the information retrieval request, the second device broadcasts the information retrieval request. The broadcasting method for the information retrieval request is, for example, the second device sends the information retrieval request to the wireless devices it is connected to, and the wireless devices broadcast the information retrieval request in their respective WLANs, so that all communication devices connected to the WLAN can receive the information retrieval request. Correspondingly, if the second device's associated device (such as the first device) and the second device are both connected to the same WLAN, the second device's associated device can also receive the information retrieval request.

[0220] In other embodiments, the second device and its associated devices may be connected to different WLANs. Taking the first device as an example, the first device is connected to a wireless device in a first WLAN (referred to as the first wireless device), and the second device is connected to a wireless device in a second WLAN (referred to as the second wireless device). The first and second WLANs are different WLANs, connected through a communication network. For this connection scenario, the information retrieval request can be broadcast. For example, the second device sends an information retrieval request to the second wireless device, which then broadcasts the request in the second WLAN and to the communication network. The communication network then forwards the request to other WLANs it is connected to. Upon receiving the request, the wireless devices in other WLANs broadcast it in their respective WLANs, allowing communication devices connected to those WLANs to access the request. Correspondingly, when a wireless device in the first WLAN receives and broadcasts an information retrieval request in the first WLAN, the first device connected to the first WLAN can also receive the request.

[0221] Step D2: The first device receives an information acquisition request.

[0222] The first device may receive the information acquisition request before or after step 402. Here, this embodiment does not limit the execution order between step D2 and step 402.

[0223] Step D3: If the gesture for the first device is the first gesture, the first device sends the first information to the second device based on the information acquisition request.

[0224] Upon receiving an information retrieval request, the first device determines whether the second device is associated with the first device based on the association information in the information retrieval request. For example, if the first device has the association information in the information retrieval request, it means that the first device is associated with the second device indicated by the address information in the information retrieval request, and the first device is an associated device of the second device; if the first device does not have the association information in the information retrieval request, it means that the first device is not associated with the second device indicated by the address information in the information retrieval request, and the first device is not an associated device of the second device.

[0225] If the second device is not associated with the first device, the first device discards the information retrieval request and does not execute the step of sending the first information to the second device, i.e., it does not respond to the information retrieval request. If the second device is associated with the first device, the first device decides whether to respond to the information retrieval request based on the recognition result of step 401 above. For example, if no gesture for the first device is recognized in step 401 (i.e., a third recognition result is obtained in step 401), the first device discards the information retrieval request and does not execute the step of sending the first information to the second device. If a gesture for the first device is recognized in step 401, if the gesture for the first device is not the first gesture, the first device discards the information retrieval request and does not execute the step of sending the first information to the second device. If the gesture for the first device is the first gesture, the first device sends the first information to the second device to respond to the information retrieval request. The method of sending the first information is, for example, the first device sends an information sending request to the second device based on the address information in the information retrieval request. The information sending request indicates that information is sent to the associated device, and the information sending request includes the first information.

[0226] The above description uses the example of a first device receiving an information transmission request from another device (i.e., a second device). In other embodiments, other communication devices may exist around the second device. A second user's gesture towards the second device may also trigger other communication devices to broadcast information acquisition requests, allowing the first device to receive information acquisition requests from other communication devices as well. Of course, even if a communication device is not one of the other communication devices around the second device, a user's second gesture towards that device may still trigger a broadcast information acquisition request from that device. Therefore, the first device may receive information acquisition requests from multiple devices. For ease of description, these multiple devices are referred to as fourth devices, meaning that multiple fourth devices include the second device and other communication devices besides the second device.

[0227] In a scenario where a first device receives information retrieval requests from multiple fourth devices, the first device obtains the second gesture distance of each fourth device from the information retrieval requests, for example, obtaining the second gesture distance of one fourth device from one information retrieval request. If the gesture for the first device is a first gesture, the first device can identify the fourth device with the smallest second gesture distance among the multiple fourth devices as its associated device and send first information to the fourth device with the smallest second gesture distance.

[0228] For example, if the second gesture distance of the second device among multiple fourth devices is the smallest and the gesture towards the first device is the first gesture, the first device sends an information interaction confirmation request to the second device. This information interaction confirmation request is used to confirm whether to perform cross-device information interaction. The second device receives the information interaction confirmation request, displays it, and the second user can confirm the request, indicating confirmation of cross-device information interaction. In response to this confirmation, the second device sends an information interaction confirmation response to the first device, indicating confirmation of cross-device information interaction. If the first device receives the information interaction confirmation response from the second device, it sends first information to the second device.

[0229] Among multiple fourth devices, the device with the smallest second gesture distance is not necessarily the device that will participate in cross-device information interaction. For example, suppose the second device among multiple fourth devices is the device participating in cross-information interaction, and another device among the multiple fourth devices does not participate in cross-device information interaction. However, if a user makes a second gesture at a distance close to the display of this device, it may trigger this device to broadcast an information retrieval request, causing the first device to receive this device's information retrieval request. If the second gesture distance in this device's information retrieval request is the smallest, it may trigger the first device to send first information to this device, leading to an error in information interaction. Based on this, in this embodiment, the first device confirms with the device with the smallest second gesture distance among multiple fourth devices whether it participates in cross-information interaction through an information interaction request. Only after the device with the smallest second gesture distance confirms will the first device send first information to that device to ensure the correctness of information interaction.

[0230] In other embodiments, for each fourth device, the first device determines whether the fourth device is associated with the first device based on the association information in the fourth device's information acquisition request. This process is similar to determining whether a second device is associated with the first device. If multiple fourth devices are associated with the first device and the gesture towards the first device is a first gesture, the first device sends first information to the fourth device with the smallest second gesture distance among these multiple fourth devices. For example, if the second device is the device with the smallest second gesture distance among these multiple fourth devices, the first device can first send an information interaction confirmation request to the second device. If it receives an information interaction confirmation response from the second device, it then sends the first information to the second device. In this way, the first device can determine the associated devices participating in cross-device information interaction from multiple associated devices and send the first information to the devices that acquire information related to cross-device information interaction, thereby ensuring the correctness of information interaction.

[0231] Step D4: The second device receives the first information.

[0232] For example, the second device receives an information sending request from the first device and obtains the first information from the information sending request.

[0233] In the data transfer method 1 described above, steps D2 and D3 are possible implementations of step 402. Steps D1 and D4 are possible implementations of step 404. In the data transfer method 1 described above, the second device actively requests the first device to obtain the first information from the first device, which can prevent the first device from broadcasting the first information and also prevent irrelevant devices from receiving and processing the first information.

[0234] Data transfer method 2: The first device actively pushes information from the first device to the second device.

[0235] As shown in Figure 9, data transfer method 2 includes steps E1 to E3.

[0236] Step E1: If the gesture for the first device is the first gesture, the first device broadcasts an information sending request, which instructs the device to send information to the associated device.

[0237] In this embodiment, the first device and its associated device (such as the second device) have the same association information, and the information transmission request includes the association information and the first information. In other embodiments, the information transmission request of any communication device (such as the first device) also includes the first gesture distance of the communication device.

[0238] If a gesture targeting the first device is identified in step 401, and the gesture is a first gesture, the first device determines first information from its information and generates a message sending request based on the first information and the associated information; alternatively, it generates a message sending request based on the first information, the first gesture distance, and the associated information. After generating the message sending request, the first device broadcasts the generated message sending request. The broadcasting method can refer to the method used by the second device to broadcast the message acquisition request in step D1 above, and will not be repeated here.

[0239] Step E2: The second device receives the information sending request.

[0240] The process of the second device receiving the information transmission request can be referred to the process of the first device receiving the information acquisition request in D1 above. Here, this embodiment of the application will not repeat step E2. The second device may receive the information transmission request before or after step 403 above. Here, this embodiment of the application does not limit the execution order between step E2 and step 403.

[0241] Step E3: If the gesture for the second device is the second gesture, the second device obtains the first information from the information sending request.

[0242] After receiving the information sending request, the second device determines whether the first device is associated with the second device based on the association information in the information sending request. The method of determination can refer to the method in step D3 above where the first device determines whether the second device is associated with the first device based on the association information in the information acquisition request.

[0243] If the first device and the second device are not associated, the second device discards the information transmission request. If the first device and the second device are associated, the second device decides whether to obtain the first information in the information transmission request based on the recognition result of step 403 above. For example, if a gesture for the second device is recognized in step 403 above, and the gesture for the second device is a second gesture, the second device obtains the first information from the information transmission request. If the gesture for the second device is not a second gesture, the second device discards the information transmission request. If no gesture for the second device is recognized in step 403 above (i.e., a third recognition result is obtained in step 403), the second device discards the information transmission request.

[0244] The above description uses the example of a second device receiving a message transmission request from a first device. In other embodiments, other communication devices may exist around the first device. A gesture from the first user towards the first device may also trigger other communication devices to broadcast message transmission requests, allowing the second device to receive message transmission requests from other communication devices as well. Of course, even if a communication device is not one of the other communication devices around the first device, a user's gesture towards that device may still trigger a broadcast message transmission request from that device. Therefore, the second device may receive message transmission requests from multiple devices. For ease of description, these multiple devices are referred to as third devices, meaning that multiple third devices include the first device and other communication devices besides the first device.

[0245] In a scenario where a second device receives information sending requests from multiple third devices, the second device obtains the first gesture distance of each third device from the information sending requests. For example, it obtains the first gesture distance of a third device from a single information sending request. If the gesture for the second device is a second gesture, the second device can designate the device with the smallest first gesture distance among the multiple third devices as its associated device and obtain information about that device's participation in cross-device information interaction from its information sending requests.

[0246] For example, if the first device has the smallest first gesture distance among multiple third devices and the gesture towards the second device is a second gesture, the second device sends an information interaction confirmation request to the first device. The first device receives the information interaction confirmation request, displays it, and the first user can confirm the request. In response to this confirmation, the first device sends an information interaction confirmation response to the second device. If the second device receives the information interaction confirmation response from the first device, it retrieves the first information from the first device's information sending request.

[0247] Among multiple third devices, the device with the smallest first gesture distance is not necessarily the device participating in cross-device information interaction. For example, suppose the first device among multiple third devices is the device participating in cross-information interaction, and another device among the multiple third devices does not participate in cross-device information interaction. However, if a user makes a first gesture at a distance close to the display of this device, it may trigger this device to broadcast an information transmission request, causing a second device to receive this device's information transmission request. If the first gesture distance in this device's information transmission request is the smallest, the second device may be triggered to retrieve the device's information from this information transmission request, leading to errors in information interaction. Based on this, in this embodiment, the second device confirms with the device with the smallest first gesture distance among multiple third devices whether it participates in cross-information interaction through an information interaction request. Only after the device with the smallest first gesture distance confirms will the second device retrieve the device's information from that device's information transmission request, ensuring the correctness of information interaction.

[0248] In other embodiments, for each third device, the second device determines whether the third device is associated with the second device based on the association information in the third device's information acquisition request. This process is similar to determining whether the first device is associated with the second device. If multiple third devices are associated with the second device and the gesture towards the second device is a second gesture, the second device obtains the information sending request from the third device with the smallest first gesture distance among these multiple third devices to acquire information related to cross-device information interaction. For example, if the first device is the device with the smallest first gesture distance among these multiple third devices, the second device can first send an information interaction confirmation request to the first device. If it receives an information interaction confirmation response from the first device, it then obtains the first information from the first device's information sending request. In this way, the second device can determine the associated devices participating in cross-device information interaction from multiple associated devices and accurately acquire the information related to cross-device information interaction.

[0249] In the data transfer method 2 described above, step E1 is one possible implementation of step 402, and steps E2 to E3 are one possible implementation of step 404. In the data transfer method 2 described above, the first device actively requests the second device to push the first information, which can avoid the second device broadcasting the information acquisition request and also avoid irrelevant devices receiving and processing the information acquisition request.

[0250] Data transfer method 3: The information interaction system also includes a fifth device, through which the first information is transferred between the first and second devices.

[0251] The fifth device serves as an intermediary in cross-device information interaction scenarios, providing information storage and forwarding services for communication devices engaging in cross-device information interaction. The fifth device can be a server. In data transfer method 3, if the gesture towards the first device is a first gesture, the first device sends first information to the second device through the fifth device (i.e., a possible implementation of step 402); if the gesture towards the second device is a second gesture, the second device obtains the first information through the fifth device (i.e., a possible implementation of step 404). As shown in Figure 10, data transfer method 3 includes steps F1 to F4.

[0252] Step F1: If the gesture for the first device is the first gesture, the first device sends an information sending request to the fifth device, and the information sending request instructs to send information to the associated device.

[0253] In this step F1, the same principle applies as step E1 above. The difference is that in this step F1, the first device sends an information transmission request to a specific device (i.e., the fifth device) instead of broadcasting an information transmission request.

[0254] Step F2: If the gesture for the second device is the second gesture, the second device sends an information retrieval request to the fifth device. The information retrieval request indicates that the device requests information from the associated device.

[0255] In this step F2, the same principle applies as step D1. The difference is that in step F2, the second device sends an information acquisition request to a specific device (i.e., the fifth device) instead of broadcasting an information acquisition request.

[0256] Step F3: Based on the information sending request and the information acquisition request, the fifth device determines that the second device is associated with the first device and sends the first information in the information sending request to the second device.

[0257] Upon receiving an information sending request from the first device and an information retrieval request from the second device, the fifth device compares the association information in the information sending request with the association information in the information retrieval request. If the two association information are the same, the fifth device determines that the second device is associated with the first device and sends the first information to the second device. Alternatively, if the two association information are the same, the fifth device sends the first information to the second device, but does not execute the association step of determining the association between the second device and the first device. If the two association information are different, the fifth device does not execute the step of sending the first information to the second device.

[0258] The above description uses the example of receiving a message sending request from an associated device (i.e., the first device) of the second device. In other embodiments, the fifth device may receive message sending requests from multiple associated devices of the second device. The fifth device can select one of these associated devices to send its message to the second device. For example, multiple third devices may be triggered by a first gesture and send message sending requests to the fifth device. If all of these third devices are associated devices of the second device, the fifth device obtains the first gesture distance of each third device from the message sending requests of the multiple third devices, determines the device with the smallest first gesture distance from the multiple third devices, and sends the information from that device's message sending request that participates in cross-device information interaction to the second device.

[0259] Taking multiple third devices, including the first device, as an example, if the first device has the smallest first gesture distance among these third devices, the fifth device sends the first information to the second device. Alternatively, if the first device has the smallest first gesture distance among these third devices, the fifth device first sends an information interaction confirmation request to the first device. If it receives an information interaction confirmation response from the first device, it then sends the first information from the first device's information sending request to the second device. In this way, the fifth device can accurately determine the associated devices that are engaging in cross-device information interaction with the second device from among the second device's multiple associated devices, and send information about the identified devices participating in the cross-device information interaction to the second device, thus ensuring the accuracy of the information interaction.

[0260] The above description uses the example of receiving an information acquisition request from an associated device (i.e., a second device) of the first device. In other embodiments, the fifth device may receive information acquisition requests from multiple associated devices of the first device. The fifth device selects one of these associated devices and sends the first device's first information to the selected associated device. For example, multiple fourth devices may be triggered by a second gesture and send information acquisition requests to the fifth device. If all of these fourth devices are associated devices of the first device, the fifth device obtains the second gesture distance of each fourth device from the information acquisition requests of the multiple fourth devices and sends the first device's first information to the device with the smallest second gesture distance among the multiple fourth devices.

[0261] Taking multiple fourth devices, including the second device, as an example, if the second device has the smallest second gesture distance among these fourth devices, the fifth device sends the first information to the second device. Alternatively, if the second device has the smallest second gesture distance among these fourth devices, the fifth device first sends an information interaction confirmation request to the second device. If it receives an information interaction confirmation response from the second device, it then sends the first information of the first device to the second device. In this way, the fifth device can accurately determine the associated device that is interacting with the first device across devices from among the multiple associated devices of the first device, and send the first information identifying the first device to the determined device, thus ensuring the accuracy of information interaction.

[0262] If a fifth device receives multiple information sending requests from third devices and multiple information receiving requests from fourth devices, and these third devices and fourth devices are associated with each other, then the fifth device will designate the device with the smallest first gesture distance among the third devices (such as the first device) as the information sender, and the device with the smallest second gesture distance among the fourth devices as the information receiver. The fifth device will then send the information from the sending request that involves cross-device information interaction to the receiver. Alternatively, the fifth device can first send information interaction confirmation requests to both the sending and receiving ends. If it receives confirmation responses from both ends, it will then send the information from the sending request that involves cross-device information interaction to the receiver. This method allows the fifth device to accurately identify the device pairs involved in cross-information interaction from among multiple associated devices, and send the information from the sending end of the device pair to the receiving end of the device pair, ensuring the accuracy of information interaction.

[0263] Step F4: The second device receives the first information.

[0264] In the data transfer method 3 described above, step F1 is a possible implementation of step 402, and steps F2 and F4 are possible implementations of step 404. In the data transfer method 3 described above, the fifth device assists in completing the cross-device information exchange between the first device and the second device, avoiding the first device's broadcast information sending request and the second device's broadcast information receiving request, thereby preventing irrelevant devices from receiving and processing information sending and receiving requests.

[0265] In the information interaction method provided in the embodiment of Figure 4 above, the device can recognize the user's gestures towards the device through received wireless signals. If the recognized gesture is a first gesture, information from the device is sent to the associated device; if the recognized gesture is a second gesture, information from the associated device is retrieved, thus completing information interaction between different devices. Therefore, by the user performing the first gesture and the second gesture respectively on the associated first and second devices, information interaction between the first and second devices can be triggered. During the information interaction process, the user does not need to operate the device, simplifying the information interaction process and improving information interaction efficiency. The first gesture and the second gesture form a combined gesture. The first gesture is used to trigger the first device to send first information to the associated device, and the second gesture is used to trigger the second device to retrieve information from the first device. Based on this, in the scenario where the first device sends information to the second device, the user can trigger information interaction between the first and second devices by performing the first gesture on the first device and the second gesture on the second device. During the information interaction process, the user does not need to operate the device, simplifying the information interaction process and improving information interaction efficiency.

[0266] The above description uses the example of a second device (an associated device) of the first device obtaining information from the first device. In other embodiments, when the first device has multiple associated devices, at least two associated devices can obtain information from the first device in the same way that the second device obtains information from the first device, thereby realizing one-to-many information interaction.

[0267] In one possible implementation, communication devices can also achieve cross-device information interaction based on computer vision. For example, a camera can be used to capture user gestures towards the communication device, triggering cross-device information interaction through these gestures. This camera-based cross-device information interaction method has the following problems: the camera's field of view is limited by its focal length, creating blind spots for close-range gestures in office settings, making them difficult to recognize; camera-based cross-device information interaction technology may also involve user privacy; cameras are sensitive to lighting conditions, making it difficult to recognize user gestures in low-light environments. Compared to camera-based cross-device information interaction, the information interaction method provided in this application recognizes user gestures towards the communication device based on wireless signals, eliminating the need to consider camera blind spots, having no requirements for lighting conditions, and not capturing user images, thus improving privacy protection.

[0268] The methods of the embodiments of this application have been described above. The apparatus of the embodiments of this application is described below. It should be understood that the apparatus described below has the same inventive concept as the above-described information interaction method and can achieve the beneficial effects that the above-described method embodiments can achieve. The apparatus and device involved in this application will be described below with reference to Figures 11 to 13. It should be understood that the technical features described in the above-described method embodiments are also applicable to the following apparatus embodiments.

[0269] Figure 11 is a schematic diagram of an information interaction device provided in an embodiment of this application. The device 1100 shown in Figure 11 is applied to a first device in an information interaction system. For example, the device 1100 can be configured as the first device involved in the above embodiments, or it can be a gesture recognition module in the first device. The device 1100 is used to execute the method steps performed by the first device in the above information interaction method. Referring to Figure 11, the device 1100 includes:

[0270] The gesture recognition unit 1101 is used to recognize the gesture of the first user toward the first device based on the first wireless signal, wherein the first wireless signal is the wireless signal received by the first device.

[0271] The sending unit 1102 is configured to send first information to a second device in the information interaction system if the gesture for the first device is a first gesture, wherein the first gesture indicates that information is sent to an associated device, the second device is an associated device of the first device, and the first information is information in the first device that participates in cross-device information interaction.

[0272] Optionally, the first gesture corresponds to the second gesture, and the second gesture is used to trigger the second device to obtain information from the associated device.

[0273] Optionally, the gesture recognition unit 1101 includes:

[0274] The first acquisition unit is used to acquire multiple first signal status information, the first signal status information representing the status of a first wireless signal used to carry a data packet when it reaches the first device;

[0275] Analysis unit is used to perform time-series feature analysis on multiple first signal state information to obtain first time-series features, which indicate the changes of the first signal state information over time;

[0276] The recognition unit is used to recognize the gesture of the first user toward the first device based on the first temporal features.

[0277] Optionally, the first wireless signal is a WiFi signal, the first signal status information includes the channel status information corresponding to the first wireless signal, and the identification unit includes:

[0278] The feature analysis subunit is used to perform multi-dimensional feature analysis on the changes in channel state information in multiple first signal state information based on the first time-series feature, to obtain the first frequency domain feature, the first time domain feature, and the first spatial domain feature. The first frequency domain feature represents the changes in channel state information in the frequency domain, the first time domain feature represents the changes in channel state information in the time domain, and the first spatial domain feature represents the changes in channel state information in the spatial domain.

[0279] The gesture recognition subunit is used to recognize the gestures of the first user toward the first device based on the first frequency domain features, the first time domain features, and the first spatial domain features.

[0280] Optionally, the first gesture corresponds to the first target frequency domain feature, the first target time domain feature, and the first target spatial domain feature. The first target frequency domain feature represents the change in the channel state information corresponding to the wireless signal in the frequency domain under the influence of the first gesture. The first target time domain feature represents the change in the channel state information corresponding to the wireless signal in the time domain under the influence of the first gesture. The first target spatial domain feature represents the change in the channel state information corresponding to the wireless signal in the spatial domain under the influence of the first gesture.

[0281] The gesture recognition subunit is configured to recognize a first gesture as a gesture of a first user to the first device if at least one of the following conditions is met: the first condition is that the first frequency domain feature matches the first target frequency domain feature; the second condition is that the first time domain feature matches the first target time domain feature; and the third condition is that the first spatial domain feature matches the first target spatial domain feature.

[0282] Optionally, the device 1100 further includes:

[0283] The first determining unit is used to determine a first distance based on a first timing feature, wherein the first distance is the distance between the spatial location that causes the change in the state information of the first signal and the first device;

[0284] The recognition unit is further configured to perform a step of recognizing a gesture of a first user toward a first device based on a first temporal feature if the first distance is less than or equal to a first threshold.

[0285] Optionally, the device 1100 further includes:

[0286] The second determining unit is used to determine first information based on the content displayed by the first device on the currently displayed interface, wherein the first information is associated with the content.

[0287] Optionally, the sending unit 1102 is further configured to broadcast an information sending request if the gesture for the first device is a first gesture, the information sending request instructing the sending of information to the associated device.

[0288] Optionally, the device 1100 further includes:

[0289] The receiving unit is used to receive an information acquisition request from the second device, wherein the information acquisition request indicates a request to acquire information from the associated device.

[0290] The sending unit 1102 is further configured to send first information to the second device based on an information acquisition request if the gesture for the first device is a first gesture.

[0291] Optionally, the device 1100 further includes:

[0292] The second acquisition unit is used to acquire the second gesture distance of each of the multiple fourth devices from the information acquisition requests of the multiple fourth devices. The multiple fourth devices include the second device, and the second gesture distance is the distance between the device and the second gesture for the device.

[0293] The sending unit 1102 is further configured to send an information interaction confirmation request to the second device if the second gesture distance of the second device among the multiple fourth devices is the smallest and the gesture for the first device is the first gesture. The information interaction confirmation request is used to confirm whether to perform cross-device information interaction.

[0294] The sending unit 1102 is also configured to send first information to the second device if it receives an information interaction confirmation response from the second device, wherein the information interaction confirmation response indicates confirmation of cross-device information interaction.

[0295] Optionally, the sending unit 1102 is further configured to send first information to the second device via the fifth device if the gesture for the first device is a first gesture. For example, the sending unit 1102 is also configured to, if the gesture for the first device is a first gesture, instruct an information sending request to send information to an associated device, the information sending request including the first information.

[0296] It should be understood that device 1100 corresponds to the first device in the above method embodiment. The modules in device 1100 and the other operations and / or functions described above are for implementing various steps and methods carried out by the first device in the method embodiment. For specific details, please refer to the above method embodiment. For the sake of brevity, they will not be repeated here.

[0297] Figure 12 is a schematic diagram of an information interaction device provided in an embodiment of this application. The device 1200 shown in Figure 12 is applied to a second device in an information interaction system. For example, the device 1200 can be configured as the second device involved in the above embodiments, or it can be a gesture recognition module in the second device. The device 1200 is used to execute the method steps performed by the second device in the above information interaction method. Referring to Figure 12, the device 1200 includes:

[0298] The gesture recognition unit 1201 is used to recognize the gesture of the second user toward the second device based on the second wireless signal, wherein the second wireless signal is the wireless signal received by the second device.

[0299] The first acquisition unit 1202 is used to acquire first information of the first device in the information interaction system if the gesture for the second device is a second gesture, wherein the second gesture indicates the acquisition of information in the associated device, the first device is the associated device of the second device, and the first information is information in the first device that participates in cross-device information interaction.

[0300] Optionally, the second gesture corresponds to the first gesture, and the second gesture is used to trigger the first device to send the first information to the associated device.

[0301] Optionally, the gesture recognition unit 1201 includes:

[0302] The second acquisition unit is used to acquire multiple second signal status information, which indicates the status of the second wireless signal used to carry a data packet when it reaches the second device.

[0303] The analysis unit is used to perform time-series feature analysis on multiple second signal state information to obtain second time-series features, which indicate the changes of the second signal state information over time.

[0304] The recognition unit is used to recognize the gestures of the second user toward the second device based on the second temporal features.

[0305] Optionally, the second wireless signal is a WiFi signal, the second signal status information includes the channel status information corresponding to the second wireless signal, and the identification unit includes:

[0306] The feature analysis subunit is used to perform multi-dimensional feature analysis on the changes in channel state information in multiple second signal state information based on the second time-series features, to obtain second frequency domain features, second time domain features, and second spatial domain features. The second frequency domain features represent the changes in channel state information in the frequency domain, the second time domain features represent the changes in channel state information in the time domain, and the second spatial domain features represent the changes in channel state information in the spatial domain.

[0307] The gesture recognition subunit is used to recognize the gestures of the second user toward the second device based on the second frequency domain features, the second time domain features, and the second spatial domain features.

[0308] Optionally, the second gesture corresponds to the second target frequency domain feature, the second target time domain feature, and the second target spatial domain feature. The second target frequency domain feature represents the change in the channel state information corresponding to the wireless signal in the frequency domain under the influence of the second gesture. The second target time domain feature represents the change in the channel state information corresponding to the wireless signal in the time domain under the influence of the second gesture. The second target spatial domain feature represents the change in the channel state information corresponding to the wireless signal in the spatial domain under the influence of the second gesture.

[0309] The gesture recognition subunit is configured to recognize the second gesture as a gesture of the second user to the second device if at least one of the fourth, fifth, and sixth conditions is met. The fourth condition is that the second frequency domain feature matches the second target frequency domain feature, the fifth condition is that the second time domain feature matches the second target time domain feature, and the sixth condition is that the second spatial domain feature matches the second target spatial domain feature.

[0310] Optionally, the device 1200 further includes:

[0311] The first determining unit is used to determine the second distance based on the second timing characteristics. The second distance is the distance between the spatial location that causes the change in the state information of the second signal and the second device.

[0312] The recognition unit is also configured to perform a step of recognizing a second user's gesture toward a second device based on a second temporal feature if the second distance is less than or equal to a first threshold.

[0313] Optionally, the first acquisition unit 1202 is further configured to:

[0314] The first device receives an information transmission request, which instructs that information be sent to the associated device.

[0315] If the gesture for the second device is the second gesture, obtain the first information from the information sending request.

[0316] Optionally, the device 1200 further includes:

[0317] The third acquisition unit is used to acquire the first gesture distance of each third device from the information sending requests of multiple third devices. The multiple third devices include the first device. The first gesture distance is the distance between the device and the first gesture to the device.

[0318] The sending unit is configured to send an information interaction confirmation request to the first device if the first device has the smallest first gesture distance among multiple third devices and the gesture for the second device is the second gesture. The information interaction confirmation request is used to confirm whether to perform cross-device information interaction.

[0319] The first acquisition unit 1202 is further configured to, if it receives an information interaction confirmation response from the first device, acquire first information from the information sending request of the first device, and the information interaction confirmation response indicates confirmation to conduct cross-device information interaction.

[0320] Optionally, the first acquisition unit 1202 is further configured to:

[0321] If the gesture for the second device is the second gesture, broadcast an information retrieval request, the information retrieval request indicating a request to retrieve information from the associated device;

[0322] Receive the first information returned by the first device based on the information acquisition request.

[0323] Optionally, the first acquisition unit 1202 is further configured to acquire first information through the fifth device if the gesture for the second device is a second gesture. For example, the first acquisition unit 1202 is further configured to send an information acquisition request to the fifth device if the gesture for the second device is a second gesture, the information acquisition request indicating a request to acquire information from the associated device; and receive the first information returned by the fifth device based on the information acquisition request.

[0324] It should be understood that device 1200 corresponds to the second device in the above method embodiments. The modules in device 1200 and the other operations and / or functions described above are for implementing various steps and methods implemented by the second device in the method embodiments. For specific details, please refer to the above method embodiments. For the sake of brevity, they will not be repeated here.

[0325] It should be understood that for either device 1100 or device 1200, when the device performs cross-device information interaction, the above-described division of functional modules is only used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0326] Figure 13 shows a schematic diagram of a communication device according to an embodiment of this application. The communication device 1300 shown in Figure 13 is configured as the aforementioned first device or second device. As shown in Figure 13, the communication device 1300 includes: a bus 1302, a processor 1304, a memory 1306, and a wireless communication module 1308. The processor 1304, the memory 1306, and the wireless communication module 1308 communicate with each other via the bus 1302. The communication device 1300 can be any type of terminal device. It should be understood that this application does not limit the number of processors and memories in the communication device 1300. The bus 1302 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one line is used to represent the bus 1302 in Figure 13, but this does not mean that there is only one bus or one type of bus. Bus 1302 may include a path for transmitting information between various components of communication device 1300 (e.g., memory 1306, processor 1304, wireless communication module 1308).

[0327] Processor 1304 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP). Memory 1306 may include volatile memory, such as random access memory (RAM). Processor 1304 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid-state drive (SSD).

[0328] The memory 1306 stores multiple executable program codes, and the processor 1304 executes these program codes to implement the information interaction method provided in the above-described method embodiments. The program codes stored in the memory 1306 include at least one first program code and at least one second program code. By executing the first program code, the processor 1304 causes the communication device 1300 to execute the method steps performed by the first device in the information interaction method provided in the above-described embodiments. By executing the second program code, the processor 1304 causes the communication device 1300 to execute the method steps performed by the second device in the information interaction method provided in the above-described embodiments.

[0329] For example, processor 1304 is configured to execute first program code to implement the following steps: based on a first wireless signal, identify a gesture of a first user toward the communication device, wherein the first wireless signal is a wireless signal received by the communication device; if the gesture toward the communication device is a first gesture, send first information to a second device in the information interaction system where the communication device is located, wherein the first gesture indicates sending information to an associated device, the second device being an associated device of the communication device, and the first information being information in the communication device participating in cross-device information interaction. In this case, communication device 1300 acts as the first device in the information interaction system. In addition to the above steps, processor 1304, by executing the first program code, can also implement other method steps executed by the first device in the above method embodiments, which will not be elaborated here.

[0330] For example, processor 1304 is configured to execute second program code to implement the following steps: based on a second wireless signal, identify a gesture of a second user toward the communication device, wherein the second wireless signal is a wireless signal received by the communication device; if the gesture toward the second device is a second gesture, obtain first information of a first device in the information interaction system in which the communication device is located, wherein the second gesture indicates obtaining information from an associated device, the first device being an associated device of the second device, and the first information being information in the first device participating in cross-device information interaction. In this case, communication device 1300 acts as the second device in the information interaction system. In addition to the above steps, processor 1304, by executing the second program code, can also implement other method steps executed by the second device in the above method embodiments, which will not be elaborated here.

[0331] The wireless communication module 1308 can be any of the wireless communication modules described above, used for wireless communication with other devices, receiving wireless signals, parsing wireless signals, collecting signal status information corresponding to the wireless signals, and providing the collected signal status information to the processor 1304, so that the processor 1304 can obtain the signal status information during the execution of the information interaction method provided in this application. The wireless signal can be a WiFi signal or a Bluetooth signal.

[0332] In other embodiments, the communication device 1300 also includes a communication interface, which is, for example, but not limited to, a transceiver module such as a network interface card or a transceiver, and is used to enable communication between the communication device 1300 and other devices or communication networks.

[0333] In an exemplary embodiment, a computer-readable storage medium is also provided, such as a memory including program code, which can be executed by a processor in a communication device to complete the method steps performed by the first device in the above-described information interaction method, so as to realize the function of the first device in the above-described information interaction system.

[0334] In an exemplary embodiment, a computer-readable storage medium is also provided, such as a memory including program code, which can be executed by a processor in a communication device to complete the method steps performed by the second device in the above-described information interaction method, so as to realize the function of the second device in the above-described information interaction system.

[0335] The computer-readable storage media in the above embodiments are non-transitory computer-readable storage media, such as ROM, RAM, compact disc read-only memory (CD-ROM), magnetic tape, floppy disk, and optical data storage devices.

[0336] This application also provides a computer program product or computer program, which includes program code and computer instructions stored in a computer-readable storage medium. The processor of the communication device reads the program code from the computer-readable storage medium and executes the program code, causing the communication device to perform the method steps executed by the first device in the above-described information interaction method, so as to realize the function of the first device in the above-described information interaction system.

[0337] This application also provides a computer program product or computer program, which includes program code and computer instructions stored in a computer-readable storage medium. The processor of the communication device reads the program code from the computer-readable storage medium and executes the program code, causing the communication device to perform the method steps executed by the second device in the above-described information interaction method, so as to realize the function of the second device in the above-described information interaction system.

[0338] In addition, embodiments of this application also provide an apparatus, which may specifically be a chip, component or module. The apparatus may include a connected processor and a memory. The memory is used to store computer execution instructions. When the apparatus is running, the processor may execute the computer execution instructions stored in the memory to cause the chip to execute the method steps performed by the first device or the second device in the information interaction method of the above method embodiments.

[0339] In this embodiment, the apparatus, device, computer-readable storage medium, computer program product or chip are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects of the corresponding methods provided above, and will not be repeated here.

[0340] In the description of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B. The "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. Furthermore, "at least one" means one or more, and "multiple" means two or more. The terms "first," "second," etc., do not limit the quantity or order of execution, and "first," "second," etc., do not necessarily imply differences.

[0341] In this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or illustration. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0342] All information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in this application have been authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions. For example, the signal status information involved in this application was obtained with full authorization.

[0343] All of the above-mentioned optional technical solutions can be combined in any way to form optional embodiments of this disclosure, and will not be described in detail here.

[0344] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. An information interaction system, characterized in that, The system includes a first device and a second device; The first device is configured to identify a gesture of a first user toward the first device based on a first wireless signal, wherein the first wireless signal is a wireless signal received by the first device; The first device is further configured to send first information to the second device if the gesture for the first device is a first gesture, wherein the first gesture indicates that information is sent to an associated device, the second device is an associated device of the first device, and the first information is information in the first device that participates in cross-device information interaction; The second device is used to identify a second user's gesture toward the second device based on a second wireless signal, wherein the second wireless signal is a wireless signal received by the second device; The second device is further configured to acquire the first information if the gesture directed to the second device is a second gesture, wherein the second gesture indicates acquisition of information from an associated device.

2. The system according to claim 1, characterized in that, The first device is also used for: Acquire multiple first signal status information, wherein the first signal status information indicates the status of a first wireless signal used to carry a data packet when it reaches the first device; A time-series feature analysis is performed on the plurality of first signal state information to obtain a first time-series feature, which indicates the change of the first signal state information over time. Based on the first timing feature, the gesture of the first user on the first device is identified.

3. The system according to claim 2, characterized in that, The first wireless signal is a WiFi signal, the first signal status information includes the channel status information corresponding to the first wireless signal, and the first device is further configured to: Based on the first timing feature, a multi-dimensional feature analysis is performed on the changes in channel state information among the plurality of first signal state information to obtain a first frequency domain feature, a first time domain feature, and a first spatial domain feature. The first frequency domain feature represents the changes in channel state information among the plurality of first signal state information in the frequency domain, the first time domain feature represents the changes in channel state information among the plurality of first signal state information in the time domain, and the first spatial domain feature represents the changes in channel state information among the plurality of first signal state information in the spatial domain. Based on the first frequency domain feature, the first time domain feature, and the first spatial domain feature, the gesture of the first user toward the first device is identified.

4. The system according to claim 3, characterized in that, The first gesture corresponds to the first target frequency domain feature, the first target time domain feature, and the first target spatial domain feature. The first target frequency domain feature represents the change in the channel state information corresponding to the wireless signal in the frequency domain under the influence of the first gesture. The first target time domain feature represents the change in the channel state information corresponding to the wireless signal in the time domain under the influence of the first gesture. The first target spatial domain feature represents the change in the channel state information corresponding to the wireless signal in the spatial domain under the influence of the first gesture. The first device is further configured to recognize the first gesture as a gesture of the first user toward the first device if at least one of the first, second, and third conditions is met, wherein the first condition is that the first frequency domain feature matches the first target frequency domain feature, the second condition is that the first time domain feature matches the first target time domain feature, and the third condition is that the first spatial domain feature matches the first target spatial domain feature.

5. The system according to any one of claims 2-4, characterized in that, The first device is also used for: Based on the first timing feature, a first distance is determined, which is the distance between the spatial location that caused the change in the first signal state information and the first device. If the first distance is less than or equal to the first threshold, the step of identifying the first user's gesture toward the first device based on the first temporal features is executed.

6. The system according to any one of claims 1-5, characterized in that, The second device is also used for: Acquire multiple second signal status information, the second signal status information representing the status of a second wireless signal used to carry a data packet when it reaches the second device; A time-series feature analysis is performed on the plurality of second signal state information to obtain a second time-series feature, which indicates the change of the second signal state information over time. Based on the second timing feature, the gesture of the second user toward the second device is identified.

7. The system according to claim 6, characterized in that, The second wireless signal is a WiFi signal, and the second signal status information includes the channel status information corresponding to the second wireless signal. The second device is further used to: Based on the second timing feature, a multi-dimensional feature analysis is performed on the changes in channel state information among the plurality of second signal state information to obtain a second frequency domain feature, a second time domain feature, and a second spatial domain feature. The second frequency domain feature represents the changes in channel state information among the plurality of second signal state information in the frequency domain, the second time domain feature represents the changes in channel state information among the plurality of second signal state information in the time domain, and the second spatial domain feature represents the changes in channel state information among the plurality of second signal state information in the spatial domain. Based on the second frequency domain feature, the second time domain feature, and the second spatial domain feature, the gestures of the second user toward the second device are identified.

8. The system according to claim 7, characterized in that, The second gesture corresponds to the second target frequency domain feature, the second target time domain feature, and the second target spatial domain feature. The second target frequency domain feature represents the change in the channel state information corresponding to the wireless signal in the frequency domain under the influence of the second gesture. The second target time domain feature represents the change in the channel state information corresponding to the wireless signal in the time domain under the influence of the second gesture. The second target spatial domain feature represents the change in the channel state information corresponding to the wireless signal in the spatial domain under the influence of the second gesture. The second device is further configured to recognize the second gesture as a gesture of the second user toward the second device if at least one of the fourth, fifth, and sixth conditions is met, wherein the fourth condition is that the second frequency domain feature matches the second target frequency domain feature, the fifth condition is that the second time domain feature matches the second target time domain feature, and the sixth condition is that the second spatial domain feature matches the second target spatial domain feature.

9. The system according to any one of claims 6-8, characterized in that, The second device is also used for: Based on the second timing characteristics, a second distance is determined, which is the distance between the spatial location that causes the change in the second signal state information and the second device; If the second distance is less than or equal to the first threshold, the step of identifying the second user's gesture toward the second device based on the second temporal features is performed.

10. The system according to any one of claims 1-9, characterized in that, The first device is also used for: Based on the content displayed by the first device in the currently displayed interface, the first information is determined, and the first information is associated with the content.

11. The system according to any one of claims 1-10, characterized in that, The first device is further configured to broadcast an information sending request if the gesture directed to the first device is a first gesture, the information sending request instructing the sending of information to an associated device; The second device is also configured to receive the information sending request, and if the gesture for the second device is a second gesture, to obtain the first information from the information sending request.

12. The system according to claim 11, characterized in that, The second device is also used for: The first gesture distance of each third device is obtained from the information sending request of the plurality of third devices, wherein the plurality of third devices includes the first device, and the first gesture distance is the distance between the device to which the first gesture is directed and the first gesture to the device to which the first gesture is directed. If the first gesture distance of the first device is the smallest among the plurality of third devices and the gesture for the second device is the second gesture, an information interaction confirmation request is sent to the first device. The information interaction confirmation request is used to confirm whether to perform cross-device information interaction. If an information interaction confirmation response is received from the first device, the first information is obtained from the information sending request of the first device, and the information interaction confirmation response indicates confirmation to conduct cross-device information interaction.

13. The system according to any one of claims 1-10, characterized in that, The second device is further configured to broadcast an information acquisition request if the gesture toward the second device is a second gesture, the information acquisition request indicating a request to acquire information from an associated device; The first device is further configured to receive the information acquisition request, and if the gesture for the first device is a first gesture, send the first information to the second device based on the information acquisition request; The second device is also used to receive the first information.

14. The system according to claim 13, characterized in that, The first device is also used for: The second gesture distance of each of the multiple fourth devices is obtained from the information acquisition requests of the multiple fourth devices, including the second device, and the second gesture distance is the distance between the device and the second gesture for the device. If the second gesture distance of the second device is the smallest among the plurality of fourth devices and the gesture to the first device is the first gesture, an information interaction confirmation request is sent to the second device. The information interaction confirmation request is used to confirm whether to perform cross-device information interaction. If an information interaction confirmation response is received from the second device, the first information is sent to the second device, wherein the information interaction confirmation response indicates confirmation of cross-device information interaction.

15. The system according to any one of claims 1-10, characterized in that, The system also includes a fifth device; The first device is further configured to send an information sending request to the fifth device if the gesture to the first device is a first gesture, the information sending request instructing the sending of information to the associated device; The second device is further configured to send an information acquisition request to the fifth device if the gesture toward the second device is a second gesture, the information acquisition request indicating a request to acquire information from the associated device; The fifth device is configured to determine that the second device is associated with the first device based on the information sending request and the information acquisition request, and send the first information in the information sending request to the second device; The second device is also used to receive the first information.

16. The system according to claim 15, characterized in that, The fifth device is also used for: The first gesture distance of each third device is obtained from the information sending request of the multiple third devices, wherein the multiple third devices are all associated devices of the second device, and the multiple third devices include the first device. The first gesture distance is the distance between the device to which the device belongs and the first gesture for the device to which the device belongs. If the first gesture distance of the first device is the smallest among the plurality of third devices, the first information in the information sending request of the first device is sent to the second device.

17. The system according to claim 15, characterized in that, The fifth device is also used for: The second gesture distance of each of the multiple fourth devices is obtained from the information acquisition requests of the multiple fourth devices. The multiple fourth devices are all associated devices of the first device. The multiple fourth devices include the second device. The second gesture distance is the distance between the device to which the device belongs and the second gesture for the device to which the device belongs. If the second gesture distance of the second device is the smallest among the plurality of fourth devices, the first information is sent to the second device.

18. An information interaction device, characterized in that, The device is applied to a first device in an information interaction system, the device comprising: A gesture recognition unit is used to recognize a gesture of a first user toward the first device based on a first wireless signal, wherein the first wireless signal is a wireless signal received by the first device. The sending unit is configured to send first information to a second device in the information interaction system if the gesture for the first device is a first gesture, wherein the first gesture indicates that information is to be sent to an associated device, the second device is an associated device of the first device, and the first information is information in the first device that participates in cross-device information interaction.

19. An information interaction device, characterized in that, The device is used as a second device in an information interaction system, and the device includes: A gesture recognition unit is used to recognize a gesture of a second user toward the second device based on a second wireless signal, wherein the second wireless signal is a wireless signal received by the second device; The first acquisition unit is configured to acquire first information of the first device in the information interaction system if the gesture for the second device is a second gesture, wherein the second gesture indicates the acquisition of information in an associated device, the first device is an associated device of the second device, and the first information is information in the first device that participates in cross-device information interaction.

20. A communication device, characterized in that, The communication device includes a processor coupled to a memory, the memory storing at least one line of program code. The processor is configured to execute the program code to perform the following steps: Based on a first wireless signal, the gesture of a first user toward the communication device is identified, wherein the first wireless signal is a wireless signal received by the communication device; If the gesture for the communication device is a first gesture, first information is sent to a second device in the information interaction system where the communication device is located. The first gesture indicates that information is sent to an associated device. The second device is an associated device of the communication device. The first information is information in the communication device that participates in cross-device information interaction.

21. A communication device, characterized in that, The communication device includes a processor coupled to a memory, the memory storing at least one line of program code. The processor is used to execute program code to perform the following steps: Based on a second wireless signal, the gesture of a second user toward the communication device is identified, wherein the second wireless signal is a wireless signal received by the communication device; If the gesture for the second device is a second gesture, the first information of the first device in the information interaction system where the communication device is located is obtained. The second gesture indicates that information in the associated device is obtained. The first device is the associated device of the second device, and the first information is the information in the first device that participates in cross-device information interaction.

22. A computer-readable storage medium, characterized in that, The storage medium stores at least one piece of program code, which is read by a processor to enable the communication device to perform the function of the first device in the system as described in any one of claims 1 to 17.

23. A computer-readable storage medium, characterized in that, The storage medium stores at least one piece of program code, which is read by a processor to cause the communication device to perform the function of the second device in the system as described in any one of claims 1 to 17.

Images