Method, system and device for establishing multi-screen collaborative connection, storage medium and chip

Abstract

The invention relates to the technical field of terminal equipment, provides a method, a system and equipment for establishing multi-screen cooperative connection, a storage medium and a chip, and can solve the problems that when the terminal equipment establishes the multi-screen cooperative connection, the operation of the connection process is tedious, and the user experience is poor. The method is applied to first equipment, the first equipment is provided with a magnetic sensor, and the method comprises the following steps: detecting the magnetic field intensity of the position of the first equipment through the magnetic sensor; when the magnetic field intensity meets a preset condition, target equipment is determined from at least one surrounding second equipment, magnets are arranged in the second equipment, and the magnets can influence the magnetic field intensity around the second equipment; and establishing a multi-screen cooperative connection with the target device.

Description

[0001] This application is a divisional application. The application number of the original application is 202310666025.4, and the original application date is June 6, 2023. The entire contents of the original application are incorporated into this application by reference. Technical Field

[0002] The present application relates to the field of terminal device technology, and in particular to a method, system, device, storage medium and chip for establishing a multi-screen collaborative connection. Background Art

[0003] With the rapid development of terminal devices, different terminal devices can realize multi-device interaction functions such as resource sharing, screen mirroring, and screen expansion after establishing multi-screen collaborative connections.

[0004] At present, different terminal devices can establish multi-screen collaborative connections based on built-in special devices (such as near field communication (NFC) sensing chips) through natural interactions such as approaching and one-touch connection, without the need for users to manually operate other related controls. However, some terminal devices may not be equipped with such special devices. Therefore, for such terminal devices, traditional methods such as scanning codes and wired connections are usually required, which cannot achieve the effect of natural interaction, and the connection process is cumbersome and the user experience is poor. Summary of the invention

[0005] The present application provides a method, system, device, storage medium and chip for establishing a multi-screen collaborative connection, which solves the problem in the prior art that when terminal devices establish a multi-screen collaborative connection, the connection process is cumbersome and the user experience is poor.

[0006] In order to achieve the above objectives, this application adopts the following technical solutions:

[0007] In a first aspect, a method for establishing a multi-screen collaborative connection is provided, which is applied to a first device, wherein the first device is provided with a magnetic sensor, and the method comprises: detecting the magnetic field strength at the location of the first device by means of the magnetic sensor; when the magnetic field strength meets a preset condition, determining a target device from at least one surrounding second device, wherein the second device is provided with a magnet, and the magnet can affect the magnetic field strength around the second device; and establishing a multi-screen collaborative connection with the target device.

[0008] The method for establishing a multi-screen collaborative connection provided in the embodiment of the present application can use the built-in universal components of the terminal device (such as magnetic sensors and magnets) to use the change in magnetic field strength when the devices are close to each other as the trigger condition for establishing a multi-screen collaborative connection between the devices, thereby realizing a natural interactive connection between the first device and the target device. This method can be implemented without setting up special devices, the connection process is simple to operate, and can improve the user experience.

[0009] In some embodiments, the preset conditions include: the magnetic field strength is greater than a first threshold; or the rate of change of the magnetic field strength within a preset time is greater than a second threshold.

[0010] In this embodiment, the first threshold and the second threshold serve as the basis for determining whether the magnetic field strength meets the preset conditions. If so, the process of triggering the establishment of a multi-screen collaborative connection is initiated; if not, the process of triggering the establishment of a multi-screen collaborative connection is not initiated, thereby avoiding false triggering.

[0011] In some embodiments, when the magnetic field strength meets a preset condition, a target device is determined from at least one surrounding second device, including: when the magnetic field strength meets the preset condition, a first indication message is broadcast, the first indication message is used to instruct the second device to send a detection signal; and the target device is determined based on the received detection signal.

[0012] In one example, the first indication message is also used to instruct the second device to send a detection signal of a preset strength, and determine the target device based on the received detection signal, including: determining the signal strength of each received detection signal; and determining the second device corresponding to the detection signal with the maximum signal strength and greater than the strength threshold as the target device.

[0013] In another example, the first indication message is also used to instruct the second device to send a detection signal carrying a sending time, and determine the target device based on the received detection signal, including: determining the receiving time of each detection signal; determining the distance between the first device and each second device based on the sending time and receiving time of each detection signal; and determining the second device with the smallest distance and less than the distance threshold as the target device.

[0014] In some embodiments, when the magnetic field strength meets a preset condition, a target device is determined from at least one surrounding second device, including: when the magnetic field strength meets the preset condition, a third indication message and a detection signal are broadcast, the third indication message is used to instruct the second device to turn on proximity detection, the proximity detection includes the second device determining the distance between the second device and the first device according to the detection signal after receiving the detection signal; receiving feedback signals sent by each second device, the feedback signal is broadcast after the second device turns on proximity detection, and the feedback signal includes the distance between the second device and the first device; according to the feedback signal, the second device corresponding to the feedback signal with the shortest distance in the feedback signal and less than the distance threshold is determined as the target device.

[0015] In this embodiment, after detecting that the magnetic field strength meets the preset conditions, the first device starts the detection process of determining the target device from at least one surrounding second device. Compared with the process of determining the target device by using high-precision proximity detection methods such as ultrasonic signals or UWB pulse signals online in the traditional technology, the method in this embodiment can accurately determine the target device, reduce false triggering, and save device power consumption.

[0016] In some embodiments, when the magnetic field strength meets a preset condition, after determining the target device from at least one surrounding second device, the method further includes: broadcasting a notification message, where the notification message is used to indicate that the corresponding second device becomes the target device.

[0017] Optionally, other second devices may also determine, based on the notification message, that they are not the target device identified and determined by the first device.

[0018] In some embodiments, after broadcasting the notification message, the method also includes: determining first posture information of the first device; broadcasting a second indication message, the second indication message is used to instruct the target device to send second posture information; receiving the second posture information; and determining a posture match between the first device and the target device based on the first posture information and the second posture information, the posture match including: the first plane where the first device is located coincides with, is parallel to, or is perpendicular to, the second plane where the second device is located.

[0019] In some embodiments, the posture match between the first device and the target device is determined based on the first posture information and the second posture information, including: when the absolute value of the difference between the corresponding angles of the first device and the target device are both within a first preset range, determining that the first plane where the first device is located is coincident or parallel to the second plane where the second device is located; and / or, when the absolute value of the difference between the corresponding angles of the first device and the target device is within a second preset range, determining that the first plane where the first device is located is perpendicular to the second plane where the second device is located.

[0020] In some embodiments, establishing a multi-screen collaborative connection with the target device includes: establishing a multi-screen collaborative connection with the target device when the posture of the first device matches that of the target device.

[0021] In this embodiment, the device posture is used as a prerequisite for triggering the establishment of a multi-screen collaborative connection. When there are multiple second devices (for example, mobile phones, tablet computers, etc.), the first device only selects the second devices that coincide with, are parallel to, or are perpendicular to its plane to establish a multi-screen collaborative connection, thereby improving the accuracy of the multi-screen collaborative connection process and avoiding false triggering.

[0022] In some embodiments, after establishing a multi-screen collaborative connection with the target device, the method also includes: determining first posture information of the first device; sending a posture information request to the target device, the posture information request is used to request the target device to send second posture information; receiving the second posture information; determining a posture match between the first device and the target device based on the first posture information and the second posture information; when the posture of the first device matches that of the target device, triggering a corresponding multi-screen collaborative function.

[0023] In some embodiments, when the postures of the first device and the target device match, the corresponding multi-screen collaboration function is triggered, including: when the first plane where the first device is located coincides with or is parallel to the second plane where the second device is located, the target device uses the accessories of the first device.

[0024] In some embodiments, the method further includes: when the absolute values ​​of the differences between the corresponding angles of the first device and the target device are both within a third preset range, the target device stopping using the accessory of the first device.

[0025] In this embodiment, the first device and the target device adjust the multi-screen collaborative function according to the posture matching between each other to adapt to different application scenarios.

[0026] In some embodiments, when the posture of the first device matches that of the target device, the corresponding multi-screen collaboration function is triggered, including: when the first plane where the first device is located and the second plane where the second device is located are perpendicular to each other, the first device acts as an external keyboard or external controller of the target device.

[0027] In this embodiment, after the first device and the target device establish a connection, different functions are triggered according to the relative postures between the devices, and the semantics contained in the device posture information are used to achieve a richer and more natural interaction method.

[0028] In a second aspect, a system for establishing a multi-screen collaborative connection is provided, comprising a first device and at least one second device, the first device being provided with a magnetic sensor and being configured to execute the method for establishing a multi-screen collaborative connection as shown in the first aspect above, the second device being provided with a magnet capable of affecting the magnetic field strength around the second device.

[0029] In a third aspect, a terminal device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for establishing a multi-screen collaborative connection as shown in the first aspect above is implemented.

[0030] In a fourth aspect, a computer-readable storage medium is provided, which stores a computer program, and when the computer program is executed by a processor, the method for establishing a multi-screen collaborative connection as shown in the first aspect above is implemented.

[0031] In a fifth aspect, a chip is provided, which includes a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the method for establishing a multi-screen collaborative connection as shown in the first aspect above is implemented.

[0032] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 This is a schematic diagram of a proximity connection between terminals provided by an embodiment of the present application;

[0034] Figure 2 Schematic diagram of a system architecture applicable to the method for establishing a multi-screen collaborative connection provided in an embodiment of the present application;

[0035] Figure 3 is a schematic diagram of the structure of a terminal device provided by an embodiment of the present application;

[0036] Figure 4 It is a structural schematic diagram of a magnet setting method of a terminal device provided by an embodiment of the present application;

[0037] Figure 5 is a structural schematic diagram of a magnet setting method of a terminal device provided in another embodiment of the present application;

[0038] Figure 6 is a schematic flowchart of a method for establishing a multi-screen collaborative connection provided by an embodiment of the present application;

[0039] Figure 7 This is a schematic diagram of an interactive process of a method for establishing a multi-screen collaborative connection provided by an embodiment of the present application;

[0040] Figure 8 This is a schematic diagram of a scenario of close connection between terminals provided by an embodiment of the present application;

[0041] Fig. 9 It is a schematic diagram of an interaction process for a first device to determine whether its posture matches that of a target device, provided in an embodiment of the present application;

[0042] Fig.10 It is a structural schematic diagram of the device posture provided in the embodiment of the present application;

[0043] Fig.11 is a schematic diagram of an interactive process of a method for establishing a multi-screen collaborative connection provided by another embodiment of the present application;

[0044] Fig. 12AThis is a schematic diagram of a scenario of screen mirroring between devices provided by an embodiment of the present application;

[0045] Fig. 12B This is a schematic diagram of a scenario of screen expansion between devices provided by an embodiment of the present application;

[0046] Fig. 12C This is a schematic diagram of a scenario of resource sharing between devices provided by an embodiment of the present application;

[0047] Fig.13 is a schematic diagram of a multi-screen collaboration function selection interface provided by an embodiment of the present application;

[0048] Fig.14 It is a schematic diagram of an interactive process of a method for triggering a multi-screen collaboration function according to a device posture provided by an embodiment of the present application;

[0049] Fig.15 This is a schematic diagram of an application scenario of a multi-screen collaboration function provided by an embodiment of the present application;

[0050] Fig.16 This is a schematic diagram of an application scenario of a multi-screen collaboration function provided by another embodiment of the present application;

[0051] Fig.17 This is a schematic diagram of an application scenario of a multi-screen collaboration function provided by another embodiment of the present application;

[0052] Fig.18 It is a schematic diagram of the structure of a chip provided in one embodiment of the present application. DETAILED DESCRIPTION

[0053] The technical solution provided in the embodiments of the present application is described below with reference to the accompanying drawings.

[0054] It should be understood that in the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is merely a way to describe the association relationship of associated objects, indicating that three relationships may exist, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0055] In this embodiment, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features.

[0056] In the description of this embodiment, unless otherwise specified, "multiple" means two or more than two. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0057] With the rapid development of terminal devices, after establishing a multi-screen collaborative connection between different terminal devices, multi-device interaction functions such as screen mirroring, screen expansion, and resource sharing can be realized. This interaction method has the advantages of high efficiency and speed, and can achieve in-depth interaction across systems and devices. It can be widely used in the field of terminal devices.

[0058] There are many ways to trigger the establishment of multi-screen collaborative connections between different terminal devices, such as close proximity between devices, code scanning, or wired methods. The method based on close proximity between devices is convenient and fast and does not require additional data cables. Therefore, it has become a widely used triggering method.

[0059] In some embodiments, the proximity between devices includes Bluetooth (BT), wireless fidelity (WiFi), ultra-wide band (UWB) technology, and the proximity of special devices. However, when the multi-screen collaborative connection is established by the proximity trigger of Bluetooth and WiFi, the ranging accuracy of Bluetooth and WiFi is too low, usually with an error of 1 to 2 meters. Using Bluetooth technology or WiFi technology as a trigger condition for the proximity between devices cannot accurately determine the target device that needs to be approached; and UWB technology, although the positioning accuracy is high, the power consumption is also relatively high, and it cannot be used always-on. Therefore, the triggering method of proximity between devices based on special devices is widely used to trigger multi-screen collaborative connections between different terminal devices.

[0060] Among various special devices, near field communication (NFC) sensing chips have been applied to terminal devices due to their low power consumption, ultra-close working distance (10cm) and high ranging accuracy.

[0061] Figure 1This is a schematic diagram of a proximity connection between terminals provided by an embodiment of the present application, which involves a process of establishing a multi-screen collaborative connection between a mobile phone and a laptop computer based on an NFC sensing chip trigger. The mobile phone includes a first NFC sensing chip, and the laptop computer includes a second NFC sensing chip. Specifically, the user touches the first NFC sensing chip of the mobile phone to the second NFC sensing chip of the laptop computer. After the first NFC sensing chip of the mobile phone detects the radio waves emitted by the second NFC sensing chip of the laptop computer, it can read the identification information of the laptop computer through the second NFC sensing chip of the laptop computer, and establish a multi-screen collaborative connection with the laptop computer based on the identification information.

[0062] It should be noted that before the laptop and mobile phone trigger the establishment of a multi-screen collaborative connection through the NFC sensing chip, you must first ensure that the WLAN, Bluetooth and NFC functions on the mobile phone are turned on, and that the laptop supports NFC chip sensing, and turns on WLAN, Bluetooth, and PC Manager, and agrees to the user license agreement function, before a close connection between the two can be achieved.

[0063] Based on the above content, it can be known that when the built-in NFC sensing chip is used to trigger the establishment of a multi-screen collaborative connection between different devices, the coordinated terminal devices must all be equipped with NFC sensing chips. However, since some terminal devices may not be equipped with this special device, for this type of terminal device, it is usually necessary to use traditional methods such as scanning codes and wired connections, which cannot achieve the effect of natural interaction, the connection process is cumbersome, and the user experience is poor.

[0064] To this end, an embodiment of the present application provides a method for establishing a multi-screen collaborative connection. The method is based on common components of terminal devices and establishes a multi-screen collaborative connection by triggering proximity between a first device with a built-in magnetic sensor and a second device with a built-in magnet. No other special devices need to be configured. Not only can it be applied to most terminal devices, but it also has lower power consumption, is easy to operate, and provides a better user experience.

[0065] Figure 2 Schematic diagram of the system architecture applicable to the method for establishing a multi-screen collaborative connection provided in the embodiment of the present application. Figure 2 As shown, the system includes a first terminal device (referred to as the first device) and at least one second terminal device (referred to as the second device).

[0066] It should be noted that, in this embodiment, the first device and each second device have a communication function. For example, the first device and the second device are both connected to WLAN, and both have Bluetooth turned on. Among them, the WLAN connected to the first device and the second device can be the same network or different networks. The Bluetooth can be classic Bluetooth (bluetooth, BT) or low-power Bluetooth (bluetooth low energy, BLE).

[0067] In the present application, the terminal device may be a mobile phone, a tablet computer, a computer with wireless transceiver function, a smart TV, a wearable device (such as a smart watch), a vehicle-mounted device, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), and other electronic devices. The embodiments of the present application do not limit the specific type of the terminal device.

[0068] Figure 3 1 is a schematic diagram of the structure of a terminal device provided by an embodiment of the present application. The terminal device may include a processor 310, an external memory interface 320, an internal memory 321, a universal serial bus (USB) interface 330, a charging management module 340, a power management module 341, a battery 342, an antenna 1, an antenna 2, a mobile communication module 350, a wireless communication module 360, an audio module 370, a speaker 370A, a receiver 370B, a microphone 370C, an earphone interface 370D, a sensor module 380, a button 390, a motor 391, an indicator 392, a camera 393, a display screen 394, and a subscriber identification module (SIM) card interface 395, etc.

[0069] It is to be understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the terminal device. In other embodiments of the present application, the terminal device may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0070] For example, when the terminal device is a mobile phone or a tablet computer, it may include all the components shown in the figure, or may include only some of the components shown in the figure.

[0071] The processor 310 may include one or more processing units. Different processing units may be independent devices or integrated into one or more processors. A memory may also be provided in the processor 310 for storing instructions and data. In some embodiments, the memory in the processor 310 is a cache memory. The memory may store instructions or data that the processor 310 has just used or circulated. If the processor 310 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 310, and thus improves the efficiency of the system.

[0072] The external memory interface 320 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the terminal device. The external memory card communicates with the processor 310 through the external memory interface 320 to implement a data storage function. For example, files such as music and videos can be stored in the external memory card.

[0073] The internal memory 321 can be used to store computer executable program codes, and the executable program codes include instructions. The processor 310 executes various functional applications and data processing of the terminal device by running the instructions stored in the internal memory 321. The internal memory 321 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.). The data storage area may store data created during the use of the terminal device (such as audio data, a phone book, etc.).

[0074] In addition, the internal memory 321 may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0075] The USB interface 330 is an interface that complies with the USB standard specification, and can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 330 can be used to connect a charger to charge the terminal device, and can also be used to transfer data between the terminal device and peripheral devices. It can also be used to connect headphones to play audio through the headphones. The interface can also be used to connect other terminal devices, such as AR devices, etc.

[0076] The charging management module 340 is used to receive charging input from the charger. The power management module 341 is used to connect the battery 342, the charging management module 340 and the processor 310. The power management module 341 receives input from the battery 342 and / or the charging management module 340 to power the processor 310, the internal memory 321, the external memory, the camera 393, the display screen 394, and the wireless communication module 360.

[0077] The wireless communication function of the terminal device can be realized through antenna 1, antenna 2, mobile communication module 350, wireless communication module 360, modem processor and baseband processor.

[0078] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the terminal device can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of the antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0079] The mobile communication module 350 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to terminal devices. The mobile communication module 350 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 350 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 350 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1.

[0080] In some embodiments, at least some functional modules of the mobile communication module 350 may be disposed in the processor 310. In some embodiments, at least some functional modules of the mobile communication module 350 and at least some modules of the processor 310 may be disposed in the same device.

[0081] The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 370A, a receiver 370B, etc.), or displays an image or video through a display screen 394. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 310 and be set in the same device as the mobile communication module 350 or other functional modules.

[0082] The wireless communication module 360 ​​can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth, global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), ultra-wideband (UWB) and other wireless communication solutions applied to terminal devices. The wireless communication module 360 ​​can be one or more devices integrating at least one communication processing module. The wireless communication module 360 ​​receives electromagnetic waves via antenna 2, modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 310. The wireless communication module 360 ​​can also receive the signal to be sent from the processor 310, modulate the frequency, amplify it, and convert it into electromagnetic waves for radiation through antenna 2.

[0083] In some embodiments, antenna 1 of the terminal device is coupled to mobile communication module 350, and antenna 2 is coupled to wireless communication module 360, so that the terminal device can communicate with the network and other devices through wireless communication technology.

[0084] The terminal device can implement audio functions through an audio module 370, a speaker 370A, a receiver 370B, a microphone 370C, an earphone interface 370D, and an application processor.

[0085] The audio module 370 is used to convert digital audio signals into analog audio signals for output, and is also used to convert analog audio inputs into digital audio signals. The audio module 370 can also be used to encode and decode audio signals. In some embodiments, the audio module 370 can be arranged in the processor 310, or some functional modules of the audio module 370 are arranged in the processor 310.

[0086] The speaker 370A, also called a "speaker", is used to convert an audio electrical signal into a sound signal. The terminal device can listen to music or listen to a hands-free call through the speaker 370A. In addition, the speaker 370A can also emit an ultrasonic signal of a preset frequency (such as f1) according to the control of the processor 310.

[0087] The receiver 370B, also called the "earpiece", is used to convert the audio electrical signal into a sound signal. When the terminal device receives a call or voice message, the voice can be heard by placing the receiver 370B close to the human ear. In addition, the receiver 370B can also transmit an ultrasonic signal of a preset frequency (such as f2) according to the control of the processor 310.

[0088] Microphone 370C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can make a sound by approaching the microphone 370C with his mouth to input the sound signal into the microphone 370C. The terminal device may be provided with at least one microphone 370C. In some embodiments, the microphone 370C may also receive ultrasonic signals, for example, ultrasonic signals of different frequencies emitted by speakers and receivers of other terminal devices.

[0089] The earphone interface 370D is used to connect a wired earphone and can be a USB interface 330 or a 3.5 mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0090] The key 390 includes a power key, a volume key, etc. The key 390 may be a mechanical key or a touch key. The terminal device may receive key input and generate key signal input related to user settings and function control of the terminal device.

[0091] Motor 391 can generate vibration prompts. Motor 391 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. For touch operations acting on different areas of the display screen 394, motor 391 can also correspond to different vibration feedback effects. Different application scenarios (for example: time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

[0092] Indicator 392 may be an indicator light that may be used to indicate charging status, power changes, messages, missed calls, notifications, and the like.

[0093] The terminal device implements the display function through a GPU, a display screen 394, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 394 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 310 may include one or more GPUs, which execute program instructions to generate or change display information.

[0094] The display screen 394 is used to display images, videos, etc. The display screen 394 includes a display panel. In some embodiments, the terminal device may include 1 or N display screens 394, where N is a positive integer greater than 1.

[0095] The SIM card interface 395 is used to connect a SIM card. The SIM card can be connected to or disconnected from the terminal device by inserting into or removing from the SIM card interface 395. The terminal device can support 1 or N SIM card interfaces, where N is a positive integer greater than 1.

[0096] The terminal device shown in the above embodiment may be a first device or a second device.

[0097] When the terminal device is the first device, the first device further includes a magnetic sensor. The magnetic sensor is used to detect the magnetic field strength at the location of the first device. The magnetic field strength is a vector used to describe the strength and direction of the magnetic field, and its unit is Gauss.

[0098] In some implementations, the magnetic sensor may be a magnetic field sensor, a Hall sensor, or a nine-axis inertial measurement unit (IMU). The nine-axis IMU includes a three-axis accelerometer, a three-axis gyroscope, and a three-axis magnetometer (magnetic sensor, M-Sensor). The three-axis accelerometer is used to sense the acceleration information of the carrier in the three degrees of freedom of roll, pitch, or yawl; the three-axis gyroscope is used to sense the attitude information of the carrier in the three degrees of freedom of roll, pitch, and yawl; the three-axis magnetometer is used to detect the magnetic field strength at the location of the carrier, which may be the vector sum of the magnetic field strengths in each direction detected by the three-axis magnetometer, or the magnetic field strength of the magnetometer in the three degrees of freedom of roll, pitch, and yawl.

[0099] Typically, a mobile phone is provided with a nine-axis IMU, so the mobile phone can be used as the first device in this embodiment. In other implementations, a tablet computer or a laptop computer provided with a nine-axis IMU can also be used as the first device in this embodiment.

[0100] When the terminal device is the second device, the second device further includes a magnet, which is used to generate a magnetic field.

[0101] In some implementations, the magnet can be a magnetic strip, or a magnetic element.

[0102] For example, see Figure 4 As shown in , terminal devices (such as mobile phones, tablet computers, and laptop computers, etc.) are all equipped with speakers. Since the sound-generating principle of the speaker is related to its magnet, the speaker includes a magnet. Therefore, the terminal devices including the speaker can be used as the second device in this embodiment.

[0103] Also, see Figure 5 As shown in , some terminal devices (such as laptops and tablet computers) usually use magnetic adsorption to connect peripheral devices to the terminal devices. For example, the tablet computer adsorbs the stylus pen on the tablet computer through a strong magnetic strip set on the side of the tablet computer. The strong magnetic strip can also generate a magnetic field. Therefore, the terminal device provided with a strong magnetic strip can also be used as the second device in this embodiment.

[0104] In some implementations, when the second device does not include a magnet, a magnetic strip may be attached to the device, and the device with the magnetic strip attached may be used as the second device.

[0105] In some further implementations, when the second device includes an IMU, the second device may determine the device posture based on the IMU, wherein the IMU may be a nine-axis IMU (see above for details), or an IMU that does not include a magnetometer, such as a six-axis IMU, or a three-axis IMU.

[0106] In the system provided in the embodiment of the present application, the first device can use the change in the magnetic field strength at its location as a trigger condition for establishing a multi-screen collaborative connection. After the magnetic field strength at its location meets the preset conditions, it can determine the target device from at least one surrounding second device and establish a connection with the target device.

[0107] Based on the system and terminal device for establishing a multi-screen collaborative connection provided in the above-mentioned embodiment, the method for establishing a multi-screen collaborative connection provided in the embodiment is specifically explained below.

[0108] Figure 6 is a schematic flow chart of a method for establishing a multi-screen collaborative connection provided by an embodiment of the present application. Figure 6 As shown, the method may include but is not limited to the following steps S601 to S603.

[0109] S601: The first device detects the magnetic field strength at the location of the first device through a magnetic sensor.

[0110] The magnetic field strength can be the vector sum of the magnetic field strengths in various directions at the location of the first device detected by the magnetic sensor, or it can be the magnetic field strength in a preset direction at the location of the first device. It can be set according to actual detection needs and is not specifically limited in this embodiment.

[0111] In some embodiments, the first device detects the magnetic field strength at the location of the first device in real time based on the magnetic sensor. In this way, missed detection can be effectively prevented, and the problem that the magnetic field strength at the location of the first device has changed at a certain moment but the magnetic sensor fails to detect it can be avoided, thereby improving the accuracy of magnetic field strength identification.

[0112] In other embodiments, the first device detects the magnetic field strength at the location of the first device based on the magnetic sensor when the first device is in a specific state such as a screen-on state (including a screen-on state when the screen is locked and a screen-on state when the screen is unlocked). This method can effectively reduce the power consumption of the device and increase the service life of the device.

[0113] S602: When the magnetic field strength meets a preset condition, the first device determines a target device from at least one surrounding second device.

[0114] In some embodiments, the preset condition magnetic field strength is greater than a first threshold. The first threshold may be between 40 and 70 Gauss, such as 50 Gauss, 60 Gauss, etc. Exemplarily, taking the first threshold of 50 Gauss as an example, when the magnetic field strength detected by the first device through the magnetic sensor is 70 Gauss, since it is greater than the first threshold, it is determined that the magnetic field strength meets the preset condition.

[0115] In other embodiments, the preset condition includes that the rate of change of the magnetic field strength within a preset time is greater than a second threshold. It can be understood that when the rate of change is greater than the second threshold, it means that the first device and the second device are rapidly approaching each other in a short time (such as 0.5 seconds). Exemplarily, the second threshold can be between 50 and 100 gauss per second, for example, 120 gauss per second, 130 gauss per second, etc.

[0116] When the magnetic field strength meets the preset conditions, the first device starts the proximity detection function and determines the target device that is closest to the first device and / or has a matching posture from the surrounding second devices. Among them, the proximity detection can be ultrasonic proximity detection or UWB proximity detection. The specific proximity detection and determination process will be explained in detail in the following embodiments, and will not be repeated in this section.

[0117] S603: The first device and the target device establish a multi-screen collaborative connection.

[0118] In some embodiments, the first device and the target device establish a communication connection based on their respective identification information, wherein the identification information may be a medium access control (MAC) address or a device name of the first device or the target device. After the first device and the target device establish a communication connection, a multi-screen collaborative relationship may be further established.

[0119] The method provided in the embodiment of the present application can realize a natural interactive connection between the first device and the target device by using the common components (such as magnetic sensors and magnets) built into the terminal device, and using the change in magnetic field strength when the devices are close to each other as a trigger condition for establishing a multi-screen collaborative connection between the devices. The method can be realized without setting up special devices, and the connection process is simple to operate, which can improve the user experience.

[0120] In the method for establishing a multi-device collaborative connection provided in this embodiment, the target device can be determined from the surrounding second devices based on a variety of different methods, for example, the first device determines the target device based on the received detection signal, or the first device determines the target device based on the distance information in the feedback signal, etc. The method provided in the embodiment of the present application is exemplarily described below in combination with different methods for determining the target device.

[0121] Example 1: The first device determines the target device according to the received detection signal.

[0122] Figure 7 is a schematic diagram of the interactive flow of a method for establishing a multi-screen collaborative connection provided by an embodiment of the present application, see Figure 7 As shown, the method includes the following steps S701 to S707.

[0123] S701: The first device detects the magnetic field strength at the location of the first device through a magnetic sensor.

[0124] Exemplarily, when the first device is a mobile phone, the mobile phone can detect the magnetic field strength at the location of the mobile phone based on the built-in nine-axis IMU. Normally, the detection value of the magnetic field strength detected by the nine-axis IMU in the mobile phone is the vector sum of the magnetic field strengths in all directions at the location of the mobile phone. Therefore, the mobile phone can directly determine the detection value as the magnetic field strength at its location. When the mobile phone uses the magnetic field strength in the first direction as the magnetic field strength at its location, the first direction can be any direction of the magnetometer in the three degrees of freedom of roll, pitch, and yawl, and the mobile phone can use the magnetic field strength in the first direction detected by the magnetometer as the magnetic field strength at its location.

[0125] S702: When the magnetic field strength meets a preset condition, the first device broadcasts a first indication message.

[0126] In some embodiments, the first device may broadcast a first indication message when the magnetic field strength is greater than a first threshold value, wherein the first indication message is used to instruct the surrounding second device to transmit a detection signal, which may be an ultrasonic signal or a UWB pulse signal.

[0127] The first device may broadcast the first indication message to surrounding devices via Bluetooth, or to devices in a local area network via WiFi, and this embodiment does not impose any specific limitation.

[0128] S703: After receiving the first indication message, the second device transmits a detection signal.

[0129] In some embodiments, the second device may transmit an ultrasonic wave signal based on a built-in speaker or receiver, or transmit a UWB pulse signal through a built-in UWB wireless communication module.

[0130] In this embodiment, the detection signal transmitted by the second device carries unique identification information of the second device, which is used to uniquely indicate the corresponding second device. The identification information includes a MAC address and / or a device name of the second device.

[0131] In some implementations, after receiving the first indication message, the second device transmits a detection signal once at a preset interval, thereby preventing the first device from failing to receive the detection signal due to some reasons after the second device transmits the detection signal.

[0132] S704: The first device determines the target device according to the received detection signal.

[0133] In this embodiment, when the detection signal is an ultrasonic signal, the first device may receive the detection signal through a built-in microphone. When the detection signal is a UWB pulse signal, the first device may receive the detection signal through a built-in UWB wireless communication module.

[0134] In some embodiments, when determining the target device, the first device may determine the target device according to the strength of the detection signal transmitted by each second device. For example, the first device may determine the second device corresponding to the detection signal with the strongest signal strength and greater than the strength threshold as the target device. The strength threshold may be in the range of 40 to 60 decibels (dB), for example, 50 dB.

[0135] It can be understood that there is usually loss in the detection signal during the propagation process, and the longer the propagation distance, the greater the loss. When the first device receives the detection signal, the smaller the strength of the detection signal is. Therefore, the second device corresponding to the detection signal with the strongest intensity and greater than the intensity threshold received by the first device is closest to the first device and is the target device in this application. When the first device determines the target device based on the intensity of the detection signal, the first indication message is also used to further instruct the surrounding second devices to transmit a detection signal of a preset intensity. In other words, the detection signal emitted by the second device needs to maintain a consistent intensity so that the first device can determine the target device based on the strength of the received detection signal when receiving the detection signal. Exemplarily, the preset intensity can be 60dB.

[0136] For example, see Figure 8 As shown in , taking the case where the first device is a mobile phone, there are three second devices around the first device, and the second devices are laptops as an example, in the process of the mobile phone determining the target device, the mobile phone detects that the strength of the detection signal of laptop A is 60dB, the strength of the detection signal of laptop B is 20dB, and the strength of the detection signal of laptop C is 10dB. Since the strength of the detection signal of laptop A is the largest and exceeds the strength threshold of 50dB, the mobile phone determines laptop A as the target device.

[0137] In other embodiments, the first device determines the distance between the first device and each second device according to the received detection signals transmitted by each second device, and then determines the second device with the smallest distance and less than the distance threshold as the target device. The distance threshold may be in the range of 1 cm to 2 cm, for example, 1 cm.

[0138] It should be noted that, in this embodiment, the first indication signal is further used to instruct the surrounding second devices to transmit a detection signal carrying timestamp information, and the timestamp information includes the sending time T1 of the detection signal. After receiving the detection signal transmitted by each second device, the first device can determine the distance between each second device according to the sending time T1 and the receiving time T2 of each signal.

[0139] Exemplary, combined Figure 8 As shown in , in the process of determining the target device based on the distance between the mobile phone and each laptop computer, the mobile phone determines that the distance between the mobile phone and laptop computer A is 0.5 cm based on the detection signal of laptop computer A, the mobile phone determines that the distance between the mobile phone and laptop computer B is 10 cm based on the detection signal of laptop computer B, and the mobile phone determines that the distance between the mobile phone and laptop computer C is 15 cm based on the detection signal of laptop computer C. Since the distance between the mobile phone and laptop computer A, 0.5 cm, is the minimum distance and the distance value is less than the distance threshold of 1 cm, the mobile phone determines laptop computer A as the target device.

[0140] S705: The first device broadcasts a notification message, where the notification message is used to indicate that the corresponding second device becomes a target device.

[0141] In this embodiment, the first device may broadcast a notification message to surrounding devices via Bluetooth, or broadcast a notification message to other devices in a local area network via WiFi. The notification message may carry identification information of the target device, and the second device determined as the target device may determine that it is the target device identified by the first device after receiving the notification message.

[0142] Optionally, other second devices may also determine, based on the notification message, that they are not the target device identified and determined by the first device.

[0143] For example, after the first device determines that the second device 1 is the target device, the first device broadcasts a notification message, which is used to indicate that the second device 1 has become the target device. Based on this, after receiving the notification message, the second device 1 determines that it is the target device, stops transmitting the detection signal, and waits to establish a connection with the first device. After receiving the notification message, the second device 2, the second device 3, etc. determine that they are not the target device and stop transmitting the detection signal.

[0144] S706, the first device determines whether its posture matches that of the target device. If so, the next step S707 is executed.

[0145] In some embodiments, see Fig. 9 As shown in , S705 specifically includes the following S706a to S706d.

[0146] It should be noted that, in this embodiment, when the first device determines whether its posture matches that of the target device, the first device and the target device need to have built-in sensor devices for determining their respective posture information, such as IMU. Among them, the IMU of the first device can be a nine-axis IMU, in which the three-axis magnetometer in the nine-axis IMU is used to detect the magnetic field strength at the location of the first device, and the three-axis gyroscope and the three-axis accelerometer are used to determine the posture information of the first device; the built-in IMU of the second device can be an IMU that does not include a three-axis magnetometer, such as a three-axis IMU (including a three-axis gyroscope) or a six-axis IMU (including a three-axis gyroscope and a three-axis accelerometer), or a nine-axis IMU.

[0147] S706a: The first device determines first posture information of the first device.

[0148] In this embodiment, the first device can determine the first posture information of the first device based on the built-in nine-axis IMU, and the first posture information is used to represent the posture of the first device, such as horizontal screen, vertical screen, etc.

[0149] In some embodiments, see Fig.10 As shown in , the posture information of the device includes the angle information of the pitch angle α, roll angle β and offset angle γ of the terminal device.

[0150] S706b: The first device broadcasts a second indication message, where the second indication message is used to instruct the target device to send its second posture information.

[0151] S706c: The target device sends second gesture information.

[0152] Exemplarily, after receiving the second indication message, the target device determines its own second posture information through a built-in sensor device, and then sends the second posture information in the form of broadcasting.

[0153] S706d: The first device receives second gesture information.

[0154] Optionally, the first device may not execute S706b to S706d, and use the first indication message to further instruct the second device to send its posture information, thereby obtaining the posture information of all second devices and filtering out the second posture information of the target device. When the first device uses the first indication message to further instruct the second device to send its posture information, the first device directly executes the following step S706e after determining the first posture information of the first device.

[0155] S706e: The first device determines whether the posture of the first device matches that of the target device according to the first posture information and the second posture information.

[0156] In this embodiment, the posture matching between the first device and the target device includes: the first plane and the second plane are coincident, parallel or perpendicular to each other. Exemplarily, the first plane is the plane where the display screen of the first device is located, and the second plane is the plane where the display screen of the target device is located.

[0157] In some embodiments, when the absolute value of the corresponding angle difference between the first posture information and the second posture information is within a preset range, it is determined that the posture of the first device matches that of the target device. The preset range includes a first preset range and a second preset range, the first preset range may be [0°, 5°] or [0°, 10°], and the second preset range may be [85°, 95°] or [90°, 95°].

[0158] When the absolute value of the corresponding angle difference between the first posture information and the second posture information is within a first preset range, for example, [0°, 5°], it indicates that the first plane coincides with or is parallel to the second plane.

[0159] When the absolute value of the corresponding angle difference between the first posture information and the second posture information is within a first preset range, for example, [85°, 95°], it indicates that the first plane and the second plane are perpendicular to each other.

[0160] It should be understood that in this embodiment, the first device and the target device are in the same Euler coordinate system. The following is a specific example to explain the process of the first device determining whether the posture matches that of the target device.

[0161] Exemplary, combined Figure 8 As shown in , the first device is a mobile phone and the target device is a laptop A. Laptop A includes surface B (screen area of ​​the laptop) and surface C (keyboard area of ​​the laptop). Assuming that the IMU is set on surface B of laptop A, the plane where the mobile phone is located is the first plane, and the plane where surface B of laptop A is located is the second plane, then the second posture information determined by laptop A through IMU can be expressed as (α A , β A , γ A). The first posture information determined by the mobile phone through the nine-axis IMU can be expressed as (α, β, γ). The mobile phone determines the second posture information (α A , β A , γ A ) is compared with its own first posture information (α, β, γ) to determine the absolute value of the corresponding angle difference.

[0162] θ α =|α-α A |,θ β =|β-β A |

[0163] Among them, θ α Represents the absolute value of the difference between the pitch angles of the mobile phone and the laptop A, θ β Indicates the absolute value of the difference between the roll angles of the mobile phone and the laptop A.

[0164] When θ α ∈[0°,5°], and θ β ∈[0°,5°], it is determined that the postures of the mobile phone and the laptop A match, and the plane where the mobile phone is located is parallel to or coincides with the B surface of the laptop A. β ∈[85°,95°], it is determined that the postures of the mobile phone and the laptop A match, and the plane where the mobile phone is located is perpendicular to the B surface of the laptop A.

[0165] It should be noted that S706 is an optional step, that is, after determining the target device, the first device may not determine whether its posture matches that of the target device, but directly execute the following step S707.

[0166] S707: The first device establishes a multi-screen collaborative connection with the target device.

[0167] In some embodiments, a communication connection can be established between the first device and the target device based on their respective identification information (such as MAC addresses), and a multi-screen collaborative relationship can be established. It can be understood that after the first device determines the target device, the first device broadcasts a first indication signal, and the target device also determines itself as the target device based on the first indication signal. Therefore, the establishment of a multi-screen collaborative connection can be initiated by the first device or by the target device.

[0168] In the method provided in this embodiment, based on the magnetic sensor built into the first device and the magnet included in the second device, after the magnetic field strength at the location of the first device meets the preset conditions, the target device is determined according to the detection signal, and a multi-screen collaborative connection is established with the target device. This method not only satisfies the requirement of being able to achieve natural interactive connection, but also has the advantages of low power consumption and strong applicability.

[0169] In addition, through the method for establishing a multi-screen collaborative connection provided in this embodiment, in an embodiment where the device posture is used as a prerequisite for triggering the establishment of a multi-screen collaborative connection, when there are multiple second devices (for example, mobile phones, tablets, etc.), the first device only selects the second device that is coincident with, parallel to, or perpendicular to its plane to establish a multi-screen collaborative connection, thereby improving the accuracy of the multi-screen collaborative connection process and avoiding false triggering.

[0170] Example 2: The first device determines the target device according to the received feedback signal.

[0171] Fig.11 FIG. 1 is a schematic diagram of an interactive flow of a method for establishing a multi-screen collaborative connection provided by another embodiment of the present application. Fig.11 As shown, the method includes the following steps S1101 to S1106.

[0172] S1101: The first device detects the magnetic field strength at the location of the first device through a magnetic sensor.

[0173] In this embodiment, the specific content of S1101 can be found in S701 above, which will not be repeated here.

[0174] S1102: When the magnetic field strength meets a preset condition, the first device broadcasts a third indication message and sends a detection signal.

[0175] In this embodiment, the first device broadcasts a third indication message when the magnetic field strength is greater than the first threshold. The third indication message is used to instruct the surrounding second devices to turn on proximity detection. The proximity detection may be that each second device determines the distance between the second device and the first device based on the received detection signal. The detection signal includes an ultrasonic signal or a UWB pulse signal.

[0176] In some embodiments, the first device may transmit an ultrasonic wave signal based on a built-in speaker or receiver, or transmit a UWB pulse signal through a built-in UWB wireless communication module.

[0177] S1103: After receiving the second indication message, the second device determines the distance between itself and the first device according to the detection signal.

[0178] The second device may receive the ultrasonic wave signal through a built-in microphone, or receive the UWB pulse signal through a built-in UWB wireless communication module.

[0179] After the first device broadcasts the third indication message and transmits the detection signal, the second device that receives the second indication message determines the distance between itself and the first device according to the received detection signal.

[0180] Exemplary, combined Figure 8As shown in , after the mobile phone broadcasts the third instruction message and transmits the detection signal, laptop A, laptop B, and laptop C all determine the distance between the mobile phone and each of them according to the detection signal. Laptop 1 determines the first distance between itself and the mobile phone according to the detection signal, laptop 2 determines the second distance between itself and the mobile phone according to the detection signal, and laptop 3 determines the third distance between itself and the mobile phone according to the detection signal.

[0181] S1104: The second device broadcasts a feedback signal.

[0182] The second device that receives the indication message broadcasts a feedback signal to the first device, where the feedback signal includes distance information between each of the second devices and the first device.

[0183] S1105: The first device determines a target device according to the feedback signal.

[0184] In some embodiments, after receiving the feedback signal, the first device determines the second device corresponding to the signal with the shortest distance in the feedback signal and less than the distance threshold as the target device according to the distance information in the feedback signal.

[0185] S1106: The first device broadcasts a notification message, where the notification message is used to indicate that the corresponding second device becomes a target device.

[0186] S1107, the first device determines whether its posture matches that of the target device. If so, the next step S1108 is executed.

[0187] In this embodiment, the specific content of S1107 can be found in S706 above, which will not be repeated here.

[0188] Similarly, in this embodiment, when the first device determines whether its posture matches that of the target device, the first device and the target device need to have built-in sensor devices for determining their respective posture information.

[0189] It should be noted that, in this embodiment, S1107 is also an optional step, that is, after determining the target device, the first device may not determine whether its posture matches that of the target device, but directly execute the following step S1108.

[0190] S1108: The first device establishes a multi-screen collaborative connection with the target device.

[0191] In this embodiment, the specific content of S1108 can be found in S707 above and will not be repeated here.

[0192] It should be noted that, in the above-mentioned embodiments, the various multi-screen collaborative functions triggered after the first device establishes a multi-screen collaborative connection with the target device can be set according to actual needs. For example, they can be written according to the specific operations that can be achieved between different devices (for example, mobile phones and laptops, mobile phones and tablet computers, tablet computers and laptops, etc.). This embodiment does not impose any specific restrictions.

[0193] In some embodiments, the multi-screen collaboration function includes multi-device interaction functions such as screen mirroring, screen extension, and resource sharing.

[0194] Screen mirroring means that after the first device and the target device establish a multi-screen collaborative connection, the first device sends the first screen image of the first device to the target device in real time in the form of a video stream, and at the same time, the target device displays the first screen image in real time in the form of a floating window. For details, see Fig. 12A As shown. Based on screen mirroring technology, it is possible to operate and use the first device in the target device interface. For example, after a laptop and a mobile phone establish a multi-screen collaborative connection, the user can use the laptop's mouse to click on the pictures and music in the mobile phone, which can locate more accurately and quickly; the user can also use the computer input method to enter text in the mobile phone chat window, thereby significantly improving typing speed and efficiency.

[0195] Screen extension means that after the first device and the target device establish a multi-screen collaborative connection, the target device can be used as an external screen of the first device. For details, see Fig. 12B As shown. By means of screen expansion, the first device can move the target content (such as files, shortcut icons, pictures, videos, application icons, floating windows, etc.) on the first device to the target device for display according to the user operation. The user can operate the target content on the target device, such as opening the target content, deleting the target content, moving the target content, modifying the target content, etc., which is not limited in this embodiment.

[0196] Resource sharing means that after the first device and the target device establish a multi-screen collaborative connection, the resource information in the first device can be shared with the target device according to the user operation. For details, see Fig. 12C The resource information includes, but is not limited to, documents, media files (such as images, audio, and video), messages (such as messages in instant messaging applications, short messages, and multimedia messages), application installation packages, contact information, web links, web content, or other data that can be accessed by a terminal device, and the present application embodiment does not limit this.

[0197] As can be seen from the above, after the first device establishes a connection with the target device, the triggered functions include one or more. When the triggered functions include one, it can be directly triggered, such as directly mirroring the content displayed on the mobile phone to the laptop. When the triggered functions include multiple functions, the selection controls of each multi-screen collaboration function can be displayed on the device, and then the corresponding multi-screen collaboration function can be started according to the user operation.

[0198] For example, see Fig.13 As shown in , after a connection is established between a mobile phone and a laptop computer, selection controls for various multi-screen collaboration functions that can be triggered are displayed on the laptop computer, for example, "mirroring", "extending" and "sharing". In response to the user's selection operation on the "extending" selection control, the multi-device collaboration between the laptop computer and the mobile phone enters the "extended" multi-screen collaboration function.

[0199] In summary, the method for establishing a multi-screen collaborative connection provided in this embodiment is to start high-precision proximity detection of ultrasonic signals or UWB pulse signals after detecting that the magnetic field strength meets the preset conditions. Compared with the process of determining the target device by using high-precision proximity detection methods such as ultrasonic signals or UWB pulse signals always-on in traditional technologies, the method in this embodiment can accurately determine the target device, reduce false triggering, and save device power consumption.

[0200] The above is a process of establishing a multi-screen collaborative connection between different terminal devices in different ways provided in this embodiment. In addition, this embodiment also provides a process of triggering a corresponding multi-screen collaborative function according to the device posture after the first device establishes a multi-screen collaborative connection with the target device.

[0201] Fig.14 This is a schematic diagram of the interaction process of a method for triggering a multi-screen collaborative function according to a device posture provided by an embodiment of the present application, which involves a process of triggering a corresponding multi-screen collaborative function according to the device posture after the first device establishes a connection with the target device. It should be noted that since this embodiment also involves specific steps for performing corresponding operations according to the device posture, the first device and the target device in this embodiment also need to have built-in sensor devices for determining their respective posture information. Fig.14 As shown, the method specifically includes the following steps S1401 to S1404.

[0202] S1401: The first device determines first posture information of the first device.

[0203] In this embodiment, the specific content of S1401 is shown in S706a above and will not be repeated here.

[0204] S1402: The first device sends a posture information request to the target device.

[0205] The posture information request is used by the target device to send second posture information of the target device to the first device according to the posture information request, and the second posture information is used to represent the posture of the target device.

[0206] S1403: After receiving the posture information request, the target device sends second posture information of the target device to the first device.

[0207] After the first device establishes a connection with the target device, the first device may send a posture information request to the target device through the established connection channel. After receiving the posture information request, the target device sends its own second posture information to the first device through the established connection channel.

[0208] S1404: The first device receives second posture information.

[0209] S1405: The first device determines whether the posture of the first device matches that of the target device according to the first posture information and the second posture information. If they match, the next step S1406 is executed.

[0210] In this embodiment, the process of the first device determining whether the posture of the target device matches is shown in the aforementioned step S706, and is not described in detail in this section.

[0211] S1406: When the first device matches the posture of the target device, the corresponding multi-screen collaborative function is triggered.

[0212] In this embodiment, different matching forms between the first device and the target device trigger different multi-screen collaborative functions.

[0213] It should be noted that, in this embodiment, when the posture state of the first device matches that of the target device, the triggered functions include but are not limited to screen mirroring, screen expansion, and resource sharing shown in the above embodiments, and also include other functions triggered according to the device posture.

[0214] In some embodiments, when the first plane of the first device is parallel to or overlaps with the second plane of the target device, the screen mirroring function is triggered. For example, taking the first device as a mobile phone and the target device as a laptop computer, if the first screen image currently displayed on the mobile phone screen is a film image, after the mobile phone and the laptop computer establish a multi-screen collaborative connection, the film image of the mobile phone is transferred to the display interface of the laptop computer in real time, and the user can continue to watch the film content on the laptop computer screen.

[0215] In other embodiments, when the first plane of the first device is parallel to or overlaps with the second plane of the target device, the triggered multi-screen collaboration function includes other functions, such as the first device using the accessories of the target device (such as a camera, speaker, microphone, mouse, and keyboard, etc.). For example, if the first device is a mobile phone and the target device is a laptop, if the first screen currently displayed on the mobile phone screen is a video chat content, then after the mobile phone and the laptop establish a multi-screen collaboration connection, the laptop's camera is called to continue the mobile phone's current video chat task, and the user can continue the video chat on the laptop. For details, see Fig.15 as shown in .

[0216] Since the mobile phone and the laptop are in the same or similar plane during the camera call process, based on the similar viewing angles of the cameras of the two devices and in conjunction with graphics algorithms, the video stream can flow smoothly when switching cameras.

[0217] Compared with the traditional technology where a user uses the video chat function of a mobile phone, when the video source needs to be switched from the camera of the mobile phone to the camera of a laptop computer, since the shooting angles corresponding to the cameras of the mobile phone and the laptop computer are different, there will be a jump between the different viewing angles of the mobile phone camera and the laptop computer camera during video streaming, resulting in a freeze in the video streaming process, affecting the user experience. The process of triggering multi-screen collaborative connection provided in this embodiment can achieve freeze-free streaming of the video stream and improve the user experience.

[0218] In some other implementations of this embodiment, after the mobile phone and the laptop computer establish a multi-screen collaborative connection and trigger the multi-screen collaborative function of the first device using the accessories of the target device, the mobile phone determines in real time whether the postures of the mobile phone and the laptop computer match according to its own first posture information and the second posture information of the laptop computer. Fig.16 As shown in , when the inclination angle of the B-side screen of the laptop exceeds a certain angle, the posture of the laptop changes, and the screen camera is no longer suitable as a video source for video chat. Therefore, the mobile phone can trigger corresponding operations according to the changed posture information. For example, when the mobile phone determines that the absolute value of the corresponding angle difference between the plane where the mobile phone is located and the plane where the B-side of the laptop is located is in the third preset range, it stops using the screen camera of the laptop and triggers the multi-screen collaboration function of switching the camera of the laptop used for video to the front camera of the mobile phone. Exemplarily, the third preset range can be [65°, 75°], for example, it can be 70°.

[0219] In some other embodiments, when the first plane of the first device and the second plane of the target device are perpendicular to each other, the triggered multi-screen collaboration function includes other functions, such as the first device acting as an external keyboard or an external controller of the target device. For example, when the first device is a mobile phone and the target device is a tablet computer, if the mobile phone determines that the plane where the mobile phone is located is parallel to the plane where the tablet computer is located based on the posture information of the mobile phone and the posture information of the tablet computer, then the triggered multi-screen collaboration function includes the mobile phone acting as an external keyboard of the tablet computer, or the tablet computer acting as an external keyboard of the mobile phone, see Fig.17 As shown in ; or the mobile phone is used as an external controller of the tablet to control the tablet.

[0220] In this embodiment, after the first device and the target device establish a connection, the multi-screen collaboration function is not triggered immediately, but after determining the posture between the first device and the target device, the corresponding multi-screen collaboration function under each posture is triggered. This embodiment triggers different functions according to the different relative postures between the devices, and uses the semantics contained in the device posture information to achieve a richer and more natural interaction method.

[0221] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0222] An embodiment of the present application also provides a terminal device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor is configured to implement the method for establishing a multi-screen collaborative connection as shown in the above-mentioned embodiments when executing the computer program.

[0223] The present application also provides a chip, see Fig.18 As shown, the chip includes a processor and a memory, in which a computer program is stored. When the computer program is executed by the processor, the method for establishing a multi-screen collaborative connection in the above-mentioned embodiments is implemented.

[0224] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the method for establishing a multi-screen collaborative connection provided in the above embodiments is implemented.

[0225] An embodiment of the present application also provides a computer program product, which includes a computer program. When the computer program is executed by a terminal device, the terminal device implements the method for establishing a multi-screen collaborative connection provided in the above embodiments.

[0226] It should be understood that the processor mentioned in the embodiments of the present application may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0227] It should also be understood that the memory mentioned in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0228] In the embodiments provided in the present application, the division of each framework or module is only a logical function division. There may be other division methods in actual implementation. For example, multiple frameworks or modules can be combined or integrated into another system, or some features can be ignored or not executed.

[0229] In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of hardware or software functional modules.

[0230] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0231] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0232] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for establishing a connection, characterized in that: Applied to a first device, the first device is provided with a magnetic sensor, the method comprises: Detecting the magnetic field strength at the location of the first device by the magnetic sensor in a first detection manner; When the magnetic field strength meets a preset condition, broadcasting a first indication message in a second detection mode, wherein the first indication message is used to instruct a surrounding second device to transmit a feedback signal; receiving a feedback signal sent by the second device in a second detection mode, and determining a target device according to the feedback information; Broadcasting a notification message in a second detection mode, where the notification message is used to indicate that the corresponding second device becomes a target device; Establishing a connection with the target device; The first detection method is different from the second detection method.

2. The method according to claim 1, characterized in that The power consumption of the first detection mode is lower than the power consumption of the second detection mode.

3. The method according to claim 1 or 2, characterized in that: The establishing a connection with the target device includes: Receive the posture information sent by the target device, determine that the posture of the first device matches that of the target device, and establish a connection with the target device.

4. The method according to any one of claims 1 to 3, characterized in that: The preset conditions include: The magnetic field strength is greater than a first threshold; or, The rate of change of the magnetic field intensity within a preset time is greater than a second threshold.

5. The method according to any one of claims 1 to 4, characterized in that: The first indication message is further used to instruct the second device to send a detection signal of a preset strength, and the determining the target device according to the received detection signal includes: Determining the signal strength of each of the received detection signals; The second device corresponding to the detection signal having the largest signal strength and greater than the strength threshold is determined as the target device.

6. The method according to any one of claims 1 to 5, characterized in that: The first indication message is further used to instruct the second device to send a detection signal carrying a sending time, and determining the target device according to the received detection signal includes: Determining a reception time of each of the detection signals; Determine the distance between the first device and each of the second devices according to the sending time and the receiving time of each detection signal; The second device with the smallest distance and less than the distance threshold is determined as the target device.

7. The method according to any one of claims 1 to 4, characterized in that: When the magnetic field strength meets the preset condition, broadcasting the detection signal in the second detection mode includes: When the magnetic field strength meets a preset condition, broadcasting a first indication message and the detection signal in a second detection mode, wherein the first indication message is used to instruct the second device to turn on proximity detection, and the proximity detection includes the second device determining the distance between the second device and the first device according to the detection signal after receiving the detection signal; receiving a feedback signal sent by each second device, where the feedback signal is broadcast by the second device after proximity detection is enabled, and the feedback signal includes a distance between the second device and the first device; According to the feedback signals, the second device corresponding to the feedback signal with the shortest distance in the feedback signals and less than a distance threshold is determined as the target device.

8. The method according to any one of claims 1 to 7, characterized in that: When the magnetic field strength meets the preset condition, after determining the target device from at least one surrounding second device, the method further includes: A notification message is broadcasted, where the notification message is used to indicate that the corresponding second device becomes the target device.

9. The method according to claim 8, characterized in that After broadcasting the notification message, the method further includes: Determining first posture information of the first device; Broadcasting a second indication message, where the second indication message is used to instruct the target device to send second posture information; receiving the second posture information; According to the first posture information and the second posture information, it is determined that the first device matches the target device, and the posture matching includes: a first plane where the first device is located and a second plane where the second device is located are coincident, parallel or perpendicular to each other.

10. The method according to claim 9, characterized in that Determining, according to the first posture information and the second posture information, that the posture of the first device matches that of the target device includes: When the absolute values ​​of the differences between the corresponding angles of the first device and the target device are both within a first preset range, determining that a first plane where the first device is located coincides with or is parallel to a second plane where the second device is located; and / or, When the absolute value of the difference between the corresponding angles of the first device and the target device is within a second preset range, it is determined that a first plane where the first device is located and a second plane where the second device is located are perpendicular to each other.

11. The method according to claim 9 or 10, characterized in that: The establishing a multi-screen collaborative connection with the target device includes: When the postures of the first device and the target device match, a multi-screen collaborative connection is established with the target device.

12. The method according to any one of claims 1 to 11, characterized in that: After establishing a multi-screen collaborative connection with the target device, the method further includes: Determining first posture information of the first device; Sending a posture information request to the target device, wherein the posture information request is used to request the target device to send second posture information; receiving the second posture information; Determining, according to the first posture information and the second posture information, a posture matching condition between the first device and the target device; When the postures of the first device and the target device match, a corresponding multi-screen collaboration function is triggered.

13. The method according to claim 12, characterized in that When the posture of the first device matches that of the target device, triggering a corresponding multi-screen collaboration function includes: When a first plane where the first device is located coincides with or is parallel to a second plane where the second device is located, the target device uses the accessory of the first device.

14. The method according to claim 13, characterized in that The method further comprises: When the absolute values ​​of the differences between the corresponding angles of the first device and the target device are both within a third preset range, the target device stops using the accessory of the first device.

15. The method according to claim 12, characterized in that When the posture of the first device matches that of the target device, triggering a corresponding multi-screen collaboration function includes: When a first plane where the first device is located is perpendicular to a second plane where the second device is located, the first device serves as an external keyboard or an external controller of the target device.

16. A system for establishing a multi-screen collaborative connection, characterized in that: It includes a first device and at least one second device, the first device is provided with a magnetic sensor and is configured to execute the method for establishing a multi-screen collaborative connection as described in any one of claims 1 to 15, and the second device is provided with a magnet, which can affect the magnetic field strength around the second device.

17. A terminal device, characterized in that: It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the method for establishing a multi-screen collaborative connection as described in any one of claims 1 to 15.

18. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for establishing a multi-screen collaborative connection according to any one of claims 1 to 15 is implemented.

19. A chip, characterized in that: The chip includes a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the method for establishing a multi-screen collaborative connection as described in any one of claims 1 to 15 is implemented.

Images