Method, system, device, storage medium and chip for establishing multi-screen coordination connection

Abstract

The application relates to the technical field of terminal devices, and provides a method, a system, a device, a storage medium and a chip for establishing a multi-screen cooperative connection, which can solve the problem that the connection process is complicated and the user experience is poor when a terminal device establishes a multi-screen cooperative connection. The method is applied to a first device provided with a magnetic sensor, and the method comprises the following steps: detecting the magnetic field intensity of the position where the first device is located through the magnetic sensor; when the magnetic field intensity meets a preset condition, determining a target device from at least one second device around the first device, the second device being provided with a magnet, and the magnet being capable of affecting the magnetic field intensity around the second device; and establishing a multi-screen cooperative connection with the target device.

Description

[0001] This application is a divisional application, the original application number is 202310666025.4, the original application date is June 6, 2023, and the entire contents of the original application are incorporated herein by reference. TECHNICAL FIELD

[0002] The present application relates to the technical field of terminal devices, in particular to a method, system, device, storage medium and chip for establishing a multi-screen cooperative connection. BACKGROUND

[0003] With the rapid development of terminal devices, different terminal devices can achieve multi-device interaction functions such as resource sharing, screen mirroring, and screen extension after establishing a multi-screen cooperative connection.

[0004] Currently, different terminal devices can trigger the establishment of a multi-screen cooperative connection through natural interaction such as proximity and one-touch connection based on built-in special devices (such as near field communication (NFC) induction chips), without the need for users to manually operate other related controls. However, some terminal devices may not be equipped with such special devices, so for this type of terminal device, traditional methods such as scanning codes and wired connections are usually used, which cannot achieve natural interaction, and the connection process is cumbersome and the user experience is poor. SUMMARY

[0005] The present application provides a method, system, device, storage medium and chip for establishing a multi-screen cooperative connection, which solves the problem of cumbersome connection process and poor user experience when terminal devices establish a multi-screen cooperative connection in the prior art.

[0006] To achieve the above-mentioned purpose, the present application adopts the following technical solutions:

[0007] In a first aspect, a method for establishing a multi-screen cooperative connection is provided, applied to a first device provided with a magnetic sensor. The method comprises: detecting the magnetic field strength of the location of the first device through the magnetic sensor; when the magnetic field strength meets a preset condition, determining a target device from at least one second device around the first device, the second device being provided with a magnet capable of affecting the magnetic field strength around the second device; and establishing a multi-screen cooperative connection with the target device.

[0008] The method for establishing a multi-screen cooperative connection provided by the present application can use the general elements (such as magnetic sensors and magnets) built into terminal devices to change the magnetic field strength when the devices are close as the trigger condition for establishing a multi-screen cooperative connection between the devices, thereby achieving natural interaction connection between the first device and the target device. This method does not require additional special devices and is easy to operate, which can improve the user experience.

[0009] In some embodiments, the preset condition comprises: the magnetic field strength is greater than a first threshold; or, a rate of change of the magnetic field strength within a preset time is greater than a second threshold.

[0010] In this embodiment, the first threshold and the second threshold are used as a basis for judging whether the magnetic field strength meets the preset condition. If the preset condition is met, the process of triggering the establishment of the multi-screen cooperation connection is started. If the preset condition is not met, the process of triggering the establishment of the multi-screen cooperation connection is not started, thereby avoiding false triggering.

[0011] In some embodiments, when the magnetic field strength meets the preset condition, the target device is determined from the at least one second device around the first device, comprising: broadcasting a first indication message when the magnetic field strength meets the preset condition, the first indication message being used to instruct the second device to send a probe signal; and determining the target device according to the received probe signal.

[0012] In one example, the first indication message is also used to instruct the second device to send a probe signal of a preset strength, and the target device is determined according to the received probe signal, comprising: determining the signal strength of each received probe signal; and determining the second device corresponding to the probe signal with the maximum signal strength and greater than a strength threshold as the target device.

[0013] In another example, the first indication message is also used to instruct the second device to send a probe signal carrying a sending time, and the target device is determined according to the received probe signal, comprising: determining the receiving time of each received probe signal; determining the distance between the first device and each second device according to the sending time and the receiving time of each probe signal; and determining the second device with the minimum distance and less than a distance threshold as the target device.

[0014] In some embodiments, when the magnetic field strength meets the preset condition, the target device is determined from the at least one second device around the first device, comprising: broadcasting a third indication message and a probe signal when the magnetic field strength meets the preset condition, the third indication message being used to instruct the second device to start proximity detection, the proximity detection comprising that the second device determines the distance between the first device and the second device according to the probe signal after receiving the probe signal; receiving a feedback signal sent by each second device, the feedback signal being broadcasted by the second device after starting the proximity detection, the feedback signal comprising the distance between the second device and the first device; and determining the second device corresponding to the feedback signal with the shortest distance and less than a distance threshold in the feedback signal as the target device according to the feedback signal.

[0015] In this embodiment, the first device starts the detection process of determining the target device from the at least one second device in the surrounding after detecting that the magnetic field strength meets the preset condition. Compared with the process of determining the target device by using the high-precision proximity detection method such as the ultrasonic signal or the UWB pulse signal in the traditional technology, the method in this embodiment can accurately determine the target device, reduce the false triggering, and save the device power consumption.

[0016] In some embodiments, after determining the target device from the at least one second device in the surrounding when the magnetic field strength meets the preset condition, the method further comprises: broadcasting a notification message, the notification message being used to instruct the corresponding second device to become the target device.

[0017] Optionally, the other second devices can also determine that they are not the target device identified and determined by the first device according to the notification message.

[0018] In some embodiments, after broadcasting the notification message, the method further comprises: determining first attitude information of the first device; broadcasting a second instruction message, the second instruction message being used to instruct the target device to send the second attitude information; receiving the second attitude information; and determining that the attitude of the first device matches the attitude of the target device according to the first attitude information and the second attitude information, the attitude matching comprising that a first plane where the first device is located coincides with, is parallel to, or is perpendicular to a second plane where the second device is located.

[0019] In some embodiments, determining that the attitude of the first device matches the attitude of the target device according to the first attitude information and the second attitude information comprises: when the absolute values of the differences between the corresponding angles of the first device and the target device are all in a first preset range, determining that the first plane where the first device is located coincides with or is parallel to the second plane where the second device is located; and / or when the absolute value of the difference between the corresponding angles of the first device and the target device is in a second preset range, determining that the first plane where the first device is located is perpendicular to the second plane where the second device is located.

[0020] In some embodiments, establishing the multi-screen collaborative connection with the target device comprises: establishing the multi-screen collaborative connection with the target device when the attitude of the first device matches the attitude of the target device.

[0021] In this embodiment, the device attitude is used as the prerequisite condition for triggering the establishment of the multi-screen collaborative connection, when there are multiple second devices (such as mobile phones, tablet computers, etc.), the first device only selects the second device whose plane coincides with, is parallel to, or is perpendicular to the plane where the first device is located to establish the multi-screen collaborative connection, thereby improving the accuracy of the process of establishing the multi-screen collaborative connection and avoiding false triggering.

[0022] In some embodiments, after the multi-screen cooperation connection is established with the target device, the method further comprises: determining first posture information of the first device; sending a posture information request to the target device, the posture information request being used to request the target device to send second posture information; receiving the second posture information; determining posture matching between the first device and the target device according to the first posture information and the second posture information; and triggering a corresponding multi-screen cooperation function when the posture of the first device matches the posture of the target device.

[0023] In some embodiments, when the posture of the first device matches the posture of the target device, triggering the corresponding multi-screen cooperation function comprises: when the first plane where the first device is located and the second plane where the second device is located are coincident or parallel, the target device uses the accessory of the first device.

[0024] In some embodiments, the method further comprises: when the absolute values of the differences between the corresponding angles of the first device and the target device are all within a third preset range, the target device stops using the accessory of the first device.

[0025] In the embodiment, the first device and the target device adjust the multi-screen cooperation function according to the posture matching between them, so as to adapt to different application scenarios.

[0026] In some embodiments, when the posture of the first device matches the posture of the target device, triggering the corresponding multi-screen cooperation function comprises: when the first plane where the first device is located and the second plane where the second device is located are perpendicular to each other, the first device serves as an external keyboard or an external controller of the target device.

[0027] In the embodiment, after the connection is established, the first device and the target device trigger different functions according to the relative postures between them, and utilize the semantics contained in the posture information of the devices to realize a more abundant and natural interaction mode.

[0028] In a second aspect, a system for establishing a multi-screen cooperation connection is provided, comprising a first device and at least one second device, the first device being provided with a magnetic sensor and being configured to execute the method for establishing a multi-screen cooperation connection as shown in the first aspect, and the second device being provided with a magnet capable of affecting the magnetic field strength around the second device.

[0029] In a third aspect, a terminal device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor executes the computer program to implement the method for establishing a multi-screen cooperation connection as shown in the first aspect.

[0030] In a fourth aspect, a computer readable storage medium is provided, the computer readable storage medium storing a computer program, and the computer program is executed by a processor to implement the method for establishing a multi-screen cooperation connection as shown in the first aspect.

[0031] In a fifth aspect, a chip is provided, which comprises a processor and a memory, and the memory stores a computer program, which, when executed by the processor, implements the method for establishing a multi-screen cooperative connection as shown in the first aspect.

[0032] It can be understood that the beneficial effects of the second aspect to the fifth aspect can be referred to the related description in the first aspect, which will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0033] Figure 1 is a schematic diagram of a close connection between terminals provided by an embodiment of the present application;

[0034] Figure 2 is a schematic diagram of a system architecture to which the method for establishing a multi-screen cooperative connection provided by an embodiment of the present application is applied;

[0035] Figure 3 is a schematic diagram of the structure of a terminal device provided by an embodiment of the present application;

[0036] Figure 4 is a schematic diagram of the structure of a magnet arrangement of a terminal device provided by an embodiment of the present application;

[0037] Figure 5 is a schematic diagram of the structure of a magnet arrangement of a terminal device provided by another embodiment of the present application;

[0038] Figure 6 is a schematic flowchart of the method for establishing a multi-screen cooperative connection provided by an embodiment of the present application;

[0039] Figure 7 is a schematic diagram of the interaction process of the method for establishing a multi-screen cooperative connection provided by an embodiment of the present application;

[0040] Figure 8 is a schematic diagram of a close connection between terminals provided by an embodiment of the present application;

[0041] Figure 9 is a schematic diagram of the interaction process of the first device determining whether the posture of the first device matches the posture of the target device provided by an embodiment of the present application;

[0042] Figure 10 is a schematic diagram of the posture of a device provided by an embodiment of the present application;

[0043] Figure 11 is a schematic diagram of the interaction process of the method for establishing a multi-screen cooperative connection provided by another embodiment of the present application;

[0044] Figure 12Ais a scenario diagram of screen mirroring between devices provided by an embodiment of the present application;

[0045] Figure 12B is a scenario diagram of screen extension between devices provided by an embodiment of the present application;

[0046] Figure 12C is a scenario diagram of resource sharing between devices provided by an embodiment of the present application;

[0047] Figure 13 is a diagram of a multi-screen collaboration function selection interface provided by an embodiment of the present application;

[0048] Figure 14 is a diagram of an interaction flow of a method of triggering a multi-screen collaboration function according to a device posture provided by an embodiment of the present application;

[0049] Figure 15 is a diagram of an application scenario of a multi-screen collaboration function provided by an embodiment of the present application;

[0050] Figure 16 is a diagram of an application scenario of a multi-screen collaboration function provided by another embodiment of the present application;

[0051] Figure 17 is a diagram of an application scenario of a multi-screen collaboration function provided by still another embodiment of the present application;

[0052] Figure 18 is a diagram of a structure of a chip provided by an embodiment of the present application. DETAILED DESCRIPTION

[0053] The technical solutions provided by the embodiments of the present application will be described below with reference to the accompanying drawings.

[0054] It should be understood that, in the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in the present text merely describes the association relationship of the associated objects, and means that there can be three relationships, for example, A and / or B, which means that there are three cases of A alone, A and B together, and B alone.

[0055] In the present embodiment, the terms "first" and "second" are used only for the purpose of description, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first" and "second" can explicitly or implicitly include one or more of the features.

[0056] In the description of the present embodiments, unless otherwise specified, the meaning of "a plurality of" is two or more. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b, or c can represent a, b, c, a-b, a-c, b-c, or a-b-c, where a, b, and c can be single or multiple.

[0057] With the rapid development of terminal devices, after establishing a multi-screen cooperative connection between different terminal devices, multi-device interaction functions such as screen mirroring, screen expansion, and resource sharing can be achieved. This interaction method has the advantages of high efficiency and quickness, and can realize in-depth interaction across systems and devices, and can be widely applied in the field of terminal devices.

[0058] There are many ways to trigger the establishment of a multi-screen cooperative connection between different terminal devices, such as device proximity, code scanning, or wired connection. The proximity between devices is convenient and fast and does not require additional data lines, so it has become a widely used triggering method.

[0059] In some embodiments, device proximity includes proximity of Bluetooth (BT), wireless fidelity (WiFi), ultra-wide band (UWB) technology, and special devices. However, when establishing a multi-screen cooperative connection by triggering proximity through Bluetooth and WiFi, the ranging accuracy of Bluetooth and WiFi is too low, usually with an error of 1-2m. Using Bluetooth technology or WiFi technology as a trigger condition for device proximity cannot accurately determine the target device that needs to be approached. While UWB technology has high positioning accuracy, it also has relatively high power consumption and cannot be used always-on. Therefore, the triggering method based on the proximity of special devices between devices is widely used to trigger the multi-screen cooperative connection between different terminal devices.

[0060] Among various special devices, near field communication (NFC) induction chips have been applied to terminal devices due to their low power consumption, ultra-short (10cm) working distance, and high ranging accuracy.

[0061] Figure 1is an embodiment provided by the present application, and relates to a process of triggering a multi-screen cooperative connection based on an NFC induction chip between a mobile phone and a notebook computer. The mobile phone comprises a first NFC induction chip, and the notebook computer comprises a second NFC induction chip. Specifically, a user contacts the first NFC induction chip of the mobile phone with the second NFC induction chip of the notebook computer. After the first NFC induction chip of the mobile phone detects radio waves emitted by the second NFC induction chip of the notebook computer, the first NFC induction chip of the mobile phone can read identification information of the notebook computer through the second NFC induction chip of the notebook computer, and establish a multi-screen cooperative connection with the notebook computer according to the identification information.

[0062] It should be noted that before the notebook computer and the mobile phone trigger the multi-screen cooperative connection through the NFC induction chip, it is necessary to first ensure that the WLAN, Bluetooth and NFC functions of the mobile phone are turned on, and the notebook computer supports the NFC chip induction, and the WLAN, Bluetooth and computer manager are turned on, and the user permission agreement function is agreed, so as to realize the close connection between the two.

[0063] Based on the above content, it can be known that when triggering the multi-screen cooperative connection between different devices through the built-in NFC induction chip, the terminal devices to be cooperated need to be provided with the NFC induction chip. However, since some terminal devices may not be equipped with the special device, for this type of terminal device, it is usually necessary to realize through the traditional code scanning, wired connection and the like, and the natural interaction effect cannot be achieved, the connection process is complicated to operate, and the user experience is poor.

[0064] Therefore, an embodiment of the present application provides a method for establishing a multi-screen cooperative connection. The method triggers the multi-screen cooperative connection between a first device provided with a magnetic sensor and a second device provided with a magnet based on general elements of terminal devices, does not need to additionally configure other special devices, can be applied to most terminal devices, has lower power consumption, is simple to operate, and has good user experience.

[0065] Figure 2 is a system architecture diagram to which the method for establishing a multi-screen cooperative connection provided by an embodiment of the present application is applied. As shown in Figure 2 , the system comprises a first terminal device (referred to as a first device) and at least one second terminal device (referred to as a second device).

[0066] It should be noted that in this embodiment, the first device and each second device have a communication function. For example, the first device and the second device are both connected to a WLAN, and both have Bluetooth turned on. The WLAN to which the first device and the second device are connected can be the same network or different networks. The Bluetooth can be classic Bluetooth (BT) or Bluetooth low energy (BLE).

[0067] In this application, the terminal device can be a mobile phone, a tablet computer (Pad), a computer with wireless transceiver function, a smart television, a wearable device (such as a smart watch), a vehicle-mounted device, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), and the like. The specific type of the terminal device is not limited in the embodiments of this application.

[0068] Figure 3 is a structural schematic diagram of a terminal device provided by an embodiment of this application. The terminal device can include a processor 310, an external memory interface 320, an internal memory 321, a universal serial bus (USB) interface 330, a charge management module 340, a power management module 341, a battery 342, an antenna 1, an antenna 2, a mobile communication module 350, a wireless communication module 360, an audio module 370, a loudspeaker 370A, a receiver 370B, a microphone 370C, a headset interface 370D, a sensor module 380, a key 390, a motor 391, an indicator 392, a camera 393, a display screen 394, and a subscriber identification module (SIM) card interface 395, and the like.

[0069] It can be understood that the structure illustrated in the embodiments of this application does not constitute a specific limitation on the terminal device. In other embodiments of this application, the terminal device can include more or fewer components than the illustration, or combine certain components, or split certain components, or different component arrangements. The illustrated components can be implemented in hardware, software, or a combination of software and hardware.

[0070] As an example, when the terminal device is a mobile phone or a tablet computer, it can include all the components illustrated, or only include part of the components illustrated.

[0071] The processor 310 can include one or more processing units. Different processing units can be independent devices or integrated in one or more processors. The processor 310 can also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 310 is a cache memory. The memory can store instructions or data that the processor 310 has just used or recycled. If the processor 310 needs to use the instructions or data again, it can be directly called from the memory. This avoids repeated access and reduces the waiting time of the processor 310, thereby improving the efficiency of the system.

[0072] The external memory interface 320 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the terminal device. The external memory card communicates with the processor 310 through the external memory interface 320 to realize data storage functions. For example, files such as music and video are saved in the external memory card.

[0073] The internal memory 321 can be used to store computer executable program codes, which include instructions. The processor 310 executes various functional applications and data processing of the terminal device by running the instructions stored in the internal memory 321. The internal memory 321 can include a program storage area and a data storage area. The program storage area can store an operating system and at least one application program required by a function (such as a sound playing function, an image playing function, etc.). The data storage area can store data created during the use of the terminal device (such as audio data, a phonebook, etc.).

[0074] In addition, the internal memory 321 can include a high-speed random access memory and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, a universal flash memory (UFS), etc.

[0075] The USB interface 330 is an interface conforming to the USB standard specification, and can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 330 can be used to connect a charger to charge the terminal device, and can also be used to transmit data between the terminal device and a peripheral device. It can also be used to connect a headset to play audio through the headset. The interface can also be used to connect other terminal devices, such as AR devices, etc.

[0076] The charging management module 340 is configured to receive charging input from a charger. The power management module 341 is configured to connect the battery 342 to the charging management module 340 and the processor 310. The power management module 341 receives input from the battery 342 and / or the charging management module 340 to power the processor 310, the internal memory 321, the external memory, the camera 393, the display 394, the wireless communication module 360, and the like.

[0077] The wireless communication function of the terminal device can be implemented by the antenna 1, the antenna 2, the mobile communication module 350, the wireless communication module 360, the modem processor, and the baseband processor, and the like.

[0078] The antenna 1 and the antenna 2 are configured to transmit and receive electromagnetic wave signals. Each antenna in the terminal device can be used to cover a single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization of the antennas. For example, the antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in combination with a tuning switch.

[0079] The mobile communication module 350 can provide a solution for wireless communication including 2G / 3G / 4G / 5G and the like applied to the terminal device. The mobile communication module 350 can include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), and the like. The mobile communication module 350 can receive electromagnetic waves from the antenna 1, filter, amplify, and the like the received electromagnetic waves, and transmit the processed electromagnetic waves to the modem processor for demodulation. The mobile communication module 350 can also amplify the signals modulated by the modem processor and convert the signals into electromagnetic waves radiated by the antenna 1.

[0080] In some embodiments, at least part of the functional modules of the mobile communication module 350 can be arranged in the processor 310. In some embodiments, at least part of the functional modules of the mobile communication module 350 and at least part of the modules of the processor 310 can be arranged in the same device.

[0081] The modem processor can include a modulator and a demodulator. The modulator is configured to modulate a low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is configured to demodulate a received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. The low-frequency baseband signal processed by the baseband processor is transmitted to the application processor. The application processor outputs a sound signal through an audio device (not limited to the speaker 370A, the receiver 370B, etc.), or displays an image or video through the display screen 394. In some embodiments, the modem processor can be a separate device. In other embodiments, the modem processor can be independent of the processor 310 and be disposed in the same device as the mobile communication module 350 or other functional modules.

[0082] The wireless communication module 360 can provide a wireless communication solution including wireless local area networks (WLAN) (such as a wireless fidelity (Wi-Fi) network), Bluetooth, a global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), Ultra-Wideband (UWB) technology, etc. The wireless communication module 360 can be one or more devices that integrate at least one communication processing module. The wireless communication module 360 receives electromagnetic waves via the antenna 2, performs frequency modulation and filtering processing on the electromagnetic wave signal, and transmits the processed signal to the processor 310. The wireless communication module 360 can also receive a signal to be transmitted from the processor 310, perform frequency modulation, amplification, and convert it into electromagnetic wave radiation via the antenna 2.

[0083] In some embodiments, the antenna 1 of the terminal device is coupled with the mobile communication module 350, and the antenna 2 is coupled with the wireless communication module 360, so that the terminal device can communicate with the network and other devices through wireless communication technology.

[0084] The terminal device can realize audio functions through the audio module 370, the speaker 370A, the receiver 370B, the microphone 370C, the earphone interface 370D, and the application processor, etc.

[0085] The audio module 370 is configured to convert a digital audio signal into an analog audio signal for output, and to convert an analog audio input into a digital audio signal. The audio module 370 can also be configured to encode and decode audio signals. In some embodiments, the audio module 370 can be disposed in the processor 310, or some of the functional modules of the audio module 370 can be disposed in the processor 310.

[0086] The speaker 370A, also referred to as a "loudspeaker", is configured to convert an audio electrical signal into a sound signal. The terminal device can listen to music or listen to a hands-free call through the speaker 370A. In addition, the speaker 370A can also be configured to emit an ultrasonic signal of a preset frequency (such as f1) under the control of the processor 310.

[0087] The receiver 370B, also referred to as a "earpiece", is configured to convert an audio electrical signal into a sound signal. When the terminal device answers a call or a voice message, the receiver 370B can be held close to the ear of a person to listen to the voice. In addition, the receiver 370B can also be configured to emit an ultrasonic signal of a preset frequency (such as f2) under the control of the processor 310.

[0088] The microphone 370C, also referred to as a "microphone", "sound receiver", is configured to convert a sound signal into an electrical signal. When making a call or sending a voice message, a user can speak into the microphone 370C by holding the mouth close to the microphone 370C, and input the sound signal into the microphone 370C. The terminal device can be provided with at least one microphone 370C. In some embodiments, the microphone 370C can also be configured to receive an ultrasonic signal, for example, to receive ultrasonic signals of different frequencies emitted by the speaker and the receiver of another terminal device.

[0089] The earphone interface 370D is configured to connect a wired earphone. The earphone interface 370D can be a USB interface 330, or a 3.5mm open mobile terminal platform (OMTP) standard interface, a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0090] The keys 390 include a power-on key, a volume key, and the like. The keys 390 can be mechanical keys. Alternatively, the keys 390 can be touch keys. The terminal device can receive a key input, and generate a key signal input related to user settings and function control of the terminal device.

[0091] The motor 391 can generate a vibration prompt. The motor 391 can be used for incoming call vibration prompt, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, playing audio, etc.) can correspond to different vibration feedback effects. The motor 391 can also correspond to different vibration feedback effects for touch operations acting on different regions of the display screen 394. Different application scenarios (such as time reminders, received messages, alarms, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also be customized.

[0092] The indicator 392 can be an indicator light, which can be used to indicate the charging state, the power change, and can also be used to indicate messages, missed calls, notifications, etc.

[0093] The terminal device realizes display function through GPU, the display screen 394 and the application processor, etc. The GPU is a microprocessor for image processing, connected with the display screen 394 and the application processor. The GPU is used to execute mathematical and geometric calculations, and is used for graphics rendering. The processor 310 can include one or more GPUs, which execute program instructions to generate or change display information.

[0094] The display screen 394 is used to display images, videos, etc. The display screen 394 includes a display panel. In some embodiments, the terminal device can include 1 or N display screens 394, N being a positive integer greater than 1.

[0095] The SIM card interface 395 is used to connect the SIM card. The SIM card can be inserted into or pulled out of the SIM card interface 395 to realize contact and separation with the terminal device. The terminal device can support 1 or N SIM card interfaces, N being a positive integer greater than 1.

[0096] The terminal device shown in the above embodiments can be a first device or a second device.

[0097] When the terminal device is the first device, the first device further includes a magnetic sensor. The magnetic sensor is used to detect the magnetic field intensity of the position where the first device is located. The magnetic field intensity is a vector, which is used to describe the strength and direction of the magnetic field, and its unit is Gauss (G).

[0098] In some implementations, the magnetic sensor can be a magnetic field sensor, a Hall sensor, or a nine-axis inertial measurement unit (IMU). The nine-axis IMU includes a three-axis accelerometer, a three-axis gyroscope, and a three-axis magnetometer (M-Sensor). The three-axis accelerometer senses the acceleration information of the carrier in the three degrees of freedom: roll, pitch, and yaw. The three-axis gyroscope senses the attitude information of the carrier in the three degrees of freedom: roll, pitch, and yaw. The three-axis magnetometer detects the magnetic field strength at the carrier's location; this magnetic field strength can be the vector sum of the magnetic field strengths detected by the magnetometer in each direction, or it can be the magnetic field strength measured by the magnetometer in the three degrees of freedom: roll, pitch, and yaw.

[0099] Typically, mobile phones are equipped with a nine-axis IMU; therefore, a mobile phone can be used as the first device in this embodiment. In other implementations, a tablet computer or laptop computer equipped with a nine-axis IMU can also be used as the first device in this embodiment.

[0100] When the terminal device is a second device, the second device also includes a magnet for generating a magnetic field.

[0101] In some implementations, the magnet can be a magnetic strip or a magnetic element.

[0102] For example, see Figure 4 As shown, terminal devices (such as mobile phones, tablets, and laptops) are all equipped with speakers. Since the sound-producing principle of a speaker is related to its magnet, all speakers contain a magnet. Therefore, any terminal device containing a speaker can be used as the second device in this embodiment.

[0103] Additionally, see Figure 5 As shown, some terminal devices (such as laptops and tablets) typically use magnetic attachment to connect peripheral devices to the terminal device. For example, a tablet uses a strong magnetic strip on its side to attach a stylus to the tablet. Since the strong magnetic strip can also generate a magnetic field, a terminal device with a strong magnetic strip can also serve as a second device in this embodiment.

[0104] In some implementations, when the second device does not contain a magnet, a magnetic strip can be attached to the device, and the device with the attached magnetic strip can be used as the second device.

[0105] In some other implementations, when the second device comprises an IMU, the second device can determine the device posture according to the IMU. In some embodiments, the IMU can be a nine-axis IMU (see the foregoing for details), or an IMU without a magnetometer, such as a six-axis IMU or a three-axis IMU.

[0106] In the system provided by the embodiments of the present application, the first device can take the change of the magnetic field strength at the location where the first device is located as a trigger condition for establishing the multi-screen cooperative connection, and after the magnetic field strength at the location where the first device is located meets a preset condition, determine a target device from at least one second device around the first device, and establish a connection with the target device.

[0107] Based on the system and the terminal device for establishing the multi-screen cooperative connection provided by the embodiments of the present application, the method for establishing the multi-screen cooperative connection provided by the embodiments of the present application is explained in detail as follows.

[0108] Figure 6 is a schematic flowchart of the method for establishing the multi-screen cooperative connection provided by an embodiment of the present application. As shown in Figure 6 The method can comprise, but is not limited to, the following steps S601-S603.

[0109] S601, the first device detects the magnetic field strength at the location where the first device is located through a magnetic sensor.

[0110] The magnetic field strength can be a vector sum of the magnetic field strength in each direction at the location where the first device is located detected by the magnetic sensor, or can be the magnetic field strength in a preset direction at the location where the first device is located, which can be set according to actual detection needs, and is not limited in the embodiments.

[0111] In some embodiments, the first device detects the magnetic field strength at the location where the first device is located in real time based on the magnetic sensor. In this way, the problem that the magnetic field strength at the location where the first device is located has changed at a certain moment but the magnetic sensor fails to detect can be effectively prevented, and the accuracy of the magnetic field strength identification is improved.

[0112] In some other embodiments, the first device detects the magnetic field strength at the location where the first device is located based on the magnetic sensor when the first device is in a specific state, such as a bright screen state (including a screen-locked bright screen state and a screen-unlocked bright screen state). In this way, the power consumption of the device can be effectively reduced, and the service life of the device is improved.

[0113] S602, the first device determines a target device from at least one second device around the first device when the magnetic field strength meets a preset condition.

[0114] In some embodiments, the preset condition is that the magnetic field strength is greater than a first threshold. The first threshold can be between 40 and 70 Gauss, for example, 50 Gauss, 60 Gauss, etc. For example, when the first device detects a magnetic field strength of 70 Gauss through the magnetic sensor, it is determined that the magnetic field strength meets the preset condition because it is greater than the first threshold.

[0115] In some other embodiments, the preset condition includes that a change rate of the magnetic field strength within a preset time is greater than a second threshold. It can be understood that when the change rate is greater than the second threshold, it indicates that the first device and the second device are rapidly approaching each other within a short time (e.g., 0.5 seconds). For example, the second threshold can be between 50 and 100 Gauss per second, for example, 120 Gauss per second, 130 Gauss per second, etc.

[0116] When the magnetic field strength meets the preset condition, the first device starts the proximity detection function to determine a target device closest to the first device and / or matching the posture from the surrounding second devices. The proximity detection can be ultrasonic proximity detection or UWB-based proximity detection. The specific proximity detection and determination process will be explained in detail in the following embodiments, and this part will not be repeated.

[0117] S603, the first device and the target device establish a multi-screen cooperative connection.

[0118] In some embodiments, the first device and the target device establish a communication connection according to the respective identification information, where the identification information can be a medium access control (MAC) address or a device name of the first device or the target device. After the first device and the target device establish the communication connection, a multi-screen cooperative relationship can be further established.

[0119] The method provided by the embodiments of the present application can use the general elements (such as the magnetic sensor and the magnet) built-in the terminal device, and use the change of the magnetic field strength when the devices are close to each other as the trigger condition for establishing the multi-screen cooperative connection between the devices, to realize the natural interactive connection between the first device and the target device. The method does not need to additionally set special devices, and the connection process is simple to operate, which can improve the user experience.

[0120] In the method for establishing the multi-device cooperative connection provided by the embodiments of the present application, the target device can be determined from the surrounding second devices based on various different ways, for example, the first device determines the target device according to the received probe signal, or the first device determines the target device according to the distance information in the feedback signal, etc. The method provided by the embodiments of the present application will be exemplarily explained in combination with different determination ways of the target device.

[0121] Example 1: The first device determines the target device according to the received probe signal.

[0122] Figure 7 is a schematic diagram of an interaction process of a method for establishing multi-screen cooperative connection provided by an embodiment of the present application, referring to Figure 7 , the method comprises the following steps S701-S707.

[0123] S701, the first device detects the magnetic field intensity at the location of the first device through a magnetic sensor.

[0124] Exemplarily, when the first device is a mobile phone, the mobile phone can detect the magnetic field intensity at the location of the mobile phone based on the built-in nine-axis IMU. Generally, the detection value of the magnetic field intensity detected by the nine-axis IMU in the mobile phone is the vector sum of the magnetic field intensity in each direction at the location of the mobile phone, therefore, the mobile phone can directly determine the detection value as the magnetic field intensity at the location of the mobile phone. When the mobile phone takes the magnetic field intensity in a first direction as the magnetic field intensity at the location of the mobile phone, the first direction can be any one of the roll, pitch and yaw degrees of freedom of the magnetometer, and the mobile phone can take the magnetic field intensity in the detected first direction of the magnetometer as the magnetic field intensity at the location of the mobile phone.

[0125] S702, the first device broadcasts a first indication message when the magnetic field intensity meets a preset condition.

[0126] In some embodiments, the first device can broadcast the first indication message when the magnetic field intensity is greater than a first threshold. The first indication message is used to instruct the surrounding second device to emit a probe signal. The probe signal can be an ultrasonic signal or a UWB pulse signal.

[0127] The first device can broadcast the first indication message in the form of broadcasting to the surrounding devices through Bluetooth, or broadcasting to the devices in the local area network through WiFi, which is not limited in the embodiment.

[0128] S703, the second device emits a probe signal after receiving the first indication message.

[0129] In some embodiments, the second device can emit an ultrasonic signal based on the built-in loudspeaker or microphone, or emit a UWB pulse signal through the built-in UWB wireless communication module.

[0130] In the embodiment, the probe signal emitted by the second device carries the unique identification information of the second device, which is used to uniquely indicate the corresponding second device. The identification information comprises the MAC address and / or the device name of the second device.

[0131] In some implementations, after receiving the first instruction message, the second device transmits a detection signal at preset intervals to prevent the first device from failing to receive the detection signal for some reason after the second device transmits the detection signal.

[0132] S704, the first device determines the target device based on the received detection signal.

[0133] In this embodiment, when the detection signal is an ultrasonic signal, the first device can receive the detection signal through its built-in microphone. When the detection signal is a UWB pulse signal, the first device can receive the detection signal through its built-in UWB wireless communication module.

[0134] In some embodiments, when determining a target device, the first device can determine the target device based on the strength of the detection signals received from each of the second devices. For example, the first device can determine the second device corresponding to the detection signal with the strongest signal strength that is greater than a strength threshold as the target device. The strength threshold can be in the range of 40 to 60 dB, for example, 50 dB.

[0135] It is understood that detection signals typically experience loss during propagation, and the longer the propagation distance, the greater the loss. Consequently, the strength of the detection signal received by the first device will be weaker. Therefore, the second device corresponding to the strongest detection signal received by the first device, which is greater than the strength threshold, is the closest to the first device and is the target device in this application. When the first device determines the target device based on the strength of the detection signal, the first indication message is also used to further instruct surrounding second devices to emit detection signals of a preset strength. In other words, the detection signals emitted by the second devices need to maintain a consistent strength so that the first device can determine the target device based on the strength of the received detection signal. For example, the preset strength can be 60dB.

[0136] For example, see Figure 8 As shown in the example, taking a mobile phone as the first device and three second devices (laptops) around it, during the process of the mobile phone identifying the target device, the mobile phone detects a detection signal strength of 60dB from laptop A, 20dB from laptop B, and 10dB from laptop C. Since the detection signal strength of laptop A is the highest and exceeds the strength threshold of 50dB, the mobile phone identifies laptop A as the target device.

[0137] In other embodiments, the first device determines the distance between itself and each of the second devices based on the detection signals received from each of the second devices, and then identifies the second device with the smallest distance that is less than a distance threshold as the target device. The distance threshold can be in the range of 1cm to 2cm, for example, it can be 1cm.

[0138] It should be noted that, in this embodiment, the first indication signal is further used to instruct surrounding second devices to transmit detection signals carrying timestamp information, including the transmission time T1 of the detection signal. After receiving the detection signals transmitted by each second device, the first device can determine the distance between itself and each second device based on the transmission time T1 and the reception time T2 of each signal.

[0139] For example, in combination Figure 8 As shown, in the process of the mobile phone determining the target device based on the distance to each laptop, the mobile phone determines the distance between itself and laptop A to be 0.5cm based on the detection signal of laptop A, 10cm based on the detection signal of laptop B, and 15cm based on the detection signal of laptop C. Since the distance between the mobile phone and laptop A is 0.5cm, which is the minimum distance and less than the distance threshold of 1cm, the mobile phone identifies laptop A as the target device.

[0140] S705, the first device broadcasts a notification message, which is used to indicate that the corresponding second device becomes the target device.

[0141] In this embodiment, the first device can broadcast a notification message to surrounding devices via Bluetooth, or to other devices within the local area network via WiFi. This notification message can carry the identification information of the target device. Upon receiving this notification message, the second device, identified as the target device, can then confirm that it is the target device identified by the first device.

[0142] Optionally, other second devices can also determine, based on the notification message, that they are not the target device identified by the first device.

[0143] For example, after the first device determines that the second device 1 is the target device, the first device broadcasts a notification message instructing the second device 1 to become the target device. Based on this, after receiving the notification message, the second device 1 determines that it is the target device, stops transmitting detection signals, and waits to establish a connection with the first device. After receiving the notification message, the second devices 2, 3, etc., determine that they are not the target devices and stop transmitting detection signals.

[0144] S706, the first device determines whether the posture of the first device matches the posture of the target device. If yes, the next step S707 is performed.

[0145] In some embodiments, as shown in Figure 9 S705 specifically includes S706a-S706d.

[0146] It should be noted that in the present embodiment, when the first device determines whether the posture of the first device matches the posture of the target device, the first device and the target device need to be built-in with sensor devices for determining the posture information of each device, such as IMU. The IMU of the first device can be a nine-axis IMU, the three-axis magnetometer in the nine-axis IMU is used to detect the magnetic field strength of the position where the first device is located, and the three-axis gyroscope and the three-axis accelerometer are used to determine the posture information of the first device; the IMU built-in the second device can be an IMU without three-axis magnetometer, such as a three-axis IMU (including three-axis gyroscope) or a six-axis IMU (including three-axis gyroscope and three-axis accelerometer), or a nine-axis IMU.

[0147] S706a, the first device determines the first posture information of the first device.

[0148] In the present embodiment, the first device can determine the first posture information of the first device according to the built-in nine-axis IMU, and the first posture information is used to represent the posture of the first device, such as horizontal screen, vertical screen, etc.

[0149] In some embodiments, as shown in Figure 10 the posture information of the device includes the angle information of the pitch angle a, the roll angle b and the yaw angle g of the terminal device.

[0150] S706b, the first device broadcasts a second indication message, and the second indication message is used to instruct the target device to send the second posture information of the target device.

[0151] S706c, the target device sends the second posture information.

[0152] For example, after receiving the second indication message, the target device determines the second posture information of the target device through the built-in sensor device, and then sends the second posture information in the form of broadcast.

[0153] S706d, the first device receives the second posture information.

[0154] Optionally, the first device can also not perform S706b-S706d, and further instruct the second device to send its attitude information using the first indication message, so as to obtain the attitude information of all the second devices, and screen the second attitude information of the target device from the attitude information of all the second devices. When the first device further instructs the second device to send its attitude information using the first indication message, the first device directly performs the following step S706e after determining the first attitude information of the first device.

[0155] S706e, the first device determines whether the attitude of the first device and the target device matches according to the first attitude information and the second attitude information.

[0156] In the embodiment, the attitude matching of the first device and the target device includes that the first plane and the second plane are coincident, parallel or perpendicular to each other. Exemplarily, the first plane is the plane where the display screen of the first device is located, and the second plane is the plane where the display screen of the target device is located.

[0157] In some embodiments, when the absolute value of the corresponding angle difference in the first attitude information and the second attitude information is in a preset range, it is determined that the attitude of the first device and the target device matches. The preset range includes a first preset range and a second preset range. The first preset range can be [0°, 5°] or [0°, 10°], and the second preset range can be [85°, 95°] or [90°, 95°].

[0158] When the absolute value of the corresponding angle difference in the first attitude information and the second attitude information is in the first preset range, for example, [0°, 5°], it means that the first plane and the second plane are coincident or parallel.

[0159] When the absolute value of the corresponding angle difference in the first attitude information and the second attitude information is in the first preset range, for example, [85°, 95°], it means that the first plane and the second plane are perpendicular to each other.

[0160] It should be understood that in the embodiment, the first device and the target device are in the same Euler coordinate system. The process of the first device determining whether the attitude of the first device and the target device matches is explained below with a specific example.

[0161] Exemplarily, in combination with the above-mentioned Figure 8 In the embodiment, the first device is a mobile phone, and the target device is a notebook computer A. The notebook computer A includes a B face (the screen area of the notebook computer) and a C face (the keyboard area of the notebook computer). Assuming that the IMU is arranged on the B face of the notebook computer A, the plane where the mobile phone is located is the first plane, and the plane where the B face of the notebook computer A is located is the second plane, the second attitude information determined by the notebook computer A through the IMU can be represented as (α A , β A , γ A). The first attitude information determined by the mobile phone through the nine-axis IMU can be represented as (α, β, γ). The mobile phone compares the second attitude information (α A , β A , γ A ) with the first attitude information (α, β, γ) of itself, and determines the absolute value of the corresponding angle difference.

[0162] θ α = |α-α A |, θ β = |β-β A |

[0163] Wherein, θ α represents the absolute value of the difference between the pitch angles of the mobile phone and the notebook computer A, and θ β represents the absolute value of the difference between the roll angles of the mobile phone and the notebook computer A.

[0164] When θ α ∈ [0°, 5°] and θ β ∈ [0°, 5°], it is determined that the attitude of the mobile phone and the notebook computer A matches, and the plane where the mobile phone is located is parallel or coincides with the B surface of the notebook computer A. When θ β ∈ [85°, 95°], it is determined that the attitude of the mobile phone and the notebook computer A matches, and the plane where the mobile phone is located is perpendicular to the B surface of the notebook computer A.

[0165] It should be noted that S706 is an optional step. That is, after the first device determines the target device, it can also not determine whether its attitude matches the target device, but directly execute the following step S707.

[0166] S707, the first device and the target device establish a multi-screen cooperative connection.

[0167] In some embodiments, the first device and the target device can establish a communication connection according to the respective identification information (such as MAC address), and establish a multi-screen cooperative relationship. It can be understood that after the first device determines the target device, the first device broadcasts a first indication signal, and the target device also determines itself as the target device according to the first indication signal, so that the establishment of the multi-screen cooperative connection can be initiated by the first device or by the target device.

[0168] In the method provided in the embodiment, based on the magnetic sensor built-in in the first device and the magnet included in the second device, after the magnetic field intensity at the location of the first device meets the preset condition, the target device is determined according to the detection signal, and a multi-screen cooperative connection is established with the target device. This method has the advantages of low power consumption and strong applicability on the basis of being able to realize natural interactive connection.

[0169] In addition, by the method for establishing the multi-screen cooperative connection provided in this embodiment, in the embodiment in which the device posture is taken as a trigger to establish the multi-screen cooperative connection, when there are multiple second devices (such as a mobile phone, a tablet computer, etc.), the first device only selects the second devices that coincide with, are parallel to, or are perpendicular to the plane where the first device is located to establish the multi-screen cooperative connection, thereby improving the accuracy of the process of establishing the multi-screen cooperative connection and avoiding false triggering.

[0170] Example 2: The first device determines the target device according to the received feedback signal.

[0171] Figure 11 is an interaction flow diagram of the method for establishing the multi-screen cooperative connection provided in another embodiment of the present application. Referring to Figure 11 The method includes the following steps S1101-S1106.

[0172] S1101, the first device detects the magnetic field strength of the location where the first device is located through a magnetic sensor.

[0173] In this embodiment, the specific content of S1101 can be referred to the foregoing S701, and will not be described here again.

[0174] S1102, the first device broadcasts a third indication message and sends a probe signal when the magnetic field strength meets a preset condition.

[0175] In this embodiment, the first device broadcasts the third indication message when the magnetic field strength is greater than a first threshold. The third indication message is used to instruct the surrounding second devices to start proximity detection. The proximity detection can be that each second device determines the distance from the first device according to the received probe signal. The probe signal includes an ultrasonic signal or a UWB pulse signal.

[0176] In some embodiments, the first device can emit the ultrasonic signal based on a built-in loudspeaker or a microphone, or emit the UWB pulse signal through a built-in UWB wireless communication module.

[0177] S1103, the second device determines the distance from the first device according to the probe signal after receiving the second indication message.

[0178] The second device can receive the ultrasonic signal through a built-in microphone, or receive the UWB pulse signal through a built-in UWB wireless communication module.

[0179] After the first device broadcasts the third indication message and emits the probe signal, the second device that receives the second indication message determines the distance from the first device according to the received probe signal.

[0180] For example, in combination with Figure 8As shown in FIG. 6, after the mobile phone broadcasts the third indication message and transmits the probe signal, the notebook computer A, the notebook computer B and the notebook computer C all determine the distance between the mobile phone and each of them according to the probe signal. The notebook computer 1 determines the first distance between the mobile phone and itself according to the probe signal, the notebook computer 2 determines the second distance between the mobile phone and itself according to the probe signal, and the notebook computer 3 determines the third distance between the mobile phone and itself according to the probe signal.

[0181] S1104, the second device broadcasts a feedback signal.

[0182] The second device receiving the indication message broadcasts a feedback signal to the first device, and the feedback signal comprises distance information between each of the second devices and the first device.

[0183] S1105, the first device determines the target device according to the feedback signal.

[0184] In some embodiments, after the first device receives the feedback signal, the first device determines the second device corresponding to the signal with the shortest distance and less than the distance threshold in the feedback signal as the target device according to the distance information in the feedback signal.

[0185] S1106, the first device broadcasts a notification message, and the notification message is used to indicate that the corresponding second device becomes the target device.

[0186] S1107, the first device determines whether the posture of the first device matches the posture of the target device. If yes, the next step S1108 is performed.

[0187] In the embodiment, the specific content of S1107 can be referred to the foregoing S706, and details are not described herein.

[0188] Similarly, in the embodiment, when the first device determines whether the posture of the first device matches the posture of the target device, the first device and the target device need to be internally provided with a sensor device for determining the posture information of each of them.

[0189] It should be noted that, in the embodiment, S1107 is also an optional step. That is, the first device can determine the target device, and then directly perform the following step S1108 without determining whether the posture of the first device matches the posture of the target device.

[0190] S1108, the first device and the target device establish a multi-screen cooperation connection.

[0191] In the embodiment, the specific content of S1108 can be referred to the foregoing S707, and details are not described herein.

[0192] It should be noted that in the above various embodiments, the various multi-screen collaboration functions triggered after the first device and the target device establish the multi-screen collaboration connection can be set according to actual needs, for example, can be written according to the specific operations that can be achieved between different devices (such as a mobile phone and a notebook computer, a mobile phone and a tablet computer, a tablet computer and a notebook computer, etc.), and the present embodiment does not make specific limitations.

[0193] In some embodiments, the multi-screen collaboration function includes multi-device interaction functions such as screen mirroring, screen extension, and resource sharing.

[0194] Screen mirroring refers to that after the first device and the target device establish the multi-screen collaboration connection, the first device sends the first screen picture of the first device to the target device in the form of a video stream in real time, and the target device displays the first screen picture in the form of a floating window in real time. For details, please refer to FIG. 1. Figure 12A Based on the screen mirroring technology, the target device interface can be operated and used with the first device. For example, after a notebook computer and a mobile phone establish the multi-screen collaboration connection, the user can use the mouse of the notebook computer to click the pictures and music in the mobile phone, and can more accurately and quickly locate; the user can also use the computer input method to input text in the chat window of the mobile phone, thereby significantly improving the typing speed and efficiency.

[0195] Screen extension refers to that after the first device and the target device establish the multi-screen collaboration connection, the target device can be used as an external screen of the first device. For details, please refer to FIG. 2. Figure 12B Through the screen extension mode, the first device can move the target content (such as a file, a shortcut icon, a picture, a video, an application icon, a floating window, etc.) on the first device to the target device for display according to user operation. The user can operate the target content on the target device, for example, open the target content, delete the target content, move the target content, modify the target content, etc., and the present embodiment does not make specific limitations.

[0196] Resource sharing refers to that after the first device and the target device establish the multi-screen collaboration connection, the resource information in the first device can be shared into the target device according to user operation. For details, please refer to FIG. 3. Figure 12C The resource information includes but is not limited to a document, a media file (for example, an image, an audio, a video), a message (for example, a message in an instant messaging application, a short message, a multimedia message), an application installation package, contact information, a web link, web content, or other data that can be accessed by a terminal device, etc., and the present embodiment does not make specific limitations.

[0197] As can be seen, after the first device establishes a connection with the target device, the triggered function includes one or more. When the triggered function includes one, it can be directly triggered, such as directly mirroring the content displayed in the mobile phone to the notebook computer. When the triggered function includes more, the selection control of each multi-screen collaboration function can be displayed on the device, and then the corresponding multi-screen collaboration function is started according to the user operation.

[0198] For example, as shown in Figure 13 For example, as shown in

[0199] In summary, the method for establishing a multi-screen collaboration connection provided in this embodiment is to start high-precision proximity detection of ultrasonic signals or UWB pulse signals after detecting that the magnetic field strength meets the preset condition. Compared with the process of determining the target device by using high-precision proximity detection methods such as ultrasonic signals or UWB pulse signals in the traditional always-on manner, the method in this embodiment can accurately determine the target device, reduce false triggering, and save device power consumption.

[0200] The above is the process of establishing a multi-screen collaboration connection between different terminal devices through different ways provided in this embodiment. In addition, this embodiment also provides a process of triggering a corresponding multi-screen collaboration function according to a device posture after a first device establishes a multi-screen collaboration connection with a target device.

[0201] Figure 14 is an interactive flowchart of a method for triggering a multi-screen collaboration function according to a device posture provided in an embodiment of the present application, and relates to a process of triggering a corresponding multi-screen collaboration function according to a device posture after a first device establishes a connection with a target device. It should be noted that, since the embodiment also involves specific steps of performing corresponding operations according to a device posture, the first device and the target device in this embodiment also need to be built-in with sensor devices for determining their own posture information. Referring to Figure 14 The method specifically includes the following steps S1401-S1404.

[0202] S1401, the first device determines first posture information of the first device.

[0203] In this embodiment, the specific content of S1401 is described in the foregoing S706a, which will not be repeated here.

[0204] S1402, the first device sends a posture information request to the target device.

[0205] The posture information request is used for the target device to send second posture information of the target device to the first device according to the posture information request, and the second posture information is used for indicating the posture of the target device.

[0206] S1403, after the target device receives the posture information request, the target device sends the second posture information of the target device to the first device.

[0207] After the first device and the target device establish a connection, the first device can send the posture information request to the target device through the established connection channel, and the target device sends the second posture information of the target device to the first device through the established connection channel after receiving the posture information request.

[0208] S1404, the first device receives the second posture information.

[0209] S1405, the first device determines whether the posture of the first device and the target device matches according to the first posture information and the second posture information. If matched, the next step S1406 is executed.

[0210] In the embodiment, the process that the first device determines whether the posture of the target device matches is described in the foregoing step S706, and this part will not be described here.

[0211] S1406, the first device triggers a corresponding multi-screen collaboration function when the posture of the first device and the target device matches.

[0212] In the embodiment, the first device and the target device match in different forms, and different multi-screen collaboration functions are triggered.

[0213] It should be noted that in the embodiment, the functions triggered when the posture of the first device and the target device matches include but are not limited to the screen mirroring, screen extension, resource sharing and the like shown in the foregoing embodiment, and also include other functions triggered according to the posture of the device.

[0214] In some embodiments, the first plane of the first device is parallel to or coincides with the second plane of the target device, and the screen mirroring function is triggered. For example, the first device is a mobile phone, and the target device is a notebook computer. If the first screen picture currently displayed on the screen of the mobile phone is a video picture, after the mobile phone and the notebook computer establish a multi-screen collaboration connection, the video picture of the mobile phone is streamed to the display interface of the notebook computer in real time, and the user can continue to watch the video content by using the screen of the notebook computer.

[0215] In some other embodiments, when the first plane of the first device is parallel to or coincides with the second plane of the target device, the triggered multi-screen cooperation function includes other functions, such as the first device using accessories (such as a camera, a speaker, a microphone, a mouse, a keyboard, and the like) of the target device. Taking the case where the first device is a mobile phone and the target device is a notebook computer as an example, if the first screen currently displayed on the screen of the mobile phone is video chat content, after the mobile phone and the notebook computer establish a multi-screen cooperation connection, the camera of the notebook computer is called to continue the current video chat task of the mobile phone, and the user can continue the video chat on the notebook computer. For details, refer to the description in Figure 15

[0216] Since the mobile phone and the notebook computer are in the same or similar plane during the process of implementing the camera calling, based on the similar camera view angles of the two devices, and in cooperation with a graphics algorithm, the video stream can be switched without stuttering.

[0217] Compared with the case where the user uses the video chat function of the mobile phone in the prior art, when the video source needs to be switched from the camera of the mobile phone to the camera of the notebook computer, since the shooting angles of the cameras of the mobile phone and the notebook computer are different, the video stream switching will cause the jump of the different view angles of the cameras of the mobile phone and the notebook computer, thereby causing the stuttering of the video stream and affecting the user experience. The process of triggering the multi-screen cooperation connection provided in this embodiment can realize the non-stuttering switching of the video stream, thereby improving the user experience.

[0218] In some other implementations of this embodiment, after the mobile phone and the notebook computer establish a multi-screen cooperation connection and trigger the multi-screen cooperation function of the first device using the accessories of the target device, the mobile phone determines whether the posture of the mobile phone and the posture of the notebook computer are matched in real time according to the first posture information of the mobile phone and the second posture information of the notebook computer. For details, refer to the description in Figure 16 When the inclination angle of the B-face screen of the notebook computer exceeds a certain angle, the posture of the notebook computer is changed, and the screen camera is no longer suitable as the video source for video chat. Therefore, the mobile phone can trigger the corresponding operation according to the changed posture information. For example, when the absolute value of the corresponding angle difference between the plane where the mobile phone is located and the plane where the B-face of the notebook computer is located is in a third preset range, the mobile phone stops using the screen camera of the notebook computer and triggers the multi-screen cooperation function of switching the camera of the notebook computer used for video chat to the front camera of the mobile phone. Exemplarily, the third preset range can be [65°, 75°], for example, 70°.

[0219] ​In some other embodiments, when the first plane of the first device and the second plane of the target device are perpendicular to each other, the triggered multi-screen cooperation function includes other functions, such as the first device acting as an external keyboard or an external controller of the target device. For example, when the first device is a mobile phone and the target device is a tablet computer, if the mobile phone determines, according to the posture information of the mobile phone and the posture information of the tablet computer, that the plane where the mobile phone is located and the plane where the tablet computer is located are parallel to each other, the triggered multi-screen cooperation function includes the mobile phone acting as an external keyboard of the tablet computer, or the tablet computer acting as an external keyboard of the mobile phone, as shown in Figure 17 ; or the mobile phone acting as an external controller of the tablet computer, for controlling the tablet computer.

[0220] In this embodiment, the first device and the target device do not trigger the multi-screen cooperation function immediately after establishing the connection, but trigger the corresponding multi-screen cooperation function in each posture after determining the posture between the first device and the target device. In this embodiment, different functions are triggered according to the different relative postures between the devices, and more abundant and natural interaction modes are realized by using the semantics contained in the device posture information.

[0221] It should be understood that the size of the serial number of each step in the above embodiments does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0222] The embodiments of the present application also provide a terminal device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor is configured to implement the method for establishing a multi-screen cooperation connection as shown in each of the above embodiments when executing the computer program.

[0223] The embodiments of the present application also provide a chip, as shown in Figure 18 , which includes a processor and a memory, and the memory stores a computer program, and the computer program is executed by the processor to implement the method for establishing a multi-screen cooperation connection in each of the above embodiments.

[0224] The embodiments of the present application also provide a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the method for establishing a multi-screen cooperation connection provided in each of the above embodiments.

[0225] The embodiments of the present application also provide a computer program product, which includes a computer program, and when the computer program is run by a terminal device, the terminal device implements the method for establishing a multi-screen cooperation connection provided in each of the above embodiments.

[0226] It should be appreciated that the processor referred to in the embodiments of the present application can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic components, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0227] It should also be understood that the memory referred to in the embodiments of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically EPROM (EEPROM) or a flash memory. The volatile memory can be a random access memory (RAM) used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced SDRAM (ESDRAM), synchlink DRAM (SLDRAM) and direct rambus RAM (DR RAM).

[0228] In the embodiments provided in the present application, the division of each framework or module is only a logical functional division, and when actually implemented, there can be another division manner, for example, a plurality of frameworks or modules can be combined or integrated into another system, or some features can be ignored or not executed.

[0229] In addition, each functional module in each embodiment of the present application can be integrated in one processing module, or each module can be physically present alone, or two or more modules can be integrated in one module. The integrated module can be realized in the form of hardware or in the form of a software functional module.

[0230] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described system, device and unit can refer to the corresponding processes in the foregoing method embodiments, which will not be described here.

[0231] In the present application, the reference to "one embodiment" or "some embodiments" means that the specific features, structures or characteristics described in connection with the embodiment are included in one or more embodiments of the present application. Therefore, the statements "in one embodiment", "in some embodiments", "in other some embodiments", "in further some embodiments" and the like appearing in different places in the specification are not necessarily all referring to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized. The terms "include", "contain", "have" and their variants mean "include but are not limited to", unless otherwise specifically emphasized.

[0232] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A method for establishing a connection, characterized in that, Applied to a first device, the first device being equipped with a magnetic sensor, the method includes: The magnetic sensor is used to detect the magnetic field strength at the location of the first device in a first manner. When the magnetic field strength meets the preset conditions, a first indication message is broadcast. The first indication message is used to instruct the surrounding second devices to emit detection signals in a second manner. Receive the detection signal sent by the second device in the second manner, and determine the target device based on the detection signal; A broadcast notification message is used to indicate that the corresponding second device becomes the target device; Establish a connection with the target device; The first method differs from the second method.

2. The method according to claim 1, characterized in that, The power consumption of running the first method is lower than that of running the second method.

3. The method according to claim 1 or 2, characterized in that, The first indication message is further used to instruct the second device to send a detection signal of a preset strength, wherein determining the target device based on the received detection signal includes: Determine the signal strength of each of the received detection signals; The second device corresponding to the detection signal with the largest signal strength that is greater than the strength threshold is identified as the target device.

4. The method according to claim 1 or 2, characterized in that, The first indication message is further used to instruct the second device to send a probe signal carrying the transmission time, wherein determining the target device based on the received probe signal includes: Determine the reception time of each of the aforementioned detection signals; The distance between the first device and each of the second devices is determined based on the transmission time and reception time of each detection signal.

5. A method for establishing a connection, characterized in that, Applied to a first device, the first device being equipped with a magnetic sensor, the method includes: The magnetic field strength at the location of the first device is detected by the magnetic sensor; When the magnetic field strength meets the preset conditions, a third indication message and a detection signal are broadcast. The third indication message is used to instruct the second device to activate proximity detection. The proximity detection includes the second device determining the distance between itself and the first device based on the detection signal after receiving it. Receive feedback signals sent by each of the second devices, wherein the feedback signals are broadcast by the second devices after they enable proximity detection, and the feedback signals include the distance between the second devices and the first device; The second device is determined to be the target device based on the feedback signal; A broadcast notification message is used to indicate that the corresponding second device has become the target device; Establish a connection with the target device.

6. The method according to claim 5, characterized in that, The step of determining the second device as the target device based on the feedback signal includes: The feedback signal is used to identify the second device corresponding to the feedback signal with the shortest distance and less than the distance threshold as the target device.

7. The method according to any one of claims 1, 2, 5, and 6, characterized in that, Establishing a connection with the target device includes: The first device receives attitude information sent by the target device, determines that the attitudes of the first device and the target device are matched, and establishes a connection with the target device.

8. The method according to any one of claims 1, 2, 5, and 6, characterized in that, The preset conditions include: The magnetic field strength is greater than a first threshold; or, The rate of change of the magnetic field strength within a preset time is greater than a second threshold. The second device with the smallest distance and less than the distance threshold is identified as the target device.

9. The method according to any one of claims 1, 2, 5, and 6, characterized in that, Following the broadcast notification message, the method further includes: Determine the first attitude information of the first device; Broadcast a second instruction message, which instructs the target device to send second attitude information; Receive the second attitude information; Based on the first attitude information and the second attitude information, the attitude matching between the first device and the target device is determined. The attitude matching includes: the first plane where the first device is located coincides with, is parallel to, or is perpendicular to the second plane where the second device is located.

10. The method according to claim 9, characterized in that, Determining the attitude matching between the first device and the target device based on the first attitude information and the second attitude information includes: When the absolute value of the angle difference between the corresponding angles of the first device and the target device is within a first preset range, it is determined that the first plane where the first device is located coincides with or is parallel to the second plane where the second device is located; and / or, When the absolute value of the difference between the corresponding angles of the first device and the target device is within a second preset range, it is determined that the first plane where the first device is located is perpendicular to the second plane where the second device is located.

11. The method according to claim 9, characterized in that, Establishing a connection with the target device includes: When the posture of the first device matches that of the target device, a multi-screen collaborative connection is established with the target device.

12. The method according to any one of claims 1, 2, 5, and 6, characterized in that, After establishing a connection with the target device, the method further includes: Determine the first attitude information of the first device; Send an attitude information request to the target device, the attitude information request being used to request the target device to send second attitude information; Receive the second attitude information; Based on the first attitude information and the second attitude information, determine the attitude matching status between the first device and the target device; When the postures of the first device and the target device are matched, the corresponding multi-screen collaboration function is triggered.

13. The method according to claim 12, characterized in that, When the postures of the first device and the target device are matched, the corresponding multi-screen collaboration function is triggered, including: When the first plane where the first device is located coincides with or is parallel to the second plane where the second device is located, the target device uses the accessories of the first device.

14. The method according to claim 13, characterized in that, The method further includes: When the absolute value of the difference between the corresponding angles of the first device and the target device is within a third preset range, the target device stops using the accessories of the first device.

15. The method according to claim 12, characterized in that, When the postures of the first device and the target device are matched, the corresponding multi-screen collaboration function is triggered, including: When the first plane where the first device is located is perpendicular to the second plane where the second device is located, the first device serves as an external keyboard or external controller for the target device.

16. A system for establishing a connection, characterized in that, The device includes a first device and at least one second device, the first device being provided with a magnetic sensor and configured to perform the method for establishing a connection as described in any one of claims 1 to 15, and the second device being provided with a magnet capable of influencing the magnetic field strength around the second device.

17. A terminal device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the method for establishing a connection as described in any one of claims 1 to 15.

18. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method for establishing a connection as described in any one of claims 1 to 15.

19. A chip, characterized in that, The chip includes a processor and a memory, the memory storing a computer program that, when executed by the processor, implements the method for establishing a connection as described in any one of claims 1 to 15.

Images