Device interaction method, device interaction system, electronic device, and storage medium

Abstract

The present application provides a device interaction method, a device interaction system, an electronic device, and a computer-readable storage medium, which are applied to a first terminal and include the steps of: when a first operation is identified, broadcasting a first message based on the first operation, the first message including first close-range connection information corresponding to the first operation; obtaining a second message broadcast by a second terminal based on the recognized second operation, the second message including second close-range connection information corresponding to the second operation; and when it is determined that the first close-range connection information and the second close-range connection information match, controlling the first terminal and the second terminal to establish a first connection.

Description

[Technical Field]

[0001] This application claims priority from Chinese Patent Application No. 202211717081.8, filed on December 29, 2022, the contents of which are incorporated herein by reference.

[0002] The embodiments of the present application relate to the field of communication technology, and in particular to a device interaction method, a device interaction system, an electronic device, and a computer-readable storage medium. [Background technology]

[0003] Smart homes and smart offices are user experiences that mobile phone manufacturers have been working hard to build over the past two years. They use short-range communication to realize multi-screen collaboration between multiple smart devices, such as screen projection, cross-screen editing, input feedback, application streaming or relay, cross-device document mutual transfer, cross-device clipboard, cross-device audio streaming, cross-device photography, cross-device document collaboration, and cross-screen display. This creates extremely valuable and attractive scenarios for users, bringing a new collaborative experience to smart devices.

[0004] The realization of these functional experiences relies on short-range communication between devices to enable quick and easy mutual discovery between devices. Currently, methods for realizing mutual discovery between devices are not user-friendly and have a high barrier to use for users. Summary of the Invention

[0005] In a first aspect, the present embodiment comprises: Apply to the first terminal, When a first operation is recognized, broadcasting a first message according to the first operation, the first message including first short-range connection information corresponding to the first operation; obtaining a second message broadcast by the second terminal according to the recognized second operation, the second message including second short-range connection information corresponding to the second operation; When it is determined that the first short-distance connection information and the second short-distance connection information match, controlling the first terminal and the second terminal to perform a first connection; The present invention provides a device interaction method including:

[0006] In a second aspect, the present embodiment comprises: at least one processor; a memory having stored therein at least one computer program, the at least one computer program, when executed by the at least one processor, causing the at least one processor to implement the device interaction method; provides electronic equipment including

[0007] In a third aspect, the present embodiment comprises A computer program is stored in the device, and when the computer program is executed by a processor, the processor is caused to perform the device interaction method. A computer-readable storage medium is provided.

[0008] In a fourth aspect, the present application provides a first terminal configured to, when a first operation is recognized, broadcast a first message according to the first operation, the first message including first short-range connection information corresponding to the first operation; obtain a second message broadcast by a second terminal according to a recognized second operation, the second message including second short-range connection information corresponding to the second operation; and, when it is determined that the first short-range connection information and the second short-range connection information match, control the first terminal and the second terminal to perform a first connection; a second terminal configured to, when a second operation is recognized, broadcast a second message according to the second operation, the second message including second short-range connection information corresponding to the second operation; obtain a first message broadcast by the first terminal according to the recognized first operation, the first message including first short-range connection information corresponding to the first operation; and, when it is determined that the first short-range connection information and the second short-range connection information match, control the second terminal and the first terminal to perform a second connection; The present invention provides a device interaction system including: [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a flowchart of a device interaction method applied to a first terminal provided by an embodiment of the present application. [Figure 2] FIG. 2 is a flowchart of a device interaction method applied to a second terminal provided by an embodiment of the present application. [Figure 3] FIG. 3 is a flowchart of a device interaction method provided by an example of an embodiment of the present application. [Figure 4] FIG. 4 is a flowchart of a device interaction method provided by an example of an embodiment of the present application. [Figure 5] FIG. 5 is a block diagram of an electronic device according to an embodiment of the present invention. [Figure 6] FIG. 6 is a block diagram of a device interaction system according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0010] In order to help those skilled in the art better understand the technical solution of the present application, the device interaction method and system, electronic equipment, and computer-readable storage medium provided in the present application are described in detail below in combination with the drawings.

[0011]

[0023] Exemplary embodiments are described more fully below with reference to the drawings, but the exemplary embodiments may be embodied in different forms and the present disclosure should not be construed as being limited to the embodiments set forth herein. The purpose of providing these examples is to make the present application clear and complete, and to fully convey the scope of the present application to those skilled in the art.

[0012] Unless contradictory, the embodiments and features of the embodiments in this application may be combined with each other in any combination.

[0013] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0014] The terms used herein are for the purpose of describing particular embodiments only and are not intended to limit the scope of the present application. As used herein, the singular forms "a," "an," and "the" are intended to include the plural unless the context clearly dictates otherwise. Additionally, the use of the terms "comprising" and / or "consisting of" indicates that a particular feature, whole, step, operation, element, and / or component is present, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, elements, components, and / or groups thereof.

[0015] Unless otherwise specified, the meaning of all terms (including technical and scientific terms) used herein is the same as that commonly understood by those skilled in the art. For example, terms defined in common dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the relevant art and this application, and it is understood that they should not be interpreted as having an idealized or overly formal meaning unless the present specification clearly limits them.

[0016] FIG. 1 is a flowchart of a device interaction method applied to a first terminal provided by an embodiment of the present application.

[0017] In a first aspect, referring to Figure 1, an embodiment of the present application provides a device interaction method applied to a first terminal, which may be any of devices such as a smart screen, a laptop, a desktop computer, a PAD, a mobile phone, a router, a smart speaker, a cloud terminal, a smart refrigerator, a storage device, smart glasses, a virtual reality (VR) device, a smart mirror, a smart wearable device, a smart cockpit, an aerial photography drone, a robot dog, and a smart small appliance.

[0018] The device interaction method includes steps 100 to 102.

[0019] Step 100: If a first operation is recognized, broadcast a first message based on the first operation, where the first message includes first short-range connection information corresponding to the first operation.

[0020] The embodiment of the present application is not limited to the first operation, and the first operation may be, for example, a first gesture, a video playback, an audio playback, or the like.

[0021] In the present embodiment, the first gesture is not limited as long as it is sufficiently distinguishable from other gestures. For example, the first gesture may be at least one of an alphabet gesture, a numeric gesture, a single symbol gesture, etc.

[0022] In some embodiments, the alphabet gestures may be the alphabets a-z, or at least one in the alphabets A-Z.

[0023] In some embodiments, the number gesture may be at least one of the numbers 0-9.

[0024] In some embodiments, the single symbol gesture may be at least one of a fork symbol gesture, a triangle symbol gesture, a pentagram symbol gesture, a circle symbol gesture, a square symbol gesture, a diamond gesture, a gesture of tapping a preset number of times in succession, a gesture of sliding multiple fingers a preset distance, etc.

[0025] In some embodiments, the first gesture may be identified by identifying the trajectory of the user's slide on a touchscreen, or may be identified from an image captured by a camera using an artificial intelligence (AI) algorithm.

[0026] In some embodiments, if a first operation is identified and the first correspondence relationship between the pre-established operation information corresponding to the first operation, the operation information corresponding to the second operation and the operation intention information includes operation information corresponding to the identified first operation, continue to perform the subsequent step, i.e., continue to perform the step of broadcasting a first message based on the first operation.

[0027] In some embodiments, the operation information corresponding to the first operation may be a preset name or encoding, command, number, code, pattern, etc. corresponding to the first operation.

[0028] In some embodiments, the operation information corresponding to the second operation may be a preset name or encoding, command, number, code, pattern, etc. corresponding to the second operation.

[0029] In some embodiments, the manipulation encoding may be a pointer array trajectory encoding corresponding to the manipulation, i.e., a sequence encoding of points sequentially passed through by a pattern drawn by the user in the pointer array.

[0030] In some embodiments, the operation encoding may be a pre-defined encoding for distinguishing different operations, ensuring that the operation encodings corresponding to different operations are different and the operation encodings corresponding to the same operation are the same.

[0031] In some embodiments, the control intention information may be a control intention encoding, for example, 0100 representing cross-screen projection.

[0032] In some embodiments, the operation intention information refers to information about the operation / function to be performed.

[0033] In some embodiments, a user may be guided through a wizard to set the first correspondence.

[0034] In some embodiments, if the first operation is recognized and the screen is currently in a sleep state, the first message is broadcast based on the first operation after the screen is unlocked.

[0035] In some embodiments, the screen may be unlocked directly, or for added security, the screen may be unlocked based on fingerprint, password, or facial recognition.

[0036] In some embodiments, if a first action is recognized and the screen is currently in a bright state, broadcast a first message based on the first action.

[0037] In some embodiments, the first message further includes first device information of the first terminal.

[0038] In some embodiments, the first device information includes at least one of first identification information of the first terminal and first hardware information of the first terminal.

[0039] In some embodiments, the first device information further includes at least one of first manufacturer information of the first terminal, Wireless Fidelity (Wifi) 5G bands supported by the first terminal, a Bluetooth Media Access Control (MAC) address of the first terminal, a WIFI address, information characterizing a Bluetooth Low Energy (BLE) version of the first terminal, information characterizing a multi-screen collaboration function supported by the first terminal, and version information characterizing a multi-screen collaboration function supported by the first terminal.

[0040] In some embodiments, the first identification information includes at least one of information characterizing the multi-screen collaboration function supported by the first terminal, and version information characterizing the multi-screen collaboration function supported by the first terminal.

[0041] In some embodiments, the first hardware information includes at least one of device type information of the first terminal, size information and resolution information of a display screen of the first terminal, and resolution information of an image captured by a camera of the first terminal.

[0042] In some embodiments, the first message further includes a first identification of the first terminal.

[0043] In some embodiments, the first message further includes a message length of the first message.

[0044] In some embodiments, the first short-range connection information includes at least one of operation information corresponding to the first operation, operation intention information corresponding to the first operation, first time information at which the first operation is identified, first user account information of the first terminal, first connection information of the first terminal, and a first connection validity period of the first connection information.

[0045] In some embodiments, the first connection information is generated after a first connection between the first terminal and the second terminal is successful, or after the first terminal and the second terminal are successfully connected for the first time after the previously generated first connection information has expired.

[0046] In some embodiments, the first connection information may include a communication address of the first terminal and a communication address of the second terminal.

[0047] In some embodiments, the first connection information may include a communication address of the first terminal, a communication address of the second terminal, and a first connection validity period.

[0048] In some embodiments, the communication address may be a Bluetooth Media Access Control (MAC) address.

[0049] In some embodiments, if the first connection validity period is currently in effect, the first short-range connection information includes the first connection information, and if the first connection validity period is not currently in effect, the first short-range connection information does not include the first connection information.

[0050] In some embodiments, the first short-range connection information includes the first connection information regardless of whether the first connection validity period is currently in effect, or the first short-range connection information includes both the first connection information and the first connection validity period.

[0051] In some embodiments, the first connection information can be configured to automatically expire when the first terminal logs out of the user account.

[0052] In some embodiments, broadcasting a first message based on a first operation includes obtaining first close-range connection information corresponding to the first operation, and broadcasting the first message.

[0053] In some embodiments, the first short-range connection information includes operation information corresponding to a first operation, and the step of obtaining the first short-range connection information corresponding to the first operation includes a step of searching for operation information corresponding to the recognized slide trajectory of the user from a second correspondence relationship between the slide trajectory of the user and the operation information.

[0054] In some embodiments, the path of the user's slide may be the path of the user sliding on a touchscreen, or the path of the user sliding in front of a camera.

[0055] In some embodiments, the first message carries operation information corresponding to the first operation, or the first message carries encoded and compressed operation information corresponding to the first operation.

[0056] In some embodiments, if the length of the operation information corresponding to the first operation is less than or equal to the maximum length of the field containing the operation information in the first message, the operation information corresponding to the first operation is attached to the first message, and if the length of the operation information corresponding to the first operation is greater than the maximum length of the field containing the operation information in the first message, the encoded and compressed operation information corresponding to the first operation is attached to the first message.

[0057] In some embodiments, the first short-range connection information includes operation intention information corresponding to a first operation, and the step of acquiring the first short-range connection information corresponding to the first operation includes a step of searching for operation intention information corresponding to the operation information corresponding to the first operation from a first correspondence relationship between operation information corresponding to the first operation, which is pre-established, and operation information and operation intention information corresponding to the second operation.

[0058] In some embodiments, the first message includes operation intention information corresponding to the first operation, or includes encoded and compressed operation intention information corresponding to the first operation.

[0059] In some embodiments, if the length of the operation intention information corresponding to the first operation is equal to or less than the maximum length of the field containing the operation intention information in the first message, the operation intention information corresponding to the first operation is attached to the first message, and if the length of the operation intention information corresponding to the first operation is greater than the maximum length of the field containing the operation intention information in the first message, the encoded and compressed operation intention information corresponding to the first operation is attached to the first message.

[0060] In some embodiments, the first message may be broadcast based on any of a mechanism or protocol such as a Bluetooth Low Energy (BLE) broadcast mechanism, an ultrasonic communication broadcast mechanism, a short-range optical communication broadcast mechanism, an infrared communication broadcast mechanism, or the Wi-Fi Constrained Application Protocol (CoAP).

[0061] In some embodiments, the first message may be broadcast based on the broadcast mechanism of BLE 4.x. The maximum payload length of a broadcast message in BLE 4.x is 31 bytes. If the broadcast message cannot carry all the necessary information, a 31-byte message may be transmitted by scanning for a response using an active scan. That is, the first terminal first broadcasts the first message, and after receiving the first response message from the second terminal, broadcasts a third message, carrying at least one of first short-range connection information, first device information, and first identification information in the first and third messages.

[0062] In some embodiments, as shown in Table 1, the first message may include first identification information, first user account information, the communication address of the first terminal, etc., and as shown in Table 2, the third message may include operation information corresponding to the first operation, operation intention information corresponding to the first operation, first time information when the first operation was identified, etc.

[0063] JPEG2025531802000002.jpg45165

[0064] JPEG2025531802000003.jpg53165

[0065] In some embodiments, the first message may be broadcast based on the broadcast mechanism of the BLE 5.x version. The maximum payload length of a broadcast message in the BLE 5.x version is 255 bytes, and required information can be included in one broadcast message.

[0066] Step 101: Obtain a second message broadcast by a second terminal based on a recognized second operation, the second message including second short-distance connection information corresponding to the second operation.

[0067] The embodiment of the present application does not limit the second operation, and the second operation may be, for example, a second gesture, a video playback, an audio playback, or the like.

[0068] In the present embodiment, the second gesture is not limited as long as it can be sufficiently distinguished from other gestures. For example, the second gesture may be at least one of an alphabet gesture, a number gesture, a single symbol gesture, etc.

[0069] In some embodiments, the alphabet gesture may be at least one of alphabets a-z, alphabets A-Z.

[0070] In some embodiments, the number gesture may be at least one of the digital digits 0-9.

[0071] In some embodiments, the single symbol gesture may be at least one of a fork symbol gesture, a triangle symbol gesture, a pentagram symbol gesture, a circle symbol gesture, a square symbol gesture, a diamond gesture, a gesture of tapping a preset number of times in succession, a gesture of sliding multiple fingers a preset distance, etc.

[0072] In some embodiments, the first operation and the second operation may be the same operation.

[0073] In some embodiments, the first operation and the second operation may be different operations, for example, the first operation is a gesture of three fingers tracing a predetermined distance from left to right, and the second operation is a gesture of three fingers tracing a predetermined distance from right to left.

[0074] In some embodiments, the first and second actions may be different gestures, or the first and second actions may be parts of a complete alphabetic gesture, a numeric gesture, or a single symbol gesture. The single symbol gesture may be, for example, any one of a fork symbol gesture, a triangle symbol gesture, a pentagram symbol gesture, a circle symbol gesture, a square symbol gesture, etc. For example, the first action is a left semicircle symbol gesture and the second action is a right semicircle symbol gesture, and the first action and the second action together form a complete circle symbol gesture.

[0075] In some embodiments, the second message further includes second device information of the second terminal.

[0076] In some embodiments, the second device information includes at least one of second identification information of the second terminal and second hardware information of the second terminal.

[0077] In some embodiments, the second device information includes at least one of second manufacturer information of the second terminal, information characterizing whether the second terminal supports WIFI 5G, a Bluetooth MAC address of the second terminal, a WIFI MAC address, device type information, information characterizing the BLE version of the second terminal, information characterizing the multi-screen collaboration function supported by the second terminal, and version information characterizing the multi-screen collaboration function supported by the second terminal.

[0078] In some embodiments, the second identification information includes at least one of information characterizing the multi-screen collaboration function supported by the second terminal, and version information characterizing the multi-screen collaboration function supported by the second terminal.

[0079] In some embodiments, the second hardware information includes at least one of size information and resolution information of the display screen of the second terminal, resolution information of an image captured by a camera of the second terminal, and the like.

[0080] In some embodiments, the second message further includes a second identification of the second terminal.

[0081] In some embodiments, the second short-range connection information includes at least one of operation information corresponding to the second operation, operation intention information corresponding to the second operation, second time information at which the second operation is identified, second user account information of the second terminal, second connection information of the second terminal, and a second connection validity period of the second connection information.

[0082] In some embodiments, the second message carries operation information corresponding to the second operation, or carries encoded and compressed operation information corresponding to the second operation.

[0083] In some embodiments, if the length of the operation information corresponding to the second operation is less than or equal to the maximum length of the field containing the operation information in the second message, the second message includes the operation information corresponding to the second operation; if the length of the operation information corresponding to the second operation is greater than the maximum length of the field containing the operation information in the second message, the second message includes the encoded and compressed operation information corresponding to the second operation.

[0084] In some embodiments, the second message includes operation intention information corresponding to the second operation, or the second message includes compressed and encoded operation intention information corresponding to the second operation.

[0085] In some embodiments, if the length of the operation intention information corresponding to the second operation is equal to or less than the maximum length of the field containing the operation intention information in the second message, the second message accompanies the operation intention information corresponding to the second operation; if the length of the operation intention information corresponding to the second operation is greater than the maximum length of the field in the second message containing the intention information for the second operation, the second message accompanies the encoded and compressed operation intention information corresponding to the second operation.

[0086] In some embodiments, the second connection information is generated after a first connection between the first terminal and the second terminal is successful, or after the first terminal and the second terminal are successfully connected for the first time after the previously generated second connection information has expired.

[0087] In some embodiments, the second connection information may include a communication address of the first terminal and a communication address of the second terminal.

[0088] In some embodiments, the second connection information may include a communication address of the first terminal, a communication address of the second terminal, and a second connection validity period.

[0089] In some embodiments, the communication address may be a Bluetooth MAC address.

[0090] In some embodiments, if the second connection validity period is currently in effect, the second short-range connection information includes the second connection information, and if the second connection validity period is not currently in effect, the second short-range connection information does not include the second connection information.

[0091] In some embodiments, the second short-range connection information includes the second connection information regardless of whether the second connection validity period is currently in effect, or the second short-range connection information includes both the second connection information and the second connection validity period.

[0092] Step 102: If it is determined that the first short-distance connection information and the second short-distance connection information match, control the first connection between the first terminal and the second terminal.

[0093] In some embodiments, the first connection includes establishing a short-range wireless connection between the first terminal and the second terminal, disconnecting the short-range wireless connection between the first terminal and the second terminal, initiating a multi-screen collaborative operation to the second terminal, or performing a multi-screen collaborative operation initiated by the second terminal.

[0094] In some embodiments, the multi-screen collaborative operation includes at least one of a cross-screen projection operation, an input feedback operation, a cross-screen copy-paste operation, a cross-screen document collaborative operation, a cross-screen camera multi-position shooting operation, an application streaming relay operation, an audio playback streaming relay operation, a call streaming relay operation, and the like.

[0095] In some embodiments, if it is determined that the first identification information and the second identification information match, and it is determined that the first short-range connection information and the second short-range connection information match, the first terminal and the second terminal are controlled to establish a first connection.

[0096] In some embodiments, determining that the first identification information and the second identification information match comprises determining that the first identification information and the second identification information are the same.

[0097] In some embodiments, determining that the first short-range connection information and the second short-range connection information match includes determining that the first connection information and the second connection information match when at least one of the following conditions is met: operation information corresponding to the first operation and operation information corresponding to the second operation match; operation intention information corresponding to the first operation and operation intention information corresponding to the second operation are identical; and the first user account information and the second user account information match.

[0098] In some embodiments, determining that the first short-range connection information and the second short-range connection information match includes determining that the first short-range connection information and the second short-range connection information match if the first connection information and the second connection information are the same and are within the first connection validity period and the second connection validity period.

[0099] In some embodiments, a match between operation information corresponding to a first operation and operation information corresponding to a second operation includes the operation information corresponding to the first operation and the operation information corresponding to the second operation being the same, or the operation information corresponding to the first operation and the operation information corresponding to the second operation satisfying a first correspondence relationship.

[0100] In some embodiments, operation information corresponding to a first operation and operation information corresponding to a second operation satisfy a first correspondence relationship, which means that operation information corresponding to a second operation that corresponds to operation information corresponding to the first operation can be found from the first correspondence relationship.

[0101] In some embodiments, the operation information corresponding to the first operation and the operation information corresponding to the second operation satisfy a first correspondence relationship means that the operation information corresponding to the second operation that corresponds to the operation information corresponding to the first operation can be found from the first correspondence relationship, and in the first correspondence relationship, the operation intention information corresponding to the operation information corresponding to the first operation, the operation intention information corresponding to the operation information corresponding to the second operation, and the operation intention information corresponding to the first operation and the operation intention information corresponding to the second operation are all the same.

[0102] In some embodiments, matching the first user account information and the second user account information includes that the first user account information and the second user account information are identical, or that the first user account information and the second user account information satisfy a second correspondence relationship between the first user account information and the second user account information that has been previously established.

[0103] In some embodiments, the first user account information and the second user account information satisfy a third correspondence relationship refers to the fact that the second user account information corresponding to the first user account information can be found from the third correspondence relationship.

[0104] In some embodiments, if it is determined that the first short-range connection information and the second short-range connection information do not match, or the first identification information and the second identification information do not match, an inquiry message is displayed to inquire whether the user should continue the subsequent judgment flow. If a continuation command from the user is received, authentication is performed with the second terminal. If the authentication is successful, the subsequent judgment flow is continued. If no command is received or a stop command is received, the flow is terminated.

[0105] In some embodiments, determining that the first identification information and the second identification information do not match comprises determining that the first identification information and the second identification information are not identical.

[0106] In some embodiments, determining that the first short-range connection information and the second short-range connection information do not match includes determining that the first short-range connection information and the second short-range connection information do not match when at least one of the following conditions is met: operation information corresponding to the first operation and operation information corresponding to the second operation do not match, operation intention information corresponding to the first operation and operation intention information corresponding to the second operation are different, the first user account information and the second user account information do not match, and the first connection information and the second connection information are different.

[0107] In some embodiments, determining that the first short-range connection information and the second short-range connection information do not match includes determining that the first short-range connection information and the second short-range connection information do not match if the first connection information and the second connection information are different or the first connection validity period and the second connection validity period are not within each other.

[0108] In some embodiments, a mismatch between the operation information corresponding to the first operation and the operation information corresponding to the second operation includes a case where the operation information corresponding to the first operation and the operation information corresponding to the second operation are different, or a case where the operation information corresponding to the first operation and the operation information corresponding to the second operation do not satisfy a first correspondence relationship.

[0109] In some embodiments, the fact that the operation information corresponding to the first operation and the operation information corresponding to the second operation do not satisfy the first correspondence relationship means that it is not possible to find operation information corresponding to the second operation that corresponds to the operation information corresponding to the first operation from the first correspondence relationship.

[0110] In some embodiments, the fact that the operation information corresponding to the first operation and the operation information corresponding to the second operation do not satisfy the first correspondence relationship refers to the fact that, although operation information corresponding to the second operation that corresponds to the operation information corresponding to the first operation can be found from the first correspondence relationship, the operation intention information corresponding to the operation information corresponding to the first operation and the operation intention information corresponding to the operation information corresponding to the second operation in the first correspondence relationship are different from the operation intention information corresponding to the first operation or the operation intention information corresponding to the second operation.

[0111] In some embodiments, the mismatch between the first user account information and the second user account information includes the first user account information and the second user account information being different, or the first user account information and the second user account information not satisfying a third correspondence relationship between the first user account information and the second user account information that has been pre-established.

[0112] In some embodiments, the first user account information and the second user account information do not satisfy the third correspondence relationship refers to the fact that the second user account information corresponding to the first user account information cannot be found from the third correspondence relationship.

[0113] In some embodiments, when the first terminal is a master device, the step of authenticating with the second terminal includes the steps of generating an authentication code and displaying the generated authentication code; receiving the authentication code sent by the second terminal; and passing the authentication if the generated authentication code and the received authentication code are the same; and failing the authentication if the generated authentication code and the received authentication code are different.

[0114] In some embodiments, when the first terminal is a slave device, the step of performing authentication with the second terminal includes the steps of displaying an authentication code input interface, receiving the authentication code entered by a user, and transmitting the authentication code entered by the user to the second terminal to perform authentication.

[0115] In some embodiments, when the first terminal is a master device, the step of authenticating with the second terminal includes the steps of generating and displaying a connection two-dimensional code, receiving a connection authentication code obtained by the second terminal scanning the connection two-dimensional code, verifying the connection authentication code, and passing the authentication if the verification is passed, or not passing the authentication if the verification is not passed.

[0116] In some embodiments, when the first terminal is a slave device, the step of authenticating with the second terminal includes scanning a connection two-dimensional code displayed by the second terminal to obtain a connection authentication code, and sending the connection authentication code to the second terminal.

[0117] In order to more clearly explain how to determine whether a first terminal and a second terminal are matched, six examples are given below, and the listed examples are not intended to limit the scope of the claims of the embodiments of the present application.

[0118] Example 1 The first message includes operation information corresponding to the first operation and operation intention information corresponding to the first operation. The second message includes operation information corresponding to the second operation and operation intention information corresponding to the second operation. Whether the first short-range connection information and the second short-range connection information match is determined based on the operation information corresponding to the first operation, the operation intention information corresponding to the first operation, the operation information corresponding to the second operation, and the operation intention information corresponding to the second operation.

[0119] If the operation information corresponding to the first operation and the operation information corresponding to the second operation are identical, and the operation intention information corresponding to the first operation and the operation intention information corresponding to the second operation are identical, it is determined that the first short-range connection information and the second short-range connection information match. If the operation information corresponding to the first operation and the operation information corresponding to the second operation are different, or if the operation intention information corresponding to the first operation and the operation intention information corresponding to the second operation are different, it is determined that the first short-range connection information and the second short-range connection information do not match. Alternatively, if the operation information corresponding to the first operation and the operation information corresponding to the second operation satisfy a first correspondence relationship, and the operation intention information corresponding to the first operation and the operation intention information corresponding to the second operation are the same, it is determined that the first short-range connection information and the second short-range connection information match. If the operation information corresponding to the first operation and the operation information corresponding to the second operation do not satisfy the first correspondence relationship, or if the operation intention information corresponding to the first operation and the operation intention information corresponding to the second operation are different, it is determined that the first short-range connection information and the second short-range connection information do not match.

[0120] Example 2 The first message includes first identification information, operation information corresponding to the first operation, and operation intention information corresponding to the first operation. The second message includes second identification information, operation information corresponding to the second operation, and operation intention information corresponding to the second operation. Whether the first short-range connection information and the second short-range connection information match is determined based on the first identification information, the operation information corresponding to the first operation, the operation intention information corresponding to the first operation, the second identification information, the operation information corresponding to the second operation, and the operation intention information corresponding to the second operation.

[0121] The first short-distance connection information and the second short-distance connection information are determined to match if the first identification information and the second identification information are the same, the operation information corresponding to the first operation and the operation information corresponding to the second operation are the same, and the operation intention information corresponding to the first operation and the operation intention information corresponding to the second operation are the same. The first short-distance connection information and the second short-distance connection information are determined to not match if the first identification information and the second identification information are different, or the operation information corresponding to the first operation and the operation information corresponding to the second operation are different, or the operation intention information corresponding to the first operation and the operation intention information corresponding to the second operation are different. Alternatively, the first short-distance connection information and the second short-distance connection information are determined to match if the first identification information and the second identification information are the same, and the operation information corresponding to the first operation and the operation information corresponding to the second operation satisfy the first correspondence relationship and the operation intention information corresponding to the first operation and the operation intention information corresponding to the second operation are the same. If the first identification information and the second identification information are different, or the operation information corresponding to the first operation and the operation information corresponding to the second operation do not satisfy the first correspondence relationship, or the operation intention information corresponding to the first operation and the operation intention information corresponding to the second operation are different, it is determined that the first short-range connection information and the second short-range connection information do not match.

[0122] Example 3 The first message includes first identification information, first user account information, operation information corresponding to the first operation, and operation intention information corresponding to the first operation. The second message includes second identification information, second user account information, operation information corresponding to the second operation, and operation intention information corresponding to the second operation. It is determined whether the first identification information and the second identification information match, and it is determined whether the first short-range connection information and the second short-range connection information match based on the first user account information, the operation information corresponding to the first operation, the operation intention information corresponding to the first operation, the second user account information, the operation information corresponding to the second operation, and the operation intention information corresponding to the second operation.

[0123] The first short-range connection information and the second short-range connection information are determined to match when the first user account information and the second user account information are identical, the operation information corresponding to the first operation and the operation information corresponding to the second operation are identical, and the operation intention information corresponding to the first operation and the operation intention information corresponding to the second operation are identical. The first short-range connection information and the second short-range connection information are determined to not match when the first account information and the second account information are different, or the operation information corresponding to the first operation and the operation information corresponding to the second operation are different, or the operation intention information corresponding to the first operation and the operation intention information corresponding to the second operation are different. Alternatively, the first short-range connection information and the second short-range connection information are determined to match when the first user account information and the second user account information satisfy a third correspondence relationship, the operation information corresponding to the first operation and the operation information corresponding to the second operation are the same, and the operation intention information corresponding to the first operation and the operation intention information corresponding to the second operation are the same. If the first account information and the second account information do not satisfy the third correspondence relationship, or the operation information corresponding to the first operation and the operation information corresponding to the second operation are different, or the operation intention information corresponding to the first operation and the operation intention information corresponding to the second operation are different, it is determined that the first short-range connection information and the second short-range connection information do not match. Alternatively, if the first user account information and the second user account information are identical, and the operation information corresponding to the first operation and the operation information corresponding to the second operation satisfy the first correspondence relationship, and the operation intention information corresponding to the first operation and the operation intention information corresponding to the second operation are identical, it is determined that the first short-range connection information and the second short-range connection information do not match. If the first account information and the second account information are different, or the operation information corresponding to the first operation and the operation information corresponding to the second operation do not satisfy the first correspondence relationship, or the operation intention information corresponding to the first operation and the operation intention information corresponding to the second operation are different, it is determined that the first short-range connection information and the second short-range connection information do not match.Alternatively, if the first user account information and the second user account information satisfy the third correspondence relationship, and the operation information corresponding to the first operation and the operation information corresponding to the second operation satisfy the first correspondence relationship, and the operation intention information corresponding to the first operation and the operation intention information corresponding to the second operation are the same, it is determined that the first short-range connection information and the second short-range connection information match. If the first account information and the second account information do not satisfy the third correspondence relationship, or the operation information corresponding to the first operation and the operation information corresponding to the second operation do not satisfy the first correspondence relationship, or the operation intention information corresponding to the first operation and the operation intention information corresponding to the second operation are different, it is determined that the first short-range connection information and the second short-range connection information do not match.

[0124] Example 4 The first message includes first connection information, operation information corresponding to the first operation, and operation intention information corresponding to the first operation. The second message includes second connection information, operation information corresponding to the second operation, and operation intention information corresponding to the second operation. Whether the first short-range connection information and the second short-range connection information match is determined based on the first connection information, operation information corresponding to the first operation, operation intention information corresponding to the first operation, second connection information, operation information corresponding to the second operation, and operation intention information corresponding to the second operation.

[0125] The first short-distance connection information and the second short-distance connection information are determined to match when the first connection information and the second connection information are identical, and the operation information corresponding to the first operation and the operation information corresponding to the second operation are identical, and the operation intention information corresponding to the first operation and the operation intention information corresponding to the second operation are identical. The first short-distance connection information and the second short-distance connection information are determined to not match when the first connection information and the second connection information are different, or the operation information corresponding to the first operation and the operation information corresponding to the second operation are different, or the operation intention information corresponding to the first operation and the operation intention information corresponding to the second operation are different. Alternatively, the first short-distance connection information and the second short-distance connection information are determined to match when the first connection information and the second connection information are identical, and the operation information corresponding to the first operation and the operation information corresponding to the second operation satisfy the first correspondence relationship, and the operation intention information corresponding to the first operation and the operation intention information corresponding to the second operation are identical. If the first connection information and the second connection information are different, or the operation information corresponding to the first operation and the operation information corresponding to the second operation do not satisfy the first correspondence relationship, or the operation intention information corresponding to the first operation and the operation intention information corresponding to the second operation are different, it is determined that the first short-range connection information and the second short-range connection information do not match.

[0126] Example 5 The first message includes first connection information, a first connection validity period, operation information corresponding to the first operation, and operation intention information corresponding to the first operation. The second message includes second connection information, a second connection validity period, operation information corresponding to the second operation, and operation intention information corresponding to the second operation. Whether the first short-range connection information and the second short-range connection information match is determined based on the first connection information, the first connection validity period, operation information corresponding to the first operation, operation intention information corresponding to the first operation, second connection information, the second connection validity period, operation information corresponding to the second operation, and operation intention information corresponding to the second operation.

[0127] If the first connection information and the second connection information are the same, the current period is within the first connection validity period or the second connection validity period, the operation information corresponding to the first operation and the operation information corresponding to the second operation are the same, and the operation intention information corresponding to the first operation and the operation intention information corresponding to the second operation are the same, it is determined that the first short-range connection information and the second short-range connection information match. If the first connection information and the second connection information are different, or the current period is not within the first connection validity period or the second connection validity period, or the operation information corresponding to the first operation and the operation information corresponding to the second operation are different, or the operation intention information corresponding to the first operation and the operation intention information corresponding to the second operation are different, it is determined that the first short-range connection information and the second short-range connection information do not match. Alternatively, it is determined that the first short-range connection information and the second short-range connection information match if the first connection information and the second connection information are identical, the current period is within the first connection validity period or the second connection validity period, the operation information corresponding to the first operation and the operation information corresponding to the second operation satisfy the first correspondence relationship, and the operation intention information corresponding to the first operation and the operation intention information corresponding to the second operation are identical.It is determined that the first short-range connection information and the second short-range connection information do not match if the first connection information and the second connection information are different, or the current period is not within the first connection validity period or the second connection validity period, or the operation information corresponding to the first operation and the operation intention information corresponding to the second operation do not satisfy the first correspondence relationship, or the operation intention information corresponding to the first operation and the operation intention information corresponding to the second operation are different.

[0128] Example 6 The first message includes first identification information, first user account information, first connection information, operation information corresponding to the first operation, and operation intention information corresponding to the first operation. The second message includes second identification information, second user account information, second connection information, operation information corresponding to the second operation, and operation intention information corresponding to the second operation. Whether the first short-range connection information and the second short-range connection information match is determined based on the first connection information, the operation information corresponding to the first operation, the operation intention information corresponding to the first operation, the second connection information, the operation information corresponding to the second operation, and the operation intention information corresponding to the second operation.

[0129] The first short-distance connection information and the second short-distance connection information are determined to match when the first connection information and the second connection information are identical, and the operation information corresponding to the first operation and the operation information corresponding to the second operation are identical, and the operation intention information corresponding to the first operation and the operation intention information corresponding to the second operation are identical. The first short-distance connection information and the second short-distance connection information are determined to not match when the first connection information and the second connection information are different, or the operation information corresponding to the first operation and the operation information corresponding to the second operation are different, or the operation intention information corresponding to the first operation and the operation intention information corresponding to the second operation are different. Alternatively, the first short-distance connection information and the second short-distance connection information are determined to match when the first connection information and the second connection information are identical, and the operation information corresponding to the first operation and the operation information corresponding to the second operation satisfy the first correspondence relationship, and the operation intention information corresponding to the first operation and the operation intention information corresponding to the second operation are identical. If the first connection information and the second connection information are different, or the operation information corresponding to the first operation and the operation information corresponding to the second operation do not satisfy the first correspondence relationship, or the operation intention information corresponding to the first operation and the operation intention information corresponding to the second operation are different, it is determined that the first short-range connection information and the second short-range connection information do not match.

[0130] The judgment conditions in the above examples can be combined with each other.

[0131] In some embodiments, if it is still not determined that the first short-range connection information and the second short-range connection information match within a preset time period after the first operation is recognized, prompt information is displayed to notify the user that the connection has failed.

[0132] In some embodiments, the first terminal and the second terminal are two equal actors, and in such a case, if the first terminal and the second terminal are determined to be a match, the first terminal and the second terminal directly control the first connection to be made.

[0133] In some embodiments, the first terminal and the second terminal are not two equal execution entities, and in such a case, before performing an operation corresponding to the first operation, the device interaction method further includes a step of determining a master-slave relationship between the first terminal and the second terminal based on the first device information and the second device information, and the step of controlling the first terminal and the second terminal to make a first connection includes a step of controlling the first terminal and the second terminal to make the first connection based on the master-slave relationship between the first terminal and the second terminal.

[0134] In some embodiments, a master-slave relationship may also be referred to as a relationship between a service end and a client, a relationship between a source end and a sink end, or the like.

[0135] In some embodiments, determining a master-slave relationship between the first terminal and the second terminal based on the first device information and the second device information includes determining that the first terminal is a master device and the second terminal is a slave device if the corresponding hardware conditions of the first hardware information are higher than the corresponding hardware conditions of the second hardware information. For example, if the size of the display screen of the first terminal is larger than the size of the display screen of the second terminal, determining that the first terminal is a master device and the second terminal is a slave device.

[0136] In some embodiments, determining a master-slave relationship between the first terminal and the second terminal based on the first device information and the second device information includes determining that the second terminal is a master device and the first terminal is a slave device if the hardware conditions corresponding to the first hardware information are lower than the hardware conditions corresponding to the second hardware information. For example, if the size of the display screen of the first terminal is smaller than the size of the display screen of the second terminal, determining that the second terminal is a master device and the first terminal is a slave device.

[0137] In some embodiments, the step of determining a master-slave relationship between the first terminal and the second terminal based on the first device information and the second device information includes a step of determining a master-slave relationship between the first terminal and the second terminal based on the first time information and the second time information when the hardware conditions corresponding to the first hardware information and the hardware conditions corresponding to the second hardware information are the same.

[0138] In some embodiments, the step of determining a master-slave relationship between the first terminal and the second terminal based on the first time information and the second time information includes the step of determining that the first terminal is a master device and the second terminal is a slave device if the first time information is earlier than the second time information.

[0139] In some embodiments, the step of determining a master-slave relationship between the first terminal and the second terminal based on the first time information and the second time information includes the step of determining that the second terminal is a master device and the first terminal is a slave device if the first time information is later than the second time information.

[0140] In some embodiments, the hardware condition for a smart terminal with device type information of PAD is higher than the hardware condition for a smart terminal with device type information of mobile phone, and the hardware condition for a smart terminal with device type information of desktop computer, laptop computer, smart screen, etc. is higher than the hardware condition for a smart terminal with device type information of PAD.

[0141] In some embodiments, for two smart devices with the same device type information, the smart device with a larger display screen size will have higher hardware requirements.

[0142] In some embodiments, smart devices with larger display screens have higher hardware requirements.

[0143] In some embodiments, the cross-screen projection operation, the cross-screen copy-paste operation, and the cross-screen document collaboration operation need to determine the master-slave relationship based on the device type information and the display screen size information.

[0144] In some embodiments, the multi-position shooting operation of a cross-screen camera requires determining a master-slave relationship based on the resolution information of the images captured by the camera.

[0145] In some embodiments, the step of controlling the first terminal and the second terminal to establish a first connection based on a master-slave relationship between the first terminal and the second terminal includes the step of initiating a first connection from a smart terminal that is a slave device to a smart terminal that is a master device.

[0146] The device interaction method provided by the present application realizes controlling the first terminal and the second terminal to make a first connection based on the identified operation, that is, adopts an operation execution method that is closer to natural user authentication, which reduces the user's usage threshold and improves the user experience.

[0147] FIG. 2 is a flowchart of a device interaction method applied to a second terminal provided by an embodiment of the present application.

[0148] In a second aspect, referring to Figure 2, an embodiment of the present application provides a device interaction method applied to a second terminal, which may be any of devices such as a smart screen, a laptop, a desktop computer, a PAD, a mobile phone, a router, a smart speaker, a cloud terminal, a smart refrigerator, a storage device, smart glasses, a virtual reality (VR) device, a smart mirror, a smart wearable device, a smart cockpit, an aerial photography drone, a robot dog, and a smart small appliance.

[0149] The device interaction method includes steps 200-202.

[0150] Step 200: If a second operation is recognized, broadcast a second message based on the second operation, where the second message includes second short-range connection information corresponding to the second operation.

[0151] The embodiment of the present application does not limit the second operation, and the second operation may be, for example, any of a second gesture, video playback, audio playback, and the like.

[0152] In the present embodiment, the second gesture is not limited as long as it can be sufficiently distinguished from other gestures. For example, the second gesture may be at least one of an alphabet gesture, a number gesture, a single symbol gesture, etc.

[0153] In some embodiments, the alphabet gesture may be at least one of alphabets a-z, alphabets A-Z.

[0154] In some embodiments, the number gesture may be at least one of the digital digits 0-9.

[0155] In some embodiments, the single symbol gesture may be at least one of a fork symbol gesture, a triangle symbol gesture, a pentagram symbol gesture, a circle symbol gesture, a square symbol gesture, a diamond gesture, a gesture of tapping a preset number of times in succession, a gesture of sliding multiple fingers a preset distance, etc.

[0156] In some embodiments, the second gesture may be identified by identifying the trajectory of the user sliding across the touchscreen, or the second gesture may be identified from an image captured by a camera using an AI algorithm.

[0157] In some embodiments, if a second operation is identified and the first correspondence relationship between the pre-established operation information corresponding to the first operation, the operation information corresponding to the second operation, and the operation intention information includes the operation information corresponding to the identified second operation, continue to perform the subsequent step, i.e., continue to perform the step of broadcasting a second message based on the second operation.

[0158] In some embodiments, the operation information corresponding to the first operation may be a preset name or encoding, command, number, code, pattern, etc. corresponding to the first operation.

[0159] In some embodiments, the operation information corresponding to the second operation may be a preset name or encoding, command, number, code, pattern, etc. corresponding to the second operation.

[0160] In some embodiments, the manipulation encoding may be a pointer array trajectory encoding corresponding to the manipulation, i.e., a sequence encoding of points sequentially passed through by a pattern drawn by the user in the pointer array.

[0161] In some embodiments, the operation encoding may be an encoding for distinguishing different pre-defined operations, ensuring that the operation encodings corresponding to different operations are different and the operation encodings corresponding to the same operation are the same.

[0162] In some embodiments, the control intention information may be a control intention encoding, for example, 0100 representing cross-screen projection.

[0163] In some embodiments, the operation intention information is information about an operation to be performed.

[0164] In some embodiments, a user may be guided through a wizard to set the first correspondence.

[0165] In some embodiments, if the second operation is recognized and the screen is currently in a sleep state, the screen is unlocked and then a second message is broadcast based on the second operation.

[0166] In some embodiments, the screen can be locked directly and can also be unlocked based on fingerprint or password or face recognition for added security.

[0167] In some embodiments, if a second action is recognized and the screen is currently in a bright state, a second message is broadcast based on the second action.

[0168] In some embodiments, the second message further includes second device information of the second terminal.

[0169] In some embodiments, the second device information includes at least one of second identification information of the second terminal and second hardware information of the second terminal.

[0170] In some embodiments, the second device information further includes at least one of second manufacturer information of the second terminal, a WIFI 5G band supported by the second terminal, a Bluetooth MAC address of the second terminal, a WIFI address, information characterizing the BLE version of the second terminal, information characterizing the multi-screen collaborative function supported by the second terminal, version information characterizing the multi-screen collaborative function supported by the second terminal, and the like.

[0171] In some embodiments, the second identification information includes at least one of information characterizing the multi-screen collaboration function supported by the second terminal, and version information characterizing the multi-screen collaboration function supported by the second terminal.

[0172] In some embodiments, the second hardware information includes at least one of device type information of the second terminal, size information and resolution information of the display screen of the second terminal, and resolution information of images captured by the camera of the second terminal.

[0173] In some embodiments, the second message further includes a first identification information of the first terminal.

[0174] In some embodiments, the second message further includes a message length for the second message.

[0175] In some embodiments, the second short-range connection information includes at least one of operation information corresponding to the second operation, operation intention information corresponding to the second operation, second time information at which the second operation is identified, second user account information of the second terminal, second connection information of the second terminal, and a second connection validity period of the second connection information.

[0176] In some embodiments, the second connection information is generated after a first connection between the first terminal and the second terminal is successful, or after the first terminal and the second terminal are successfully connected for the first time after the previously generated first connection information has expired.

[0177] In some embodiments, the second connection information may include a communication address of the first terminal and a communication address of the second terminal.

[0178] In some embodiments, the second connection information may include a communication address of the first terminal, a communication address of the second terminal, and a second connection validity period.

[0179] In some embodiments, the communication address may be a Bluetooth MAC address.

[0180] In some embodiments, the second short-range connection information includes the second connection information if the current connection is within the second connection validity period, and does not include the second connection information if the current connection is not within the second connection validity period.

[0181] In some embodiments, the second short-range connection information includes the second connection information regardless of whether the second connection validity period is currently in effect, or the second short-range connection information both includes the second connection information and the second connection validity period.

[0182] In some embodiments, the second connection information can be configured to automatically expire when the second terminal logs out of the user account.

[0183] In some embodiments, broadcasting a second message based on a second operation includes obtaining second close range connection information corresponding to the second operation, and broadcasting the second message.

[0184] In some embodiments, the second short-range connection information includes operation information corresponding to a second operation, and obtaining the second short-range connection information corresponding to the second operation includes searching for operation information corresponding to the identified user slide trajectory from a second correspondence relationship between the user slide trajectory and the operation information.

[0185] In some embodiments, the path of the user's slide may be the path of the user sliding on a touchscreen or the path of the user sliding in front of a camera.

[0186] In some embodiments, the second message is accompanied by operation information corresponding to the second operation, or the second message is accompanied by encoded and compressed operation information corresponding to the second operation.

[0187] In some embodiments, if the length of the operation information corresponding to the second operation is less than or equal to the maximum length of the field containing the operation information in the second message, the second message includes the operation information corresponding to the second operation; if the length of the operation information corresponding to the second operation is greater than the maximum length of the field containing the operation information in the second message, the second message includes the encoded and compressed operation information corresponding to the second operation.

[0188] In some embodiments, the second short-range connection information includes operation intention information corresponding to the second operation, and the step of acquiring the second short-range connection information corresponding to the second operation includes a step of searching for operation intention information corresponding to the operation information corresponding to the second operation from the operation information corresponding to the first operation that has been pre-established and a first correspondence relationship between the operation information and operation intention information corresponding to the second operation.

[0189] In some embodiments, the second message includes operation intention information corresponding to the second operation, or the second message includes compressed and encoded operation intention information corresponding to the second operation.

[0190] In some embodiments, if the length of the operation intention information corresponding to the second operation is equal to or less than the maximum length of the field containing the operation intention information in the second message, the second message accompanies the operation intention information corresponding to the second operation, and if the length of the operation intention information corresponding to the second operation is greater than the maximum length of the field in the second message containing the second operation intention information, the second message accompanies the encoded and compressed operation intention information corresponding to the second operation.

[0191] In some embodiments, the second message may be broadcast based on any of the following mechanisms or protocols: a BLE broadcast mechanism, an ultrasonic communication broadcast mechanism, a short-range optical communication broadcast mechanism, an infrared communication broadcast mechanism, or WIFI CoAP.

[0192] In some embodiments, the second message may be broadcast based on the broadcast mechanism of BLE 4.x. The maximum payload length of a broadcast message in BLE 4.x is 31 bytes. If the broadcast message cannot carry all the necessary information, the second terminal may use active scanning to scan for a response and transmit a 31-byte message. That is, the second terminal may first broadcast the second message, and after receiving the second response message from the first terminal, broadcast a fourth message to carry at least one of second short-range connection information, second device information, and second identification information.

[0193] In some embodiments, as shown in Table 3, the second message may include second identification information, second user account information, the communication address of the second terminal, etc., and as shown in Table 4, the fourth message may include operation information corresponding to the second operation, operation intention information corresponding to the second operation, second time information when the second operation was identified, etc.

[0194] JPEG2025531802000004.jpg48165

[0195] JPEG2025531802000005.jpg55165

[0196] In some embodiments, the second message may be broadcast based on the broadcast mechanism of the BLE 5.x version. The maximum payload length of a broadcast message in the BLE 5.x version is 255 bytes, and required information can be included in one broadcast message.

[0197] Step 201: Obtain a first message broadcast by a first terminal based on an identified first operation, where the first message includes first short-range connection information corresponding to the first operation.

[0198] The embodiment of the present application is not limited to the first operation, and the first operation may be, for example, any of a first gesture, a video playback, an audio playback, and the like.

[0199] In the present embodiment, the first gesture is not limited as long as it is sufficiently distinguishable from other gestures. For example, the first gesture may be at least one of an alphabet gesture, a numeric gesture, a single symbol gesture, etc.

[0200] In some embodiments, the alphabet gesture may be at least one of alphabets a-z, alphabets A-Z.

[0201] In some embodiments, the number gesture may be at least one of the digital digits 0-9.

[0202] In some embodiments, the single symbol gesture may be at least one of a fork symbol gesture, a triangle symbol gesture, a pentagram symbol gesture, a circle symbol gesture, a square symbol gesture, a diamond gesture, a gesture of tapping a preset number of times in succession, a gesture of sliding multiple fingers a preset distance, etc.

[0203] In some embodiments, the second device information includes at least one of second identification information of the second terminal and second hardware information of the second terminal.

[0204] In some embodiments, the second device information includes at least one of second manufacturer information of the second terminal, information characterizing whether the second terminal supports WIFI 5G, a Bluetooth MAC address of the second terminal, a WIFI MAC address, device type information, information characterizing the BLE version of the second terminal, information characterizing the multi-screen collaboration function supported by the second terminal, and version information characterizing the multi-screen collaboration function supported by the second terminal.

[0205] In some embodiments, the second identification information includes at least one of information characterizing the multi-screen collaboration function supported by the second terminal, and version information characterizing the multi-screen collaboration function supported by the second terminal.

[0206] In some embodiments, the second hardware information includes at least one of size information and resolution information of the display screen of the second terminal, resolution information of an image captured by a camera of the second terminal, and the like.

[0207] In the present embodiment, the first operation is not limited as long as it is sufficiently distinguishable from other operations. For example, the first operation may be at least one of an alphabet operation, a digital operation, a single code operation, etc.

[0208] In some embodiments, the alphabetic manipulation may be at least one of alphabets a-z, alphabets A-Z.

[0209] In some embodiments, the digital manipulation may be at least one of the digits 0-9.

[0210] In some embodiments, the single symbol action may be at least one of, for example, a hook symbol action, a triangle symbol action, a pentagram symbol action, a circle symbol action, a square symbol action, a diamond symbol action, a preset number of consecutive taps, a multi-finger slide action a preset distance, etc.

[0211] In some embodiments, the first operation and the second operation may be the same operation.

[0212] In some embodiments, the first operation and the second operation may be different operations, for example, the first operation is tracing a preset distance gesture with three fingers from left to right, and the second operation is tracing a preset distance gesture with three fingers from right to left.

[0213] In some embodiments, the first and second operations may be different gestures, and may be parts of a complete alphabetic gesture, a numeric gesture, or a single symbol gesture. The single symbol gesture may be, for example, a fork symbol gesture, a triangle symbol gesture, a pentagram symbol gesture, a circle symbol gesture, a square symbol gesture, etc. For example, the first operation may be a left semicircular symbol gesture and the second operation may be a right semicircular symbol gesture, and the first and second operations together form a complete circle symbol gesture.

[0214] In some embodiments, the first message further includes first device information of the first terminal.

[0215] In some embodiments, the first device information includes at least one of first identification information of the first terminal and first hardware information of the first terminal.

[0216] In some embodiments, the first device information includes at least one of first manufacturer information of the first terminal, information characterizing whether the first terminal supports WIFI 5G, a Bluetooth MAC address of the first terminal, a WIFI MAC address, device type information, information characterizing the BLE version of the first terminal, information characterizing a multi-screen collaboration function supported by the first terminal, and version information characterizing a multi-screen collaboration function supported by the first terminal.

[0217] In some embodiments, the first identification information includes at least one of information characterizing the multi-screen collaboration function supported by the first terminal, and version information characterizing the multi-screen collaboration function supported by the first terminal.

[0218] In some embodiments, the first hardware information includes at least one of size information and resolution information of a display screen of the first terminal, resolution information of an image captured by a camera of the first terminal, and the like.

[0219] In some embodiments, the first message further includes a first identification information of the first terminal.

[0220] In some embodiments, the first short-range connection information includes at least one of operation information corresponding to the first operation, operation intention information corresponding to the first operation, first time information at which the first operation is identified, first user account information of the first terminal, first connection information of the first terminal, and a first connection validity period.

[0221] In some embodiments, the first message is accompanied by operation information corresponding to the first operation, or the first message is accompanied by encoded and compressed operation information corresponding to the first operation.

[0222] In some embodiments, if the length of the operation information corresponding to the first operation is less than or equal to the maximum length of the field containing the operation information of the first message, the first message includes the operation information corresponding to the first operation, and if the length of the operation information corresponding to the first operation is greater than the maximum length of the field of the first message containing the first operation information, the first message includes the encoded and compressed operation information corresponding to the first operation.

[0223] In some embodiments, the first message is accompanied by operation intention information corresponding to the first operation, or the first message is accompanied by encoded and compressed operation intention information corresponding to the first operation.

[0224] In some embodiments, if the length of the operation intention information corresponding to the first operation is equal to or less than the maximum length of the field containing the operation intention information in the first message, the first message accompanies the operation intention information corresponding to the first operation, and if the length of the operation intention information corresponding to the first operation is greater than the maximum length of the field in the first message containing the first operation intention information, the first message accompanies the encoded and compressed operation intention information corresponding to the first operation.

[0225] In some embodiments, the first connection information is generated after a first connection between the first terminal and the second terminal is successful, or after the first terminal and the second terminal are successfully connected for the first time after the previously generated first connection information has expired.

[0226] In some embodiments, the first connection information may include a communication address of the first terminal and a communication address of the second terminal.

[0227] In some embodiments, the first connection information may include a communication address of the first terminal, a communication address of the second terminal, and a first connection validity period.

[0228] In some embodiments, the communication address may be a Bluetooth MAC address.

[0229] In some embodiments, if the first connection validity period is currently in effect, the first short-range connection information includes the first connection information, and if the first connection validity period is not currently in effect, the first short-range connection information does not include the first connection information.

[0230] In some embodiments, the first short-range connection information includes the first connection information regardless of whether it is currently within the first connection validity period, or the first short-range connection information both includes the first connection information and the first connection validity period.

[0231] Step 202: If it is determined that the first short-distance connection information and the second short-distance connection information match, control the second terminal and the first terminal to establish a second connection.

[0232] In some embodiments, the second connection includes establishing a short-range wireless connection between the first terminal and the second terminal, disconnecting the short-range wireless connection between the first terminal and the second terminal, performing a multi-screen collaborative operation initiated by the first terminal, or initiating a multi-screen collaborative operation to the first terminal.

[0233] In some embodiments, the multi-screen collaborative operation includes at least one of a cross-screen projection operation, an input feedback operation, a cross-screen copy-paste operation, a cross-screen document collaborative operation, a cross-screen camera multi-position shooting operation, an application streaming relay operation, an audio playback streaming relay operation, a call streaming relay operation, and the like.

[0234] In some embodiments, the first connection and the second connection may be the same connection or may be corresponding connections. For example, the first connection and the second connection may be the establishment of a proximity wireless connection between a first terminal and a second terminal, or the disconnection of a proximity wireless connection between the first terminal and the second terminal, but the first connection may initiate a multi-screen collaborative operation and the second connection may perform the initiated multi-screen collaborative operation, or the second connection may initiate a multi-screen collaborative operation and the first connection may perform the initiated multi-screen collaborative operation.

[0235] In some embodiments, if it is determined that the first identification information and the second identification information match, and it is determined that the first short-range connection information and the second short-range connection information match, the second terminal and the first terminal are controlled to establish a second connection.

[0236] In some embodiments, the step of determining that the first short-range connection information and the second short-range connection information match includes determining that the first short-range connection information and the second short-range connection information match if at least one of the following conditions is satisfied: operation information corresponding to the first operation and operation information corresponding to the second operation match, operation intention information corresponding to the first operation and operation intention information corresponding to the second operation are the same, the first user account information and the second user account information match, and the first connection information and the second connection information are the same.

[0237] In some embodiments, determining that the first short-range connection information and the second short-range connection information match includes determining that the first short-range connection information and the second short-range connection information match if the first connection information and the second connection information are the same and are within the first connection validity period and the second connection validity period.

[0238] In some embodiments, if it is determined that the first identification information and the second identification information match, and it is determined that the first short-range connection information and the second short-range connection information match, the second terminal and the first terminal are controlled to establish a second connection.

[0239] In some embodiments, determining that the first identification information and the second identification information match comprises determining that the first identification information and the second identification information are the same.

[0240] In some embodiments, a match between operation information corresponding to a first operation and operation information corresponding to a second operation includes a situation in which the operation information corresponding to the first operation and the operation information corresponding to the second operation are the same, or a situation in which the operation information corresponding to the first operation and the operation information corresponding to the second operation satisfy a first correspondence relationship.

[0241] In some embodiments, operation information corresponding to a first operation and operation information corresponding to a second operation satisfy a first correspondence relationship, which means that operation information corresponding to a second operation that corresponds to the operation information corresponding to the first operation can be found from the first correspondence relationship.

[0242] In some embodiments, the operation information corresponding to the first operation and the operation information corresponding to the second operation satisfy a first correspondence relationship means that the operation information corresponding to the second operation that corresponds to the operation information corresponding to the first operation can be found from the first correspondence relationship, and the operation intention information corresponding to the operation information corresponding to the first operation, the operation intention information corresponding to the operation information corresponding to the second operation, the operation intention information corresponding to the first operation, and the operation intention information corresponding to the second operation are all the same in the first correspondence relationship.

[0243] In some embodiments, matching the first user account information and the second user account information includes that the first user account information and the second user account information are the same, or that the first user account information and the second user account information satisfy a third correspondence relationship between the first user account information and the second user account information that has been previously established.

[0244] In some embodiments, the first user account information and the second user account information satisfy a third correspondence relationship refers to the fact that the second user account information corresponding to the first user account information can be found from the third correspondence relationship.

[0245] In some embodiments, if it is determined that the first short-range connection information and the second short-range connection information do not match, or the first identification information and the second identification information do not match, an inquiry message is displayed to inquire whether the user should continue the subsequent judgment flow. If a command to continue is received from the user, authentication is performed with the first terminal. If the authentication is successful, the subsequent judgment flow is continued; if no command is received or a stop command is received, the flow is terminated.

[0246] In some embodiments, determining that the first identification information and the second identification information do not match includes determining that the first identification information and the second identification information are not identical.

[0247] In some embodiments, the step of determining that the first short-range connection information and the second short-range connection information do not match includes determining that the first short-range connection information and the second short-range connection information do not match when at least one of the following conditions is met: operation information corresponding to the first operation and operation information corresponding to the second operation do not match, operation intention information corresponding to the first operation and operation intention information corresponding to the second operation are different, the first user account information and the second user account information do not match, and the first connection information and the second connection information are different.

[0248] In some embodiments, determining that the first short-range connection information and the second short-range connection information do not match includes determining that the first short-range connection information and the second short-range connection information do not match if the first connection information and the second connection information are different or the first connection validity period and the second connection validity period are not within each other.

[0249] In some embodiments, a mismatch between the operation information corresponding to the first operation and the operation information corresponding to the second operation includes a case where the operation information corresponding to the first operation and the operation information corresponding to the second operation are different, or a case where the operation information corresponding to the first operation and the operation information corresponding to the second operation do not satisfy a first correspondence relationship.

[0250] In some embodiments, the operation information corresponding to the first operation and the operation information corresponding to the second operation not satisfying the first correspondence relationship means that the operation information corresponding to the second operation that corresponds to the operation information corresponding to the first operation cannot be found from the first correspondence relationship.

[0251] In some embodiments, the operation information corresponding to the first operation and the operation information corresponding to the second operation do not satisfy the first correspondence relationship means that, although operation information corresponding to the second operation that corresponds to the operation information corresponding to the first operation can be found from the first correspondence relationship, the operation intention information corresponding to the operation information corresponding to the first operation and the operation intention information corresponding to the operation information corresponding to the second operation in the first correspondence relationship are different from the operation intention information corresponding to the first operation or the operation intention information corresponding to the second operation.

[0252] In some embodiments, a mismatch between the first user account information and the second user account information includes the first user account information and the second user account information being different, or the first user account information and the second user account information not satisfying a third correspondence relationship between the first user account information and the second user account information that has been pre-established.

[0253] In some embodiments, the first user account information and the second user account information do not satisfy the third correspondence relationship refers to the fact that the second user account information corresponding to the first user account information cannot be found from the third correspondence relationship.

[0254] In some embodiments, when the second terminal is a master device, the step of authenticating with the first terminal includes the steps of generating an authentication code and displaying the generated authentication code; receiving the authentication code sent by the first terminal, and passing the authentication if the generated authentication code is the same as the received authentication code; If the generated authentication code and the received authentication code are different, the authentication is not passed.

[0255] In some embodiments, when the second terminal is a slave device, the step of performing authentication with the first terminal includes the steps of displaying an authentication code input interface, receiving an authentication code input by a user, and transmitting the authentication code input by the user to the first terminal to perform authentication.

[0256] In some embodiments, when the second terminal is a master device, the step of authenticating with the first terminal includes the steps of generating and displaying a connection two-dimensional code, receiving a connection authentication code obtained by the first terminal scanning the connection two-dimensional code, verifying the connection authentication code, and passing the authentication if the verification is passed, or not passing the authentication if the verification is not passed.

[0257] In some embodiments, when the second terminal is a slave device, the step of authenticating with the first terminal includes scanning a connection two-dimensional code displayed by the first terminal to obtain a connection authentication code, and sending the connection authentication code to the first terminal.

[0258] In some embodiments, if it is still not determined that the first terminal and the second terminal are matched within a preset time when the second operation is recognized, prompt information is presented to indicate that the user's connection has failed.

[0259] In some embodiments, the first terminal and the second terminal are two equal actors, and in such a case, if the first terminal and the second terminal are determined to be a match, the second terminal directly controls the first terminal to establish a second connection.

[0260] In some embodiments, the first terminal and the second terminal are not two equal execution entities, and in such a case, before performing an operation corresponding to the first operation, the device interaction method further includes a step of determining a master-slave relationship between the first terminal and the second terminal based on the first device information and the second device information, and the step of controlling the second terminal and the first terminal to make a second connection includes a step of controlling the second terminal and the first terminal to make a second connection based on the master-slave relationship between the first terminal and the second terminal.

[0261] In some embodiments, a master-slave relationship may also be referred to as a relationship between a service end and a client, a relationship between a source end and a sink end, or the like.

[0262] In some embodiments, determining a master-slave relationship between the first terminal and the second terminal based on the first device information and the second device information includes determining that the first terminal is a master device and the second terminal is a slave device if the hardware conditions corresponding to the first hardware information are higher than the hardware conditions corresponding to the second hardware information. For example, if the size of the display screen of the first terminal is larger than the size of the display screen of the second terminal, determining that the first terminal is a master device and the second terminal is a slave device.

[0263] In some embodiments, determining a master-slave relationship between the first terminal and the second terminal based on the first device information and the second device information includes determining that the second terminal is the master device and the first terminal is the slave device when the hardware conditions corresponding to the first hardware information are lower than the hardware conditions corresponding to the second hardware information. For example, when the size of the display screen of the first terminal is smaller than the size of the display screen of the second terminal, determining that the second terminal is the master device and the first terminal is the slave device.

[0264] In some embodiments, the step of determining a master-slave relationship between the first terminal and the second terminal based on the first device information and the second device information includes a step of determining a master-slave relationship between the first terminal and the second terminal based on the first time information and the second time information when the hardware conditions corresponding to the first hardware information and the hardware conditions corresponding to the second hardware information are the same.

[0265] In some embodiments, the step of determining a master-slave relationship between the first terminal and the second terminal based on the first time information and the second time information includes the step of determining that the first terminal is a master device and the second terminal is a slave device if the first time information is earlier than the second time information.

[0266] In some embodiments, the step of determining a master-slave relationship between the first terminal and the second terminal based on the first time information and the second time information includes the step of determining that the second terminal is a master device and the first terminal is a slave device if the first time information is later than the second time information.

[0267] In some embodiments, the hardware condition for a smart terminal with device type information of PAD is higher than the hardware condition for a smart terminal with device type information of mobile phone, and the hardware condition for a smart terminal with device type information of desktop computer, laptop computer, smart screen, etc. is higher than the hardware condition for a smart terminal with device type information of PAD.

[0268] In some embodiments, for two smart devices with the same device type information, the smart device with a larger display screen size has higher hardware requirements.

[0269] In some embodiments, smart devices with larger display screens have higher hardware requirements.

[0270] In some embodiments, the cross-screen projection operation, the cross-screen copy-paste operation, and the cross-screen document collaboration operation need to determine the master-slave relationship based on the device type information and the display screen size information.

[0271] In some embodiments, the multi-position shooting operation of a cross-screen camera requires determining a master-slave relationship based on the resolution information of the images captured by the camera.

[0272] In some embodiments, the step of controlling the first terminal to establish a second connection with the second terminal based on a master-slave relationship between the first terminal and the second terminal includes initiating a second connection from the smart terminal that is a slave device to the smart terminal that is a master device.

[0273] The device interaction method provided by the present application realizes controlling the second terminal and the first terminal to establish a second connection based on the identified operation, that is, adopting an operation execution method that is closer to natural user authentication, which reduces the user's usage threshold and improves the user experience.

[0274] In some embodiments, after the first terminal and the second terminal establish a short-range wireless connection or perform a multi-screen collaborative operation, the first terminal can also establish a short-range wireless connection with other smart terminals or perform a multi-screen collaborative operation, and the second terminal can also establish a short-range wireless connection with other smart terminals or perform a multi-screen collaborative operation.

[0275] In order to more completely illustrate the entire implementation process of the device interaction method provided in the present application, two examples are given below, but the listed examples are not intended to limit the scope of the claims of the present application.

[0276] Example 1 This example describes an establishment operation for realizing a short-range wireless connection based on gesture recognition, and the method includes steps 300 to 306, as shown in FIG.

[0277] Step 300: The first terminal pre-establishes a first correspondence relationship between gesture encoding and operation intention information, and the second terminal pre-establishes a first correspondence relationship between gesture encoding and operation intention information.

[0278] In this example, the operation intention information corresponding to the gesture encoding corresponding to the circular gesture is an operation for establishing a short-range wireless connection.

[0279] Step 301: When the first terminal recognizes a circular gesture, it searches the first correspondence relationship for operation intention information corresponding to the gesture encoding corresponding to the circular gesture, i.e., performs an operation to establish a short-range wireless connection. When the second terminal recognizes a circular gesture, it searches the first correspondence relationship for operation intention information corresponding to the gesture encoding corresponding to the circular gesture, i.e., performs an operation to establish a short-range wireless connection.

[0280] Step 302: The first terminal broadcasts a first message based on a broadcast mechanism of BLE 4.0. The first message includes first device information of the first terminal and first short-range connection information corresponding to the circular gesture. The first device information includes first identification information. The first short-range connection information includes first time information when the circular gesture is identified, a gesture encoding corresponding to the circular gesture, and operation information for establishing a short-range wireless connection. The second terminal broadcasts a second message based on a broadcast mechanism of BLE 4.0. The second message includes second device information of the second terminal and second short-range connection information corresponding to the circular gesture. The second device information includes second identification information. The second short-range connection information includes second time information when the circular gesture is identified, a gesture encoding corresponding to the circular gesture, and operation information for establishing a short-range wireless connection.

[0281] Step 303: The first terminal determines whether the first identification information and the second identification information match, and determines whether the first short-range connection information and the second short-range connection information match, and if the first identification information and the second identification information match and the first short-range connection information matches, executes step 304, and if the first identification information and the second identification information do not match or the first short-range connection information and the second short-range connection information do not match, executes step 305. The second terminal determines whether the first identification information and the second identification information match, and determines whether the first short-range connection information and the second short-range connection information match, and if the first identification information and the second identification information match and the first short-range connection information matches, executes step 304, and if the first identification information and the second identification information do not match or the first short-range connection information and the second short-range connection information do not match, executes step 305.

[0282] Step 304: The first terminal and the second terminal establish a short-range wireless connection.

[0283] Step 305: The first terminal determines whether the time interval between the current time and the first time information is greater than the preset time, and if the time interval is greater than the preset time, executes step 306; if the time interval is not greater than the preset time, continues to execute step 301; the second terminal determines whether the time interval between the current time and the second time information is greater than the preset time, and if so, executes step 306; if not, continues to execute step 301.

[0284] Step 306: Display prompt information to notify the user that the connection has failed.

[0285] Example 2 In this example, a method for performing multi-screen collaborative operations based on gesture recognition is described, and as shown in FIG. 4, the method includes steps 400-410.

[0286] Step 400: The first terminal pre-establishes a first correspondence relationship between gesture encoding and operation intention information, and the second terminal pre-establishes a first correspondence relationship between gesture encoding and operation intention information.

[0287] In this example, the operation intention information corresponding to the gesture encoding corresponding to the circular gesture is an operation for establishing a short-range wireless connection.

[0288] Step 401: When the first terminal recognizes a circular gesture, it searches the first correspondence relationship for operation intention information corresponding to the gesture encoding corresponding to the circular gesture, i.e., performs an operation to establish a short-range wireless connection. When the second terminal recognizes a circular gesture, it searches the first correspondence relationship for operation intention information corresponding to the gesture encoding corresponding to the circular gesture, i.e., performs an operation to establish a short-range wireless connection.

[0289] Step 402: The first terminal broadcasts a first message based on the broadcast mechanism of BLE 4.0. The first message includes first device information of the first terminal and first short-range connection information corresponding to the circular gesture. The first device information includes first identification information and size information of the display screen of the first terminal. The first short-range connection information includes first time information when the circular gesture is identified, a gesture encoding corresponding to the circular gesture, and operation information for establishing a short-range wireless connection. The second terminal broadcasts a second message based on the broadcast mechanism of BLE 4.0. The second message includes second device information of the second terminal and second short-range connection information corresponding to the circular gesture. The second device information includes second identification information and size information of the display screen of the second terminal. The second short-range connection information includes second time information when the circular gesture is identified, a gesture encoding corresponding to the circular gesture, and operation information for establishing a short-range wireless connection.

[0290] Step 403: The first terminal determines whether the first identification information and the second identification information match, and determines whether the first short-range connection information and the second short-range connection information match. If the first identification information and the second identification information match and the first short-range connection information and the second short-range connection information match, executes step 404. If the first identification information and the second identification information do not match or the first short-range connection information and the second short-range connection information do not match, executes step 409. The second terminal determines whether the first identification information and the second identification information match, and determines whether the first short-range connection information and the second short-range connection information match. If the first identification information and the second identification information match and the first short-range connection information and the second short-range connection information match, executes step 404. If the first identification information and the second identification information do not match or the first short-range connection information and the second short-range connection information do not match, executes step 409.

[0291] Step 404: The first terminal compares the size information of the display screen of the first terminal with the size information of the display screen of the second terminal, and if the size information of the display screen of the first terminal is larger than the size information of the display screen of the second terminal, executes step 405. If the size information of the display screen of the first terminal is smaller than the size information of the display screen of the second terminal, executes step 406. If the size information of the display screen of the first terminal and the size information of the display screen of the second terminal are the same, executes step 407. The second terminal compares the size information of the display screen of the first terminal with the size information of the display screen of the second terminal, and if the size information of the display screen of the first terminal is larger than the size information of the display screen of the second terminal, executes step 405. If the size information of the display screen of the first terminal is smaller than the size information of the display screen of the second terminal, executes step 406. If the size information of the display screen of the first terminal and the size information of the display screen of the second terminal are the same, executes step 407.

[0292] Step 405: The first terminal or the second terminal determines that the first terminal is the master device and the second terminal is the slave device, and continues to execute step 408.

[0293] Step 406: The first terminal or the second terminal determines that the second terminal is the master device and the first terminal is the slave device, and continues to execute step 408.

[0294] Step 407: The first terminal determines whether the first time information is earlier than the second time information, and if the first time information is earlier than the second time information, executes step 405. If the first time information is not earlier than the second time information, executes step 406. The second terminal determines whether the first time information is earlier than the second time information, and if the first time information is earlier than the second time information, executes step 405. If the first time information is not earlier than the second time information, executes step 406.

[0295] Step 408: The slave devices at the first terminal and the second terminal initiate a multi-screen collaborative operation to the master device.

[0296] Step 409: The first terminal determines whether the time interval between the current time and the first time information is greater than the preset time, and if so, executes step 410. If not, continues to execute step 401. The second terminal determines whether the time interval between the current time and the second time information is greater than the preset time, and if so, executes step 410. If not, continues to execute step 401.

[0297] Step 410: Display prompt information to indicate that the user's connection has failed.

[0298] In a third aspect, referring to FIG. 5, an embodiment of the present application provides an electronic device including at least one processor 501 and a memory 502 storing at least one computer program, the electronic device realizing any of the above device interaction methods when the at least one computer program is executed by the at least one processor 501.

[0299] In some embodiments, the electronic equipment further includes at least one I / O interface 503 connected between the processor 501 and the memory 502 and configured to realize information interaction between the processor 501 and the memory 502.

[0300] The processor 501 is a device capable of processing data, including, but not limited to, a central processing unit (CPU). The memory 502 is a device capable of storing data, including, but not limited to, random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), and flash memory (FLASH). The I / O interface (read / write interface) 503 is connected between the processor 501 and the memory 503 and can realize information interaction between the processor 501 and the memory 502, including, but not limited to, a data bus 504.

[0301] In some embodiments, the processor 501, memory 502 and I / O interface 503 are connected to each other and to other components of the computing facility via a bus 504.

[0302] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, the computer program, when executed by a processor, causing the processor to implement the device interaction method described above.

[0303] FIG. 6 is a block diagram of a device interaction system according to an embodiment of the present invention.

[0304] 6 in a fifth aspect, an embodiment of the present application provides a device interaction system including a first terminal 601 and a second terminal 602. The first terminal 601 is configured to broadcast a first message including first close-range connection information corresponding to the first operation based on the first operation when a first operation is recognized, obtain a second message including second close-range connection information corresponding to the second operation to be broadcast by the second terminal based on the recognized second operation, and control the first terminal and the second terminal to establish a first connection when it is determined that the first close-range connection information and the second close-range connection information match.

[0305] The second terminal 602 is configured to broadcast, based on the second operation, a second message including second short-range connection information corresponding to the second operation when the second operation is recognized; obtain a first message including first short-range connection information corresponding to the first operation broadcast by the first terminal based on the recognized first operation; and control the second terminal and the first terminal to establish a second connection when it is determined that the first short-range connection information and the second short-range connection information match.

[0306] In some embodiments, the first connection and the second connection may be the same connection or may be corresponding connections. For example, the first connection and the second connection may be the establishment of a proximity wireless connection between the first terminal and the second terminal or the disconnection of a proximity wireless connection between the first terminal and the second terminal, the first connection may be for initiating a multi-screen collaborative operation and the second connection may be for performing the initiated multi-screen collaborative operation, or the second connection may be for initiating a multi-screen collaborative operation and the first connection may be for performing the initiated multi-screen collaborative operation.

[0307] In some embodiments, the first message further includes first device information of the first terminal, and the second message further includes second device information of the second terminal, and the first terminal 601 is configured to determine a master-slave relationship between the first terminal and the second terminal based on the first device information and the second device information, and control the first terminal and the second terminal to establish a first connection based on the master-slave relationship between the first terminal and the second terminal.

[0308] The second terminal 602 is further configured to determine a master-slave relationship between the first terminal and the second terminal based on the first device information and the second device information, and to control a second connection between the second terminal and the first terminal based on the master-slave relationship between the first terminal and the second terminal.

[0309] In some embodiments, the first message further includes first identification information of the first terminal, and the second message further includes second identification information of the second terminal, and the first terminal 601 is further configured to control the first connection between the first terminal and the second terminal when it determines that the first identification information and the second identification information match and determines that the first short-range connection information and the second short-range connection information match.

[0310] The second terminal 602 is further configured to determine that the first identification information and the second identification information match, and if it determines that the first short-range connection information and the second short-range connection information match, control the second terminal and the first terminal to establish a second connection.

[0311] The specific implementation process of the device interaction system is the same as the specific implementation process of the device interaction method, and will not be described again here.

[0312] Those skilled in the art will appreciate that all or some of the steps of the methods, systems, and functional modules / means in the devices disclosed above can be implemented as software, firmware, hardware, or a suitable combination thereof. In hardware embodiments, the division between the functional modules / means mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may be performed jointly by several components. Some or all of the physical components may be implemented as software executed by a processor (e.g., a central processor, digital signal processor, or microprocessor), or as hardware, or as an integrated circuit such as a dedicated integrated circuit. Such software may be located on a computer-readable medium, which may include computer storage media (or non-transitory media) and communication media (or transitory media). As is well known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (e.g., computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium used to store the desired information and which can be accessed by a computer. Additionally, those skilled in the art will know that communication media typically include computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transmission mechanism and can include any information delivery media.

[0313] Although exemplary embodiments are disclosed herein and specific terms are employed, these should be used and interpreted in a general, illustrative sense only, and not for limiting purposes. It will be apparent to those skilled in the art that in some instances, unless expressly indicated otherwise, features, characteristics, and / or elements described in combination with particular embodiments may be used alone, or features, characteristics, and / or elements described in combination with other embodiments may be used in combination. Accordingly, those skilled in the art will recognize that various changes in form and detail may be made without departing from the scope of the present application as defined by the appended claims.

Claims

1. A device interaction method applied to a first terminal, comprising: If a first operation is recognized, broadcasting a first message according to the first operation, the first message including first short-range connection information corresponding to the first operation; obtaining a second message broadcast by the second terminal based on the recognized second operation, the second message including second close-range connection information corresponding to the second operation; When it is determined that the first short-distance connection information and the second short-distance connection information match, controlling the first terminal and the second terminal to perform a first connection; A device interaction method comprising:

2. The first message further includes first device information of the first terminal, and the second message further includes second device information of the second terminal; The device interaction method further includes, before the step of controlling the first terminal and the second terminal to perform a first connection, determining a master-slave relationship between the first terminal and the second terminal based on the first device information and the second device information; the step of controlling the first terminal and the second terminal to establish a first connection includes a step of controlling the first terminal and the second terminal to establish the first connection based on a master-slave relationship between the first terminal and the second terminal, The device interaction method of claim 1 .

3. the first message further includes first identification information of the first terminal, and the second message further includes second identification information of the second terminal; When it is determined that the first short-distance connection information and the second short-distance connection information match, the step of controlling the first terminal and the second terminal to perform a first connection includes: When it is determined that the first identification information and the second identification information match, and when it is determined that the first short-range connection information and the second short-range connection information match, controlling the first terminal and the second terminal to perform a first connection. The device interaction method of claim 1 .

4. The step of broadcasting a first message based on the first operation includes: searching for operation intention information corresponding to the operation information corresponding to the first operation from a first correspondence relationship between operation information corresponding to the first operation, operation information corresponding to the second operation, and operation intention information, which are set in advance; broadcasting the first message; The device interaction method of claim 1 , comprising:

5. the first short-distance connection information includes at least one of operation information corresponding to the first operation, operation intention information corresponding to the first operation, first time information at which the first operation is identified, first user account information of the first terminal, first connection information of the first terminal, and a first connection validity period of the first connection information; The second short-distance connection information includes at least one of operation information corresponding to the second operation, operation intention information corresponding to the second operation, second time information when the second operation is identified, second user account information of the second terminal, second connection information of the second terminal, and a second connection validity period of the second connection information. The device interaction method of claim 1 .

6. The step of determining that the first short-range connection information and the second short-range connection information match includes: operation information corresponding to the first operation matches operation information corresponding to the second operation, the operation intention information corresponding to the first operation matches the operation intention information corresponding to the second operation, The first user account information and the second user account information match, or The first connection information and the second connection information are the same. determining that the first short-distance connection information and the second short-distance connection information match when at least one of the following conditions is satisfied: The device interaction method of claim 5.

7. the first short-distance connection information includes first connection information of the first terminal and a first connection validity period of the first connection information, and the second short-distance connection information includes second connection information of the second terminal and a second connection validity period of the second connection information, The step of determining that the first short-range connection information and the second short-range connection information match includes: determining that the first short-range connection information and the second short-range connection information match when the first connection information and the second connection information are identical and are within the first connection validity period and the second connection validity period; The device interaction method of claim 1 .

8. The first connection is Establishing a short-range wireless connection between the first terminal and the second terminal, disconnecting the short-range wireless connection between the first terminal and the second terminal, initiating a multi-screen collaborative operation on the second terminal, or performing a multi-screen collaborative operation initiated by the second terminal. Contains any of the following: The device interaction method of claim 1 .

9. at least one processor; a memory having stored therein at least one computer program, the at least one computer program implementing the device interaction method according to any one of claims 1 to 8 when executed by the at least one processor; Electronic equipment, including

10. A computer program is stored, the computer program causing the processor to perform the device interaction method according to any one of claims 1 to 8 when executed by the processor. A computer-readable storage medium.

11. a first terminal configured to, when a first operation is recognized, broadcast a first message according to the first operation, the first message including first close-range connection information corresponding to the first operation; obtain a second message broadcast by a second terminal according to a recognized second operation, the second message including second close-range connection information corresponding to the second operation; and, when it is determined that the first close-range connection information and the second close-range connection information match, control the first terminal and the second terminal to perform a first connection; a second terminal configured to, when a second operation is recognized, broadcast a second message according to the second operation, the second message including second close-range connection information corresponding to the second operation; obtain a first message broadcast by the first terminal according to the recognized first operation, the first message including first close-range connection information corresponding to the first operation; and, when it is determined that the first close-range connection information and the second close-range connection information match, control the second terminal and the first terminal to perform a second connection; A device interaction system including:

Images