A terminal and device binding method

Abstract

Some embodiments of the present application show a terminal and device binding method, the method comprising: in response to a user input adding a device instruction, controlling the display to display a discovery device list; in response to a user input selecting a first device control instruction, if the first device is a display device, controlling the display to display a binding guide page, and obtaining the number of target controls in the discovery device list; if the number is greater than 1, sending a screenshot request to the first device; obtaining the screen picture from the server and displaying the screen picture in the screenshot display window; if a user input confirmation binding operation is received, binding the first device with the user account. The embodiments of the present application determine that the user selected device is a display device and the number of discovered display devices is greater than 1, obtain the screenshot picture of the selected device and show it to the user, so that the user can quickly and accurately find the target device and bind it, simplify the operation process and save user time.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to a method for binding a terminal and a device. Background Technology

[0002] In an AIoT (Artificial Intelligence & Internet of Things) system, the television acts as the central manager of the home, and the operational process of connecting to the AIoT system requires high accuracy to improve the user experience. When the television and mobile phone are connected to the same network, the mobile application can discover all televisions on the same local area network through the DLNA (Digital Living Network Alliance) service, and the user can select a target television to bind with the mobile phone.

[0003] However, as people's living standards improve, many households have more than one television, and multiple televisions are also displayed in television stores or showrooms. Mobile applications may also detect multiple televisions. Each television is likely to use a default name, such as "Smart TV" or "Living Room TV." Even if multiple televisions have different names, users cannot intuitively match the television names with the actual televisions.

[0004] When a user discovers a TV they want to pair with via a mobile application, even though the TV names may differ, the target TV cannot be correctly distinguished from the TV list displayed in the application. The user can use the remote control to operate the target TV, access its settings, and then view its corresponding TV name, allowing them to select the target TV from the list. However, this search path is deep and cumbersome, wasting user time. If all the TVs displayed in the list have the same name, the user can sequentially select TVs from the list for family pairing and then verify the pairing by checking if the target TV can be controlled via a device card. However, this process may involve repeated pairing and unpairing of TV devices, making accurate pairing impossible, cumbersome, and time-consuming. Summary of the Invention

[0005] Some embodiments of this application provide a terminal and device binding method. After determining that the device selected by the user is a display device and the number of display devices found is greater than 1, a screenshot of the selected device is obtained and displayed to the user, so that the user can quickly and accurately find the target device and bind it, simplifying the operation process and saving the user's time.

[0006] Firstly, some embodiments of this application provide a terminal, including:

[0007] monitor;

[0008] The controller is configured as follows:

[0009] After logging into a user account, in response to the user's input command to add a device, the system controls the display to show a list of discovered devices, which includes at least one device control.

[0010] In response to a user input instruction to select a first device control among the at least one device controls, if the first device control corresponds to a display device, the system controls the display to show a binding guide page and obtains the number of target controls in the discovered device list. The binding guide page includes a screenshot display window, and the target controls are controls whose corresponding devices are display devices.

[0011] If the number is greater than 1, a screenshot request is sent to the first device, so that the first device captures a screen image based on the screenshot request and uploads the screen image to the server.

[0012] The screen image is retrieved from the server and displayed in the screenshot display window;

[0013] If a confirmation binding operation is received from the user, the first device is bound to the user account so that the user can control the first device through the terminal.

[0014] In some embodiments, the controller executes the sending of a screenshot request to the first device, so that the first device captures a screen image based on the screenshot request and uploads the screen image to the server, and is further configured to:

[0015] If the second device corresponding to the second device control in the at least one device control has the same IP address as the first device, a screenshot request is sent to the server, so that the server sends the screenshot request to the first device based on the device identifier and receives the screenshot data uploaded by the first device after capturing a screen image based on the screenshot request. The screenshot request includes a device identifier and an image identifier, and the screenshot data includes a screen image and an image identifier.

[0016] In some embodiments, the controller executes the sending of a screenshot request to the first device, so that the first device captures a screen image based on the screenshot request and uploads the screen image to the server, and is further configured to:

[0017] If the IP address of the second device corresponding to the second device control in the at least one device control is different from that of the first device, a screenshot request is sent to the first device based on the IP address of the first device, so that the first device can capture a screen image and upload the screenshot data based on the screenshot request. The screenshot request includes an image identifier, and the screenshot data includes a screen image and an image identifier.

[0018] In some embodiments, the controller is configured to retrieve the screen image from the server as follows:

[0019] Send an image request to the server, the image request including an image identifier;

[0020] Receive the screen image sent by the server based on the image identifier.

[0021] In some embodiments, the controller is configured to:

[0022] If the number is not greater than 1, the placeholder identifier of the screenshot display window is set to a preset value so that the screenshot display window is not displayed on the binding guide page.

[0023] In some embodiments, the screenshot request includes a preset instruction, which instructs the first device to perform a preset operation corresponding to the preset instruction, and the preset operation is used to change the user interface of the display device.

[0024] In some embodiments, if the controller receives a confirmation binding operation from the user, it binds the first device to the user account, which is further configured as follows:

[0025] If it is detected that the login user account of the first device is the same as the login user account of the terminal, the binding control is displayed on the binding guide page;

[0026] If a user inputs an instruction to select the binding control is received, the first device is bound to the user account logged in on the terminal.

[0027] In some embodiments, the controller is configured to:

[0028] If it is detected that the first device is not logged into a user account, a first guidance box is displayed on the binding guidance page. The first guidance box includes guidance information, which is used to guide the user to log in to the user account logged in on the first device.

[0029] In some embodiments, the controller is configured to:

[0030] If it is detected that the user account logged in on the first device is different from the user account logged in on the terminal, a second guide box is displayed on the binding guide page. The second guide box includes adding a family control and rebinding the control.

[0031] If a user inputs an instruction to join a family control, the user account logged in on the terminal will be added to the family of the user account logged in on the first device, so that the user account logged in on the terminal can establish a connection with the first device;

[0032] If the user inputs a selection to rebind the control, a first guide box is displayed on the binding guide page.

[0033] Secondly, some embodiments of this application provide a device binding method, including:

[0034] After logging into a user account, in response to the user's input command to add a device, the system controls the display to show a list of discovered devices, which includes at least one device control.

[0035] In response to a user input instruction to select a first device control among the at least one device controls, if the first device control corresponds to a display device, the system controls the display to show a binding guide page and obtains the number of target controls in the discovered device list. The binding guide page includes a screenshot display window, and the target controls are controls whose corresponding devices are display devices.

[0036] If the number is greater than 1, a screenshot request is sent to the first device, so that the first device captures a screen image based on the screenshot request and uploads the screen image to the server.

[0037] The screen image is retrieved from the server and displayed in the screenshot display window;

[0038] If a confirmation binding operation is received from the user, the first device is bound to the user account so that the user can control the first device through the terminal.

[0039] Some embodiments of this application provide a terminal and device binding method. Upon logging into a user account and receiving a user's instruction to add a device, a list of discovered devices is displayed. This list includes at least one device control, which refers to the control corresponding to the device discovered by the terminal. After receiving a user's instruction to select a first device control from the at least one device control, if the first device corresponding to the first device control is a display device, a binding guide page is displayed, and the number of displayed devices in the discovered device list is obtained. The binding guide page includes a screenshot display window. If the number of displayed devices is greater than one, a screenshot request is sent to the first device, causing the first device to capture a screen image based on the screenshot request and upload the screen image to a server. The screen image is retrieved from the server and displayed in the screenshot display window. Upon receiving a confirmation binding operation from the user after determining from the screen image that they want to bind a device, the first device is bound to the user account, allowing the user to control the first device through the terminal. This application embodiment is applied to scenarios where a user account logged into a terminal binds to a device. After determining that the user has selected a display device and the number of discovered display devices is greater than one, a screenshot image of the selected device is obtained and displayed to the user, enabling the user to quickly and accurately find and bind the target device, simplifying the operation process and saving user time. Attached Figure Description

[0040] Figure 1 A flowchart of a device bonding method provided according to some embodiments is shown;

[0041] Figure 2 A schematic diagram illustrating a device discovery list display according to some embodiments is shown;

[0042] Figure 3 A schematic diagram of the homepage of an application provided according to some embodiments is shown;

[0043] Figure 4 A schematic diagram of a device discovery list provided according to some embodiments is shown;

[0044] Figure 5 A schematic diagram of a cross-network screenshot method provided according to some embodiments is shown;

[0045] Figure 6 A schematic diagram of a method for taking screenshots on the same network according to some embodiments is shown;

[0046] Figure 7 A schematic diagram of a user interface for a display device according to some embodiments is shown;

[0047] Figure 8 A schematic diagram of another display device user interface provided according to some embodiments is shown;

[0048] Figure 9 A schematic diagram of a bound pop-up notification box is shown according to some embodiments;

[0049] Figure 10 A schematic diagram of a screenshot animation provided according to some embodiments is shown;

[0050] Figure 11 A timing diagram of a cross-network screenshot method provided according to some embodiments is shown;

[0051] Figure 12 A timing diagram of a method for taking screenshots on the same network according to some embodiments is shown;

[0052] Figure 13 A schematic diagram of a first type of binding onboarding page provided according to some embodiments is shown;

[0053] Figure 14 A schematic diagram of a second type of binding onboarding page provided according to some embodiments is shown;

[0054] Figure 15 A schematic diagram of a successful connection interface provided according to some embodiments is shown;

[0055] Figure 16 A schematic diagram of the homepage of another application provided according to some embodiments is shown;

[0056] Figure 17 A schematic diagram of a third type of binding onboarding page provided according to some embodiments is shown;

[0057] Figure 18 A schematic diagram of a fourth type of binding onboarding page is shown according to some embodiments;

[0058] Figure 19 A schematic diagram of a fifth type of binding onboarding page provided according to some embodiments is shown;

[0059] Figure 20 A timing diagram of a device binding method provided according to some embodiments is shown. Detailed Implementation

[0060] To make the objectives and implementation methods of this application clearer, the exemplary implementation methods of this application will be clearly and completely described below with reference to the accompanying drawings of the exemplary embodiments of this application. Obviously, the exemplary embodiments described are only some embodiments of this application, and not all embodiments.

[0061] It should be noted that the brief descriptions of terms in this application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0062] The terms "first" and "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar or related objects or entities, and do not necessarily imply a specific order or sequence, unless otherwise specified. It should be understood that such terms are interchangeable where appropriate.

[0063] The term “comprising” and any variations thereof are intended to cover but not exclude inclusion, for example, a product or device that includes a range of components is not necessarily limited to all of the components that are clearly listed, but may include other components that are not clearly listed or that are inherent to such product or device.

[0064] The terminal provided in this application can have various implementation forms, such as display devices and smart devices. Smart devices include mobile terminals, tablet computers, computers, laptop computers, etc.

[0065] The display device provided in this application can have various implementation forms, such as a television, a smart television, a laser projection device, a monitor, an electronic bulletin board, an electronic table, etc.

[0066] In AIoT (Artificial Intelligence & Internet of Things) systems, the television acts as the central manager of the home, and the operational process of connecting to the AIoT system requires high accuracy to improve the user experience. When the television and mobile phone are connected to the same network, the mobile application can discover all televisions on the same local area network through the DLNA (Digital Living Network Alliance) service, and the user can select a target television to bind to the mobile phone. However, as people's living standards improve, many households have more than one television, and television stores or showrooms also display multiple televisions. The mobile application may also discover multiple televisions. Each television will most likely use a default television name, such as "Smart TV" or "Living Room TV." Even if multiple televisions have different names, users cannot intuitively match the television names with the actual televisions.

[0067] When a user discovers a TV they want to pair with via a mobile application, even though the TV names may differ, the target TV cannot be correctly distinguished from the TV list displayed in the application. The user can use the remote control to operate the target TV, access its settings, and then view its corresponding TV name, allowing them to select the target TV from the list. However, this search path is deep and cumbersome, wasting user time. If all the TVs displayed in the list have the same name, the user can sequentially select TVs from the list for family pairing and then verify the pairing by checking if the target TV can be controlled via a device card. However, this process may involve repeated pairing and unpairing of TV devices, making accurate pairing impossible, cumbersome, and time-consuming.

[0068] To address the above technical problems, embodiments of this application further improve some functions of the terminal. For example... Figure 1 As shown, the controller performs the following steps:

[0069] Step S101: After logging into the user account, in response to the user's input instruction to add a device, control the display to show a list of discovered devices, the list of discovered devices including at least one device control;

[0070] In some embodiments, the system receives a user input instruction to open the application and controls the display to show the application's homepage, which includes adding device controls and an account display area.

[0071] If historical account login information or locally saved account login information is detected, the user account will be logged in based on the account login information and the account information will be displayed in the account display area;

[0072] The system receives user input indicating the option to add a device control and controls the display to show a list of discovered devices.

[0073] If no historical account login information is detected or the account login information has been saved, the account display area will show "not logged in".

[0074] The system receives a user input command to select and add a device control, and controls the display to show the user login interface.

[0075] After receiving the login information entered by the user, such as username and password, and verifying it successfully, the system controls the display to show the list of discovered devices.

[0076] The "Discover Devices" section displays all nearby white goods and display devices, such as LAN white goods, Bluetooth white goods, and LAN display devices. White goods (home appliances) refer to electrical products that can replace people's housework, mainly including washing machines, some kitchen appliances, and appliances that improve living environment and material living standards, such as air conditioners and refrigerators.

[0077] like Figure 2 As shown, white goods devices can be discovered via LAN proprietary protocols and Bluetooth proprietary protocols. Display devices, such as televisions, can be discovered via LAN DLNA protocol. After discovering devices, it is necessary to update the discovered device list, filter out already bound devices, and then display the discovered device list.

[0078] In some embodiments, the step of controlling the display to show the list of discovered devices includes:

[0079] Obtain the device identifier of the discovered device and the device identifier of the bound device;

[0080] The device ID is a unique identifier for each display device. The last eight bits of the device ID are the display device's wired MAC address (Media Access Control Address, also known as the physical address or hardware address) or wireless MAC address. For example, if the display device's device ID is 8610030090000370000007127d00a247, then 7d00a247 is the corresponding MAC address.

[0081] Device information, including a device identifier, is obtained by discovering the services and protocols used by the discovered device. The device identifier of a bound device can be obtained directly from the user's local machine or from a server using their account.

[0082] Determine whether the device identifier of the discovered device is the same as the device identifier of the bound device;

[0083] If the device identifier of the discovered device differs from that of the bound devices, the corresponding device control for the discovered device will be displayed in the list of discovered devices. The device control is named using the name of the discovered device plus a field from the device identifier, such as the last four digits of the device identifier. For example, the device control might be named "Smart TV (a247)".

[0084] If the device identifier of the discovered device is the same as that of the bound device, there is no need to display the device control corresponding to the discovered device in the list of discovered devices.

[0085] All discovered devices are filtered to identify already bound devices, preventing users from performing invalid operations such as repeatedly binding already bound devices.

[0086] For example, the application's homepage is like Figure 3As shown, the homepage includes an account display area 31 and a device binding area 32. The device binding area includes an add device control 321, indicating that the user account has been bound to a device. The device binding area 32 also includes a device card 322 corresponding to the bound device. Users can control the bound device through the device card. When a user account is logged in, the account display area 31 displays the logged-in user account information; when a user account is not logged in, the account display area 31 displays the not logged-in information. Figure 3 This assumes the user account is already logged in. Upon receiving the user's instruction to click "Add Device Control," the following will be displayed: Figure 4 The list of discovered devices is shown. The list includes three living room TV controls and one smart air conditioner control.

[0087] Step S102: Receive user input indicating that the user is selecting a first device control among the at least one device controls;

[0088] For example, in Figure 4 In the middle, it receives the user's instruction to click the first living room TV control.

[0089] Step S103: In response to the user's instruction to select a first device control among the at least one device controls, determine whether the first device corresponding to the first device control is a display device;

[0090] In some embodiments, the step of determining whether the first device corresponding to the first device control is a display device includes:

[0091] Obtain the device type identifier of the first device;

[0092] Determine whether the device type identifier is a display device type identifier, for example, the display device type identifier could be dlna;

[0093] If the device type identifier is a display device type identifier, then the first device corresponding to the first device control is determined to be a display device;

[0094] If the device type identifier is not a display device type identifier, it is determined that the first device corresponding to the first device control is not a display device.

[0095] In some embodiments, the step of determining whether the first device corresponding to the first device control is a display device includes:

[0096] Determine the type of service and protocol used by the first device;

[0097] Determine whether the service and protocol type is the same as the service and protocol type used by the display device, wherein the protocol used by the display device is the DLNA protocol;

[0098] If the service and protocol type is the service and protocol type used by the display device, then the first device corresponding to the first device control is determined to be the display device;

[0099] If the service and protocol type is the service and protocol type used by the display device, then the first device corresponding to the first device control is determined not to be a display device.

[0100] It should be noted that the device information of the discovered device is obtained by discovering the services and protocols used by the device. The device information includes the device type, device identifier, device name, device icon, device IP (Internet Protocol), device wired physical address, and device wireless physical address.

[0101] If the first device corresponding to the first device control is a display device, execute step S104: control the display to display the binding guide page, and obtain the number of target controls in the discovery device list;

[0102] The binding guide page includes a screenshot display window, which is used to display a screenshot captured by the display device. The target control is a device control whose device is the display device.

[0103] In some embodiments, the screenshot display window is provided with a placeholder identifier. When the placeholder identifier of the screenshot display window is a preset value, such as 0, it indicates that the binding guide page does not display the screenshot display window. When the placeholder identifier of the screenshot display window is not a preset value, such as 1, it indicates that the binding guide page displays the screenshot display window.

[0104] In some embodiments, when displaying the list of discovered devices, the number of different device types is counted and saved locally. The number of device types is updated when a new device is added or a device is removed from the list. The number of target controls (or display devices) in the list of discovered devices can be obtained directly from the local machine.

[0105] In some embodiments, the step of obtaining the number of target controls in the discovery device list includes:

[0106] Iterate through the device controls in the device list;

[0107] Determine whether the device corresponding to the device control is a display device;

[0108] If the device corresponding to the device control is a display device, then increment the number of target controls by 1 and continue to determine whether the device corresponding to the next device control is a display device;

[0109] If the device corresponding to the device control is not a display device, continue to determine whether the device corresponding to the next device control is a display device.

[0110] After all device controls in the device list have been identified, the number of target controls is obtained.

[0111] Step S105: Determine whether the quantity is greater than 1;

[0112] If the number is greater than 1, execute step S106: send a screenshot request to the first device, so that the first device captures a screen image based on the screenshot request and uploads the screen image to the server;

[0113] In some embodiments, if the number is greater than 1, the placeholder identifier of the screenshot display window is set to a non-preset value, for example, the placeholder identifier is set to 1, so that the screenshot display window is displayed on the binding guide page.

[0114] In some embodiments, the screenshot request includes an image identifier. After sending the screenshot request to the first device, the first device captures a screen image based on the screenshot request and uploads the screen image and the image identifier together to the server.

[0115] The image identifier corresponds to or is associated with the screen image, or the image identifier is a random identifier that can identify the screen image. The terminal can then use this image identifier to obtain the corresponding, associated, or identified screen image.

[0116] In some embodiments, the image identifier format is: voice_remote+yyyyMMddHHmmss+random, which means starting with voice_remote, followed by the current time in the format yyyyMMddHHmmss, and then concatenating a 6-digit decimal random number. For example, the image identifier could be voice_remote20230620170524636462.

[0117] In some embodiments, if the number is greater than 1, it is determined whether the IP address of the second device corresponding to the second device control in the at least one device control is the same as that of the first device, that is, whether the IP address of the device control in the discovered device list is the same as that of the first device. Here, the second device control refers to a device control in the discovered device list other than the first device control.

[0118] The device information, including the device IP, is obtained by discovering the services and protocols used by the discovered device. Comparing the IP of the second device with the IP of the first device determines whether the IPs of the second and first devices are the same.

[0119] If the second device corresponding to the second device control in the at least one device control has the same IP address as the first device, a cross-network screenshot operation is performed.

[0120] The steps for taking a screenshot across networks include:

[0121] A screenshot request is sent to the server, so that the server sends the screenshot request to the first device based on the device identifier and receives the screenshot data uploaded by the first device after capturing a screen image based on the screenshot request. The screenshot request includes a device identifier and an image identifier, and the screenshot data includes a screen image and an image identifier.

[0122] To clarify the cross-network screenshot process in this application, an embodiment is disclosed below:

[0123] Obtain the device identifier of the first device and generate an image identifier;

[0124] The device identifier (deviceid) and image identifier (pushdata) are added as request parameters to the screenshot request. The screenshot request also includes the specific operation type (action) and the unique identifier (uuid) of this request. The unique identifier of the message is used to count the number of screenshot requests sent by the terminal and to display the number of screenshot requests received by the device.

[0125] For example, the request parameters for a screenshot request are as follows:

[0126] "uuid":"288179e0-0f48-11ee-96ed-4b2ea44b9229",

[0127] "action":"com.xxx.smartimages.UPLOAD_CAPTURE",

[0128] "pushdata":"mobilekey":"voice_remote20230620170524636462",

[0129] "deviceid":"86100300900000400000008300ffbf33".

[0130] Among them, mobilekey represents the unique identifier corresponding to the screen image, i.e., the image identifier.

[0131] Send the screenshot request to the server via a cross-network interface, such as: mobilemsg / push / v5.3 / remoteMonitor;

[0132] After receiving a screenshot request, the server sends a success message to the terminal and forwards the screenshot request to the device corresponding to the device identifier in the screenshot request, i.e., the first device. The screenshot request sent by the server may not include the device identifier; that is, it only sends the image identifier, operation type, and a unique identifier of the request to the first device.

[0133] Upon receiving a screenshot request, the first device performs the operation corresponding to the operation type: capturing the current user interface and generating a screen image, then uploading the screen image and its identifier to the server. The unique identifier of the request can be saved locally for later screenshot counting.

[0134] In some embodiments, the server and the display device communicate via MQTT (Message Queuing Telemetry Transport). Figure 5 As shown, the terminal sends a screenshot request to the MQTT server. The screenshot request includes a device identifier, an image identifier, and the operation type. The MQTT server pushes an MQTT message (screenshot request) to the device corresponding to the device identifier. This MQTT message includes the image identifier and the operation type. Upon receiving the MQTT message, the display device's MQTT client broadcasts it. Upon receiving the broadcast message, the screenshot application on the display device captures the current user interface, generates a screen image, and uploads the screen image and image identifier to the server.

[0135] In this embodiment of the application, considering the possibility that there may be devices with the same IP address as the currently selected device in the device list, a cross-network screenshot method is used to ensure the accuracy of the sent command in order to avoid sending screenshot commands to non-currently selected devices via IP address.

[0136] If the IP address of the second device corresponding to the second device control in the at least one device control is different from that of the first device, a same-network screenshot operation is performed.

[0137] The steps for taking a screenshot from the same network include:

[0138] A screenshot request is sent to the first device based on the IP address of the first device, so that the first device can capture a screenshot image and upload the screenshot data based on the screenshot request. The screenshot request includes an image identifier, and the screenshot data includes a screen image and an image identifier.

[0139] To clarify the process of taking screenshots on the same network in this application, an example is disclosed below:

[0140] The terminal obtains the IP address of the first device and generates an image identifier;

[0141] The terminal adds the image identifier (pushdata) as a request parameter to the screenshot request. The screenshot request also includes the specific operation type (action) and the unique identifier (uuid) of this request.

[0142] For example, the request parameters for a screenshot request are as follows:

[0143] "uuid":"288179e0-0f48-11ee-96ed-4b2ea44b9229",

[0144] "action":"com.xxx.smartimages.UPLOAD_CAPTURE",

[0145] "pushdata":"mobilekey":"voice_remote20230620170524636462".

[0146] The terminal requests services via the local area network, for example, ip:port / ..commonstarttvapp, and sends a screenshot request to the first device.

[0147] After receiving a screenshot request, the first device performs the operation corresponding to the operation type, namely, capturing the current user interface and generating a screen image, and uploading the screen image and image identifier to the server.

[0148] In some embodiments, such as Figure 6 As shown, the terminal sends a screenshot request to the display device. The screenshot request includes an image identifier and the operation type. Upon receiving the screenshot request, the display device notifies the screen mirroring service, such as the Micro Screen Mirroring SDK (Software Development Kit). The screen mirroring service broadcasts the screenshot request. Upon receiving the broadcast screenshot request, the screenshot application on the display device captures the current user interface, generates a screen image, and uploads the screen image along with the image identifier to the server.

[0149] In this embodiment of the application, if it is found that there is no device in the device list with the same IP as the selected added device, a screenshot request can be sent directly through the local area network. The screenshot process link of the local area network is shorter than that of the cross-network screenshot process link, and the screenshot time efficiency is higher. Same-network screenshot is preferred.

[0150] In some embodiments, the screenshot request includes a preset instruction, which instructs the first device to perform a preset operation corresponding to the preset instruction, and the preset operation is used to change the user interface of the display device.

[0151] Add new data content to the action field of the screenshot request. This data content is used to instruct the first device to perform the preset operation corresponding to the preset instruction.

[0152] In some embodiments, the preset operation includes displaying a binding pop-up reminder or prompt text before performing a screenshot operation. Displaying the binding pop-up reminder or prompt text before performing a screenshot operation ensures that the screenshot captured by the display device includes the binding pop-up reminder or prompt text. Specifically, a timer is started after the binding pop-up reminder is displayed, and when the timer expires, the display of the binding pop-up reminder or prompt text is canceled. For example, the user interface of the display device is as follows: Figure 7 and Figure 8 As shown.

[0153] In some embodiments, the bound pop-up reminder box includes a screenshot consent control and a disagree control. If the user inputs an instruction to select the consent control, the operation of capturing the current user interface, generating a screen image, and uploading the screen image and image identifier to the server is performed. If the user inputs an instruction to select the disagree control, the operation of capturing the current user interface, generating a screen image, and uploading the screen image and image identifier to the server is not performed, and if the terminal does not obtain a screen image within a preset time, a prompt message indicating that a screen image has not been obtained can be displayed. For example, the bound pop-up reminder box is as follows: Figure 9 As shown.

[0154] In some embodiments, the preset operation includes displaying a screenshot animation. Displaying a screenshot animation after taking a screenshot can help users distinguish whether the device control selected in the device discovery list is the target display device, especially when the user is close to the target display device or can directly view the target display device's screen. The screenshot animation may involve gradually shrinking the screen image to a corner of the screen after capturing it. Alternatively, it may involve shaking the screen image on the screen after capturing it. Or, it may involve displaying the screen image at a different angle or at a reduced size for a preset duration on the current user interface after capturing it. For example, the user interface of the display device may be as follows: Figure 10 As shown.

[0155] In some embodiments, the preset operation includes playing a screenshot notification sound. Playing a screenshot notification sound when performing a screenshot operation can help users distinguish whether the device control selected in the discovered device list is the target display device when the user is close to the target display device.

[0156] In some embodiments, the preset operation includes performing the screenshot operation again when the screenshot operation is performed and the screenshot animation is displayed. This method allows the screen image captured by the display device to contain specified frame or arbitrary frame data from the screenshot animation. The second captured screen image and image identifier are uploaded to the server, and the image can be captured as follows: Figure 10 The user interface shown.

[0157] To address the issue of difficulty in quickly distinguishing between the selected device and the target device when multiple display devices are displaying identical images, this application embodiment adds a preset instruction to the screenshot request. This instruction prompts the user interface to be changed before or after the screenshot is taken, effectively differentiating the screenshot operation during the binding process from simple screenshots or remote monitoring screenshots. The purpose of the screenshot binding in this application embodiment is to distinguish the display device, not to view the current user interface or execution results of the display device.

[0158] Step S107: Obtain the screen image from the server and display the screen image in the screenshot display window;

[0159] The step of obtaining the screen image from the server includes:

[0160] Send an image request to the server, the image request including an image identifier;

[0161] Receive the screen image sent by the server based on the image identifier.

[0162] To clarify the process of taking screenshots on the same network in this application, an example is disclosed below:

[0163] After sending a screenshot request to the server or the first device, the terminal receives a success message from the server or the first device. Upon receiving this success message, the terminal polls for the screen image or the screen image URL (uniform resource locator) based on the image identifier. The screen image can then be retrieved using the screen image URL. Polling for the screen image means sending an image request at preset intervals until the screen image is retrieved, at which point the polling stops.

[0164] When the terminal performs polling, it uses the image identifier in the screenshot request to retrieve the specified screen image through the interface. For example, the interface is https: / / xxxxxx.com / v5 / image / imageurl / by / id?sessionID=voice_remote20230620170524636462&_=744987, where the value of sessionID is the image identifier generated when the terminal sends the screenshot request.

[0165] In some embodiments, such as Figure 11 As shown, when the conditions for cross-network screenshotting are met, the terminal's application sends a screenshot request to the server's message subsystem. The screenshot request includes a device identifier and an image identifier. Upon receiving the screenshot request, the message subsystem sends a success message to the terminal and then sends a screenshot request to the device corresponding to the device identifier (display device), which includes the image identifier. Upon receiving the screenshot request, the display device captures the user interface and generates a screen image, uploading the image identifier and the screen image to the server's AI (Artificial Intelligence) data subsystem. Upon receiving the data uploaded by the display device, the AI ​​data subsystem sends a success message to the display device. After receiving the success message, the terminal begins polling the screen image URL through an interface, sending image requests with the image identifier until the server sends the screen image URL.

[0166] In some embodiments, such as Figure 12 As shown, when the conditions for performing a screenshot within the same network are met, the terminal's application sends a screenshot request to the display device based on the display device's IP address. The screenshot request includes an image identifier. Upon receiving the screenshot request, the display device sends a success message to the terminal, captures the user interface, generates a screen image, and uploads the image identifier and screen image to the server's AI data subsystem. After receiving the uploaded data from the display device, the AI ​​data subsystem sends an upload success message to the display device. Upon receiving the success message, the terminal begins polling the screen image URL via an interface, sending image requests with the image identifier until the server sends the screen image URL.

[0167] In some embodiments, the screenshot display window further includes a screen refresh control. Upon receiving a user input indicating that the user should select the screen refresh control, a screenshot request is sent to the first device again, causing the first device to capture a screenshot image again based on the screenshot request and upload the screen image and image identifier to the server.

[0168] Step S108: Receive confirmation of binding operation from the user, bind the first device to the user account, so that the user can control the first device through the terminal.

[0169] The purpose of binding the first device to the user account is to allow the user to control the first device via a terminal that is logged into the user account. Control methods include turning the device on and off, adjusting the volume, and searching for media resources.

[0170] In some embodiments, if the first device corresponding to the first device control is a display device, it is detected whether the first device is logged into a user account.

[0171] If a user account is not logged in on the first device, a first guidance box is displayed on the binding guidance page. The first guidance box includes guidance information to guide the user to log in to the user account logged in on the first device. The user can log in to the user account on the first device according to the guidance information.

[0172] For example, such as Figure 13 As shown, the binding guide page includes a screenshot display window 131 and a first guide box 132. The screenshot display window 131 includes a screen refresh control 1311.

[0173] If a user account is detected to be logged in on the first device, determine whether the user account logged in on the first device is the same as the user account logged in on the terminal.

[0174] If it is detected that the user account logged in to the first device is the same as the user account logged in to the terminal, the binding control and the cancellation control are displayed on the binding guide page;

[0175] If a user inputs an instruction to select the binding control is received, the first device is bound to the user account logged in on the terminal.

[0176] If a user inputs an instruction to select the cancel control, the system controls the display to show the list of discovered devices.

[0177] For example, in Figure 4 In the middle, upon receiving the user's command to click the first living room TV control, a binding guide page is displayed. For example... Figure 14 As shown, the binding guide page includes a screenshot display window 141, a binding control 142, and a cancellation control 143. After the screenshot display window 141 displays a screen image, if the user confirms that the currently displayed screen image is the same as the user interface of the target display device, they can select the binding control 142. Upon receiving the user's input to select the binding control 142, the system binds the living room TV to the user account logged into the terminal and displays a connection success interface, such as... Figure 15 As shown. On the successful connection screen, you can select the room where the device belongs and change the device name. After a successful connection, a device card for the living room TV will be added to the bound devices section of the application's home page, such as... Figure 16 As shown. Selecting this device card allows you to control the living room TV.

[0178] If it is detected that the user account logged in on the first device is different from the user account logged in on the terminal, a second guide box is displayed on the binding guide page. The second guide box includes adding a family control and rebinding the control.

[0179] If a user inputs an instruction to join a family control, the user account logged in on the terminal will be added to the family of the user account logged in on the first device, so that the user account logged in on the terminal can establish a connection with the first device;

[0180] If the user inputs a selection to rebind the control, a first guide box is displayed on the binding guide page.

[0181] The user account logged into the first device is different from the user account logged into the terminal, indicating that the first device is bound to the user account logged into the first device. To control the first device via the terminal, the user account logged into the terminal can be added to the same household as the user account logged into the first device, establishing a connection between the terminal and the first device, thus enabling control via the terminal. Alternatively, the first device can be unbound from the user account logged into the first device; that is, the terminal user account can be logged into the first device, binding the terminal user account to the first device.

[0182] For example, such as Figure 17 As shown, the binding guide page includes a screenshot display window 171 and a second guide box 172. The second guide box includes "Add Family Control" 1721 and "Rebind Control" 1722. If the user inputs the instruction to select "Add Family Control" 1721, the user account XX logged in on the terminal is added to the family of the user account AA logged in on the display device, and a connection success screen is displayed, such as... Figure 15 As shown. If a user inputs a command to rebind control 1722, the binding guide page is displayed, as shown. Figure 18 As shown, the binding guide page includes a screenshot display window 131 and a first guide box 132. The first guide box 132 displays the user account logged into the device.

[0183] If the quantity equals 1, the placeholder flag of the screenshot display window is set to a preset value and the flag is set to 0, for example, so that the screenshot display window is not displayed on the binding guide page. For example, the binding guide page is as follows: Figure 19 As shown.

[0184] In some embodiments, such as Figure 20As shown, the terminal application discovers devices and sends device information to the terminal's binding service layer. The binding service layer filters already bound devices and controls the display to show the list of discovered devices. Upon receiving a user selection of a device control from the discovered device list, a binding guide page is displayed, and it checks if the number of devices displayed in the discovered device list is greater than one. If the number is greater than one, it checks if a device with the same IP address as the selected device exists in the list. If a device with the same IP address exists, a screenshot request is sent to the server across the network. If no device with the same IP address exists, a screenshot request is sent to the display device within the same network. Upon receiving the screenshot request, the display device captures the user interface, generates a screen image, and uploads the image to the server. After receiving the result from the server or the display device, the binding service layer polls for the captured screen images, sends an image request to the server, and receives the screen image URL. The screen image is displayed on the binding guide page. If the number of displayed devices is not greater than one, the display is controlled to show a binding guide page without a screen image.

[0185] Some embodiments of this application provide a device binding method applicable to a terminal, the terminal including a display and a controller. The controller is configured to: after logging into a user account, in response to a user inputting an instruction to add a device, control the display to display a list of discovered devices, the list including at least one device control; in response to a user inputting an instruction to select a first device control among the at least one device controls, if the first device control corresponds to a display device, control the display to display a binding guide page, and obtain the number of target controls in the discovered device list, the binding guide page including a screenshot display window, the target controls being controls whose corresponding devices are display devices; if the number is greater than 1, send a screenshot request to the first device, so that the first device captures a screen image based on the screenshot request and uploads the screen image to a server; obtain the screen image from the server and display the screen image in the screenshot display window; if a user confirmation binding operation is received, bind the first device to the user account, so that the user can control the first device through the terminal. This application embodiment is applied to the scenario of binding a user account to a device during terminal login. After determining that the device selected by the user is a display device and the number of display devices found is greater than 1, a screenshot of the selected device is obtained and displayed to the user, so that the user can quickly and accurately find the target device and bind it, simplifying the operation process and saving the user's time.

[0186] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0187] For ease of explanation, the above description has been provided in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or to limit the embodiments to the specific forms disclosed above. Various modifications and variations can be obtained based on the above teachings. The selection and description of the above embodiments are for the purpose of better explaining the principles and practical applications, thereby enabling those skilled in the art to better utilize the described embodiments and various different variations of embodiments suitable for specific use considerations.

Claims

1. A terminal, characterized in that, include: monitor; The controller is configured as follows: After logging into the user account, in response to the user's command to add a device, the device identifier of the discovered device and the device already bound to the user account is obtained; If the device identifier of the discovered device is different from the device identifier of the device already bound to the user account, the display is controlled to show a list of discovered devices, the list of discovered devices including at least one device control; In response to a user input instruction to select a first device control among the at least one device controls, if the first device control corresponds to a display device, the system controls the display to show a binding guide page and obtains the number of target controls in the discovered device list. The binding guide page includes a screenshot display window, and the target controls are controls whose corresponding devices are display devices. If the number is greater than 1, a screenshot request is sent to the first device, so that the first device captures a screen image based on the screenshot request and uploads the screen image to the server. The screen image is retrieved from the server and displayed in the screenshot display window; If a confirmation binding operation is received from the user, the first device is bound to the user account so that the user can control the first device through the terminal.

2. The terminal according to claim 1, characterized in that, The controller executes the sending of a screenshot request to the first device, so that the first device captures a screen image based on the screenshot request and uploads the screen image to the server, and is further configured to: If the second device corresponding to the second device control in the at least one device control has the same IP address as the first device, a screenshot request is sent to the server, so that the server sends the screenshot request to the first device based on the device identifier and receives the screenshot data uploaded by the first device after capturing a screen image based on the screenshot request. The screenshot request includes a device identifier and an image identifier, and the screenshot data includes a screen image and an image identifier.

3. The terminal according to claim 1, characterized in that, The controller executes the sending of a screenshot request to the first device, so that the first device captures a screen image based on the screenshot request and uploads the screen image to the server, and is further configured to: If the IP address of the second device corresponding to the second device control in the at least one device control is different from that of the first device, a screenshot request is sent to the first device based on the IP address of the first device, so that the first device can capture a screen image and upload the screenshot data based on the screenshot request. The screenshot request includes an image identifier, and the screenshot data includes a screen image and an image identifier.

4. The terminal according to claim 2 or 3, characterized in that, The controller is configured to retrieve the screen image from the server. Send an image request to the server, the image request including an image identifier; Receive the screen image sent by the server based on the image identifier.

5. The terminal according to claim 1, characterized in that, The controller is configured as follows: If the number is not greater than 1, the placeholder identifier of the screenshot display window is set to a preset value so that the screenshot display window is not displayed on the binding guide page.

6. The terminal according to claim 1, characterized in that, The screenshot request includes a preset instruction, which instructs the first device to perform a preset operation corresponding to the preset instruction. The preset operation is used to change the user interface of the display device.

7. The terminal according to claim 1, characterized in that, If the controller receives a confirmation binding operation from the user, it binds the first device to the user account, and is further configured as follows: If it is detected that the login user account of the first device is the same as the login user account of the terminal, the binding control is displayed on the binding guide page; If a user inputs an instruction to select the binding control is received, the first device is bound to the user account logged in on the terminal.

8. The terminal according to claim 7, characterized in that, The controller is configured as follows: If it is detected that the first device is not logged into a user account, a first guidance box is displayed on the binding guidance page. The first guidance box includes guidance information, which is used to guide the user to log in to the user account logged in on the first device.

9. The terminal according to claim 7, characterized in that, The controller is configured as follows: If it is detected that the user account logged in on the first device is different from the user account logged in on the terminal, a second guide box is displayed on the binding guide page. The second guide box includes adding a family control and rebinding the control. If a user inputs an instruction to join a family control, the user account logged in on the terminal will be added to the family of the user account logged in on the first device, so that the user account logged in on the terminal can establish a connection with the first device; If the user inputs a selection to rebind the control, a first guide box is displayed on the binding guide page.

10. A device binding method applied to a terminal, the terminal including a display and a controller, characterized in that, The controller is configured as follows: After logging into the user account, in response to the user's command to add a device, the device identifier of the discovered device and the device already bound to the user account is obtained; If the device identifier of the discovered device is different from the device identifier of the device already bound to the user account, the display is controlled to show a list of discovered devices, the list of discovered devices including at least one device control; In response to a user input instruction to select a first device control among the at least one device controls, if the first device control corresponds to a display device, the system controls the display to show a binding guide page and obtains the number of target controls in the discovered device list. The binding guide page includes a screenshot display window, and the target controls are controls whose corresponding devices are display devices. If the number is greater than 1, a screenshot request is sent to the first device, so that the first device captures a screen image based on the screenshot request and uploads the screen image to the server. The screen image is retrieved from the server and displayed in the screenshot display window; If a confirmation binding operation is received from the user, the first device is bound to the user account so that the user can control the first device through the terminal.

Images