Display device and device control method

Abstract

A display apparatus (200) and an apparatus control method. The device control method comprises: a display device (200) establishing communication with a first controlled device based on a first protocol (S701); creating a second endpoint and a second service function corresponding to the first controlled device, wherein the second service function corresponds to a service capability supported by the first controlled device (S702); adding the second endpoint and the second service function to the endpoint information (S703); providing a second endpoint and a second service function to the master device (100) after network distribution with the master device (100) based on the Matter protocol (S704); receiving a first instruction sent by the master control device (100) to the first controlled device, converting the first instruction into a second instruction supporting the first protocol, and sending the second instruction to the first controlled device (S705); therefore, the control of the main control device (100) on the non-Matter device can be realized.

Description

Display device and device control method

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This disclosure claims priority to Chinese applications filed on August 14, 2023, with application number 202311016639.4; filed on December 27, 2023, with application number 202311831382.8; and filed on December 29, 2023, with application number 202311865403.8, the entire contents of which are incorporated by reference into this disclosure. Technical Field

[0003] The present disclosure relates to the technical field of Internet of Things, and in particular to a display device and a device control method. Background Art

[0004] The Matter protocol is an open-source standard for smart homes, developed, certified, and promoted by the Connectivity Standards Alliance. IoT devices that support the Matter protocol are called Matter devices, and Matter devices can interconnect and communicate with each other. A master device can configure network configuration for controlled devices based on the Matter protocol, adding them to the master device's Matter network, or virtual domain (Fabric). Once a controlled device joins the master device's virtual domain, the master device gains control over it, effectively granting it full control over the controlled device.

[0005] However, for non-Matter devices that don't support the Matter protocol, the master device can't configure the network for them based on the Matter protocol, nor can it add them to its virtual domain, making it impossible to control them. Consequently, the IoT based on the Matter protocol is incompatible with non-Matter devices.

[0006] Summary of the Invention

[0007] In a first aspect, the present disclosure provides a display device comprising: a display configured to display an image and / or a user input interface; a user input interface configured to receive instructions from a user; a communication device configured to communicate with an external device according to a predetermined protocol; a memory configured to: store endpoint information, the endpoint information comprising a first endpoint and a first business function corresponding to the first endpoint, each of the first endpoints corresponding to a device type defined in the Matter protocol, the first endpoint comprising a designated endpoint corresponding to a bridging device type defined in the Matter protocol, the designated endpoint corresponding to the first business function comprising the display device supporting as a bridging device; at least one processor connected to the display, the user input interface, the communication device and the memory, and configured to execute computer instructions so that the The display device executes: establishing communication with a first controlled device based on the first protocol, the first controlled device is a non-Matter device and supports the first protocol that represents the non-Matter protocol; creating a second endpoint corresponding to the first controlled device and a second business function corresponding to the second endpoint, the second business function corresponding to the business capability supported by the first controlled device; adding the second endpoint and the second business function to the endpoint information, and providing the second endpoint and the second business function to the master device after networking with the master device based on the Matter protocol; receiving a first instruction sent by the master device to the first controlled device, converting the first instruction into a second instruction that supports the first protocol, and sending the second instruction to the first controlled device.

[0008] In the second aspect, the present disclosure also provides a device control method, including: establishing communication with a first controlled device based on a first protocol, the first controlled device is a non-Matter device and supports the first protocol that characterizes the non-Matter protocol; creating a second endpoint corresponding to the first controlled device and a second business function corresponding to the second endpoint, the second business function corresponding to the business capability supported by the first controlled device; adding the second endpoint and the second business function to the endpoint information, and providing the second endpoint and the second business function to the master device after networking with the master device based on the Matter protocol; receiving a first instruction sent by the master device to the first controlled device, converting the first instruction into a second instruction that supports the first protocol, and sending the second instruction to the first controlled device. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] FIG1 is a schematic diagram of a usage scenario of a master device and a controlled device according to some embodiments;

[0010] FIG2 is a configuration diagram of a display device according to some embodiments;

[0011] FIG3 is a diagram illustrating an operating system configuration of a display device according to some embodiments;

[0012] FIG4 is a flow chart of network configuration between a master device and a display device according to some embodiments;

[0013] FIG5 is a schematic diagram of controlling a non-Matter device according to some embodiments;

[0014] FIG6 is a schematic diagram of an endpoint information list according to some embodiments;

[0015] FIG7 is a flowchart illustrating a display device supporting a master device to control a non-Matter device according to some embodiments;

[0016] FIG8 is a schematic diagram illustrating a display device supporting a master device to control a non-Matter device according to some embodiments;

[0017] 9 is a flow chart showing a device creating a second endpoint and a second service function according to some embodiments;

[0018] FIG10 is a schematic diagram of a first controlled device list according to some embodiments;

[0019] FIG11 is a flow chart showing a display device creating a second service function according to some embodiments;

[0020] FIG12 is a schematic diagram of adding a page according to some embodiments;

[0021] 13 is a flow chart showing a device creating a second endpoint and a second service function according to some embodiments;

[0022] FIG14 is a schematic diagram of a controlled device list according to some embodiments;

[0023] FIG15 is another schematic diagram of a controlled device list according to some embodiments;

[0024] FIG16 is a schematic diagram of a usage scenario of a display device according to some embodiments;

[0025] FIG17 is a schematic diagram of controlling a first controlled device through a display device according to some embodiments;

[0026] FIG18 is another schematic diagram of controlling a first controlled device through a display device according to some embodiments;

[0027] FIG19 is a flow chart showing a display device continuing to add non-Matter devices according to some embodiments;

[0028] FIG20 is a schematic diagram of adding a second device according to some embodiments;

[0029] FIG21 is a schematic diagram of deleting a target device on a display device side according to some embodiments;

[0030] FIG22 is another flowchart of supporting a master device controlling a non-Matter device according to some embodiments;

[0031] FIG23 is another schematic diagram of controlling a first controlled device through a display device according to some embodiments;

[0032] FIG24 is a schematic diagram illustrating deleting a non-Matter device from a master device according to some embodiments;

[0033] FIG25 is a schematic diagram of another usage scenario of a display device according to some embodiments;

[0034] FIG26 is a flow chart of performing network configuration on a second controlled device according to some embodiments;

[0035] FIG27 is a flow chart illustrating generating network configuration device information through voice recognition according to some embodiments;

[0036] FIG28 is a schematic diagram of a process of a master device scanning a pairing code according to some embodiments;

[0037] FIG29 is a flowchart of pairing based on network distribution device information within a first preset range according to some embodiments;

[0038] FIG30 is a flowchart of determining pairing failure based on the first pairing duration according to some embodiments;

[0039] FIG31 is a flowchart of refreshing the first pairing duration according to added network configuration device information according to some embodiments;

[0040] FIG32 is a flowchart of pairing based on network distribution device information within a second preset range according to some embodiments;

[0041] FIG33 is a flow chart illustrating pairing based on network pairing feedback information according to some embodiments;

[0042] FIG34 is a flow chart of a device control method according to some embodiments;

[0043] FIG35 is a schematic diagram of an options interface according to some embodiments;

[0044] FIG36 is a flowchart of a process of establishing a communication connection between a display device and a controlled device according to some embodiments;

[0045] FIG37 is a schematic diagram illustrating a display device establishing a communication connection with a controlled device according to some embodiments;

[0046] FIG38 is another schematic diagram of establishing a communication connection between a display device and a controlled device according to some embodiments;

[0047] FIG39 is another schematic diagram of establishing a communication connection between a display device and a controlled device according to some embodiments;

[0048] FIG40 is a schematic diagram of viewing and controlling a controlled device according to some embodiments;

[0049] FIG41 is a schematic diagram of viewing and controlling a controlled device according to some embodiments;

[0050] FIG42 is another schematic diagram of viewing and controlling a controlled device according to some embodiments;

[0051] FIG43 is another schematic diagram of viewing and controlling a controlled device according to some embodiments;

[0052] FIG44 is a schematic diagram of a shortcut key configuration interface according to some embodiments;

[0053] FIG45 is a schematic diagram of determining a control object according to some embodiments;

[0054] FIG46 is another schematic diagram of determining a control object according to some embodiments;

[0055] FIG47 is a schematic diagram of controlling multimedia resources of a display device and a controlled device according to some embodiments;

[0056] FIG48 is another schematic diagram of controlling multimedia resources of a display device and a controlled device according to some embodiments;

[0057] FIG49 is a schematic structural diagram of a device control apparatus according to some embodiments. DETAILED DESCRIPTION

[0058] The Matter protocol is an open-source standard for smart homes, developed, certified, and promoted by the Connectivity Standards Alliance. IoT devices that support the Matter protocol are called Matter devices, and Matter devices can interconnect and communicate with each other. A master device can configure network configuration for controlled devices based on the Matter protocol, adding them to the master device's Matter network, or virtual domain (Fabric). Once a controlled device joins the master device's virtual domain, the master device gains control over it, effectively granting it full control over the controlled device.

[0059] The master device and controlled device provided in the embodiments of the present disclosure may have various implementation forms. For example, the master device and the controlled device may be display devices or non-display devices. The embodiments of the present disclosure are described by taking the example that both the master device and the controlled device are display devices with display functions.

[0060] The master device (Controller) has a corresponding virtual domain (Fabric), which includes the master device and all controlled devices controlled by the master device, that is, the IoT devices that join the virtual domain of the master device. The master device and the controlled devices belonging to the same virtual domain are connected to the same local area network (that is, the designated network). The devices in the virtual domain are identified by the node operation certificate (NOC) they hold to identify the virtual domain they belong to and their identity in the virtual domain. When the master device configures the network for the controlled device, it assigns a NOC certificate to the controlled device to mark the addition of the controlled device to its own virtual domain. The NOC certificate held by each device in the virtual domain includes: a virtual domain identifier for uniquely identifying the virtual domain, such as a Fabric ID, and a node identifier for uniquely identifying the device in the virtual domain, such as a Node ID. In other words, the virtual domains of different master devices have different Fabric IDs, and the Node IDs of each device in the same virtual domain are different, but the Node IDs of devices belonging to different virtual domains can be the same. The same LAN may include one or more virtual domains, and there may be overlaps between the virtual domains.

[0061] Figure 1 is a schematic diagram of a usage scenario of a master device and a controlled device according to an embodiment. Taking the master device adding a display device to its own virtual domain as an example, as shown in Figure 1, the master device 100 adds the display device 200 to its own virtual domain, such as Fabric1. The master device 100 has control authority over the display device 200, that is, the user can control the display device 200 through the master device 100. The NOC certificate held by the master device 100 includes Fabric ID: 1 and Node ID: 1. The NOC certificate allocated by the master device 100 to the display device 200 includes Fabric ID: 1 and Node ID: 2.

[0062] The display device 200 can have various implementation forms, for example, it can be a smart TV, laser projection equipment, monitor, electronic bulletin board, electronic table, etc., or it can be a device with a display screen such as a mobile phone, tablet computer, smart watch, etc.

[0063] FIG. 2 is a block diagram illustrating a configuration of a display device 200 according to an exemplary embodiment.

[0064] The display device 200 includes at least one of a tuner and demodulator 210 , a communication device 220 , a detector 230 , an external device interface 240 , a processor 250 , a display 260 , an audio output interface 270 , a memory, a power supply, and a user input interface.

[0065] In some embodiments, the display device 200 can establish communication with the main control device 100 and other devices via the communication device 220 to send and receive control signals and data signals. In some embodiments, the detector 230 includes a sound collector. The display device 200 can collect user voice via the sound collector and transmit the collected user voice to the processor 250 for processing. In some embodiments, the processor 250 and the tuner 210 can be located in different separate devices. That is, the tuner 210 can also be located in an external device to the main device where the processor 250 is located, such as an external set-top box. In some embodiments, the processor 250 controls the operation of the display device and responds to user operations through various software control programs stored in the memory. The processor 250 controls the overall operation of the display device 200. In some embodiments, the processor 250 includes at least one of a central processing unit (CPU), a video processor, an audio processor, a graphics processing unit (GPU), random access memory (RAM), read-only memory (ROM), first to nth interfaces for input / output, and a communication bus. In some embodiments, the display 260 includes a display screen component for presenting web pages, a driver component for driving image display, a component for receiving image signals output from the processor 250, and a component for displaying video content, image content, and a menu control interface, as well as a user control UI interface. In some embodiments, the display 260 may be a liquid crystal display, an OLED display, or a projection display, or may be a projection device and projection screen. In some embodiments, a user may enter user commands through a graphical user interface (GUI) displayed on the display 260, and the user input interface receives the user input commands through the GUI. Alternatively, the user may enter user commands by inputting specific sounds or gestures, and the user input interface receives the user input commands by recognizing the sounds or gestures through sensors. In some embodiments, a "user interface" is a media interface for interaction and information exchange between an application or operating system and a user, which converts information between its internal form and a form acceptable to the user. A common form of user interface is a graphical user interface (GUI), which refers to a user interface related to computer operations that is displayed graphically. It can be an interface element such as an icon, window, control, etc. displayed on the display screen of an electronic device, where the control can include at least one of the visual interface elements such as icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, etc.

[0066] Referring to FIG. 3 , in some embodiments, the operating system of the display device 200 is divided into four layers: from top to bottom, the application layer (hereinafter referred to as the "application layer"), the application framework layer (hereinafter referred to as the "framework layer"), the Android runtime and system library layer (hereinafter referred to as the "system runtime layer"), and the kernel layer. In some embodiments, at least one application runs in the application layer. These applications can be window programs, system settings programs, or clock programs that come with the operating system, such as a live broadcast application for playing live programs; or they can be applications developed by third-party developers.

[0067] In some embodiments, the main control device 100 can have various implementation forms, for example, it can be a smart TV, a laser projection device, a monitor, an electronic whiteboard (electronic bulletin board), an electronic table, etc., or it can be a device with a display screen, such as a mobile phone, a tablet computer, a smart watch, etc. When the main control device 100 is a device with a display screen, reference can be made to the display device 200 shown in Figures 2 and 3, and detailed description is omitted here.

[0068] The main control device 100 and the display device 200 are both Matter devices that support the Matter protocol. The main control device 100 and the display device 200 can be networked according to the process shown in Figure 4. The specific steps are as follows:

[0069] S401, the display device 200 starts a device discovery service.

[0070] In some embodiments, the display device 200 can allow the control device 100 to discover itself in the network through the Domain Name System (DNS).

[0071] In some embodiments, the display device 200 can allow the control device 100 to discover itself in the network through a multicast DNS (mDNS).

[0072] In some embodiments, the display device 200 can be configured to support Near Field Communication (NFC). If the main control device 100 is configured to support NFC, the main control device 100 can be brought close to or touched with the display device 200, or the display device 200 can be brought close to or touched with the main control device 100, so that the main control device 100 can discover the display device 200 through NFC.

[0073] In some embodiments, the display device 200 can be configured to support Bluetooth Low Energy (BLE). If the master device 100 is configured to support BLE, the display device 200 can allow the master device 100 to discover itself by sending Bluetooth broadcasts to the surrounding area.

[0074] S402 : After discovering the display device 200 , the main control device 100 establishes a first connection with the display device 200 and obtains device information of the display device 200 .

[0075] After discovering the display device 200 , the master device 100 may establish a first connection based on a communication protocol supported by both devices. For example, if the master device 100 discovers the display device 200 based on BLE, the master device 100 may establish a BLE connection with the display device 200 .

[0076] When discovering the display device 200, the master device 100 can obtain the device information carried in the broadcast or request of the display device 200. After establishing a first connection with the display device 200, the master device 100 can also request the device information from the display device 200 through the first connection, or the display device 200 can actively send the device information to the master device 100 through the first connection. The device information can be basic device information of the display device 200, such as the device name.

[0077] S403: The main control device 100 and the display device 200 establish a first secure session based on the first connection.

[0078] After acquiring the device information of the display device 200 , the main control device 100 may display the device information of the display device 200 on the device search page to show the user the discovered display device 200 and basic information of the display device 200 .

[0079] If the user wants to control the display device 200 through the master device 100 , the user may input an adding instruction based on the item of the display device 200 on the device search page to instruct the master device 100 to add the display device 200 as a controlled device.

[0080] In response to the add instruction, the master device 100 starts the process of adding the display device 200 to the virtual domain. To ensure the security of the process, the master device 100 establishes a first secure session with the display device 200 to perform the process of adding the display device 200 to the virtual domain in the first secure session.

[0081] The first security session can be a Passcode-Authenticated Session Establishment (PASE) session. The main control device 100 can request password information from the display device 200 based on the first connection, or the display device 200 can actively send password information to the display device 200 based on the first connection. The password information is used to establish the PASE session.

[0082] S404 : The main control device 100 performs device authentication on the display device 200 .

[0083] In the first secure session, the main control device 100 first performs device authentication on the display device 200 before adding the display device 200 to the virtual domain to ensure the security of the added display device 200.

[0084] For example, the master device 100 can request a Digital Access Certificate (DAC) from the display device 200. A DAC certificate is a certificate specified by the Matter protocol and issued by a specialized organization. Each device has a unique DAC, which must be burned into the device in advance during factory production. Thus, the master device 100 can authenticate the display device 200 based on the DAC certificate of the display device 200.

[0085] The display device 200 sends a DAC certificate to the master device 100 in response to the request of the master device 100. After receiving the DAC certificate, the master device 100 authenticates the DAC certificate. For example, the master device 100 can send the DAC certificate to a server, such as a Distributed Control Language (DCL) server. The DCL server is a server of the alliance proposed by Matter and can be used to verify the legitimacy of the DAC certificate. If the DAC certificate is legal, the display device 200 has passed the Matter certification; if the DAC certificate is not legal or the DAC certificate sent by the display device 200 is not received, the display device 200 has not passed the Matter certification.

[0086] S405 , after the display device 200 passes the device authentication, the main control device 100 allocates a NOC certificate to the display device 200 , and performs network configuration for the display device 200 when the display device 200 has not joined the designated network.

[0087] If the display device 200 passes the device authentication, the master device 100 assigns the NOC certificate of the virtual domain in which the display device 200 is located. The NOC certificate includes the Fabric ID of the virtual domain and the Node ID of the display device 200 in the virtual domain. Please refer to Figure 1. The master device 100 assigns the NOC certificate to the display device 200, marking it to join the virtual domain in which the display device 200 is located.

[0088] The designated network can be a network accessed by the master device 100. If the display device 200 has not yet joined the designated network, that is, the display device 200 has not yet accessed the same network as the master device 100, for example, the display device 200 is discovered by the master device 100 through DNS, BLE, NFC, etc., then the master device 100 sends network configuration information to the display device 200, such as the network name and password of the designated network, so that the display device 200 can access the designated network according to the network configuration information.

[0089] If the display device 200 is discovered by the master device 100 through mDNS, it can be considered that the display device 100 has connected to the designated network, that is, has connected to the same network as the master device 100.

[0090] S406 , the main control device 100 and the display device 200 establish a second connection based on the designated network, and establish a second secure session based on the second connection according to the NOC certificates held by each of them.

[0091] The designated network may be a WiFi network, and the main control device 100 and the display device 200 may establish a second connection based on the WiFi network to transmit control instructions and a larger amount of data through the second connection. To ensure the security of transmitting control instructions and data based on the second connection, a second secure session may be established based on the second connection to transmit control instructions and data under the second secure session.

[0092] The second secure session can be a Certificate-Authenticated Session Establishment (CASE) session, and the master device 100 and the display device 200 can establish the second secure session based on the NOC certificates they each hold. In a CASE session, all instructions and data transmitted between the master device 100 and the display device 200 need to be encrypted.

[0093] S407 : In the second secure session, the main control device 100 controls the display device 200 .

[0094] In the second secure session, the main control device 100 sends corresponding control instructions to the display device 200 in response to user operations to control the display device 200 .

[0095] Control permissions can be View, Operate, Manage, or Administer. View permissions allow you to read or subscribe to data, Operate permissions allow you to write operation data and invoke operation commands, Manage permissions allow you to write configuration data and invoke configuration commands, and Administer permissions allow you to write management data and invoke management commands.

[0096] The master device 100 may be configured to have the highest control authority, such as Administer.

[0097] From the above process, it can be seen that the master device 100 can add the controlled device to the virtual domain, configure the network for the controlled device, and control the controlled device on the basis that the controlled device needs to support the Matter protocol, that is, the controlled device is a Matter device. For non-Matter (no-Matter) devices that do not support the Matter protocol, the master device 100 cannot configure the network for the non-Matter device based on the Matter protocol, nor can it add the non-Matter device to its own virtual domain, and thus cannot control the non-Matter device. Therefore, the Internet of Things formed based on the Matter protocol is not compatible with non-Matter devices.

[0098] In order to solve the above problems, the present disclosure provides a display device 200, which is a Matter device, that is, it supports the Matter protocol, and the display device 200 also supports non-Matter protocols, such as the first protocol. After the main control device 100 adds the display device 200 to the virtual domain, the display device 200 can be used as a bridge (Bridge) to connect non-Matter devices. After the display device 200 establishes communication with the non-Matter device, the main control device 100 can control the non-Matter device through the display device 200, which is equivalent to bridging the non-Matter device to the virtual domain of the main control device 100. In this way, the display device 200 can be called a bridge device (Bridge Device), and the non-Matter device can be called a bridged device (Bridged Device).

[0099] Referring to the overall architecture shown in Figure 5, based on the overall architecture, the control device 100 can control non-Matter devices. As shown in Figure 5, the overall architecture includes a controller, a Matter device, and a non-Matter device.

[0100] The controller may be a main control device 100, such as a mobile phone or audio equipment installed with a corresponding App.

[0101] The Matter device can be a display device 200, and the display device 200 can be integrated with a Matter Software Development Kit (SDK), a bridge service (Bridge Server) SDK and other functions. Among them, the display device 200 can realize device discovery, network configuration, control and other functions with the Controller through the Matter protocol through the Matter SDK. The Bridge Server SDK is an implementation based on a non-Matter protocol, such as an implementation based on the first protocol. The first protocol can include the Message Queuing Telemetry Transport Protocol (MQTT), other private protocols, such as User Datagram Protocol Broadcasting (UDP) broadcast, mDNS and other network communication protocols.

[0102] In the Matter protocol, each Matter device is a node. Each node consists of many endpoints, each of which corresponds to at least one business function (cluster). This business function corresponds to the business capabilities supported by the Matter device, and the business capabilities supported by the Matter device are defined by the Matter device. For example, a Matter device supports business capabilities such as on / off, and the corresponding business function is the on / off cluster, which is the ability to turn the device on and off.

[0103] The display device 200 stores endpoint information in the memory, which includes a first endpoint and a first business function corresponding to the first endpoint. Each first endpoint corresponds to a device type (Device Type) defined in the Matter protocol, and the first business function corresponds to the business capability supported by the display device 200.

[0104] In some embodiments, the display device 200 can manage and store endpoint information in the form of a list. Reference can be made to the endpoint information list shown in Figure 6. Taking the display device 200 as a TV as an example, the endpoint information of the display device 200 includes endpoint 0, and the device type corresponding to the endpoint is the root node (Root Node). The endpoint corresponds to the cluster for implementing the root node (not shown in the figure). The display device 200 also includes endpoint 1, which is the device type of the implemented TV. As a node of the TV function, the device type can be a basic controlled implementation, such as a basic video player (basic video player) / casting video player (casting video player). The endpoint corresponds to the cluster for implementing the TV function (not shown in the figure).

[0105] The first endpoint also includes a designated endpoint, which corresponds to a bridging device type defined in the Matter protocol, such as Aggregator. The first service function corresponding to the designated endpoint includes the display device 200 supporting a cluster as a bridging device. For example, as the designated endpoint being the Nth endpoint of the display device 200, as shown in FIG6 , the endpoint information of the display device 200 also includes endpoint n, corresponding to the device type Aggregator, which corresponds to a cluster used to implement bridging (not shown in the figure).

[0106] Other endpoints of the display device 200 may be defined and configured according to the Matter standard based on the manufacturer's implementation.

[0107] Non-Matter devices can be devices such as lights and TVs that do not support the Matter protocol. Non-Matter devices integrate non-Matter protocol implementations, such as bridge devices (Bridge Device SDK). The Bridge Device SDK is also based on non-Matter protocol implementations, such as the first protocol. The first protocol is not described here.

[0108] The main control device 100 can control the non-Matter device through the display device 200, including: the display device 200 discovers and connects to the non-Matter device, and controls the non-Matter device through the display device 200.

[0109] Example 1

[0110] The process of the display device 200 discovering and connecting with a non-Matter device may not be controlled by the master device 100. During this process, the display device 200 may not have been networked with the master device 100, or may have completed network configuration with the master device 100 and joined the virtual domain of the master device 100.

[0111] The display device 200 can support the master device 100 to control non-Matter devices according to the process shown in FIG7 . Combined with the underlying interaction diagram shown in FIG8 , the process of the display device 200 supporting the master device 100 to control non-Matter devices is specifically described:

[0112] S701: Establish communication with a first controlled device based on a first protocol.

[0113] The first controlled device is a non-Matter device and supports the first protocol.

[0114] The display device 200 and non-Matter devices perform device discovery through the Bridge Server and Bridge Device respectively. Before establishing communication with a non-Matter device, the display device 200 initializes the protocol stack of the Matter SDK and initializes the Bridge Server, and the Bridge Server subscribes to the Bridge Device.

[0115] After establishing a connection (communication) with a non-Matter device, the display device 200 stores and manages the device information and device status of these non-Matter devices. The device status of the non-Matter device includes the online status and the offline status. If the non-Matter device is in the online state, the display device 200 can interact with the non-Matter device; if the non-Matter device is in the offline state, the display device 200 cannot interact with the non-Matter device. After starting the Bridge Server, the display device 200 determines the device status of each non-Matter device, such as the currently online non-Matter device, the subsequent newly online non-Matter device, and the subsequent offline non-Matter device. If the display device 200 finds other non-Matter devices, it can query whether the non-Matter device has been connected based on the stored device information of the non-Matter device. If not, the display device 200 and the non-Matter device are connected based on the implementation of the Bridge Server and the Bridge Device, and after the connection is established, the device information of the non-Matter device is stored and managed.

[0116] Taking a non-Matter device as the first controlled device and the display device 200 finding out that the non-Matter device has never been connected as an example, a process of establishing communication between the display device 200 and the first controlled device is described.

[0117] In some embodiments, the display device 200 may search for surrounding devices based on the first protocol in response to a user's search instruction, and establish communication with the first controlled device after discovering the first controlled device.

[0118] When the user needs to add control over the first controlled device, the user can send a search instruction to the display device 200 to instruct the display device 200 to search for the first controlled device.

[0119] For example, a user can send a search instruction to the display device 200, which instructs the display device 200 to search for all surrounding devices. After discovering the surrounding devices, the display device 200 can display a device discovery list, which can include items of all discovered non-Matter devices. If the user finds the first controlled device that he wants to control among these discovered non-Matter devices, he can input an instruction based on the item of the first controlled device to instruct the display device 200 to establish a connection with the first controlled device.

[0120] For another example, a user may send a search command to the display device 200, instructing the display device 200 to discover a first controlled device. The user may send the search command to the display device 200 through a first operation. For example, if both the display device 200 and the first controlled device support NFC, the first operation may be to bring the first controlled device close to or touch the display device 200. After discovering the first controlled device, the display device 200 may establish communication with the first controlled device based on the first protocol.

[0121] In some embodiments, the display device 200 can establish communication with the first controlled device in response to a pairing request sent by the first controlled device. Without the user performing additional operations on the display device 200, the display device 200 can automatically receive the pairing request sent by the first controlled device and establish communication with the first controlled device based on the first protocol. After receiving the pairing request sent by the first controlled device, the display device 200 can display an inquiry message, which is used to ask the user whether to establish a connection with the first controlled device, and then establish communication with the first controlled device based on the first protocol when the user indicates that the connection can be established. After receiving the pairing request sent by the first controlled device, the display device 200 can also directly establish communication with the first controlled device based on the first protocol without the user's perception, so as to reduce the user's operation.

[0122] When establishing communication with the first controlled device, the display device 200 can perform device verification on the first controlled device. After the first controlled device passes the verification, communication with the first controlled device is established to ensure the device reliability of the first controlled device and the information security between the display device 200 and the first controlled device.

[0123] Based on different first protocols, the process of establishing a connection between the display device 200 and the first controlled device is also different, which is exemplified by combining the following first protocols.

[0124] The first protocol is the MQTT protocol, and the shared topic (Shared Topic) in the MQTT protocol can be used for device discovery. Based on the shared topic of the MQTT protocol, the first controlled device can publish to a shared topic, and the display device 200 can discover the first controlled device by subscribing to the shared topic. For example, the first controlled device can publish to the shared topic "device uuid / + / status", and the display device 200 can obtain the device list of the shared topic by subscribing to "device uuid / + / status". The device list includes the devices that publish to the shared topic, and these devices include the first controlled device. In this way, the display device 200 can discover the first controlled device. Among them, the shared topic can be customized, and this embodiment does not limit it. After discovering the first controlled device, the display device 200 establishes communication with the first controlled device.

[0125] The first protocol is a specialized device discovery service. This specialized device discovery service can be implemented using UDP broadcast, mDNS protocol, or the like. The first controlled device can broadcast its presence through these specialized device discovery service mechanisms, and the display device 200 can discover the first controlled device by listening to the broadcast messages of these device discovery services. For example, after the display device 200 listens to the broadcast messages sent by the first controlled device, it determines that it has discovered the first controlled device.

[0126] S702: Create a second endpoint corresponding to the first controlled device and a second service function corresponding to the second endpoint, where the second service function corresponds to a service capability supported by the first controlled device.

[0127] Since the display device 200 has a designated endpoint, namely, the Aggregator, the display device 200 has the ability to create and add endpoints of non-Matter devices.

[0128] The display device 200 creates a corresponding endpoint (ie, a second endpoint) and a service function (ie, a second service function) corresponding to the endpoint for the first controlled device in the form of a standard Matter device type.

[0129] The display device 200 may create a second endpoint and a second service function according to the process shown in FIG8 . The specific steps are as follows:

[0130] S801: Acquire device information of a first controlled device, where the device information includes a device type of the first controlled device and service capabilities supported by the first controlled device.

[0131] In some embodiments, the first controlled device may send device information to the display device 200 before establishing communication with the display device 200, such as by including the device information in an outbound broadcast sent by the first controlled device. In some embodiments, the display device 200 may request device information from the first controlled device after establishing communication with the first controlled device. In some embodiments, the first controlled device may proactively send device information to the display device 200 after establishing communication with the display device 200.

[0132] The device information includes at least the device type and supported service capabilities of the first controlled device. The device information may also include other attribute information of the first controlled device, such as the device name, the device's unique ID, the device's location, the manufacturer, and the like.

[0133] S802: Create a second endpoint corresponding to the first controlled device according to the Matter protocol, the device type of the first controlled device, and the service capabilities supported by the first controlled device, and create a second service function corresponding to the second endpoint according to the service capabilities supported by the first controlled device.

[0134] In some embodiments, after obtaining the device information of the first controlled device, the display device 200 directly creates the second endpoint based on the Matter protocol, the device type of the first controlled device, and the service capabilities supported by the first controlled device, and creates the second service function based on the service capabilities supported by the first controlled device. In this way, the display device 200 can create the second endpoint and the second service function for the first controlled device without the user having to perform additional operations on the display device 200.

[0135] If there are multiple first controlled devices and each first controlled device supports at least one service capability, the display device 200 may create second endpoints for all first controlled devices and create second service functions for the service capabilities supported by the first controlled devices.

[0136] Alternatively, the display device 200 may create a second endpoint for a target first controlled device among the multiple first controlled devices, and create a second service function for at least one target service capability supported by the target first controlled device, according to preset conditions. The preset conditions may include default / user-preset device addition conditions and service function addition conditions. For example, the user-preset device addition condition may be to add a device of the light and switch type, and the service function addition condition may be to add an on / off function.

[0137] In some embodiments, after the display device 200 obtains the device information of the first controlled device, if the user wants to control the first controlled device through the main control device 100, and wants to control some / all business capabilities supported by the first controlled device, then in response to the user's selection, a second endpoint is created according to the Matter protocol, the device type of the first controlled device and the business capabilities supported by the first controlled device, and a second business function is created according to the business capabilities supported by the first controlled device. In this way, the display device 200 can create a second endpoint for the first controlled device in a targeted manner, and create a second business function for the business capabilities supported by the first controlled device. As a result, the main control device 100 can only control the non-Matter devices selected by the user and the business functions selected by the user through the display device 200, so as to effectively limit the control scope of the main control device 100 over non-Matter devices and avoid the main control device 100 from controlling non-Matter devices without restriction.

[0138] The display device 200 may create a second endpoint and a second service function in a targeted manner according to the process shown in FIG9 . The specific steps are as follows:

[0139] S901: Display a first controlled device list according to the device name.

[0140] The device information obtained by the display device 200 from the first controlled device includes the device name of the first controlled device. After establishing communication with the first controlled device and obtaining the device information of the first controlled device, the display device 200 displays the first controlled device list according to the device name.

[0141] The first controlled device list includes an item for the first device, with the corresponding device name displayed on the item for the first device. The first device is a non-Matter device that establishes communication with the display device 200 based on the first protocol and does not have an endpoint created on the display device 200. The first device includes the first controlled device, that is, the first controlled device list includes an item for the first controlled device, with the device name of the first controlled device displayed on the item.

[0142] After obtaining the device information of the first device, the display device 200 stores and manages the device information of the first device. The process of the display device 200 obtaining the device information of the first device and establishing communication with the first device can refer to step S701 and will not be repeated here. The display device 200 can display a first controlled device list based on the device information of each first device. Referring to the first controlled device list shown in Figure 10, the first controlled device list includes items of each first device and displays the device name of the first device, such as switch (Switch), humidifier and lamp, where the lamp is the first controlled device.

[0143] In some embodiments, the first controlled device list may also include an item for a third device, with the corresponding device name displayed on the item for the third device. The third device is a non-Matter device that has established communication with the display device 200 based on the first protocol and has created an endpoint on the display device 200. In order to distinguish between the first and third devices, a first identifier may be carried on the item for the first device, and a second identifier may be carried on the item for the third device. Referring to Figure 10, the first controlled device list also includes a video player, which is a third device. A first identifier, such as an add button, is displayed on the items for the switch, humidifier, and lamp to prompt the user through the first identifier to create an endpoint for the corresponding device on the display device 200, that is, to add the corresponding device to a controlled device of the master device 100. A second identifier, such as a delete button, is displayed on the item for the video player to prompt the user through the second identifier to delete the endpoint of the corresponding device from the display device 200, that is, to cancel the control of the master device 100 over the corresponding device.

[0144] In some embodiments, the first controlled device list may further include an item for a fourth device, on which the corresponding device name is displayed. The fourth device is a Matter device that establishes communication with the display device 200 based on the Matter protocol. In order to identify the fourth device, a third identifier may be carried on the item for the fourth device. Referring to Figure 10, the first controlled device list also includes a mobile phone, which is the fourth device. A third identifier, such as "matter", is displayed on the item for the mobile phone to prompt the user that the corresponding device is a Matter device through the third identifier.

[0145] In some embodiments, the first device, the third device, and the fourth device may be displayed in partitions, as shown in the first controlled device list in Figure 10. In this way, the user can quickly find the corresponding device by browsing different partitions and perform corresponding operations.

[0146] In some embodiments, the display device 200 may directly display the first controlled device list after establishing communication with the first controlled device and obtaining the device name of the first controlled device.

[0147] If the display device 200 obtains the device information of the first controlled device before establishing communication with the first controlled device, the display device 200 directly displays the first controlled device list after establishing communication with the first controlled device. For example, after discovering surrounding devices, the display device 200 may display the device discovery list, and in response to a user input of an add instruction based on the item of the first controlled device, establish communication with the first controlled device, and directly display the first controlled device list after establishing communication with the first controlled device.

[0148] If the display device 200 acquires the device information of the first controlled device after establishing communication with the first controlled device, the display device 200 directly displays the first controlled device list after acquiring the device information of the first controlled device.

[0149] In some embodiments, the display device 200 may display the first controlled device list in response to a user instruction. For example, the display device 200 may set an entry for the first controlled device list in a menu, and the user may select the entry through a control device of the display device 200, such as a remote control, to cause the display device 200 to display the first controlled device list.

[0150] S902, in response to an add instruction input by the user based on the project of the first controlled device, create a second endpoint according to the Matter protocol, the device type of the first controlled device and the business capabilities supported by the first controlled device, and create a second business function according to the business capabilities supported by the first controlled device.

[0151] If the user wants to control the first controlled device through the main control device 100, the user can determine the first controlled device by browsing the first controlled device list, and input an add instruction to the display device 200 based on the item of the first controlled device, so as to instruct the display device 200 to create a second endpoint and a second business function for the first controlled device. Taking the first controlled device list shown in Figure 10 as an example, the user can input an add instruction based on the add button corresponding to the light. Of course, if the user wants to control other first devices, such as a switch, through the main control device 100, the user can input an add instruction based on the add button corresponding to the switch. The display device 200 can configure a multi-select function for the first controlled device list, that is, the user can select multiple first devices at the same time to send an add instruction based on these first devices, instructing the display device 200 to create a second endpoint and a second business function for these first devices respectively.

[0152] The display device 200 creates a second endpoint and a second service function for the first controlled device in response to the user's adding instruction.

[0153] In some embodiments, the display device 200 creates a second endpoint for the first controlled device in response to the user's addition instruction, and creates a second business function corresponding to all business capabilities supported by the first controlled device, or creates a second business function according to preset conditions. The preset conditions may include default / user-preset business function addition conditions. For example, if the user-preset business function addition condition is to add an on / off function, the display device 200 only creates corresponding on / off functions for the on / off business capabilities supported by the first controlled device.

[0154] In some embodiments, the display device 200 can create a corresponding second service function for the second endpoint according to the user's selection. The display device 200 can create the second service function according to the process shown in Figure 11, and the specific steps are as follows:

[0155] S1101 , in response to an add instruction, creating a second endpoint according to the Matter protocol, the device type of the first controlled device, and the service capabilities supported by the first controlled device, and displaying an add page according to the service capabilities supported by the first controlled device.

[0156] In response to the user input of the add instruction based on the project of the first controlled device, the display device 200 creates a second endpoint for the first controlled device and displays an add page corresponding to the first controlled device, the add page including the service capabilities supported by the first controlled device.

[0157] Taking the first controlled device as a lamp, and the business capabilities supported by the lamp including on / off capability and Dimmable Light as an example, if the user enters an add instruction based on the add button corresponding to the lamp in Figure 10, the display device 200 displays the add page as shown in Figure 12, which includes on / off and Dimmable Light items, as well as the add button corresponding to each item.

[0158] S1102 : In response to a selection instruction input by the user based on the target business capability, create a corresponding second business function.

[0159] The user browses the add page to determine the service capability of the first controlled device that they want to control through the master device 100, namely the target service capability, and inputs a selection instruction based on the item of the target service capability to instruct the display device 200 to create a second service function corresponding to the target service capability. The display device 200 will not create corresponding service functions for service capabilities supported by the first controlled device that are not selected by the user.

[0160] Referring to the adding page shown in FIG12 , if the user wants to control the on / off capability of the light through the main control device 100, the user can input a selection instruction based on the adding button corresponding to the on / off capability. The display device 200 responds to the selection instruction and creates an on / off function corresponding to the on / off capability, but does not create a corresponding Dimmable Light function for the Dimmable Light capability.

[0161] The display device 200 may create a second endpoint and a second service function according to the process shown in FIG13 . The specific steps are as follows:

[0162] S1301: Query the device type list in the Matter protocol.

[0163] The Device Type list includes a first definition of a device type, which includes a device ID corresponding to the device type and a device name corresponding to the device ID. The device name corresponds to at least one service capability. The at least one service capability includes a service capability that matches the device name and service capabilities corresponding to each device name that precedes the device name in the Device Type list.

[0164] For example, if the device types include lighting and media, the device type list can refer to Table 1, as follows:

[0165] Table 1 Device types defined in the Matter protocol

[0166] Table 1 shows that the device IDs corresponding to the device type light include 0x0100, 0x0101, 0x010C, and 0x010D. The device name corresponding to 0x0100 is "On / Off Light," and its corresponding at least one service capability includes "On / Off." The device name corresponding to 0x0101 is "Dimmable Light," and its corresponding at least one service capability includes "On / Off" and "Dimmable." The device name corresponding to 0x010C is "Color Temperature Light," and its corresponding at least one service capability includes "On / Off," "Dimmable," and "Color Temperature." The device name corresponding to 0x010D is "Extended Color Light," and its corresponding at least one service capability includes "On / Off," "Dimmable," "Color Temperature," and "Extended Color."

[0167] S1302: Determine a target device identifier and a target device name corresponding to the first controlled device according to the device type list, the device type of the first controlled device, and the service capabilities supported by the first controlled device.

[0168] The at least one service capability corresponding to the target device name corresponding to the first controlled device includes a service capability supported by the first controlled device. The service capability supported by the first controlled device is the service capability to be controlled by the master device 100. As can be seen from the foregoing, if the user does not specifically indicate the service capability to be added through the add instruction, the service capability supported by the first controlled device may be all service capabilities supported by the first controlled device, or may be the service capability determined by the display device 200 according to preset conditions. If the user indicates the service capability to be added through the add instruction, the service capability supported by the first controlled device is the service capability indicated by the user.

[0169] Taking the device type of the first controlled device as light, and the service capability supported by the first controlled device as on / off capability as an example, according to the device type of the first controlled device and the device type list, it can be determined that the device type of the first controlled device corresponds to the Matter protocol as lighting. According to the service capabilities supported by the first controlled device and the part corresponding to lighting in the device type list, it can be determined that the target device name corresponding to the first controlled device is On / Off Light, so that the service capability corresponding to On / Off Light covers the service capability supported by the first controlled device here, and the corresponding target device identifier is 0x0100.

[0170] Taking the device type of the first controlled device as a lamp, and the service capabilities supported by the first controlled device as on / off and Color Temperature as an example, based on the device type of the first controlled device and the device type list, it can be determined that the device type of the first controlled device corresponds to the Matter protocol as lighting. Based on the service capabilities supported by the first controlled device and the part corresponding to lighting in the device type list, it can be determined that the target device name corresponding to the first controlled device is Color Temperature Light, so that the service capabilities corresponding to Color Temperature Light cover the service capabilities supported by the first controlled device, and the corresponding target device identifier is 0x010C.

[0171] S1303: Create a second endpoint according to the target device identifier and the target device name.

[0172] For example, if display device 200 currently includes the first endpoint (endpoint 0-endpoint N) shown in Figure 6, the second endpoint created by display device 200 for the first controlled device is endpoint N+1. The device type corresponding to endpoint N+1 is specifically the target device name corresponding to the target device identifier. For example, if the target device identifier is 0x0100 and the target device name is On / Off Light, the device type corresponding to endpoint N+1 is On / Off Light corresponding to 0x0100.

[0173] S1304: Query the business function list in the Matter protocol.

[0174] Although the device name of the second endpoint created based on the above steps corresponds to at least one business capability, the capability range of the at least one business capability may be greater than the business capability supported by the first controlled device. In order to ensure the effectiveness of the control of the first controlled device by the master device 100 and enable the master device 100 to accurately control the business capabilities supported by the first controlled device, it is necessary to create a corresponding second business function for the first controlled device corresponding to the second endpoint.

[0175] The service function list includes a second definition of the service function, which includes a mapping between the service function and the service capability. In some embodiments, the service function list may correspond to different device types, i.e., each device type corresponds to a service function list, which includes a mapping between all service capabilities and service functions corresponding to the device type.

[0176] Taking the device type of On / Off Light as an example, the service function list can be referred to Table 2, as follows:

[0177] Table 2 On / Off Light Cluster Requirements

[0178] S1305 : Determine, based on the correspondence between the service functions and the service capabilities, a second service function corresponding to the service capability supported by the first controlled device.

[0179] Taking the example of the first controlled device supporting service capabilities including On / Off and Level Control, the display device 200 can determine the second service capabilities as On / Off Cluster and Level Control Cluster based on Table 2. In addition, the second service capabilities also include the service capabilities that On / Off Light must support based on the Matter protocol, as shown in Table 2, such as Identify Cluster, Groups Cluster, and Scenes Cluster.

[0180] S1306: Add a second service function for the second endpoint.

[0181] After determining the second service functions, these second service functions are added to the second endpoint to complete the creation of endpoint information for the second endpoint. The endpoint information at least includes the Device ID corresponding to the second endpoint and the supported clusters.

[0182] If there are multiple first controlled devices, the display device 200 may create a second endpoint and a second service function for each first controlled device according to the above process.

[0183] S703: Add the second endpoint and the second service function to the endpoint information.

[0184] After creating the second endpoint and the second business function for the first controlled device, the endpoint information (the second endpoint and the second business function) corresponding to the first controlled device is added to the endpoint information stored in the memory. Referring to Figure 6, the endpoint information corresponding to the first controlled device is added to the endpoint information list to store and manage the added endpoint information of the first controlled device.

[0185] S704: After network configuration with the master device based on the Matter protocol, provide the master device with a second endpoint and a second service function.

[0186] The process of the display device 200 performing network configuration with the main control device 100 and joining the virtual domain of the main control device 100 may refer to steps S401 to S407 and will not be described in detail here.

[0187] After joining the virtual domain of the master device 100 , the display device 200 can be controlled by the master device 100 as a controlled device of the master device 100 .

[0188] In some embodiments, the display device 200 may provide endpoint information to the display device 200 during the network configuration process with the master device 100 .

[0189] The endpoint information may include a first endpoint and a first service function, and a second endpoint and a second service function. In this way, the main control device 100 can directly obtain the endpoint information of all endpoints on the display device 200, so that the main control device 100 can directly use the obtained endpoint information when controlling each endpoint.

[0190] The endpoint information may only include the first endpoint and the first business function. When the master device 100 queries the specified endpoint with the device type of Aggregator in the endpoint information, it can determine that the display device 200 is a bridging device, and there may be a bridged device (non-Matter device) with the display device 200 as a bridging device. Therefore, the master device 100 can actively send a read request to the display device 200 to request the second endpoint and the second business function from the display device 200. In this way, the display device 200 can provide the second endpoint and the second business function to the master device 100 in a targeted manner when the master device 100 needs to control the first controlled device, so as to reduce the amount of data transmitted to the master device 100 each time.

[0191] Moreover, the process of the display device 200 providing the endpoint information to the main control device 100 does not require the user to perform additional operations, and the synchronization of the endpoint information can be completed without the user's perception, thereby simplifying the user's operation.

[0192] In some embodiments, the display device 200 may proactively provide endpoint information to the master device 100 after being networked with the master device 100 .

[0193] The endpoint information may include a first endpoint and a first service function, and a second endpoint and a second service function. In this way, the main control device 100 can directly obtain the endpoint information of all endpoints on the display device 200, so that the main control device 100 can directly use the obtained endpoint information when controlling each endpoint.

[0194] This endpoint information may only include the first endpoint and the first service function. When the master device 100 finds a designated endpoint with the device type "Aggregator" in the endpoint information, it can proactively send a read request to the display device 200 to request the second endpoint and the second service function. This is not detailed here. In this way, the display device 200 can specifically provide the master device 100 with the second endpoint and the second service function when the master device 100 needs to control the first controlled device, thereby reducing the amount of data transmitted to the master device 100 each time.

[0195] In some embodiments, the display device 200 may send the second endpoint and the second service function to the master device 100 in response to a read request sent by the master device 100 , where the read request is used to request to read endpoint information of the first controlled device.

[0196] When the main control device 100 needs to browse the bridged device using the display device 200 as the bridge device, it can send a read request to the display device 200 to request the second endpoint and the second service function from the display device 200.

[0197] In the above process, the endpoint information provided by the display device 200 to the main control device 100 may also include other attribute information such as the device name of the device corresponding to the endpoint.

[0198] After acquiring the endpoint information of the display device 200 , the main control device 100 may display a list of controlled devices according to the endpoint information.

[0199] If the endpoint information only includes the first endpoint and the first service function, the controlled device list includes an entry for display device 200. Assuming the master device 100 is a mobile phone and the display device 200 is TV1, referring to the controlled device list shown in FIG14 (1), the controlled device list includes an entry for TV1. Based on the specified endpoint included in the endpoint information, the entry for TV1 can be configured as an expandable entry. The user can expand the bridged devices with TV1 as the bridge device by selecting the entry for TV1. In response to a user clicking on the entry for TV1, the master device 100 can send a read request to TV1 to request the second endpoint, the second service function, and other attribute information of the first controlled device, such as the device name. After receiving the second endpoint, the second service function, and other attribute information from TV1, the master device 100 displays the entry for the first controlled device corresponding to the entry for TV1 based on this information. For example, if the first controlled device is TV2, referring to the controlled device list shown in FIG14 (2), after the user expands the entry for TV1, the entry for TV2 is displayed below the entry for TV1.

[0200] If the endpoint information includes a first endpoint and a first service function, and a second endpoint and a second service function, the controlled device list includes items of the display device 200 and the first controlled device, as shown in ② in FIG. 14 .

[0201] In some embodiments, to distinguish whether the display device 200 and the first controlled device support the Matter protocol, a first identifier can be displayed on the item of the first controlled device to prompt the user through the first identifier that the corresponding device is a non-Matter device. Accordingly, the device corresponding to the item without the first identifier is a Matter device.

[0202] The main control device 100 may also display a second identifier on the item on the display device 200 to remind the user that the corresponding device is a Matter device through the second identifier. Accordingly, the device corresponding to the item without the second identifier is a non-Matter device.

[0203] The master device 100 can also display a first identifier on the item for the first controlled device and a second identifier on the item for the display device 200 to more clearly distinguish between Matter devices and non-Matter devices. As shown in FIG14 , a first identifier, such as "non-matter," is displayed on the item for TV2, and a second identifier, such as "matter," is displayed on the item for TV1.

[0204] In some embodiments, the main control device 100 displays the item of the display device 200 in the list of controlled devices, and displays the item of the first controlled device in the first area corresponding to the item of the display device 200. The device corresponding to the item displayed in the first area is a non-Matter device controlled by the main control device 100 with the display device 200 as a bridge device. In this way, the user can more accurately and quickly determine the bridging relationship between the Matter device and the non-Matter device, which can facilitate the user's management of each Matter device and non-Matter device. For example, if the user no longer wants to control a non-Matter device through the main control device 100, the Matter device corresponding to the non-Matter device can be determined based on the display position of the non-Matter device in the list of controlled devices, and then the non-Matter device can be deleted by operating the Matter device.

[0205] In some embodiments, when the master device 100 displays the items of the display device 200 and the first controlled device in the controlled device list, it can categorize and display them according to the spatial area where each device is placed (such as living room, bedroom, etc.) and the function of each device (light, TV, kitchenware, bathroom, etc.). In order to distinguish whether each device supports the Matter protocol, a corresponding first identifier or second identifier can be displayed on the device item.

[0206] Still taking TV1 and TV2 in the above example as an example, you can refer to the list of controlled devices shown in Figure 15. TV1 corresponds to the first area 1501. The position of the first area 1501 on the screen is to the right rear of the TV1 item, which is in line with the user's usual display habits of subordinate relationships. The TV2 item is displayed in the first area 1501 to indicate that TV2 is the bridged device with TV1 as the bridge device.

[0207] S705 : Receive a first instruction sent by the master control device to the first controlled device, convert the first instruction into a second instruction supporting the first protocol, and send the second instruction to the first controlled device.

[0208] After pairing with the display device 200, the master device 100 can send control instructions to each endpoint of the display device 200 via the Matter protocol to control the display device 200 to execute the business function corresponding to the corresponding endpoint in response to the control instruction. Specifically, if the master device 100 wants to control the first controlled device to implement a second business function, the master device 100 can send a control instruction, such as a first instruction, to the second endpoint corresponding to the first controlled device in the display device 200 to control the display device 200 to execute the second business function corresponding to the second endpoint in response to the first instruction. Specifically, the display device 200 executes the second business function corresponding to the second endpoint in response to the first instruction by sending the first instruction to the second endpoint, and the Bridge Server calls a non-Matter interface to convert the first instruction into a second instruction based on the first protocol, and then sends the second instruction to the first controlled device via the first protocol. The first controlled device receives the second instruction through the Bridge Device and performs corresponding processing to respond to the control of the master device 100.

[0209] The process of the master device 100 sending the first instruction to the first controlled device via the display device 200 is a message transmission and processing process. Specifically, the master device 100 sends the first instruction to the display device 200. The display device 200 passes the received message to the Matter SDK, which then passes the message to the Bridge Server for message conversion and distribution to the Bridge Device. The Bridge Device then sends the received message to the first controlled device for processing. The Matter SDK then returns the result to the master device 100.

[0210] Refer to the schematic diagram of the usage scenario of the display device 200 shown in Figure 16. As shown in Figure 16, after the display device 200 establishes communication with the first controlled device 300, a second endpoint and a second service function are created for the first controlled device 300. Compared to Figure 1, after the master device 100 configures the network for the display device 200 and adds the display device 200 to Fabric 1, it can obtain endpoint information from the display device 200. This endpoint information includes the second endpoint and the second service function corresponding to the first controlled device 300. The master device 100 can send control instructions to each endpoint in the display device 200 to enable the display device 200 to implement the service function corresponding to the endpoint. As shown in Figure 16, the first controlled device 300 has not joined Fabric 1 of the master device 100. However, if the master device 100 sends a first instruction to the first controlled device 300, such as an off instruction, to the display device 200 via the Matter protocol, the display device 200 will act as a bridge device and send the off instruction to the second endpoint corresponding to the first controlled device 300, such as endpoint N+1: 0x0100, On / Off Light, where endpoint N+1 can correspond to the On / Off Cluster and the Level Control Cluster. The display device 200 can match the off instruction to a second instruction of the first protocol, such as the off instruction of the MQTT protocol, through the Bridge Server, and send the converted off instruction to the first controlled device 300 through communication with the first controlled device 300. The first controlled device 300 receives the off instruction via the Bridge Device and performs corresponding processing, such as turning off the light, in response to the control of the master device 100.

[0211] In some embodiments, when a user needs to control a controlled device through the master device 100, they can control the master device 100 to display a list of controlled devices, which includes items for the display device 200 and the first controlled device 300. By browsing the list of controlled devices, the user can determine the target device they want to control, such as the first controlled device 300. The user can click the item for the first controlled device 300 to instruct the master device 100 to display a control page or control options for the first controlled device 300, where the control page includes control options for the first controlled device. The control options for the first controlled device 300 correspond one-to-one to the second service function created for the first controlled device 300. For example, if the second service function created for the first controlled device 300 includes On / Off Cluster and Level Control Cluster, the control options include an On / Off option and a Level Control option. The user can enter a first instruction based on the control options. For example, they can click the On / Off option to enter the first instruction to the master device 100. The master device 100 sends the first instruction to the display device 200 and instructs the first instruction to act on the first controlled device 300. The display device 200 sends the first instruction to the second endpoint corresponding to the first controlled device 300 and sends the converted second instruction to the first controlled device 300 to cause the first controlled device 300 to respond to the second instruction.

[0212] In this way, if the first controlled device is a non-Matter device and cannot directly join the virtual domain of the master device 100 to be directly controlled by the master device 100, the first controlled device can be added as an endpoint of the display device 200, and the display device 200 provides the master device 100 with endpoint information, so that the first controlled device is displayed to the master device 100. The master device 100 can send control instructions to the endpoint corresponding to the first controlled device just as it does to the endpoint configured for the display device 200 itself. The display device 200 converts the control instructions into instructions of the protocol supported by the first controlled device, and sends the converted instructions to the first controlled device so that the first controlled device can recognize and respond to the instructions, thereby responding to the control of the master device 100. In this way, it can effectively be compatible with non-Matter devices.

[0213] Example 1

[0214] Taking the master device 100 as a mobile phone, the display device 200 as TV A, and the first controlled device as TV B as an example, the process of the master device 100 controlling the first controlled device through the display device 200 is described in conjunction with FIG17 .

[0215] TV B publishes to a shared topic, such as "device uuid / + / status," via the MQTT protocol. TV A discovers TV B by subscribing to the shared topic. TV A can establish communication with TV B and obtain device information from TV B, such as TV B's device type and supported service capabilities, such as power on / off, channel control, channel control, and button control. TV B's device information may also include other attribute information of TV B, such as device name, device ID, device location, manufacturer, etc. Based on TV B's device type and supported service capabilities, TV A creates a corresponding second endpoint for TV B, such as endpoint N+1. The corresponding device type is media, and the corresponding clusters include a power on / off cluster, a channel control cluster, a channel control cluster, and a button control cluster. TV A also stores the second endpoint and second service function corresponding to TV B in the endpoint information.

[0216] As shown in Figure 17 (1), after TV A and the mobile phone have established a network connection and joined the mobile phone's virtual domain, the mobile phone can read endpoint information from TV A. If the user wants to control a controlled device via their mobile phone, they can instruct the mobile phone to display a list of controlled devices, as shown in Figure 17 (2). This list includes entries for TV A and TV B, with TV A's entry labeled "matter" and TV B's entry labeled "non-matter." Furthermore, TV B is displayed in the first area corresponding to TV A, indicating that TV B is the bridged device, with TV A acting as the bridge device. If the user wants to control TV B, such as turning it off, they can click the entry for TV B to display the mobile phone's control page for TV B. As shown in Figure 17 (3), TV B's control page includes controls for power on / off, channel control, and key control. The user can enter a first command for power on / off, which instructs to shut down the device. As shown in Figure 17 (4), the mobile phone sends this first command to TV A via the Matter protocol. TV A converts the first command into a second command supporting the MQTT protocol and sends this second command to TV B. After receiving the second instruction, TV B turns off to respond to the control of the mobile phone.

[0217] Example 2

[0218] Taking the master device 100 as a mobile phone, the display device 200 as a TV, and the first controlled device as a lamp as an example, the process of the master device 100 controlling the first controlled device through the display device 200 is described in conjunction with FIG18 .

[0219] The light turns on its BLE function and broadcasts to its surroundings. To add the light as a controlled device, the user can use the remote control to send a search command to the TV, instructing it to search for nearby devices. If the TV discovers the light, it will display the device discovery page shown in Figure 18 (1). This page includes an entry for the light and may also include entries for other discovered devices, such as speakers. If the user wants to add the light as a controlled device, they can select the light entry using the remote control to instruct the TV to establish communication with the light. After establishing communication with the light, the TV can obtain device information from the light, such as the device type and supported service capabilities, such as on / off and level control. The device information may also include other light-related attributes, such as the device name, device ID, and location. After establishing communication with the light, the TV will display the first controlled device list shown in Figure 18 (2). This list includes entries for the light and first devices, such as switches. The list also displays an add button for each entry. If the user wants to add the light as a controlled device, they can enter the add command using the corresponding add button for the light. In response to the add instruction, the TV displays the light add page (see ③ in Figure 18). This page includes items describing the light's service capabilities, such as on / off and level control capabilities. The user can select some or all of these capabilities and add them as controllable capabilities. For example, the user can enter an add instruction using the add button corresponding to the on / off capability to indicate the addition of the on / off capability. In response to the add instruction, the TV creates a second service function for the on / off capability, namely, an on / off cluster. The TV stores the second endpoint and second service function corresponding to the light in the endpoint information.

[0220] As shown in Figure 18 (4), after the TV and mobile phone are networked and join the phone's virtual domain, the phone can read endpoint information from the TV. If the user wants to control a controlled device via their phone, they can instruct the phone to display a list of controlled devices. As shown in Figure 18 (5), this list includes items for the TV and the light. The TV item is labeled "matter," and the light item is labeled "non-matter." Furthermore, both the TV and the light are displayed in the living room area, indicating their locations. If the user wants to control the light, such as turning it off, they can click the light item to instruct the phone to display function controls in the corresponding location. As shown in Figure 18 (6), the light's function controls include an on control 1801 and an off control 1802. Using the off control 1802, the user can enter a first command, indicating a shutdown. As shown in Figure 18 (7), the phone sends this first command to the TV via the Matter protocol. The TV converts the first command into a second command supporting the BLE protocol and sends this second command to the light. Upon receiving the second command, the light turns off, responding to the phone's control.

[0221] In some embodiments, after the display device 200 is networked with the master device 100 and joins the virtual domain of the master device 100, if a new non-Matter device is discovered, the display device 200 can continue to add the new non-Matter device and provide the new non-Matter device to the master device 100. The display device 200 can continue to add non-Matter devices after being networked with the master device 100 according to the process shown in Figure 19. The specific steps are as follows:

[0222] S1901: Establish communication with a second device based on a first protocol.

[0223] The second device is a non-Matter device newly discovered by the display device 200 after the display device 200 is networked with the master device 100 and joins the virtual domain of the master device 100 . The second device supports the first protocol.

[0224] S1902: Create a third endpoint corresponding to the second device and a third service function corresponding to the third endpoint, where the third service function corresponds to a service capability supported by the second device.

[0225] Steps S1901-S1902 may refer to steps S701-S702 and are not described in detail here.

[0226] S1903: Add the third endpoint and the third service function to the endpoint information, and provide the third endpoint and the third service function to the master control device.

[0227] For example, if display device 200 currently includes a first endpoint (endpoint 0-endpoint N) and a second endpoint (endpoint N+1) as shown in Figure 6, and the second device is a switch, the third endpoint created by display device 200 for the second device is endpoint N+2, and the device type corresponding to endpoint N+2 can be 0x0200, which corresponds to On / Off Switch.

[0228] In some embodiments, after creating the third endpoint and the third business function, the display device 200 can actively send an update request to the main control device 100. The update request carries the third endpoint and the third business function, as well as other attribute information related to the second device, such as the device name, etc., and is used to request the main control device 100 to update the endpoint information corresponding to the display device 200 to add the third endpoint and the third business function.

[0229] In response to the update request, the master device 100 can directly update the endpoint information of the display device 200 to add the third endpoint and the third service function. In response to the update request, the master device 100 can also display an update notification, which is used to notify the user that there is a newly added non-Matter device, such as the second device, on the display device 200, and ask the user whether to update and add control of the second device. If the user enters an instruction to agree to the addition, the master device 100 responds to the instruction and updates the endpoint information of the display device 200 to add the third endpoint and the third service function.

[0230] If the master device 100 updates the endpoint information of the display device 200, the controlled device list is updated according to the updated endpoint information, and the updated controlled device list is newly added with the second device option. The process of the user controlling the second device through the master device 100 can refer to step S705 and is not repeated here.

[0231] Example 3

[0232] Taking the main control device 100 as a mobile phone, the display device 200 as TV A, the first controlled device as TV B, and the second device as a lamp as an example, based on Example 1 and combined with Figure 20, the process of adding a second device to the display device 200 after being networked with the main control device 100 is explained.

[0233] TV A can discover the light based on the BLE protocol, establish communication with the light, and create a third endpoint and a third business function for the light. This process can be referred to in Example 2 and will not be described in detail here. After TV A adds the third endpoint and the third business function to the endpoint information, it can send an update request to the mobile phone. The update request can carry the endpoint information of the light and other related attribute information. Based on the update request, the mobile phone can display an update notification 2001 as shown in ① in Figure 20. The update notification 2001 is in the form of a pop-up window, including information such as "The light is connected, do you want to add control to this device?" The update notification 2001 can also include an agree button and a reject button. If the user enters an instruction to agree to add based on the agree button, the mobile phone responds to the instruction and updates the endpoint information of TV A to add the third endpoint and the third business function to the endpoint information. If the user controls the phone to display the list of controlled devices, the phone displays the updated list of controlled devices, as shown in Figure 20 (2). Compared to the list of controlled devices shown in Figure 17 (2), this updated list includes a new item for the light. The light item is labeled "non-matter" and displayed in the first area of ​​TV A, indicating that it is a bridged device with TV A as the bridge device. If the user wants to control the light using the phone, they can refer to the process of controlling TV B using the phone in Example 1, which is not repeated here.

[0234] In some embodiments, if the user wants to delete control of a non-Matter device, the user can input a deletion instruction through the item of the target device in the first controlled device list of the display device 200. The display device 200 responds to the deletion instruction by deleting the endpoint information corresponding to the target device from the endpoint information and sending a deletion request to the main control device 100. The deletion request carries the device information of the target device and is used to request deletion of the endpoint information corresponding to the target device from the endpoint information corresponding to the display device 200.

[0235] In response to the deletion request, the main control device 100 deletes the endpoint information corresponding to the target device from the endpoint information to update the endpoint information corresponding to the display device 200, and displays a deletion notification, which is used to notify the user that the control authority over the target device has been terminated, and updates the controlled device list according to the updated endpoint information. The updated controlled device list no longer includes the item of the target device.

[0236] Example 4

[0237] Based on Example 3 and in conjunction with FIG. 21 , the process of deleting a target device on the display device 200 side is described.

[0238] If the user wishes to remove control of TV B, they can control TV A to display the first controlled device list. TV A can display the first controlled device list, as shown in step 1 of Figure 21. This list includes items for TV B and the light, with a delete button displayed for each item. If the user wishes to remove control of TV B, they can enter a delete command using the delete button corresponding to the item for TV B. In response to this delete command, TV A deletes the endpoint information corresponding to TV B from the endpoint information and updates the first controlled device list. The updated first controlled device list may be shown in step 2 of Figure 21.

[0239] In response to the deletion instruction, TV A also sends a deletion request to the mobile phone, requesting that the endpoint information corresponding to TV B be removed from the endpoint information corresponding to TV A. In response to the deletion request, the mobile phone deletes the endpoint information corresponding to TV B, thereby updating the endpoint information. If the user controls the mobile phone to display the list of controlled devices, the mobile phone displays an updated list of controlled devices, as shown in ③ in Figure 21, based on the updated endpoint information. Compared to the list of controlled devices shown in ② in Figure 20, this updated list does not include TV B.

[0240] In some embodiments, if a non-Matter device disconnects from the display device 200, the display device 200 deletes the endpoint information corresponding to the non-Matter device from the endpoint information and notifies the main control device 100 to delete the endpoint information corresponding to the non-Matter device. The above process is not repeated here.

[0241] Example 2

[0242] The process of the display device 200 discovering and connecting with the non-Matter device can be controlled by the main control device 100. Before executing this process, the display device 200 has completed the network configuration with the main control device 100 and joined the virtual domain of the main control device 100.

[0243] The display device 200 can support the master device 100 to control non-Matter devices according to the process shown in FIG22 . The specific steps are as follows:

[0244] S2201 , after performing network configuration with the master device based on the Matter protocol, searching for surrounding devices in response to a search instruction sent by the master device.

[0245] The master device 100 and the display device 200 are networked and can add the display device 200 to the virtual domain to control the display device 200. If the user wants to add a non-Matter device through the display device 200, the user can control the master device 100 to display a list of controlled devices, which includes an item for the display device 200. The user can input instructions based on the item for the display device 200 to instruct the master device 100 to display the control page of the display device 200. The control page of the display device 200 can be configured with a search entry, and the user can input instructions based on the search entry. In response to the instruction, the master device 200 will send a search instruction to the display device 200 to control the display device 200 to search for surrounding devices.

[0246] The process in which the display device 200 actively searches for surrounding devices and finds the first controlled device can be referred to the relevant part in step S701 and will not be described in detail here.

[0247] S2202: After discovering the first controlled device, obtain the device name of the first controlled device and send the device name to the master control device.

[0248] The first controlled device is a non-Matter device and supports the first protocol.

[0249] In some embodiments, after discovering the first controlled device, the display device 200 may directly establish communication with the first controlled device. For this process, reference may be made to the relevant part of step S701 and will not be repeated here.

[0250] In some embodiments, after discovering the first controlled device, the display device 200 obtains the device name of the first controlled device and may also obtain other attribute information of the first controlled device, and sends the device name of the first controlled device to the master device 100. The master device 100 may display the device discovery results on the control page of the first controlled device based on the obtained attribute information such as the device name of the first controlled device. The user browses the control page through the master device 100 and identifies the non-Matter device they want to control, such as the first controlled device.

[0251] S2203: Establish communication with the first controlled device in response to the control instruction sent by the master control device.

[0252] The user can click the item of the first controlled device in the device discovery result on the control page to send a connection instruction to the main control device 100. In response to the connection instruction, the main control device 100 sends a connection instruction to the display device 200 to control the display device 200 to establish communication with the first controlled device.

[0253] The process of establishing communication between the display device 200 and the first controlled device may refer to the relevant part of step S701 and will not be described in detail here.

[0254] S2204: Create a second endpoint corresponding to the first controlled device and a second service function corresponding to the second endpoint, where the second service function corresponds to a service capability supported by the first controlled device.

[0255] Step S2204 can refer to step S702, steps S801-S802, steps S901-S902, steps S1101-S1102, and steps S1301-S1306, which are not repeated here.

[0256] The process in which the display device 200 creates the second endpoint and the second service function corresponding to the first controlled device may be controlled by the main control device 100 .

[0257] In one implementation, the display device 200 can synchronize relevant data generated during the execution process to the main control device 100, and the main control device 100 updates the control page of the display device 200 based on these relevant data. The user can send corresponding control instructions to the display device 200 through the main control device 100 based on the updated control page to instruct the display device 200 to create a second endpoint and a second business function.

[0258] In another implementation, the display device 200 can synchronize the relevant pages generated during the execution process to the master device 100, and the master device 100 synchronously displays these relevant pages. Since the relevant pages synchronously displayed by the master device 100 generally cannot provide the functions of the controls, the master device 100 can calculate the target control affected by the user based on the position clicked by the user on the screen and the position of each control on the screen, and generate a control instruction corresponding to the target control, and send the control instruction to the display device 200 to control the display device 200 to implement the corresponding function of the target control, thereby controlling the display device 200 to create the second endpoint and the second business function.

[0259] S2205: Add the second endpoint and the second service function to the endpoint information, and provide the second endpoint and the second service function to the main control device.

[0260] S2206: Receive a first instruction sent by the master control device to the first controlled device, convert the first instruction into a second instruction supporting the first protocol, and send the second instruction to the first controlled device.

[0261] Steps S2205-S2206 can refer to steps S704-S705 and are not repeated here.

[0262] Example 5

[0263] Taking the master device 100 as a mobile phone, the display device 200 as TV A, and the first controlled device as TV B as an example, the process of the master device 100 controlling the first controlled device through the display device 200 is described in conjunction with FIG23 .

[0264] The mobile phone pairs with TV A and adds TV A to the virtual domain. If the user wants to add a controlled device through TV A, the user can control the mobile phone to display a list of controlled devices, as shown in Figure 23 (1), which includes TV A. The user enters a selection based on TV A's entry. In response, the mobile phone displays TV A's control page, as shown in Figure 23 (2), which includes a search entry 2301. The user can enter a search command based on search entry 2301. The mobile phone responds by sending a search command to TV A, instructing TV A to actively search for nearby devices. For example, if TV B broadcasts via the BLE protocol, TV A will discover TV B if it scans the broadcast. After discovering TV B, TV A obtains TV B's device information and sends it to the mobile phone. After receiving TV B's device information, the mobile phone displays the device discovery results on TV A's control page, as shown in Figure 23 (3). The device discovery results include TV B's entry and a connect button for that entry. If the user wants to add control of TV B, they can click the connect button corresponding to TV B's entry to enter a click command. In response to the click instruction, the mobile phone sends a connection instruction to TV A, instructing TV A to establish communication with TV B. The process of TV A establishing communication with TV B, obtaining TV B's device information, creating a second endpoint and a second service function for TV B, and the mobile phone controlling TV B through TV A can all be referred to in Example 1 and will not be repeated here.

[0265] In some embodiments, a user can delete control of a non-Matter device through the master device 100. The user can control the master device 100 to display a list of controlled devices, which includes entries for the display device 200 and the first controlled device. The list of controlled devices can also include a delete button for the entry for the first controlled device, or a delete button can be configured within the control page of the first controlled device. If the user wishes to delete control of the first controlled device, the user can enter a delete command using the delete button corresponding to the entry for the first controlled device. In response to the delete command, the master device 100 deletes the endpoint information corresponding to the first controlled device from the endpoint information corresponding to the display device 200, thereby updating the endpoint information. The master device 100 updates the controlled device list based on the updated endpoint information, and the updated controlled device list no longer includes the entry for the first controlled device. In response to the delete command, the master device 100 also sends a delete instruction to the display device 200, instructing the display device 200 to delete the endpoint information corresponding to the first controlled device from the endpoint information. For example, the master device 100 deletes the relevant device information of the non-Matter device through the Bridge Server and notifies the Matter SDK. The Matter SDK calls the interface to delete the corresponding endpoint of the non-Matter device.

[0266] Example 6

[0267] Taking the master device 100 as a mobile phone, the display device 200 as TV A, and the first controlled device as TV B as an example, the process of deleting non-Matter devices on the master device 100 side is explained in conjunction with Figure 24.

[0268] If the user wants to delete the control of non-Matter devices, such as deleting the control of TV B, the user can control the mobile phone to display the list of controlled devices. The mobile phone can display the list of controlled devices as shown in ① in Figure 24, which includes items for TV A and TV B. In addition, the list of controlled devices also displays a delete button corresponding to the item. The user can input a click instruction based on the delete button corresponding to the item of TV B. In response to the click instruction, the mobile phone deletes the endpoint information corresponding to TV B and updates the displayed list of controlled devices. The updated list of controlled devices can be shown in ② in Figure 24, which does not include the item of TV B. In response to the click instruction, as shown in ① in Figure 24, the mobile phone also sends a deletion instruction to TV A, which is used to instruct TV A to delete the endpoint information of TV B from the endpoint information.

[0269] FIG25 illustrates another use case for display device 200. As shown in FIG25 , a user can operate display device 200 via main control device 100 or control device 10. Control device 10 can be a remote control. Communication between the remote control and display device 200 can include infrared protocol communication, Bluetooth protocol communication, or other short-range communication methods, allowing wireless or wired control of display device 200. Users can control display device 200 by inputting user commands via buttons on the remote control, voice input, or control panel input.

[0270] In some embodiments, the main control device 100 (such as a mobile phone, tablet computer, computer, laptop computer, etc.) can also be used to control the display device 200. For example, the display device 200 is controlled using an application running on the main control device 100.

[0271] In some embodiments, the display device 200 may not use the above-mentioned main control device 100 or control device 10 to receive instructions, but may receive user control through touch or gestures.

[0272] In some embodiments, the display device 200 can also be controlled in a manner other than the control device 10 and the main control device 100. For example, the user's voice command control can be directly received through a module for obtaining voice commands configured inside the display device 200, or the user's voice command control can be received through a voice terminal device set outside the display device 200.

[0273] In some embodiments, the display device 200 also communicates data with the server 400. The display device 200 may be connected to a local area network (LAN), a wireless local area network (WLAN), or other networks. The server 400 may provide various content and interactions to the display device 200. The server 400 may be a single cluster or multiple clusters, and may include one or more types of servers.

[0274] In actual applications, Matter devices need to be networked through the control end before they can operate normally. The control end can be other home appliances, such as the aforementioned display device 200. The control end can complete the connection through Bluetooth connection, shooting and scanning the QR code on the Matter device, thereby enabling the control end to network the Matter device. However, due to different structural designs of homes, the display device 200 may be a certain distance away from the Matter device to be networked. When the display device 200 is far away from the Matter device, it cannot be connected to the Bluetooth of the Matter device, and since the display device 200 occupies a large space, it is not convenient to scan the QR code on the Matter device, resulting in inconvenience in the network interaction between the control end such as the display device 200 and the Matter device.

[0275] In order to solve the problem of difficult network interaction between the control end and the Matter device, in some embodiments, a first communication interface and a second communication interface can be set for the display device 200, wherein the first communication interface is used to establish a first communication connection with the main control device 100, and the second communication interface is used to establish a second communication connection with the second controlled device.

[0276] The first communication connection can be established via a WiFi connection or the same hotspot network. The master device 100 can be a mobile device such as a smart phone or a smart watch, so that after the display device 200 is connected to the master device 100, the master device 100 can scan the authentication code of the second controlled device.

[0277] In some embodiments, the display device 200 and the control device 100 may have an application installed on them for Matter connection. The display device 200 and the control device 100 may simultaneously launch the application to pair up using their identity information and establish a first communication channel. In this embodiment, to ensure a stable network environment for the first communication channel, the display device 200 and the control device 100 may be in the same network environment.

[0278] In some embodiments, the at least one processor 250 of the display device 200 may be further configured to execute computer instructions so that the display device 200 performs the steps shown in FIG26 to complete network configuration with the second controlled device:

[0279] S100: In response to a network configuration instruction input by a user, the device information of the second controlled device is sent to the master control device.

[0280] In some embodiments, a user can input a network configuration instruction to the display device 200 through various input methods, such as voice, gestures, a touchpad, or a remote control. The network configuration instruction may include device information of the second controlled device, including the device name, device model, device installation location, and device readiness status. As shown in FIG27 , when a user inputs the network configuration instruction "Please turn on the light in the master bedroom" into the display device 200 via voice, the processor 250 performs voice recognition on the network configuration instruction, thereby obtaining the second controlled device's location as "master bedroom" and device type as "light," and then sends the network configuration information to the master control device 100 via the first communication channel.

[0281] The periphery of the display device 200 may include one or more devices having Matter protocol functions, wherein the second controlled device is the Matter device specified in the network configuration instruction.

[0282] In some embodiments, before receiving the network distribution command, the display device 200 may also establish a communication connection with the power system in the area where the display device 200 is located, which may be a home interior, an office area, a home exhibition area, etc. The display device 200 may determine the readiness status of the second controlled device based on the device information of the second controlled device through the power system, such as the power level of the second controlled device, whether the second controlled device is faulty, etc., and send the device information to the master control device 100 based on the readiness status.

[0283] S200: Acquire authentication code information through the first communication channel.

[0284] The authentication code information is generated by the master device 100 scanning the pairing code corresponding to the device information. After the master device 100 obtains the device information of the second controlled device sent by the display device 200, the user can activate the camera of the master device 100 and call the scanning box to scan the pairing code of the second controlled device through the master device 100.

[0285] The master device 100 can display the device information of the second controlled device on its display through the Matter connection application, prompting the user to hold the master device 100 close to the second controlled device and scan the pairing code on the second controlled device with the master device 100. The master device 100 can then parse the pairing code to obtain the authentication code information. The authentication code information is the authentication information of the second controlled device. Each Matter device has a unique authentication code information that is used to identify the corresponding Matter device.

[0286] The main control device 100 may return the authentication code information to the display device 200 through the first communication channel, so that the display device 200 can search for the second controlled device according to the authentication code information.

[0287] In some embodiments, as shown in FIG28 , the display device 200 may send a code scanning instruction to the master device 100 via the first communication channel while sending the device information of the second controlled device to the master device 100. After receiving the device information of the second controlled device and the code scanning instruction, the master device 100 may establish a code scanning task based on the device information to prompt the user to complete scanning of the pairing code corresponding to the device information through the master device 100. After generating the code scanning task, the display device 200 may send a prompt message to the master device 100 at intervals of a first duration to remind the user to complete the scanning task through the master device 100 as soon as possible. After the master device 100 completes the scanning, the code scanning task may be cancelled.

[0288] In some embodiments, in order to prevent non-network-pairing devices from performing network pairing on the second controlled device, the pairing code can also be set with a decoding key. After the main control device 100 scans the pairing code, it can obtain the scanned code information obtained by scanning the pairing code, and perform a decoding operation on the scanned code information using a pre-stored decoding key to obtain the identification code information, thereby improving the security of the second controlled device.

[0289] In some embodiments, the main control device 100 can convert the pairing code into a digital signal through optical recognition technology and extract the identification code information from the digital signal. In the above process, the main control device 100 can obtain the pairing code through the image acquisition interface and identify the type of the pairing code, for example, whether the pairing code is a QR code or a barcode, and then use the image processing recognition algorithm to identify the shape of the pairing code and the black and white bars in the pairing code, etc., and then convert it into a digital signal according to the position of these black and white bars to extract the identification code information.

[0290] S300: Perform wireless communication scanning on network distribution devices within a first preset range according to the authentication code information.

[0291] After receiving the authentication code information, the display device 200 needs to search for the second controlled device based on the authentication code information, establish a second communication channel with the second controlled device via the second communication interface, and thus complete the network configuration of the second controlled device. In this embodiment, the display device 200 can scan for network configuration devices via wireless communication.

[0292] In some embodiments, wireless communication may include technologies such as Bluetooth modules, infrared scanning, radio or electromagnetic waves, but different wireless communication technologies have different coverage scanning ranges. Therefore, the display device 200 can perform wireless communication scanning on the network distribution devices within a first preset range, and the first preset range is the coverage range of the communication method with the largest wireless communication coverage range.

[0293] Since different wireless communication technologies cover different scanning ranges, for network distribution devices that are less than the first distance away from the display device 200, infrared scanning, electromagnetic waves and other technologies can be used to perform scanning. For network distribution devices that are greater than the first distance away from the display device 200, Bluetooth modules, radio and other wireless communication technologies can be used to perform scanning, and the network distribution devices around the display device 200 are paired one by one through scanning to determine the second controlled device.

[0294] It should be noted that the network configuration devices are all Matter devices, which are equipped with a communication device for being scanned by the display device 200. When the Matter device is in working state or standby state, the communication device can continuously send communication signals to the outside to facilitate the display device 200 to perform scanning and pairing on it.

[0295] In some embodiments, the display device 200 can detect the wireless communication operation status of the display device 200 based on the identification code information sent by the main control device 100. If the wireless communication operation status is not started, the display device 200 can switch the wireless communication operation status to the started state.

[0296] When the display device 200 detects that the wireless communication operation state is enabled, it begins to perform wireless communication scanning for network devices within the first preset range and obtains network device information of the scanned network devices. To select the second controlled device from the network devices, the display device 200 can perform pairing based on the network information using the authentication code information.

[0297] Among them, in order to facilitate pairing, as shown in Figure 29, the display device 200 can generate a device information table based on the network configuration device information obtained by scanning. When the first network configuration device information appears in the device information table, the display device 200 can start pairing through the identification code information. During the pairing process, the display device 200 continuously performs wireless communication scanning on the network configuration devices around the display device 200, and adds its network configuration device information one by one to the device information table to save the time used for pairing and improve the network configuration efficiency.

[0298] The network configuration device information may include the identification code information of the network configuration device. The display device 200 can compare the identification code information of the second controlled device with the identification code information of other network configuration devices. If the identification code information of the other network configuration devices is the same as the identification code information of the second controlled device, it means that the pairing is successful.

[0299] In some embodiments, the display device 200 may record a first pairing duration of the display device 200 when the display device 200 performs a wireless communication device scan on the network pairing device. The display device 200 may pair the network pairing device information according to the identification code information within the first pairing duration.

[0300] FIG30 is a flowchart of determining whether pairing of the display device 200 has failed according to some embodiments, including the following steps:

[0301] S3001, pairing network device information through authentication code information;

[0302] S3002: Record the first pairing duration;

[0303] S3003: Determine whether the first pairing duration is greater than a first duration threshold;

[0304] S3004: If it is greater than, pairing fails;

[0305] S3005: If not, continue the pairing operation;

[0306] During implementation, a first duration threshold can be set to assist in determining the pairing result. When the first pairing duration exceeds the first duration threshold, it means that within the first duration threshold, the display device 200 has not been paired to a network configuration device with the same identification code information. Since during the pairing process, the display device 200 is still scanning the network configuration device and adding network configuration device information, the first pairing duration also includes the waiting time for the display device 200 to scan. To this end, as shown in FIG31 , the display device 200 can also start recording the first pairing duration when pairing the most recently added network configuration device information, and re-time the first pairing duration when new network configuration device information is added.

[0307] S400: If a first scanning result is generated, a second communication channel is established with a second controlled device, and network configuration is performed on the second controlled device through the second communication channel.

[0308] When display device 200 generates a first scan result, it indicates that network configuration device information for the second controlled device has been obtained within the display device 200's scanning range. Therefore, display device 200 can directly establish a second communication channel with the second controlled device. The second communication channel can use a different wireless communication method than the first communication channel. For example, the first communication channel can be a wireless network connection, while the second communication channel can be a Bluetooth communication connection. After the display device 200 establishes the second communication channel with the second controlled device, network configuration can be performed on the second controlled device via the second communication channel.

[0309] In some embodiments, the display device 200 can also obtain authentication code information by inputting an authentication code. The user can control the display device 200 to display an authentication code input box through the control device 10, and input the authentication code information through the keyboard on the control device 10 so that the display device 200 obtains the authentication code information.

[0310] When the display device 200 does not generate the first scanning result, it indicates that the display device 200 fails to pair with the second controlled device. At this time, the display device 200 generates a second scanning result.

[0311] The second scan result indicates that the display device 200 has not scanned any other network-pairing devices paired with the second controlled device. This indicates that none of the other network-pairing devices located near the display device 200 are the second controlled device. In some embodiments, to expand the scanning range for the second controlled device, after generating the second scan result, the display device 200 may send the second scan result to the master device 100. Upon receiving the second scan result, the master device 100 may activate its wireless communication operating module, such as its Bluetooth module, based on the second scan result.

[0312] After the master device 100 activates the Bluetooth module, it also has the ability to scan for network devices. At this point, the master device 100 can use Bluetooth communication to scan for other network devices in the second preset area and obtain device information for these network devices. The second preset area is the maximum wireless communication scanning coverage of the master device 100.

[0313] In some embodiments, the display device 200 may further generate a terminal scanning instruction based on the second scanning result. The terminal scanning instruction is used to instruct the main control device 100 to start the wireless communication operation module. The display device 200 may send the terminal scanning instruction to the main control device 100. The main control device 100 will respond to the terminal scanning instruction, start the wireless communication operation module, and begin to perform wireless communication scanning in the coverage area where the main control device 100 is located.

[0314] Since the main control device 100 and the display device 200 are separated by a certain distance, the main control device 100 can scan the range that the display device 200 cannot scan, thereby expanding the range of scanning the second controlled device. Therefore, as shown in Figure 32, the main control device 100 can obtain the network configuration device information outside the scanning range of the display device 200, and return this network configuration device information to the display device 200 as the network configuration feedback information corresponding to the terminal scanning instruction.

[0315] After receiving the device information sent by the master device 100 , the display device 200 may continue to add the device information sent by the master device 100 on the basis of the original device information table, and continue to perform pairing on the device information.

[0316] In some embodiments, the first and second preset ranges may have overlapping scanning areas. When a network distribution device is located in the overlapping scanning area, for ease of description, this embodiment defines the network distribution device located in the overlapping scanning area as an overlapping device. Both the display device 200 and the master control device 100 may scan and obtain network distribution device information for the overlapping device, resulting in multiple acquisitions of network distribution device information for the overlapping device, reducing network distribution efficiency.

[0317] The display device 200 may execute the process shown in FIG33 to increase the network distribution efficiency, as shown in FIG33 , including the following steps:

[0318] S3301: Obtain distribution network feedback information;

[0319] S3302: Determine whether the network distribution device information of the overlapping device is included;

[0320] S3303: If yes, filter out the network distribution device information of the overlapping devices;

[0321] S3304: performing pairing on the remaining configuration network feedback information;

[0322] S3305: If not, perform pairing on the configuration network feedback information.

[0323] After receiving the network configuration feedback information, the display device 200 can filter the network configuration feedback information based on the network configuration device information in the device information table. The network configuration feedback information may include one or more network configuration device information. The display device 200 can obtain the network configuration device information in the current device information table and traverse the network configuration feedback information based on the network configuration device information in the current device information table. If the network configuration feedback information includes network configuration device information that already exists in the device information table, it indicates that the network configuration device corresponding to the network configuration device information is a duplicate device. In this case, the display device 200 retains the network configuration device information in the device information table and deletes the network configuration device information of the duplicate device from the network configuration feedback information.

[0324] After the screening is completed, the display device 200 adds the network configuration device information that is not the overlapping device in the network configuration feedback information to the device information table. At the same time, the display device 200 performs pairing on the network configuration device information in the network configuration feedback information according to the identification code information. When there is successfully paired network configuration device information in the network configuration feedback information, the display device 200 can mark the network configuration device corresponding to the successfully paired network configuration device information as the second controlled device.

[0325] In some embodiments, the network configuration feedback information can also be paired through the main control device 100. Since the main control device 100 has scanned the pairing code with the network configuration device, the main control device 100 also includes identification code information. For this reason, the main control device 100 can also perform the above-mentioned steps of filtering out the network configuration device information of the overlapping devices in the network configuration feedback information, and pair the network configuration device information in the network configuration feedback information through the identification code information.

[0326] After the master device 100 successfully pairs the network configuration device information using the authentication code information, the display device 200 can send a network configuration instruction to the master device 100, thereby establishing a wireless communication connection between the master device 100 and the second controlled device. The wireless communication connection may include a Bluetooth communication connection. The master device 100 has already enabled the Bluetooth communication module when scanning the network configuration device information. Therefore, the master device 100 can directly send a Bluetooth connection request to the second controlled device and wait for the second controlled device to respond to the Bluetooth connection request.

[0327] After receiving the Bluetooth connection request, the second controlled device automatically determines its connection status. If the second controlled device is currently establishing a Bluetooth communication connection with another device, it cannot establish a Bluetooth communication connection with the master device 100. In this case, the second controlled device can provide busy information to the master device 100. The busy information indicates that the second controlled device is currently connecting to another device and the Bluetooth communication connection is busy. The busy information may also include device information about the device the second controlled device is currently connecting to, as well as the duration of the connection, so that the user can obtain the current status of the second controlled device.

[0328] In some embodiments, since the volume of the master device 100 is smaller than that of the display device 200, the user can hold the master device 100 and move it. Therefore, after the display device 200 generates the second scanning result, the movement information can be sent to the master device 100 through the first communication channel. The movement information can be displayed on the display of the master device 100 to prompt the user to move the master device 100 toward the second controlled device so that the master device 100 can be paired with the second controlled device.

[0329] In some embodiments, the display device 200 may also obtain the location information of the master device 100 and the distance between the master device 100 and the second controlled device through the first communication channel. In the above process, the master device 100 may send a GPS positioning request to the server 400 to obtain the positioning information of the master device 100 fed back by the server 400, and send the positioning information to the display device 200. The display device 200 may generate corresponding movement information based on the positioning information of the master device 100 and the position of the second controlled device, thereby guiding the user to move the master device 300 according to the corresponding movement information.

[0330] In some embodiments, when the user is using the master device 100, the scanning range of the master device 100 may also change as the master device 100 moves. To this end, the display device 200 can generate a movement trajectory of the master device 100 based on the positioning information of the master device 100 fed back by the server 400, and define a second preset range based on the movement trajectory. The second preset range is the maximum range scanned by the master device 100 when moving along the movement trajectory. The display device 200 can then control the master device 100 to obtain the network configuration device information within the second preset range based on the authentication code information.

[0331] In some embodiments, the display device 200 may also time the pairing time of the master device 100 and record the second pairing time of the master device 100 through the first communication channel. Since the device model of the master device 100 is different from that of the display device 200, the processor may set a second time threshold to determine the pairing status of the master device 100. When the second pairing time exceeds the second time threshold, it indicates that the master device 100 has not been paired with the network-matching device within the second preset range. At this time, the master device 100 may generate a prompt message, which is used to prompt the master device 100 that the pairing failed within the second time threshold. The content of the prompt message may be in text or voice form, for example, "Pairing failed, please try again later!" After generating the prompt message, the master device 100 may send the prompt message to the display device 200. After receiving the prompt message, the display device 200 may re-execute the pairing according to the prompt message.

[0332] In some embodiments, the present disclosure provides a master device 100, which includes a display, a communication device, and a processor, wherein the communication device is used to establish a first communication channel with a display device 200, and the processor is configured to:

[0333] In response to a network configuration instruction input by a user, the main control device 100 receives device information of a second controlled device sent by the display device 200 through the first communication channel, wherein the main control device 100 and the display device 200 are in the same network environment.

[0334] Identification code information is sent to the display device 200 through the first communication channel, so that the display device 200 performs wireless communication scanning on the network configuration devices within a first preset range according to the identification code information, the identification code information is generated according to the pairing code corresponding to the scanned device information, and when the display device 200 generates the first scanning result, the display device 200 establishes a second communication channel with the second controlled device, and performs network configuration on the second controlled device through the second communication channel.

[0335] In some embodiments, after the master device 100 receives device information from the second controlled device, the user can activate a keyboard key or touch function configured on the master device 100. The processor can monitor the user's key events or touch events and generate corresponding network configuration instructions based on the key events or touch events. In response to the network configuration instructions, the processor can control the display of the master device 100 to display a scan box. At this time, the user can manually adjust the position and angle of the master device 100 to align the scan box with the pairing code, thereby improving the accuracy of the master device 100 in recognizing the pairing code.

[0336] In some embodiments, the main control device 100 may further receive a network configuration instruction sent by the display device 200 , and the processor may respond to the network configuration instruction and execute the above method steps.

[0337] In some embodiments, the master device 100 may send a GPS positioning request to the server 400 to obtain positioning information of the master device 100 fed back by the server 400, and send the positioning information to the display device 200. The display device 200 may generate corresponding movement information based on the positioning information of the master device 100 and the position of the second controlled device, thereby guiding the user to move the master device 100 according to the corresponding movement information.

[0338] The present disclosure also provides a server 400, including a communication module and a processing module. The communication module is used to establish a communication connection with the main control device 100, and the processing module is configured to:

[0339] The device information of the second controlled device sent by the display device 200 is received, and the main control device 100 and the display device 200 are in the same network environment.

[0340] The device information is sent to the main control device 100 .

[0341] The authentication code information returned by the main control device 100 is obtained, where the main control device 100 scans the pairing code corresponding to the device information to obtain the authentication code information.

[0342] The identification code is sent to the display device 200, so that the display device 200 performs a wireless communication scan on the network configuration devices within a first preset range according to the identification code information, and when the display device 200 generates a first scanning result, the display device 200 establishes a second communication channel with the second controlled device, and performs a network configuration operation on the second controlled device through the second communication channel.

[0343] In some embodiments, the display device 200 can execute the step of obtaining the network configuration device information of the second controlled device on the main control device 100 through the above-mentioned server 400. In this embodiment, the main control device 100 and the display device 200 may not establish a first communication channel with each other, but only interact through the server 400.

[0344] The previous section describes the network configuration process of the display device, the first controlled device, and the second controlled device in the embodiment of the present disclosure. Next, the process of controlling the controlled device through the display device is described:

[0345] As shown in Figure 2 above, in the display device 200, the communication device 220 can be configured to establish a communication connection with at least one controlled device, and to receive a control signal sent by the user through the control button of the remote control device. The processor 250 can respond to the control signal and control the display device 260 to perform corresponding operations, such as determining the control object corresponding to the control button.

[0346] In some embodiments, the display device may determine the control object corresponding to the control button in response to the control signal after receiving the control signal sent by the user through the control button of the remote control device. If the control object corresponding to the control button is the target controlled device, the first control function corresponding to the control button is searched in the first button function mapping relationship corresponding to the target controlled device, and the target controlled device is any controlled device among the at least one controlled device mentioned above. If the first button function mapping relationship includes the first control function corresponding to the control button, the control signal is converted into a control instruction corresponding to the first control function, and the control instruction is sent to the target controlled device, so that the target controlled device responds to the control instruction and executes the first control function. In summary, after receiving the control signal sent by the user through the control button, the display device can make a logical judgment on whether the control button is used to control the controlled device or the display device, and whether the control button has a mapping relationship with the control function of the controlled device.

[0347] This can avoid conflicts with the display device when controlling the controlled device through the control buttons, and is also applicable to situations where the control functions of multiple controlled devices have the same mapping relationship with the control buttons. Thus, the present disclosure can control the controlled device through the control buttons set by the user and improve the efficiency of controlling the controlled device through the display device.

[0348] The implementation may be as shown in FIG34 , including the following steps: S3410 to S3460:

[0349] S3410: Establish a communication connection with at least one controlled device.

[0350] As mentioned above, the Matter protocol is an open standard smart home protocol. In actual applications, the Matter protocol can be used to establish a communication connection between the display device and the controlled device, and enable the display device to control other devices (i.e., controlled devices). The controlled devices may include: refrigerators, air conditioners, lights, door locks, speakers, and the like, such as the first controlled device and the second controlled device mentioned above.

[0351] In some embodiments, the display device can be configured with relevant Matter application software, for example, by implementing the Matter software development kit (SDK) and the common portion of the Matter protocol stack in the application software through the Java Native Interface (JNI). Thus, the user can establish a communication connection between the display device and the controlled device by operating the application software, and enable the display device to control the other device (i.e., the controlled device).

[0352] It should be noted that the above-mentioned method of establishing a communication connection between the display device and the controlled device is only an example and is not limited in this embodiment.

[0353] Refer to the schematic diagram of an option interface shown in FIG35 , and the flowchart of the process of establishing a communication connection between a display device and a controlled device shown in FIG36 .

[0354] The user can use the Matter application to trigger the communication connection between the display device and the controlled device. As shown in Figure 35, the user interface of the display device includes a main interface 501 and related interfaces called up by the Matter application 502. The user can use the remote control device to focus the user interface on the Matter application 502 to trigger the Matter application 502 to call up the options interface 503.

[0355] The user can use the remote control device to set the focus of the option interface 503 to the "Device Connection" option. Thus, as shown in Figure 36, the controlled device executes S3601: sending broadcasts such as Bluetooth Low Energy (BLE), Multicast DNS (mDNS) or Near-field communication (NFC), so that the display device can discover one or more controlled devices in the above manner.

[0356] When the display device discovers the controlled device, it obtains relevant information of the controlled device and executes S3602: performing a Password-Authenticated Session Establishment (PASE) process.

[0357] Refer to Figures 37 to 39 for schematic diagrams of establishing a communication connection between a display device and a controlled device.

[0358] As shown in FIG37 , after the user selects the “Device Connection” option, the user can select a controlled device discovered by the display device in the device connection interface 504. For example, if the controlled devices discovered by the display device include: controlled device A, controlled device B, and controlled device C, the user can use the remote control device to set the focus of the device connection interface 504 to the “Controlled Device A” option.

[0359] During the PASE process, the display device can establish an encrypted session with the controlled device using a QR code or a user-entered pairing code. It should be understood that the QR code and pairing code contain parameter information about the controlled device; each controlled device's pairing code / QR code is unique, meaning that each controlled device has a different pairing code / QR code.

[0360] As shown in FIG. 38 , after the user selects the “controlled device A” option, the device connection interface 504 may prompt the user to enter the pairing code of the controlled device A.

[0361] Alternatively, as shown in FIG39 , after the user selects the “controlled device A” option, the device connection interface 504 may prompt the user to use the camera in the display device to scan the QR code corresponding to the controlled device A. The QR code corresponding to the controlled device A may be physical (i.e., the QR code may be displayed on the surface of the controlled device A) or electronic (i.e., the QR code may be viewed on the display interface of the controlled device A), which is not limited in this embodiment.

[0362] Continuing with Figure 36 , the display device then executes S3603: Requesting the controlled device to send a Device Attestation Certificate (DAC). Upon receiving the request, the controlled device executes S3604: Sending the Device Attestation Certificate to the display device. It is understood that the DAC is a certificate specified in the Matter protocol, and each controlled device has a unique DAC programmed into it.

[0363] After receiving the DAC sent by the controlled device, the display device executes S3605: Verify the legitimacy of the device authentication certificate, that is, verify whether the controlled device has passed Matter authentication, etc. If the controlled device passes the verification, the display device executes S3606: Generate and send a Node Operational Certificate (NOC) to the controlled device.

[0364] After receiving the NOC, the controlled device installs it and executes S3607: Joining the display device's network. The display device then executes S3608: Ending the Password Authentication Session Establishment process and beginning the Certificate Authenticated Session Establishment (CASE) process. Once the CASE encryption process is initiated, the display device and the controlled device have successfully established a communication connection, enabling the sending and receiving of commands between them.

[0365] After a successful communication connection is established between the display device and the controlled device, the display device can obtain the controlled device's device information and control functions and display them in the user interface. The controlled device's device information may include: device name, device type, manufacturer, etc.; the controlled device's control functions refer to the functions that the controlled device can perform, such as turning on and off, adjusting brightness, adjusting temperature, etc.

[0366] Refer to Figures 40 to 43 for a schematic diagram of viewing and controlling a controlled device.

[0367] As shown in FIG. 40 , the user can use the remote control device to set the focus of the option interface 503 on the “Device List” option, thereby viewing the controlled devices that have currently established a communication connection with the display device.

[0368] In some embodiments, as shown in FIG. 41 , the display device may further group the controlled devices according to different standards, and display the groups in the device grouping interface 505 after the user selects the “Device List” option.

[0369] In some embodiments, controlled devices can be grouped by device type and household room. For example, controlled devices can be divided into categories such as lights, large appliances, and door locks based on device type; or divided into categories such as living room, kitchen, master bedroom, and second bedroom based on the household room where the controlled device is located.

[0370] It should be noted that the above-mentioned method of grouping the controlled devices is only an example and is not limited in this embodiment.

[0371] As shown in FIG. 42 , after the user selects a type in the group, he or she can view the device information and control functions of the corresponding controlled device in the device control interface 506 and control the controlled device.

[0372] For example, assuming the user selects the "living room" option, the user can view information such as the device name, device type, and manufacturer of the connected controlled devices (such as controlled device A and controlled device B) in the living room in the device control interface 506.

[0373] As shown in Figure 43, the user can also use the remote control device to focus on the corresponding device control interface 506 to view the control functions of the corresponding controlled device. For example, if the focus is on the device control interface 506 corresponding to controlled device A, the control functions of controlled device A will be displayed, including power on or off, brightness adjustment, and color temperature adjustment; if the focus is on the device control interface 506 corresponding to controlled device B, the control functions of controlled device B will be displayed, including opening or closing the password lock.

[0374] In some embodiments, a user can control a controlled device by focusing the remote control device on a control function of the corresponding controlled device in the device control interface 506. For example, when the user focuses the remote control device on the brightness adjustment control of controlled device A, the user can use the control buttons (e.g., direction buttons) of the remote control device to adjust the brightness of controlled device A; when the user focuses the remote control device on the password lock switch control of controlled device B, the user can use the control buttons (e.g., confirm button) of the remote control device to turn controlled device B on or off.

[0375] In some embodiments, in addition to controlling the controlled device in the above manner, the user can also customize shortcut keys corresponding to the control functions of the controlled device in the control buttons of the remote control device, thereby controlling the controlled device through shortcut keys.

[0376] Refer to FIG44 for a schematic diagram of a shortcut key configuration interface.

[0377] As shown in FIG44 , if the user places the focus on the “Configure shortcut keys” option through the remote control device (as shown in FIG43 ), the display device can display the shortcut key configuration options corresponding to each control function of the controlled device in the shortcut key configuration interface 507 .

[0378] In some embodiments, if the user places the focus on the "Configure Shortcut Keys" option of the controlled device A through the remote control device, the user can enter corresponding configuration operations for the power switch, brightness adjustment, and color temperature adjustment control functions of the controlled device A in the shortcut key configuration interface 507, thereby configuring corresponding shortcut keys for the control functions. For example, the power switch can be configured with the on / off button of the remote control device, the brightness adjustment can be configured with the volume + / volume- of the remote control device, the color temperature adjustment can be configured with the channel + / channel- of the remote control device, and so on.

[0379] Assume that after the user completes the shortcut key configuration of the controlled device, the first key function mapping relationship between the control key created for the controlled device A and the control function of the controlled device A is shown in Table 3:

[0380] Table 3

[0381] And, the first key function mapping relationship between the control key created for the controlled device B and the control function of the controlled device B is shown in Table 4:

[0382] Table 4

[0383] It should be noted that the above method of configuring shortcut keys is only an example and is not limited to this in this embodiment.

[0384] S3420: Receive a control signal sent by the user through a control button of the remote control device.

[0385] S3430: In response to the control signal, determine the control object corresponding to the control button.

[0386] The user can control the controlled device or the display device by pressing the control button of the remote control device. After the user presses the control button, the remote control device sends a control signal corresponding to the control button to the display device.

[0387] After the display device receives the above control signal, since the control button may be configured by the user as a shortcut key for controlling the controlled device, it is first necessary to determine the control object corresponding to the above control button, that is, whether the device to be controlled by the user is the display device or the controlled device.

[0388] The control object corresponding to the control button may be determined according to the position of the focus in the user interface.

[0389] In the case where the user interface includes device control interfaces 506 corresponding to multiple controlled devices, the controlled device corresponding to the device control interface 506 where the focus position is located can be determined as the target controlled device, and the target device is the control object corresponding to the control button; and if the focus position of the user interface is on the main interface 501 corresponding to the display device, the control object of the control button can be determined to be the display device.

[0390] Refer to Figures 45 and 46 for a schematic diagram of determining a control object.

[0391] In some embodiments, assuming that the current user interface of the display device is in the state shown in Figure 45, it can be seen that the current focus position is on the device control interface 506 corresponding to the controlled device A, and the controlled device A can be determined as the target controlled device, that is, the controlled device A is the control object corresponding to the control button.

[0392] Assume that the current user interface of the display device is in the state shown in Figure 46. It can be seen that the current focus position is on application C in the main interface 501 corresponding to the display device, that is, the current focus position is not on any device control interface 506, so the display device can be determined as the control object corresponding to the control button.

[0393] The method for determining the control object corresponding to the above control button may also be determined according to the connection status between the controlled device and the display device.

[0394] In some embodiments, if the number of controlled devices is less than a preset threshold, the last controlled device to establish a communication connection with the display device may be determined as the target controlled device. For example, if there is only one controlled device (e.g., controlled device A) in the user's home that needs to establish a communication connection with the display device, then that controlled device may be determined as the target controlled device if it is connected to the display device.

[0395] If there is no controlled device that has established a communication connection with the display device, that is, the display device has not established a communication connection with any controlled device, the display device may be determined as the control object of the control button.

[0396] In other words, if the user needs to control a target controlled device among the controlled devices using control buttons, the focus can be set to the device control interface 506 corresponding to the target controlled device; alternatively, the last controlled device to establish a communication connection with the display device can be set as the target controlled device. If the user needs to control the display device using control buttons, the focus can be set to the main interface 501 corresponding to the display device, that is, the focus can be set not to any device control interface 506.

[0397] S3440: If the control object corresponding to the control button is the target controlled device, search for the first control function corresponding to the control button in the first button function mapping relationship corresponding to the target controlled device.

[0398] The target controlled device is any controlled device among the at least one controlled device.

[0399] S3450: If the first button function mapping relationship includes a first control function corresponding to the control button, convert the control signal into a control instruction corresponding to the first control function.

[0400] Next, after determining the control object, it is necessary to determine the control function corresponding to the control button. In the case where the control object is a target controlled device, the first button function mapping relationship corresponding to the target controlled device may include the control functions corresponding to some control buttons, or may include the control functions corresponding to all control buttons. Therefore, it is necessary to first determine whether the first button function mapping relationship includes the first control function corresponding to the control button.

[0401] S3460: Send a control instruction to the target controlled device, so that the target controlled device responds to the control instruction and performs the first control function.

[0402] If the control object corresponding to the control button is the target controlled device, the first control function corresponding to the control button is searched in the first button function mapping relationship corresponding to the target controlled device.

[0403] Referring to Table 3 above, if the target controlled device is controlled device A and the control button is "Volume +", it can be seen that the first button function mapping relationship includes the control button "Volume +", indicating that the control button can control controlled device A.

[0404] If the control button is found in the first button function mapping relationship, the first control function corresponding to the control button can be further determined, and the control signal can be converted into a control instruction corresponding to the first control function.

[0405] Continuing to refer to Table 3, it can be determined that the first control function corresponding to the control button "Volume +" is "Brightness Up", thus, the control signal of the control button "Volume +" is converted into a control instruction corresponding to the first control function "Brightness Up".

[0406] After determining the control instruction, the display device can send the control instruction to the target controlled device, so that the target controlled device responds to the control instruction and performs the first control function, thereby achieving control of the controlled device through the display device.

[0407] In some embodiments, after the control instruction corresponding to the first control function "brightness increase" is sent to the controlled device A, the controlled device A may respond to the control instruction and execute the brightness increase function.

[0408] When the focus position of the user interface is on the main interface corresponding to the display device, or the display device has not established a communication connection with any controlled device, or the corresponding control button is not found in the first button function mapping relationship corresponding to the target controlled device, the display device can be determined as the control object of the control button.

[0409] Continuing to refer to Table 3, if the target controlled device is controlled device B and the control button is "Channel+", it can be seen that the first button function mapping relationship does not include the above-mentioned control button "Channel+", indicating that the control button cannot control the controlled device B. In this case, the display device can be determined as the control object of the control button "Channel+".

[0410] In the case where the control object of the control button is a display device, the display device can determine the second control function corresponding to the control button according to its corresponding second button function mapping relationship and execute the second control function. A possible second button function mapping relationship corresponding to the display device is shown in Table 5:

[0411] Table 5

[0412] As shown in Table 5, it can be seen from the second key function mapping relationship that the second control function corresponding to the control key “channel+” is “channel plus one”, and the display device can execute the second control function.

[0413] Through the above scheme, after receiving a control signal sent by the user via a control button, the display device can perform a logical judgment on whether the control button is used to control the controlled device or the display device, and whether the control button has a mapping relationship with the control function of the controlled device. This can avoid situations where a conflict may occur with the display device when controlling the controlled device via the control button, and can also be applied to situations where the control functions of multiple controlled devices have the same mapping relationship with the control button. The embodiments of the present disclosure can realize the control of the controlled device through the control button set by the user, and improve the efficiency of controlling the controlled device via the display device.

[0414] 47 and 48 show schematic diagrams of a method for controlling multimedia resources of a display device and a controlled device provided by the present disclosure.

[0415] If the controlled device currently connected to the display device includes controlled device C, the second multimedia resource corresponding to controlled device C can be played in the device control interface 506. For example, as shown in FIG47 , if controlled device C is a camera, the user can view the audio and video images captured by the camera in the device control interface 506.

[0416] It is understandable that the first multimedia resource corresponding to the display device can also be played in the main interface 501 of the display device. However, if the first multimedia resource and the second multimedia resource are played at the same time, resource conflict will occur.

[0417] Thus, in some embodiments, as shown in FIG47 , when a user inputs a mute command for a second multimedia resource, the second multimedia resource can be controlled to be in a mute state, and the first multimedia resource played in the main interface can be controlled to be in a non-mute state. Alternatively, when a user inputs a non-mute command for a first multimedia resource, the first multimedia resource can be controlled to be in a non-mute state, and the second multimedia resource can be controlled to be in a mute state.

[0418] In some embodiments, as shown in FIG48 , when a user inputs a mute command for a first multimedia resource, the first multimedia resource may be muted, and a second multimedia resource played in the device control interface 506 may be unmuted. Alternatively, when a user inputs an unmuted command for a second multimedia resource, the second multimedia resource may be unmuted, and the first multimedia resource may be muted.

[0419] Therefore, assuming that the current first multimedia resource is in a muted state and the second multimedia resource is in a non-muted state, if the user wants to play the sound of the first multimedia resource, then the user only needs to input a non-muted command for the first multimedia resource (or input a mute command for the second multimedia resource), and the display device will automatically control the first multimedia resource to a non-muted state and the second multimedia resource to a muted state.

[0420] Through the above scheme, resource conflicts between the multimedia resources played by the display device and the controlled device can be avoided, and the user only needs to input a mute command / unmute command for the first multimedia resource or the second multimedia resource, and the display device can automatically control the mute state of the other multimedia resource, thereby facilitating the user's operation.

[0421] In summary, after receiving a control signal sent by a user via a control button, the display device of the disclosed embodiment can perform a logical judgment on whether the control button is used to control the controlled device or the display device, and whether the control button has a mapping relationship with the control function of the controlled device. This can avoid situations where a conflict may occur with the display device when controlling the controlled device via the control button, and can also be applied to situations where the control functions of multiple controlled devices have the same mapping relationship with the control button. This embodiment can achieve control of the controlled device via the control button set by the user, and improve the efficiency of controlling the controlled device via the display device.

[0422] Furthermore, this embodiment can control the mute status of multiple multimedia resources according to the user input of a mute command for a multimedia resource, thereby avoiding resource conflicts between the multimedia resources played by the display device and the controlled device and facilitating user operations.

[0423] Based on the same concept, an embodiment of the present disclosure also provides a device control device. As shown in Figure 49, the device control device 1200 can be applied to a display device. The device control device 1200 may include: a connection module 1210, a receiving module 1220, a determination module 1230, a search module 1240, a conversion module 1250, and a sending module 1260.

[0424] The connection module 1210 is configured to establish a communication connection with at least one controlled device.

[0425] The receiving module 1220 is configured to receive a control signal sent by a user through a control button of a remote control device.

[0426] The determination module 1230 is configured to determine the control object corresponding to the control button in response to the control signal.

[0427] The search module 1240 is configured to search for the first control function corresponding to the control button in the first button function mapping relationship corresponding to the target controlled device if the control object corresponding to the control button is a target controlled device; the target controlled device is any controlled device among the at least one controlled device.

[0428] The conversion module 1250 is configured to convert the control signal into a control instruction corresponding to the first control function if the first key function mapping relationship includes a first control function corresponding to the control key.

[0429] The sending module 1260 is configured to send a control instruction to a target controlled device, so that the target controlled device responds to the control instruction and performs a first control function.

[0430] In some embodiments, the determination module 1230 is specifically configured to: when the user interface includes device control interfaces corresponding to multiple controlled devices, determine the controlled device corresponding to the device control interface where the focus is located as the target controlled device.

[0431] In some embodiments, the determining module 1230 is specifically configured to: when the number of at least one controlled device is less than a preset threshold, determine the controlled device that last establishes a communication connection with the display device as the target controlled device.

[0432] In some embodiments, the determination module 1230 is also used to: if the focus position of the user interface is on the main interface corresponding to the display device, or the display device has not established a communication connection with any controlled device, then determine that the control object of the control button is the display device; determine the second control function corresponding to the control button according to the second button function mapping relationship corresponding to the display device, and execute the second control function.

[0433] As shown in FIG. 49 , the device control apparatus 1200 may further include: a creation module 1270 .

[0434] In some embodiments, the remote control device includes multiple control buttons, and the creation module 1270 is used to create a first button function mapping relationship between the control button and the control function of each controlled device according to the configuration operation input by the user.

[0435] As shown in FIG. 49 , the device control apparatus 1200 may further include: a control module 1280 .

[0436] In some embodiments, the user interface includes a main interface and a device control interface, the main interface is used to play the first multimedia resource corresponding to the display device, and the device control interface is used to play the second multimedia resource corresponding to the controlled device; the control module 1280 is used to: respond to the user input of a mute instruction for the second multimedia resource, control the second multimedia resource to a mute state, and control the first multimedia resource played in the main interface to a non-mute state; or, respond to the user input of a mute instruction for the first multimedia resource, control the first multimedia resource to a mute state, and control the second multimedia resource played in the device control interface to a non-mute state.

[0437] In some embodiments, the controlled device establishes a communication connection with the display device based on the Matter protocol through a QR code and / or a pairing code input by a user, and the QR code and the pairing code include parameter information of the controlled device.

[0438] In some embodiments, the user interface includes a device control interface, which is used to display device information of a controlled device and control functions of the controlled device.

[0439] An embodiment of the present disclosure provides a computer-readable storage medium storing at least one executable instruction. When the executable instruction is executed on a display device / device control apparatus, the display device / device control apparatus executes the device control method in any of the above method embodiments.

[0440] The executable instructions can be specifically used to enable the display device / device control apparatus to execute the above-mentioned device control method.

Claims

1. A display device, comprising: A display configured to display an image and / or a user input interface: A user input interface configured to receive instructions from a user; a communication device configured to communicate with an external device according to a predetermined protocol; The memory is configured to: store endpoint information, the endpoint information including a first endpoint and a first service function corresponding to the first endpoint, each of the first endpoints corresponds to a device type defined in the Matter protocol, the first endpoints include a designated endpoint corresponding to a bridge device type defined in the Matter protocol, and the designated endpoint corresponds to the first service function including that the display device supports serving as a bridge device; At least one processor, connected to the display, the user input interface, the communication device and the memory, is configured to execute computer instructions to cause the display device to perform: Establishing communication with a first controlled device based on the first protocol, where the first controlled device is a non-Matter device and supports a first protocol representing a non-Matter protocol; Creating a second endpoint corresponding to the first controlled device and a second service function corresponding to the second endpoint, where the second service function corresponds to a service capability supported by the first controlled device; Add the second endpoint and the second service function to the endpoint information, and provide the second endpoint and the second service function to the master device after performing network configuration with the master device based on the Matter protocol; A first instruction sent by the master control device to the first controlled device is received, the first instruction is converted into a second instruction supporting the first protocol, and the second instruction is sent to the first controlled device.

2. The display device according to claim 1, wherein the at least one processor is specifically configured to execute computer instructions so that the display device establishes communication with the first controlled device based on the first protocol by performing the following: In response to a search instruction of a user, searching for surrounding devices based on the first protocol, and establishing communication with the first controlled device after discovering the first controlled device; Alternatively, in response to the pairing request sent by the first controlled device, communication is established with the first controlled device.

3. The display device according to claim 1, wherein the at least one processor is specifically configured to execute computer instructions so that the display device creates a second endpoint corresponding to the first controlled device and a second service function corresponding to the second endpoint by performing the following: Acquire device information of the first controlled device, where the device information includes a device type of the first controlled device and service capabilities supported by the first controlled device; A second endpoint corresponding to the first controlled device is created according to the Matter protocol, the device type of the first controlled device, and the service capabilities supported by the first controlled device, and a second service function corresponding to the second endpoint is created according to the service capabilities supported by the first controlled device.

4. The display device according to claim 3, wherein the at least one processor is specifically configured to execute computer instructions so that the display device creates a second endpoint corresponding to the first device according to the Matter protocol, the device type of the first device, and the service capabilities supported by the first device, and creates a second service function corresponding to the second endpoint according to the service capabilities supported by the first device by performing the following: After acquiring the device information of the first controlled device, the second endpoint is created directly according to the Matter protocol, the device type of the first controlled device and the business capabilities supported by the first controlled device, and the second business function is created according to the business capabilities supported by the first controlled device.

5. The display device according to claim 3, wherein the device information further comprises a device name of the first controlled device, and the at least one processor is further configured to execute computer instructions to cause the display device to execute: After acquiring the device information of the first controlled device, the display is controlled to display the first controlled device list according to the device name; wherein, The first controlled device list includes an item of a first device, a corresponding device name is displayed on the item of the first device, the first device is a non-Matter device that establishes communication with the display device based on the first protocol and does not create an endpoint on the display device; the first device includes the first controlled device; The at least one processor is specifically configured to execute computer instructions so that the display device creates a second endpoint corresponding to the first device according to the Matter protocol, the device type of the first device, and the service capabilities supported by the first device, and creates a second service function corresponding to the second endpoint according to the service capabilities supported by the first device by performing the following: In response to an add instruction input by a user based on the project of the first controlled device, the second endpoint is created according to the Matter protocol, the device type of the first controlled device and the business capabilities supported by the first controlled device, and the second business function is created according to the business capabilities supported by the first controlled device.

6. The display device according to claim 5, wherein the at least one processor is specifically configured to execute computer instructions so that the display device creates the second endpoint according to the Matter protocol, the device type of the first controlled device, and the service capabilities supported by the first controlled device, and creates the second service function according to the service capabilities supported by the first controlled device in response to an add instruction input by a user based on the project of the first controlled device by executing the following: In response to the adding instruction, creating the second endpoint according to the Matter protocol, the device type of the first controlled device and the service capabilities supported by the first controlled device, and controlling the display to display an adding page according to the service capabilities supported by the first controlled device, wherein the adding page includes the service capabilities supported by the first controlled device; In response to a selection instruction input by the user based on the target service capability, the corresponding second service function is created.

7. The display device according to claim 3, wherein the at least one processor is specifically configured to execute computer instructions so that the display device creates a second endpoint corresponding to the first controlled device according to the Matter protocol, the device type of the first controlled device, and the service capabilities supported by the first controlled device, and creates a second service function corresponding to the second endpoint according to the service capabilities supported by the first controlled device by performing the following: Querying a device type list in the Matter protocol, the device type list including a first definition of a device type, the first definition including a device identifier corresponding to the device type and a device name corresponding to the device identifier, the device name corresponding to at least one service capability; Determine a target device identifier and a target device name corresponding to the first controlled device according to the device type list, the device type of the first controlled device, and the service capability supported by the first controlled device; Creating the second endpoint according to the target device identifier and the target device name; Querying a business function list in the Matter protocol, the business function list including a second definition of the business function, the second definition including a correspondence between the business function and the business capability; Determining, according to the correspondence between the service functions and service capabilities, the second service function corresponding to the service capability supported by the first controlled device; Add the second service function to the second endpoint.

8. The display device according to claim 1, wherein the at least one processor is specifically configured to execute computer instructions so that the display device provides the second endpoint and the second service function to the main control device by performing the following: After performing network configuration with the master device based on the Matter protocol, actively provide the master device with the endpoint information, where the endpoint information includes the first endpoint and the first service function, and the second endpoint and the second service function; Alternatively, in response to a read request sent by the master control device, the second endpoint and the second service function are sent to the master control device, wherein the read request is used to request to read the endpoint information of the first controlled device.

9. The display device according to claim 1, wherein the at least one processor is further configured to execute computer instructions to cause the display device to perform: After network configuration with the master device based on the Matter protocol, communication is established with the second device based on the first protocol; wherein, The second device is a non-Matter device and supports the first protocol; Creating a third endpoint corresponding to the second device and a third service function corresponding to the third endpoint; wherein the third service function corresponds to a service capability supported by the second device; The third endpoint and the third service function are added to the endpoint information, and the third endpoint and the third service function are provided to the main control device.

10. The display device according to claim 1, wherein the at least one processor is specifically configured to execute computer instructions so that the display device establishes communication with the first controlled device based on the first protocol by: After networking with the master device based on the Matter protocol, searching for surrounding devices in response to a search instruction sent by the master device; After discovering the first controlled device, obtaining a device name of the first controlled device, and sending the device name to the master control device; In response to the control instruction sent by the master control device, communication is established with the first controlled device.

11. The display device according to claim 1, wherein the at least one processor is further configured to execute computer instructions to cause the display device to perform: In response to a network configuration instruction input by a user, sending device information of a second controlled device to the main control device; Acquire authentication code information through the first communication channel, where the authentication code information is generated by the master control device scanning a pairing code corresponding to the device information; Performing wireless communication scanning on the network distribution devices within the first preset range according to the identification code information; If the first scanning result is generated, a second communication channel is established with the second controlled device, and network configuration is performed on the second controlled device through the second communication channel.

12. The display device according to claim 11, wherein the at least one processor is specifically configured to execute computer instructions so that the display device obtains the authentication code information through the first communication channel by performing the following: Sending a scan instruction to the main control device through the first communication channel; Obtaining scanning information obtained by the master device scanning the pairing code; A decoding operation is performed on the scanned code information to obtain the authentication code information.

13. The display device according to claim 11, wherein the at least one processor is specifically configured to execute computer instructions so that the display device performs wireless communication scanning on the network distribution devices within the first preset range according to the identification code information by performing the following: Switching the wireless communication operation state of the display device to a start state according to the authentication code information; Performing wireless communication scanning on the network configuration devices within the first preset range to obtain network configuration device information; Perform pairing on the network configuration device information according to the authentication code information; If there is network configuration device information that is successfully paired, the first scanning result is generated, and the network configuration device corresponding to the network configuration device information is marked as a second controlled device.

14. The display device according to claim 13, wherein the at least one processor is further configured to execute computer instructions to cause the display device to perform: Recording a first pairing duration of the display device; if the first pairing duration exceeds a first duration threshold and the first scanning result is not generated, generating a second scanning result; Switching the wireless communication operation state of the master control device to a startup state according to the second scanning result; The main control device performs wireless communication scanning on the network distribution devices within the second preset range.

15. The display device according to claim 14, wherein the at least one processor is specifically configured to execute computer instructions so that the display device performs wireless communication scanning on the network distribution devices within the second preset range through the main control device by performing the following: Sending a terminal scanning instruction to the main control device according to the second scanning result; Acquire network configuration feedback information sent by the master control device, where the network configuration feedback information is network configuration device information acquired by the master control device within the second preset range after responding to the terminal scanning instruction; If there is successfully paired network configuration device information in the network configuration feedback information, marking the network configuration device corresponding to the network configuration device information as the second controlled device; A network configuration instruction is sent to the master control device, where the network configuration instruction is used to instruct the master control device to establish a wireless communication connection with a second controlled device.

16. The display device according to claim 15, wherein the at least one processor is specifically configured to execute computer instructions so that the display device obtains the network configuration feedback information sent by the main control device by performing the following: Obtaining a moving track of the master control device; Delimiting the second preset range according to the movement trajectory; According to the identification code information, the network configuration device information within the second preset range is acquired through the main control device.

17. The display device according to claim 15, wherein the at least one processor is further configured to execute computer instructions to cause the display device to perform: Recording a second pairing duration of the master control device through the first communication channel; If the second pairing time exceeds a second time threshold, search failure prompt information generated by the main control device is obtained.

18. The display device according to any one of claims 1 to 17, wherein the at least one processor is further configured to execute computer instructions to cause the display device to execute: After establishing a communication connection with at least one controlled device, in response to a control signal sent through a control button of the remote control device, a control object corresponding to the control button is determined; wherein, The controlled device includes the first controlled device and the second controlled device; If the control object corresponding to the control button is a target controlled device, searching for the first control function corresponding to the control button in the first button function mapping relationship corresponding to the target controlled device; wherein the target controlled device is any controlled device among the at least one controlled device; If the first button function mapping relationship includes the first control function corresponding to the control button, converting the control signal into a control instruction corresponding to the first control function; The control instruction is sent to the target controlled device, so that the target controlled device responds to the control instruction and performs the first control function.

19. The display device according to claim 18, wherein the at least one processor is further configured to execute computer instructions to cause the display device to perform: In the case where the user interface includes a plurality of device control interfaces corresponding to the controlled devices, the controlled device corresponding to the device control interface where the focus position is located is determined as the target controlled device.

20. The display device according to claim 18, wherein the at least one processor is further configured to execute computer instructions to cause the display device to perform: When the number of the at least one controlled device is less than a preset threshold, the controlled device that last establishes a communication connection with the display device is determined as the target controlled device.

21. The display device according to claim 19, wherein the at least one processor is further configured to execute computer instructions to cause the display device to perform: If the focus position of the user interface is on the main interface corresponding to the display device, or the display device has not established a communication connection with any of the controlled devices, determining that the control object of the control button is the display device; The second control function corresponding to the control button is determined according to the second button function mapping relationship corresponding to the display device, and the second control function is executed.

22. According to the display device according to claim 18, the remote control device includes a plurality of the control buttons, and the at least one processor is further configured to execute computer instructions so that the display device performs: creating the first button function mapping relationship between the control button and the control function of each controlled device for each controlled device according to the configuration operation input by the user.

23. The display device according to claim 18, wherein the user interface comprises a main interface for playing a first multimedia resource corresponding to the display device, and a device control interface for playing a second multimedia resource corresponding to the controlled device; and the at least one processor is further configured to execute computer instructions to cause the display device to execute: In response to a user input of a mute instruction for the second multimedia resource, controlling the second multimedia resource to be in a mute state, and controlling the first multimedia resource played in the main interface to be in a non-mute state; or, In response to a mute instruction input by a user for the first multimedia resource, the first multimedia resource is controlled to be in a mute state, and the second multimedia resource played in the device control interface is controlled to be in a non-mute state.

24. The display device according to claim 18, wherein the controlled device establishes a communication connection with the display device based on the Matter protocol via a QR code and / or a pairing code input by a user, and the QR code and the pairing code include parameter information of the controlled device. 25 . The display device according to claim 18 , wherein the user interface comprises a device control interface, and the device control interface is used to display device information of the controlled device and control functions of the controlled device.

26. A device control method, applied to the display device according to any one of claims 1 to 25, comprising: Establishing communication with a first controlled device based on a first protocol, where the first controlled device is a non-Matter device and supports a first protocol representing a non-Matter protocol; Creating a second endpoint corresponding to the first controlled device and a second service function corresponding to the second endpoint, where the second service function corresponds to a service capability supported by the first controlled device; Add the second endpoint and the second service function to the endpoint information, and provide the second endpoint and the second service function to the master device after performing network configuration with the master device based on the Matter protocol; A first instruction sent by the master control device to the first controlled device is received, the first instruction is converted into a second instruction supporting the first protocol, and the second instruction is sent to the first controlled device.

Images