Remote control method, electronic equipment and communication system

Abstract

The invention discloses a remote control method, electronic equipment and a communication system. The method is applied to a communication system comprising first equipment (controlled equipment), second equipment (master control equipment) and a server. And after the controlled equipment is shut down, the controlled equipment establishes long connection with the server through the auxiliary processor, and when the controlled equipment is in a startup state, the controlled equipment establishes communication connection with the server through the main processor. After the controlled device is connected with the server through the main processor / auxiliary processor, a corresponding remote control instruction is received. Besides, before responding to the remote control instruction, the controlled device can execute any one or more of the following operations: performing security authentication with the server, performing security authentication with the main control device, regularly refreshing a password corresponding to a cloud account used for establishing connection with the server, and encrypting and storing the cloud account and the corresponding password thereof. And the safety of remote control is improved.

Description

Technical Field

[0001] The present application relates to the field of terminals, and in particular to a remote control method, electronic equipment and a communication system. Background Art

[0002] With the development of terminal technology, users can remotely control controlled devices through the main control device to perform various tasks. How to prevent the controlled devices from being maliciously controlled by illegal devices and ensure the security of remote control is an urgent problem to be solved. Summary of the invention

[0003] The present application discloses a remote control method, an electronic device and a communication system. The method is applied to a communication system including a first device (controlled device), a second device (master device) and a server. After the controlled device is turned off, the controlled device establishes a long connection with the server through the sub-processor, and when the controlled device is in the on state, a communication connection is established with the server through the main processor. After the controlled device is connected to the server through the main processor / sub-processor, it receives and responds to remote control instructions. In addition, before responding to the remote control instruction, the controlled device may also perform any one or more of the following: security authentication with the server, security authentication with the master device, and periodically refresh the password corresponding to the cloud account used to establish a connection with the server, and encrypt and store the cloud account and its corresponding password, thereby improving the security of remote control.

[0004] In a first aspect, the present application provides a remote control method, which is applied to a communication system including a first device, a second device and a server, wherein the first device includes a main processor and a sub-processor, and the method includes: the main processor of the first device is in a working state, and the authentication information of the first device is sent to the server through the main processor; the server sends the authentication information of the first device to the second device; the second device sends an authentication request to the first device through the server, and the authentication request carries first data encrypted using a public key of the first device, and the first data is calculated by a preset algorithm for any one or more of the following data: the authentication information, a random number of the first device, and a random number of the second device; the main processor of the first device is in a non-working state, and the first data is calculated by the sub-processor using the private key of the first device and the preset algorithm to obtain the first data for the authentication information, and a first key is determined based on the first data; the second device sends a first instruction encrypted using the first key to the first device through the server; the first device decrypts the first instruction through the sub-processor using the first key to obtain the first instruction, and in response to the first instruction, starts the main processor through the sub-processor.

[0005] After the method described in the first aspect is implemented, the security of the communication information between the controlled device (first device) and the main control device (second device) can be guaranteed, thereby improving the security of remote control.

[0006] In combination with the method described in the first aspect, the authentication information of the first device includes any one or more of the following: an authentication code of the first device, an identifier of the first device, or a public key of the first device.

[0007] In this way, the second device and the first device can subsequently negotiate the first key based on one or more data, thereby ensuring the reliability of the first key.

[0008] In combination with the method described in the first aspect, the first device determines the first key based on the first data through the sub-processor specifically including: the first device determines the first data as the first key through the sub-processor; or, the first device generates the first key through the first data and a second algorithm, and sends the second algorithm to the first device through the server.

[0009] In combination with the method described in the first aspect, the first device sends the authentication information of the first device to the server through the main processor, specifically including: the first device sends a registration request to the server through the main processor, and the registration request carries the authentication information of the first device encrypted using the public key of the server.

[0010] In this way, the security of the communication information between the first device and the server can be guaranteed.

[0011] In combination with the method described in the first aspect, the server responds to the registration request, uses the server's private key to decrypt to obtain the authentication information of the first device, and sends the first account and the first password to the first device.

[0012] In combination with the method described in the first aspect, after the server sends the first account and the first password to the first device, the method also includes: when the main processor of the first device is in a working state, the first device uses the first account and the first password to establish a second connection with the server through the main processor; when the main processor of the first device is in a non-working state, the first device uses the first account and the first password to establish a first connection with the server through the secondary processor.

[0013] In this way, the controlled device can establish a connection with the server through different processors in different states, and thus remote control can be achieved in more scenarios. For example, when the controlled device is turned off, the secondary processor can be used to remotely wake up the main processor and then control the controlled device to start up.

[0014] In combination with the method described in the first aspect, the performance of the main processor is better than the performance of the secondary processor, and the first connection is disconnected when the second connection is established.

[0015] In combination with the method described in the first aspect, after the server sends the first account and the first password to the first device, the method also includes: when the usage time of the first password exceeds the preset time, the server refreshes the first password to the second password and sends the second password to the first device.

[0016] This allows passwords to be refreshed regularly, further ensuring secure communications between the controlled device and the server, and further improving the security of remote control.

[0017] In combination with the method described in the first aspect, after the server sends the second password to the first device, the method also includes: the first device receives the second password through the main processor and sends the second password to the secondary processor; or, the first device receives the second password through the secondary processor.

[0018] In combination with the method described in the first aspect, after the first device receives the second password through the secondary processor, the method also includes: when the main processor of the first device is in a working state, the first device uses the first account and the first password to request the server to establish the second connection through the main processor; the server determines that the first password is different from the second password; the server refreshes the second password to a third password, and sends the third password to the first device; the first device receives the third password through the main processor, and sends the third password to the secondary processor.

[0019] In combination with the method described in the first aspect, before the server refreshes the first password to the second password, the method further includes: the first device sends signature verification information to the server, the signature verification information is generated by encrypting the public key of the server; and the server uses the private key of the server to verify the signature verification information.

[0020] In combination with the method described in the first aspect, before the first device starts the main processor through the secondary processor, the method also includes: the first device also receives the device account and the password corresponding to the device account sent by the second device through the server; the first device determines that the device account is the same as the account of the first device, and the password corresponding to the device account is the same as the password corresponding to the account of the first device.

[0021] In this way, we can further authenticate the instructions of the master device based on the device account and corresponding password of the controlled device, further ensuring the security of remote control.

[0022] In a second aspect, the present application provides a method, which is applied to a first device including a main processor and a sub-processor, and the method includes: the main processor of the first device is in a working state, and the authentication information of the first device is sent to a server through the main processor, so that the server sends the authentication information of the first device to a second device; the first device receives an authentication request sent by the second device through the server, and the authentication request carries first data encrypted using a public key of the first device, and the first data is calculated by a preset algorithm for any one or more of the following data: the authentication information of the first device, a random number of the first device, and a random number of the second device; the main processor of the first device is in a non-working state, and the first data is calculated by the sub-processor using the private key of the first device and the preset algorithm to obtain the first data for the authentication information, and a first key is determined based on the first data; the first device receives a first instruction encrypted using the first key and sent by the second device through the server; the first device decrypts the first instruction through the sub-processor using the first key to obtain the first instruction, and in response to the first instruction, starts the main processor through the sub-processor.

[0023] In combination with the method described in the second aspect, the authentication information of the first device includes any one or more of the following: an authentication code of the first device, an identification of the first device.

[0024] In combination with the method described in the second aspect, the first device determines the first key based on the first data through the sub-processor, specifically including: the first device determines the first data as the first key through the sub-processor; or, the first device generates the first key through the first data and a second algorithm, and sends the second algorithm to the first device through the server.

[0025] In combination with the method described in the second aspect, the first device sends the authentication information of the first device to the server through the main processor specifically includes: the first device sends a registration request to the server through the main processor, and the registration request carries the authentication information of the first device encrypted using the public key of the server.

[0026] In combination with the method described in the second aspect, the method further includes: when the authentication information of the first device is obtained by decrypting the private key of the server, the first device receives the first account and the first password sent by the server.

[0027] In combination with the method described in the second aspect, the first device receives the first account and the first password sent by the server, specifically including: when the main processor of the first device is in a working state, the first device uses the first account and the first password to establish a second connection with the server through the main processor; when the main processor of the first device is in a non-working state, the first device uses the first account and the first password to establish a first connection with the server through the secondary processor.

[0028] In combination with the method described in the second aspect, the performance of the main processor is better than the performance of the secondary processor, and the first connection is disconnected when the second connection is established.

[0029] In combination with the method described in the second aspect, after the server sends the first account and the first password to the first device, the method also includes: when the usage time of the first password exceeds the preset time, the first device receives the second password sent by the server.

[0030] In combination with the method described in the second aspect, the first device receives the second password sent by the server, specifically including: the first device receives the second password sent by the server through the main processor, and sends the second password to the secondary processor; or, the first device receives the second password sent by the server through the secondary processor.

[0031] In combination with the method described in the second aspect, after the first device receives the second password sent by the server through the secondary processor, the method also includes: when the main processor of the first device is in a working state, the first device uses the first account and the first password to request the server to establish the second connection through the main processor; the first device receives the third password sent by the server through the main processor, and sends the third password to the secondary processor.

[0032] In combination with the method described in the second aspect, before the first device receives the second password sent by the server, the method also includes: the first device sends signature verification information to the server, the signature verification information is generated after being encrypted by the public key of the server, and the signature verification information is used by the server to verify using the private key of the server.

[0033] In combination with the method described in the second aspect, before the first device starts the main processor through the secondary processor, the method also includes: the first device also receives the device account and the password corresponding to the device account sent by the second device through the server; the first device determines that the device account is the same as the account of the first device, and the password corresponding to the device account is the same as the password corresponding to the account of the first device.

[0034] In combination with the method described in the second aspect, the first device encrypts and stores any one or more of the following: authentication information of the first device, the first key, the public key of the server, the public key of the first device, the private key of the first device, the account of the first device or the password corresponding to the account of the first device.

[0035] In a third aspect, the present application provides a remote control method, which is applied to a second device, and the method includes: the second device receives authentication information of the first device forwarded by the server; the first authentication information is that the server receives the authentication information sent by the first device when the main processor of the first device is in a working state; the second device sends an authentication request to the first device through the server, and the authentication request carries first data encrypted using the public key of the first device, and the first data is calculated by a preset algorithm for any one or more of the following data: the authentication information, a random number of the first device, and a random number of the second device; the authentication request is used when the main processor of the first device is in a non-working state, and the authentication information is calculated by the secondary processor using the private key of the first device and the preset algorithm to obtain the first data, and a first key is determined based on the first data; the second device sends a first instruction encrypted by the first key to the first device through the server, so that the first device decrypts the first instruction through the secondary processor using the first key to obtain the first instruction, and in response to the first instruction, starts the main processor through the secondary processor.

[0036] In a fourth aspect, the present application provides an electronic device, comprising multiple memories and multiple processors; the memory is coupled to the processor, the memory is used to store computer program code, the computer program code comprises computer instructions, and the processor calls the computer instructions to enable the electronic device to execute a method as described in any one of the second aspect or the third aspect.

[0037] In a fifth aspect, the present application provides a chip, which is applied to an electronic device, and the chip is used to call computer instructions so that the electronic device executes the method described in any one of the second aspect or the third aspect.

[0038] In a sixth aspect, the present application provides a computer-readable storage medium comprising instructions, which, when executed on an electronic device, causes the electronic device to execute a method as described in any one of the second aspect or the third aspect.

[0039] In the seventh aspect, the present application provides a communication system, which includes a first device, a second device and a server; the first device includes multiple memories and multiple processors; the memory is coupled to the processor, the memory is used to store computer program code, the computer program code includes computer instructions, and the processor calls the computer instructions to enable the first device to perform the method described in any one of the second aspects; the second device includes multiple memories and multiple processors; the memory is coupled to the processor, the memory is used to store computer program code, the computer program code includes computer instructions, and the processor calls the computer instructions to enable the second device to perform the method described in any one of the third aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A schematic diagram of the architecture of a communication system 10 provided in an embodiment of the present application;

[0041] Figure 2 A schematic diagram of the hardware architecture of a main control device provided in an embodiment of the present application;

[0042] Figure 3A A schematic diagram of the hardware architecture of a controlled device provided in an embodiment of the present application;

[0043] Figure 3B A schematic diagram of a software architecture of a controlled device provided in an embodiment of the present application;

[0044] Figure 4 A schematic diagram of the hardware architecture of a server provided in an embodiment of the present application;

[0045] Figure 5A-5D A schematic diagram of an operation interface for normal shutdown of a group of controlled devices provided in an embodiment of the present application;

[0046] Figure 6A-6H A schematic diagram of a set of operation interfaces for remotely controlling the startup of a controlled device through a master device provided in an embodiment of the present application;

[0047] Figure 7A-7C A schematic diagram of a set of operation interfaces for controlling the power on of a controlled device through local operation provided in an embodiment of the present application;

[0048] Figure 8A-8F A schematic diagram of a scenario in which a group of sub-processors of a controlled device are powered off and then powered on again, provided in an embodiment of the present application;

[0049] Fig. 9 A flow chart of a remote control method provided by this application;

[0050] Fig.10 The interaction process between various devices involved in a remote control method provided in this application;

[0051] Fig.11 An interactive process for realizing remote control through a secondary processor of a main control device provided in this application;

[0052] Fig.12 This is an interactive process for implementing remote control through a main processor of a main control device according to an embodiment of the present application;

[0053] Fig.13 An interactive process for synchronizing information between a primary and secondary processors is provided in an embodiment of the present application. DETAILED DESCRIPTION

[0054] The technical solutions in the embodiments of the present application will be described clearly and in detail below in conjunction with the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in the text is only a description of the association relationship of the associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0055] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.

[0056] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0057] The term "user interface (UI)" in the following embodiments of the present application refers to a medium interface for interaction and information exchange between an application or operating system and a user, which realizes the conversion between the internal form of information and the form acceptable to the user. The user interface is a source code written in a specific computer language such as Java and extensible markup language (XML). The interface source code is parsed and rendered on an electronic device, and finally presented as content that can be recognized by the user. The commonly used form of user interface is a graphical user interface (GUI), which refers to a user interface related to computer operation displayed in a graphical manner. It can be a visual interface element such as text, icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, etc. displayed on the display screen of an electronic device.

[0058] The present application discloses a remote control method, an electronic device and a communication system. The method is applied to a communication system including a master control device, a controlled device and a server. After the controlled device is turned off, the controlled device establishes a long connection with the server through the sub-processor. When the controlled device is in the on state, a communication connection is established with the server through the main processor. The controlled device can receive and respond to remote control instructions through the connection between the main processor / sub-processor and the server. In addition, before responding to the remote control instruction, the controlled device must also perform security authentication with the server, perform security authentication with the master control device, and regularly refresh the password corresponding to the cloud account used to establish a connection with the server, and encrypt and store the cloud account and its corresponding password.

[0059] Next, we will introduce several important concepts involved in this application, including:

[0060] Long connection is the relative concept of short connection. Short connection means that the controlled device and the server establish a connection each time they need to send data, and disconnect after the data is sent. That is, each established connection is only used to complete one data sending operation. Long connection means that the controlled device and the server do not disconnect after establishing the connection and sending data, but continue to maintain the connection. When data needs to be sent later, the data can be sent directly on the maintained connection. When the controlled device and the server maintain a long connection, the controlled device and the server can send multiple data packets continuously on one connection. Moreover, during the long connection, if no data packets are sent for a long time, the controlled device and the server are usually required to detect the current communication link to detect the failure of the communication link in time.

[0061] The reconnection function includes: after the secondary processor inactively disconnects the long connection with the server, the secondary processor can automatically re-request to establish the long connection with the server. For example, if the secondary processor suddenly disconnects and disconnects the long connection with the server, after power is restored, the secondary processor will automatically trigger a request to re-request to establish the long connection with the server.

[0062] In the embodiment of the present application, in addition to after the sub-processor of the controlled device determines that the controlled device is shut down, the controlled device establishes a long connection with the server through the sub-processor to achieve remote startup, as long as the sub-processor determines that the main processor is in a non-working state, the controlled device establishes a long connection with the server through the sub-processor to achieve remote functions.

[0063] The main processor is in a non-working state including but not limited to: the controlled device is powered off, dormant, sleeping, or powered on but faulty. Specifically, when the controlled device is in a dormant or sleeping state, the controlled device establishes a long connection with the server through the secondary processor to achieve remote wake-up, or when the controlled device is powered on but faulty, the controlled device establishes a long connection with the server through the secondary processor to achieve remote restart.

[0064] The main processor is in a working state including: the state where the controlled device is powered on but not faulty. Specifically, when the controlled device is powered on, the main processor is powered on, so it is ready for operation. Further, if the controlled device is not faulty, the main processor can operate normally. Generally, whether the main processor is in a non-working state or a working state, the sub-processor is in a working state.

[0065] It can be seen that after implementing the remote control method provided by this application, the security of remote control can be guaranteed in many aspects, and specifically the following technical effects can be brought about:

[0066] (1) Ensure the security of communication information between the controlled device and the server.

[0067] (2) Ensure the security of communication information between the controlled device and the master device.

[0068] (3) Ensure the security of privacy information stored in the main processor and sub-processor of the controlled device.

[0069] Next, a schematic diagram of the architecture of the communication system 10 to which the remote control method provided in the present application is applied, as well as the product forms of various devices included in the communication system 10, etc. are introduced.

[0070] refer to Figure 1 , Figure 1 The schematic diagram of the architecture of a communication system 10 provided in an embodiment of the present application is exemplified.

[0071] like Figure 1As shown, the communication system 10 may include a master device, a server, and a controlled device. Figure 1 The number of each device is shown as one by way of example, but the number of the master device, the server, and the controlled device can be one or more, respectively, and this is not limited in the embodiments of the present application.

[0072] The master device is also called the second device. The master device can be equipped with Or portable terminal devices with other operating systems, the product forms of the main control device include but are not limited to: mobile phones, tablet computers, desktop computers, laptop computers, handheld computers, notebook computers, ultra-mobile personal computers (ultra-mobile personal computers, UMPC), netbooks, personal digital assistants (personal digital assistants, PDA), wearable devices, vehicle-mounted devices or other types of electronic devices, etc.

[0073] In the embodiment of the present application, the main control device can establish a communication connection with the server and perform data communication based on the communication connection. Specifically, it includes the following:

[0074] (1) After detecting a remote control operation input by a user, the master device can respond to the operation by sending a command for remotely controlling the controlled device (such as a power-on, sleep wake-up, hibernation wake-up, restart, or shutdown command) to the server based on the communication connection.

[0075] (2) The master device can also receive status information of the controlled device (such as shutdown, hibernation, sleep, fault, power status, etc.) forwarded by the server based on the communication connection.

[0076] The controlled device is also called the first device. The controlled device can be a Or portable terminal devices with other operating systems, the product forms of the controlled devices include but are not limited to: tablet computers, desktop computers, laptop computers, notebook computers, super mobile personal computers, netbooks, as well as augmented reality (AR) devices, virtual reality (VR) devices, artificial intelligence (AI) devices, wearable devices, vehicle-mounted devices, smart home devices and / or smart city devices, etc.

[0077] In the embodiment of the present application, the controlled device can establish a communication connection with the server through the main processor / sub-processor, and perform data communication based on the communication connection. Specifically, it includes the following:

[0078] (1) The controlled device can establish a long connection with the server through the secondary processor, and the controlled device can also receive remote control instructions or send the status of the controlled device based on the long connection.

[0079] Specifically, when the controlled device is turned off, dormant, sleeping, or turned on but malfunctioning, the main processor of the controlled device is in a non-working state, while the sub-processor and EC are in a working state. Therefore, the controlled device can establish a long connection with the server through the sub-processor and communicate data based on the long connection. For example, the sub-processor can receive remote control instructions of the main control device forwarded by the server based on the long connection (such as power on, sleep wake-up, sleep wake-up, and restart instructions), and then the sub-processor notifies the EC to start the processor based on the remote control instruction to perform the corresponding task. For another example, the controlled device can also send status information of the controlled device (such as shutdown, dormancy, sleep, malfunction, power status information, etc.) to the server based on the long connection established between the sub-processor and the server.

[0080] (2) Figure 1 As not shown, the controlled device may establish a long connection with the server through the secondary processor, and the controlled device may also receive remote control instructions or send the status of the controlled device based on the long connection.

[0081] Specifically, when the controlled device is turned on and not faulty, the main processor of the controlled device is in working state. Therefore, the controlled device can establish a communication connection with the server through the main processor, and perform data communication based on the long connection. For example, the main processor can receive remote control instructions of the main control device forwarded by the server based on the communication connection (such as shutdown, hibernation, sleep, restart, or play music, etc.), and then the main processor directly executes the corresponding task based on the remote control instruction. For another example, the controlled device can also send status information of the controlled device (such as power-on, power status, etc.) to the server based on the communication connection established between the main processor and the server.

[0082] The specific implementation of the controlled device establishing a communication connection with the server through the secondary processor / main processor and performing the above data communication can be referred to in the following text. Figure 9-13 The description of the method is not described in detail here.

[0083] In addition, in the controlled device, the main processor and the sub-processor may be connected by a bus, the main processor and the EC may be connected by a bus, and the sub-processor and the EC may be connected by a bus.

[0084] Based on the bus connection between the main processor and the sub-processor, the main processor can synchronize information and send bus heartbeat messages to the sub-processor. The synchronized information includes: connection configuration information and device configuration information. The synchronized information is sent in the form of a bus message, so the connection configuration information can also be called a connection configuration message, and the device configuration information can also be called a device configuration message. The connection configuration information is used for the sub-processor to establish a long connection with the server using the same server login information as the main processor. The device configuration information is used for the sub-processor to access the network after obtaining the network configuration information, and is also used for the sub-processor to learn the capabilities of the controlled device, so that it can parse and respond to the remote control command when receiving the remote control command.

[0085] Based on the bus connection between the main processor and the EC, the EC can control the main processor to start working based on the remote control instructions received from the secondary processor. And based on the bus connection between the main processor and the EC, the EC can query the status information of the controlled device from the main processor, including but not limited to: power on, power off, sleep, hibernation, fault status, or power status, etc.

[0086] Based on the bus connection between the sub-processor and the EC, the sub-processor can send corresponding remote control instructions to the EC based on the remote control instructions received from the server. And based on the bus connection between the sub-processor and the EC, the EC can send the status information of the controlled device to the sub-processor, including but not limited to: the status of power on, power off, sleep, hibernation, fault, or power status, etc.

[0087] For the connection relationship and data interaction between the modules in the controlled device, please refer to the following article for specific implementation. Figure 3B The description of the software architecture of the controlled device shown in the figure and the following description of the software architecture of the controlled device Figure 9-13 The description of the method is not described in detail here.

[0088] The server is equivalent to a remote transfer device between the master device and the controlled device, and is used to provide remote control services for the master device. It includes: receiving instructions sent by the master device, and sending the instructions to the controlled device, so that the controlled device performs the corresponding task according to the instructions. It also includes: receiving status information sent by the controlled device, and sending the status information to the master controlled device, so that the master device can display the status of the controlled device to the user.

[0089] Based on the above introduction to the architecture of the communication system 10 , the software and hardware architecture of each device included in the communication system 10 is introduced next.

[0090] refer to Figure 2 , Figure 2 A schematic diagram of the hardware architecture of the main control device is shown.

[0091] like Figure 2 The main control device may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a camera 193, a display screen 194, etc. The sensor module 180 may include a pressure sensor 180A and a touch sensor 180B.

[0092] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the main control device. In other embodiments of the present application, the main control device may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0093] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0094] The controller can be the nerve center and command center of the main control device. The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of fetching and executing instructions.

[0095] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0096] In the embodiment of the present application, the processor 110 is used to control the corresponding software and hardware modules to execute the following text Figure 9-13 The methods introduced can be combined with the following Figure 9-13 The relevant description is not repeated here.

[0097] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0098] The USB interface 130 is an interface that complies with USB standard specifications, and may specifically be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 may be used to connect a charger to charge the main control device.

[0099] It is understandable that the interface connection relationship between the modules illustrated in the embodiments of the present application is only a schematic illustration and does not constitute a structural limitation on the main control device. In other embodiments of the present application, the main control device may also adopt a different interface connection method in the above embodiments, or a combination of multiple interface connection methods.

[0100] The charging management module 140 is used to receive charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 can receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 can receive wireless charging input through a wireless charging coil of the main control device. While the charging management module 140 is charging the battery 142, it can also power the main control device through the power management module 141.

[0101] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the external memory, the display screen 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0102] The wireless communication function of the main control device can be implemented through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor.

[0103] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the main control device can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of the antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0104] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the main control device. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0105] The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a receiver 170B, etc.), or displays an image or video through a display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0106] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the main control device. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, demodulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0107] In some embodiments, the antenna 1 of the main control device is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the main control device can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0108] In the embodiment of the present application, the main control device can establish a communication connection with the server through the aforementioned mobile communication module 150 or the wireless communication module 160, and then send a remote control instruction for the controlled device to the server based on the communication connection.

[0109] In the embodiment of the present application, the main control device implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0110] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD). The display screen panel can also be made of an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), miniled, microled, micro-oled, quantum dot light emitting diodes (QLED), etc. In some embodiments, the main control device may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0111] In the embodiment of the present application, the main control device can display the following UI embodiment, for example, on the display screen 194: Figure 6A-6F , Figure 8B , Fig.8D and Fig.8F The user interface shown and so on will not be described in detail here.

[0112] The main control device can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0113] ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise and brightness of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 193.

[0114] Camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the main control device may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0115] The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the main control device is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0116] Video codecs are used to compress or decompress digital videos. The master device can support one or more video codecs. In this way, the master device can play or record videos in multiple coding formats, such as: Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0117] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and continuously self-learn. NPU can realize applications such as intelligent cognition of the main control device, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0118] The internal memory 121 may include one or more random access memories (RAM) and one or more non-volatile memories (NVM).

[0119] Random access memory may include static random-access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM, for example, the fifth generation DDR SDRAM is generally referred to as DDR5 SDRAM), etc.

[0120] Non-volatile memory may include disk storage devices and flash memory.

[0121] Flash memory can be divided into NOR FLASH, NAND FLASH, 3D NAND FLASH, etc. according to the operating principle; single-level cell (SLC), multi-level cell (MLC), triple-level cell (TLC), quad-level cell (QLC), etc. according to the storage unit potential level; universal flash storage (UFS), embedded multi media card (eMMC), etc. according to the storage specification.

[0122] The random access memory can be directly read and written by the processor 110, and can be used to store executable programs (such as machine instructions) of the operating system or other running programs, and can also be used to store user and application data.

[0123] The non-volatile memory may also store executable programs and user and application data, etc., and may be loaded into the random access memory in advance for direct reading and writing by the processor 110 .

[0124] The external memory interface 120 can be used to connect to an external non-volatile memory to expand the storage capacity of the main control device. The external non-volatile memory communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, files such as music and videos are stored in the external non-volatile memory.

[0125] The main control device can realize audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone interface 170D, and the application processor.

[0126] The pressure sensor 180A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be set on the display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. The capacitive pressure sensor can be a parallel plate including at least two conductive materials. When a force acts on the pressure sensor 180A, the capacitance between the electrodes changes. The main control device determines the intensity of the pressure according to the change in capacitance. When a touch operation acts on the display screen 194, the main control device detects the touch operation intensity according to the pressure sensor 180A. The main control device can also calculate the touch position according to the detection signal of the pressure sensor 180A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities can correspond to different operation instructions. For example: when a touch operation with a touch operation intensity less than the first pressure threshold acts on the short message application icon, an instruction to view the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold acts on the short message application icon, an instruction to create a new short message is executed.

[0127] The touch sensor 180B is also called a "touch panel". The touch sensor 180B can be arranged on the display screen 194, and the touch sensor 180B and the display screen 194 form a touch screen, also called a "touch screen". The touch sensor 180B is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180B can also be arranged on the surface of the main control device, which is different from the position of the display screen 194.

[0128] In the present application, the main control device can detect the operation on the display screen 194 through the pressure sensor 180A and the touch sensor 180B, such as the following Figure 6A-6F Touch operations shown, etc.

[0129] The key 190 includes a power key, a volume key, etc. The key 190 can be a mechanical key or a touch key. The main control device can receive key input and generate key signal input related to user settings and function control of the main control device.

[0130] refer to Figure 3A , Figure 3A A schematic diagram of the hardware architecture of the controlled device is shown.

[0131] like Figure 3A The controlled device may include: a main processor 201, a sub-processor 202, an EC 203, a communication module 204, a memory 205, a display screen 206 and a universal serial bus 207. It is understood that the hardware architecture illustrated in the embodiment of the present application does not constitute a specific limitation on the controlled device. In other embodiments of the present application, the controlled device may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0132] The main processor 201 may include one or more processing units, for example: the main processor 201 may include an AP, a modem processor, a GPU, an ISP, a controller, a memory, a video codec, a DSP, a baseband processor, and / or an NPU, etc. Among them, different processing units may be independent devices or integrated into one or more processors. Among them, the controller may be the nerve center and command center of the controlled device. The controller may generate an operation control signal according to the instruction opcode and the timing signal to complete the control of fetching and executing instructions. Usually, when the controlled device is turned off, dormant, or asleep, the main processor 201 is powered off, and the main processor 201 is powered on only when the controlled device is turned on.

[0133] A high-speed cache memory may also be provided in the main processor 201. In some embodiments, the high-speed cache memory may store instructions or data that have just been used or are cyclically used by the main processor 201. If the main processor 201 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the main processor 201, and thus improves the efficiency of the system.

[0134] The main processor 201 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, and / or a universal serial bus (USB) interface, etc.

[0135] The subprocessor 202 is a low-power microcontroller unit (MCU) that supports wireless communication. Usually, the subprocessor is always powered on unless the controlled device runs out of power or the battery is damaged. The subprocessor can be integrated with a communication module 204 to access the network based on the communication module 204 when the controlled device is not turned on, and then establish a long connection with other devices such as a server.

[0136] A flash memory may also be provided in the secondary processor 202. The flash memory is a non-volatile memory, that is, data will not be lost even if the power is turned off. In some embodiments, the flash memory may store information synchronized from the main processor 201 to the secondary processor 202, including but not limited to: connection configuration information and device configuration information. For an introduction to the connection configuration information and the device configuration information, please refer to the following description, which will not be repeated here.

[0137] EC 203 is equivalent to a single-chip microcomputer. Usually, the sub-processor is always powered on unless the controlled device runs out of power or the battery is damaged.

[0138] EC 203 also includes a non-volatile memory for storing independently running software. EC 203 is used for power management, battery management, keyboard control, touchpad control, etc. Specifically, EC 203 can provide power to main processor 201, sub-processor 202, communication module 204, memory 205, display screen 206, etc. EC 203 can also be used to monitor parameters such as battery capacity.

[0139] The communication function of the controlled device can be realized through an antenna, a communication module 204, a modem processor, and a baseband processor.

[0140] The antenna is used to transmit and receive electromagnetic wave signals. Each antenna in the controlled device can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of the antenna. For example, the antenna can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0141] The modem processor may include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device, or displays an image or video through a display screen. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the main processor 201 and be set in the same device as the communication module 204 or other functional modules.

[0142] The communication module 204 may include one or more, for example, the main processor 201 and the sub-processor 202 each integrate a communication module 204. The communication module 204 can provide solutions for wireless communications including WLAN (such as Wi-Fi network), BT, GNSS, FM, NFC, IR, etc. applied to the controlled device. The communication module 204 can be one or more devices integrating at least one communication processing module. The communication module 204 receives electromagnetic waves via an antenna, demodulates and filters the electromagnetic wave signals, and sends the processed signals to the main processor 201 / sub-processor 202. The communication module 204 can also receive the signal to be sent from the main processor 201 / sub-processor 202, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through the antenna.

[0143] In the embodiment of the present application, the antenna of the controlled device is coupled to the communication module 204, so that the controlled device can communicate with the server through wireless communication technology. The wireless communication technology may include WLAN, BT, GNSS, NFC, FM, and / or IR technology, etc. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0144] The controlled device realizes the display function through GPU, display screen 206, and application processor. GPU is a microprocessor for image processing, connecting display screen 206 and application processor. GPU is used to perform mathematical and geometric calculations for graphics rendering. Main processor 201 may include one or more GPUs, which execute program instructions to generate or change display information.

[0145] The display screen 206 is used to display images, videos, etc. The display screen 206 includes a display panel. The display panel can be a liquid crystal display (LCD). The display screen panel can also be made of an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), miniLED, microLed, micro-oled, quantum dot light emitting diodes (QLED), etc. In some embodiments, the controlled device may include 1 or N display screens 206, where N is a positive integer greater than 1.

[0146] The memory 205 may include one or more RAMs and one or more NVMs. The RAM can be directly read and written by the main processor 201, and can be used to store executable programs (such as machine instructions) of the operating system or other running programs, and can also be used to store user and application data, etc. The NVM can also store executable programs and store user and application data, etc., and can be loaded into the random access memory in advance for direct reading and writing by the main processor 201.

[0147] RAM may include static random-access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM, for example, the fifth generation DDR SDRAM is generally referred to as DDR5 SDRAM), etc.;

[0148] NVM can include disk storage devices and flash memory. Flash memory can be divided into NOR FLASH, NAND FLASH, 3D NAND FLASH, etc. according to the operating principle. It can be divided into single-level cell (SLC), multi-level cell (MLC), triple-level cell (TLC), quad-level cell (QLC), etc. according to the potential level of the storage unit. It can be divided into universal flash storage (UFS) and embedded multi media card (eMMC) according to the storage specification.

[0149] The bus 207 may be connected via any one or more types of interfaces such as the aforementioned I2C interface, PCM interface, UART interface, MIPI interface, GPIO interface, and / or USB interface. It is understood that Figure 3B The modules are connected via bus 207. Based on the bus connection between the main processor 201 and the sub-processor 202, the main processor 201 can synchronize information to the sub-processor 202. Based on the bus connection between the main processor 201 and the EC 203, the EC 203 can control the main processor 201 to start working based on the remote control instruction received from the sub-processor 202; and based on the bus connection between the main processor 201 and the EC 203, the EC 203 can query the status information of the controlled device from the main processor 201. Based on the bus connection between the sub-processor 202 and the EC 203, the sub-processor 202 can send the corresponding remote control instruction to the EC 203 based on the remote control instruction received from the server; and based on the bus connection between the sub-processor 202 and the EC 203, the EC 203 can send the status information of the controlled device to the sub-processor 202.

[0150] The key 208 includes a power button, etc. The key 208 may be a mechanical key or a touch key. The controlled device may receive key input and generate key signal input related to user settings and function control of the controlled device.

[0151] In the embodiment of the present application, the sub-processor 202 / main processor 201 of the controlled device can establish a communication connection with the server through the communication module 204, and perform data communication based on the communication connection. The specific implementation of the sub-processor 202 / main processor 201 of the controlled device establishing a communication connection with the server and performing the data communication can refer to the description of the method flow below, which will not be described in detail here.

[0152] Figure 3B It is a schematic diagram of the software architecture of the controlled device in the embodiment of the present application.

[0153] like Figure 3B As shown, the software architecture of the controlled device mainly includes a software module running on the main processor, a software module running on the secondary processor, and a software module running on the EC.

[0154] 1. The software modules running on the main processor include but are not limited to the following modules: bus communication management module and system manager.

[0155] The bus communication management module is used to establish a bus connection with other modules such as the bus communication management module in the configuration management module, and to send and receive bus messages based on the bus connection. Bus messages include but are not limited to the following types of messages: bus heartbeat messages, connection configuration messages, device configuration messages, and instructions.

[0156] The system manager is used to manage the connection between the main processor and the server. Specifically, when the controlled device is in the power-on state, the system manager can control the main processor to establish a connection with the server, and then receive the remote control instructions sent by the main control device forwarded by the server based on the connection.

[0157] 2. The software modules running on the secondary processor include but are not limited to the following modules: configuration management module, wake-up management module and remote connection management module, etc.

[0158] 2-1. The configuration management module includes but is not limited to the following sub-modules: bus communication management module, bus heartbeat management module, connection configuration module and device configuration management module, etc.

[0159] (1) The bus communication management module is used to establish a bus connection with other modules, such as the bus communication management module running in the software module of the main processor, and send and receive bus messages based on the bus connection, parse the bus messages, and distribute the parsed messages of different types to the corresponding modules. Specifically, it includes:

[0160] The bus communication management module distributes the bus heartbeat message parsed from the bus message to the bus heartbeat management module;

[0161] The bus communication management module also distributes the connection configuration message parsed from the bus message to the connection configuration module; optionally, the bus communication management module also directly passes the instruction parsed from the bus message, such as turning off the reconnection function, to the login module, or the instruction to turn off the reconnection function is encapsulated in the connection configuration message and needs to be parsed by the connection configuration management module before being passed to the remote login management module. Optionally, the bus communication management module also directly passes the instruction parsed from the bus message, such as releasing resources, to the wake-up service management module, or the instruction to release resources is encapsulated in the connection configuration message and needs to be parsed by the connection configuration management module before being passed to the wake-up service management module.

[0162] The bus communication management module also distributes the device configuration message parsed from the bus message to the device configuration management module; optionally, the bus communication management module also directly passes the instructions such as logging into the server parsed from the bus message to the wake-up service management module, or the instructions for logging into the server are encapsulated in the device configuration message and need to be parsed by the device configuration management module before being passed to the wake-up service management module.

[0163] (2) The bus heartbeat management module is used to parse the bus heartbeat message and maintain the bus connection, which specifically includes:

[0164] If it is parsed from the bus heartbeat message that the bus heartbeat message is received for the first time during a boot process, it is considered that the bus heartbeat state at this time is heartbeat establishment;

[0165] If it is parsed from the bus heartbeat message that the bus heartbeat message is not received during a startup process and after the heartbeat is established, it is considered that the bus heartbeat state at this time is heartbeat loss;

[0166] If it is parsed from the bus heartbeat message that a bus heartbeat message is received again during a boot process and after the heartbeat is lost, it is considered that the bus heartbeat state at this time is heartbeat recovery;

[0167] In addition, the bus heartbeat management module will also transmit the heartbeat status to the wake-up management module, so that the wake-up management module determines whether to start the remote connection management module according to the heartbeat status to execute establishment / disconnection of a long connection with the server.

[0168] (3) The connection configuration module is used to parse the connection configuration message, which includes server login information, or may also include: instructions for closing the reconnection function, instructions for releasing resources, etc. Specifically including:

[0169] If the server login information is parsed from the connection configuration message, the connection configuration module calls the interface of the remote connection management module to synchronize it to the network management in the remote connection management module for storage;

[0170] If the instruction to turn off the reconnection function is parsed from the connection configuration message, the connection configuration module notifies the remote login module to turn off the reconnection function. Optionally, the instruction to turn off the reconnection function can be sent directly to the remote login module by the bus communication management module without parsing and forwarding it from the connection configuration message through the connection configuration management module.

[0171] If instructions for releasing resources, etc. are parsed from the connection configuration message, the connection configuration module calls the interface of the wake-up management module to notify the remote login module to release the resources of temporary data. In addition, the wake-up service management module will also disconnect the connected network. Optionally, instructions for releasing resources, etc. can be sent directly from the bus communication management module to the wake-up management module without parsing and forwarding from the connection configuration message through the connection configuration management module.

[0172] (4) The device configuration management module is used to parse the device configuration message, which includes but is not limited to: device capability description file (also known as Profile file), network configuration information (such as WiFi account and password, etc.), device identification (such as serial number (SN), universally unique identifier (UUID)). Or it may also include: server login instructions, factory reset instructions, etc.

[0173] If a Profile file is parsed from the device configuration message, the device configuration management module stores the Profile file;

[0174] If the network configuration information is parsed from the device configuration message, the device configuration management module stores the network configuration information;

[0175] If the device identification is parsed from the device configuration message, the device configuration management module stores the device identification;

[0176] If the instruction to log in to the server is parsed from the device configuration message, the interface of the wake-up management module is called and passed to the wake-up service management module, so that the wake-up service management module executes the login module in the remote connection management module to establish a long connection with the server. Optionally, the instruction to log in to the server can be sent directly to the wake-up service management module by the bus communication management module, without the need for the device configuration management module to parse the device configuration message and forward it to the wake-up service management module.

[0177] 2-2. The wake-up management module running on the secondary processor includes but is not limited to the following sub-modules: a wake-up service management module, an EC control module, and the like.

[0178] (1) The wake-up service management module can be used to:

[0179] Receive the instruction of logging into the server transmitted by the bus management module or the device configuration management module, obtain the network configuration information from the device configuration management module based on the instruction to connect to the network, and obtain the Profile file from the device configuration management module and pass it to the login module in the remote connection management module, so that the login module in the remote connection management module is started to obtain the server login information from the networking module, establish a long connection with the server based on the server login information, and subscribe to the remote control service from the server based on the Profile file.

[0180] Receive the heartbeat status transmitted by the bus heartbeat management module, and manage the long connection between the remote connection management module and the server based on the heartbeat status (such as establishing a long connection, disconnecting a long connection). Specifically, when the controlled device is abnormally shut down, hibernates, sleeps or fails, resulting in the main processor not sending a login server to the secondary processor, the heartbeat status is heartbeat loss. At this time, the wake-up service management module obtains network configuration information from the device configuration management module to connect to the network, and obtains the Profile file from the device configuration management module and passes it to the login module in the remote connection management module, so that the login module in the remote connection management module is started to obtain the server login information from the networking module, establish a long connection with the server based on the server login information, and subscribe to the remote control service from the server based on the Profile file. When the controlled device is turned on, woken up from hibernation, woken up from sleep or restarted, the heartbeat status is heartbeat establishment / heartbeat recovery. At this time, the wake-up service management module controls the login module in the remote connection management module to disconnect the long connection with the server.

[0181] Receive the instruction of releasing resources transmitted by the bus management module or the connection configuration module, and notify the login module in the remote connection management module to release the resources of temporary data. In addition, the wake-up service management module will also disconnect the connected network.

[0182] Receive remote control commands (such as power on, wake up from sleep, wake up from sleep, restart, etc.) transmitted by the cloud service connection management module, and control the device software modules (such as control modules) running on the EC to perform corresponding tasks through the EC control module.

[0183] (2) The EC control module can be used to control the startup of the main chip to achieve startup, hibernation wakeup, sleep wakeup, restart, etc. according to the instructions such as startup, hibernation wakeup, sleep wakeup, restart, etc. transmitted by the wakeup service management module.

[0184] 2-3. The remote connection management module running on the secondary processor includes but is not limited to the following sub-modules: a networking management module, an encryption / decryption module, a login module, a device control module, and the like.

[0185] (1) The network management module is used to receive and store the information sent by the connection configuration module, and to encrypt and store the information through the encryption / decryption module.

[0186] (2) The encryption / decryption module is used to encrypt the connection configuration information based on the device identification and root key synchronized by the main processor of the controlled device and store it in the networking management module. Alternatively, it is also used to encrypt part of the device configuration information sent from the device configuration management module and return it to the device configuration management module for storage.

[0187] (3) The login module is used to receive the remote connection enablement instruction transmitted by the wake-up service management module, and based on the instruction, obtain the server login information from the networking management module to log in to the server. The login module is also used to receive the reconnection function disablement instruction transmitted directly by the bus management module or transmitted through the connection configuration module to disable the reconnection function. The login module is also used to receive the resource release instruction transmitted directly by the bus management module or transmitted through the connection configuration module and the wake-up service management module to clear the temporary data in the RAM.

[0188] (4) The device control module is used to receive the remote control command sent by the server and pass it to the wake-up service management module through callback to control the EC control module to execute the task corresponding to the command.

[0189] 3. The software modules running in the EC include but are not limited to the following submodules: Control module: The control module is used to receive control instructions sent by the EC control module in the secondary processor, and then control the startup of the software module of the main processor.

[0190] refer to Figure 4 , Figure 4 A schematic diagram of the hardware architecture of a server provided in an embodiment of the present application.

[0191] like Figure 4 As shown, the server may include: one or more processors 301, memory 302, communication interface 303, transmitter 305, receiver 306, coupler 307 and antenna 308. These components may be connected via bus 304 or other means. Figure 4 Take the connection via bus 304 as an example.

[0192] The processor 301 can be used to read and execute computer-readable instructions. Specifically, the processor 301 can be used to call a program stored in the memory 302, such as a method provided in an embodiment of the present application for forwarding a remote control instruction for a controlled device sent by a master device to a controlled device, implementing a program on the server side, and executing the instructions contained in the program.

[0193] The memory 302 is coupled to the processor 301 and is used to store various software programs and / or multiple sets of instructions. Specifically, the memory 302 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more disk storage devices, flash memory devices or other non-volatile solid-state storage devices.

[0194] The memory 302 may store an operating system (hereinafter referred to as system), such as embedded operating systems such as uCOS, VxWorks, RTLinux, etc. The memory 302 may also store a network communication program, which may be used to communicate with the electronic device 100, etc. The memory 302 may also store device information and login information of the controlled device.

[0195] The communication interface 303 can be used for the server to communicate with other communication devices, such as a master device, a controlled device, etc. Specifically, the communication interface 303 can be a 3G communication interface, a long-term evolution (LTE) (4G) communication interface, a 5G communication interface, a WLAN communication interface, a WAN communication interface, etc. Not limited to a wireless communication interface, the server can also be configured with a wired communication interface 303 to support wired communication.

[0196] In some embodiments of the present application, the transmitter 305 and the receiver 306 can be regarded as a wireless modem. The transmitter 305 can be used to transmit and process the signal output by the processor 301. The receiver 306 can be used to receive the signal. In the server, the number of the transmitter 305 and the receiver 306 can be one or more. The antenna 308 can be used to convert the electromagnetic energy in the transmission line into electromagnetic waves in the free space, or to convert the electromagnetic waves in the free space into electromagnetic energy in the transmission line. The coupler 307 can be used to divide the mobile communication signal into multiple paths and distribute it to multiple receivers 306. It can be understood that the antenna 308 of the network device can be implemented as a large-scale antenna array.

[0197] Based on the above introduction to the embodiments of the present application involving communication systems, electronic devices, servers and other devices, the remote control method provided by the present application is introduced in conjunction with the UI embodiment.

[0198] Figure 5A-5D The following is a schematic diagram of an operation interface for the normal shutdown of a group of controlled devices.

[0199] like Figure 5A As shown, Figure 5A The interface shown is any interface displayed by the controlled device in the power-on state, and the interface displays a start icon 512. In addition, the controlled device is also integrated with a physical power button 511.

[0200] The controlled device may receive an operation on the start icon 512 (eg, a user controls a cursor to click on the start icon 512). In response to the operation, the controlled device displays a power option 513. For details, refer to Figure 5B The content shown.

[0201] like Figure 5B As shown, the controlled device displays a power option 513 .

[0202] The controlled device may receive an operation on the power option 513 (eg, a user controls the cursor to click on the power option 513). In response to the operation, the controlled device expands and displays multiple sub-options. For details, refer to Figure 5C The content shown.

[0203] like Figure 5C As shown, the controlled device displays multiple sub-options included in the power option 513, such as a shutdown option 514, a sleep option, a restart option, and the like.

[0204] The controlled device can receive an operation acting on the shutdown option 514 (for example, the user controls the cursor to click the shutdown option 514), and in response to the operation, the controlled device shuts down, specifically including cutting off the power supply of most hardware except the sub-processor and EC, such as disconnecting the power supply of the main processor, display screen, etc.

[0205] like Figure 5D As shown in the figure, after the controlled device is turned off, the display screen is powered off and does not display anything. In addition, after the controlled device is turned off, the sub-processor and EC of the controlled device are powered on and work. The sub-processor can establish a long connection with the server to receive the remote control instructions of the main control device forwarded by the server. In addition, the sub-processor will also enable the reconnection function to automatically request the server to re-establish a long connection in the case of abnormal disconnection.

[0206] Understandably, Figure 5A-5D Only one shutdown method is shown as an example. Optionally, the shutdown of the controlled device can be triggered by directly operating the power button 511, or the controlled device can be abnormally shut down by cutting off the power supply, or the controlled device can be remotely controlled to shut down by the master device, etc. The embodiment of the present application does not limit the method of controlling the shutdown of the controlled device, which will not be described in detail here.

[0207] Figure 6A-6H A schematic diagram of an operation interface for remotely controlling the startup of a controlled device through a master control device is shown as an example.

[0208] like Fig. 6A As shown, Fig. 6AThe desktop of the main control device includes a setting application icon 611 and may also include other application icons, such as smart life application icons and the like.

[0209] The master device may receive an operation on the setting application icon 611, and in response to the operation, the master device displays a setting details page. Figure 6B The content shown.

[0210] like Figure 6B As shown, Figure 6B The setting details page of the master device includes a plurality of setting sub-options, such as more connection options 612 .

[0211] The master device may receive an operation on the more connection option 612, and in response to the operation, the master device displays a more connection details page, see Figure 6C The content shown.

[0212] like Figure 6C As shown, Figure 6C The more connection details page includes a variety of connection sub-options, such as option 613 for connecting to my PC.

[0213] The master device may receive an operation on the option 613 of connecting to my PC. In response to the operation, the master device displays the PC to which the master device is connected. Fig.6D The content shown.

[0214] like Fig.6D As shown, the master device displays a list 614 of controlled devices pre-added by the master device, such as Xiao Li's MagicBook 14, Xiao Li's MagicBook 16, and Xiao Li's watch, etc. There are many ways to add a controlled device, such as scanning a QR code of the controlled device to obtain the ID of the controlled device, or obtaining the ID of the controlled device through Bluetooth, Wi-Fi transmission, etc.

[0215] The master device can receive Fig.6D In response to the operation, the main control device displays the remote control interface for Xiao Li's MagicBook 14. For details, see Fig. 6E Among them, Xiao Li's MagicBook 14 is the controlled device described in this application, and is also the aforementioned Figure 5A-5D Introduce the controlled device.

[0216] like Fig. 6EAs shown, the main control device displays a remote control interface for Xiao Li's MagicBook 14, which displays the current status of Xiao Li's MagicBook 14, such as being in a shutdown state, and also displays a power-on icon 615, as well as other options, such as a hibernation option, a sleep option, a restart option, and more options.

[0217] The master device can receive Fig. 6E In response to the operation on the power-on icon 615 shown, the master device sends a remote power-on instruction to the controlled device “Xiao Li’s MagicBook 14” through the server, thereby controlling the controlled device to power on.

[0218] like Fig. 6F As shown, when the master device receives the Fig. 6E The operation on the power-on icon 615 shown can also pop up a window 616 for inputting the device account and password of the controlled device.

[0219] The master device can receive an operation on the confirmation control after receiving the device account and password of Xiao Li's MagicBook 14 input by the user. In response to the operation, the master device will send the device account and password to the controlled device through the server, which will then be used by the controlled device to verify whether the device account and password are correct. Only when they are correct will the master device respond to the power-on command.

[0220] Optional, Fig. 6F The window 616 shown in FIG. 614 can also be detected by the master device acting on the aforementioned Fig.6D After the operation on "Xiao Li's MagicBook 14" is shown, it is also displayed that the main control device is notified after the controlled device verifies the device account and password. Fig. 6E The control interface shown.

[0221] like Figure 6G As shown, based on the connection between the secondary processor and the server, after the controlled device receives the remote power-on command forwarded by the main control device through the server, the secondary processor of the controlled device starts the main processor through the EC, thereby starting the controlled device.

[0222] like Figure 6H As shown, after the controlled device is started up, it is in the power-on state, and the desktop of the controlled device is displayed. In addition, after the controlled device is turned on, the main processor of the controlled device notifies the secondary processor to turn off the reconnection function and then establish a connection with the server, thereby automatically disconnecting the long connection previously established between the secondary processor and the server. The controlled device can also receive remote control instructions of the main control device forwarded by the server based on the connection between the main processor and the secondary processor.

[0223] Figure 7A-7CA group of operation interface schematic diagrams for controlling the startup of a controlled device through local operation are exemplified.

[0224] like Fig. 7A As shown, Fig. 7A The interface showing the controlled device is in shutdown state.

[0225] The controlled device may receive an operation on the power key 511 , and the EC of the specific controlled device may detect the operation and wake up the main processor in response to the operation, that is, control the main processor to be powered on to start working.

[0226] like Figure 7B As shown, Figure 7B An interface showing that the controlled device is starting up.

[0227] like Figure 7C As shown, Figure 7C The interface after the controlled device has completed startup, that is, switched to the power-on state, is shown. In addition, after the controlled device is powered on, the main processor of the controlled device notifies the secondary processor to turn off the reconnection function and then establish a connection with the server, thereby automatically disconnecting the long connection previously established between the secondary processor and the server. The controlled device can also receive remote control instructions of the main control device forwarded by the server based on the connection between the main processor and the secondary processor.

[0228] Figure 8A-8F The schematic diagram exemplarily shows a scenario in which the secondary processors of a group of controlled devices are powered off and then powered on again.

[0229] like Fig. 8A As shown, Fig. 8A A schematic diagram showing the controlled device is shut down and a long connection is established with the server through the secondary processor. Fig. 8A When the status is shown, the master device can display Figure 8B The remote control interface shown.

[0230] like Figure 8B As shown, Figure 8B The remote control interface displayed by the main control device is shown in the state where the controlled device is turned off and a long connection is established with the server through the secondary processor. The content displayed on the remote control interface is the same as the above Fig. 6E The content shown is the same, and the user can remotely control the startup of the controlled device by operating on the startup icon 615 in the remote control interface.

[0231] like Figure 8C As shown, Figure 8C The schematic diagram shows that the controlled device is shut down, and the secondary processor is powered off and disconnected from the long connection with the server. Figure 8C When the status is shown, the master device can display Fig.8DThe remote control interface shown.

[0232] like Fig.8D As shown, Fig.8D The remote control interface displayed by the master device is shown when the controlled device is turned off and the secondary processor is powered off. The remote control interface displays a prompt message 811 indicating that the controlled device is offline. The prompt message 811 includes, for example, "PC is offline and cannot be remotely controlled temporarily." Optionally, in a possible implementation, the master device may also change all remote control-related icons and controls to an inoperable state, such as a gray display state. Fig.8D Not shown yet.

[0233] like Fig. 8E As shown, Fig. 8E The figure shows that the controlled device is shut down, and the secondary processor is powered off and then powered on again to restore the long connection with the server. Fig. 8E When the status is shown, the master device can display Fig.8F The remote control interface shown.

[0234] like Fig.8F As shown, Fig.8F The remote control interface displayed by the master device is shown when the controlled device is turned off and the sub-processor is powered on again after power failure, thereby restoring the long connection with the server. The remote control interface displays a prompt message 812 prompting the controlled device to go online, and the prompt message 812 includes, for example: PC is online and can be remotely controlled. Optionally, in a possible implementation, the master device may also restore all remote control related icons and controls to an operable state.

[0235] This is because, after the controlled device established a long connection with the server, the controlled device turned on the reconnection function. Therefore, after the sub-processor of the controlled device suddenly lost power and then restored power, the sub-processor of the controlled device can automatically trigger a request to re-establish a long connection with the server, thereby restoring the long connection.

[0236] In the embodiments of the present application, the above UI embodiments only introduce the remote control function provided by the setting application of the active device. In addition, the remote control function can also be implemented through other system-level applications installed in the active device, such as Smart Life, or other third-party applications. Regardless of which application provides the remote control function, its specific implementation method can be the remote control method provided by the present application, and specific reference is made to the description of the method embodiment below.

[0237] refer to Fig. 9 , Fig. 9 A remote control method provided by the present application is exemplified.

[0238] like Fig. 9 As shown, the method comprises the following steps:

[0239] S61, the main processor of the controlled device pre-stores the authentication code and the third key of the controlled device, and the second key of the server.

[0240] Among them, the authentication code of the controlled device is generated by the main chip of the controlled device. The authentication code can uniquely identify the controlled device. The authentication code can include authCode and authCodeId. authCode is randomly generated by the main processor, so duplicate codes may be generated. In this case, authCodeId needs to be added to uniquely identify the controlled device. The authentication code of the controlled device is used when the main processor requests registration from the server, and is also used when the main processor / sub-processor requests the server to refresh the password corresponding to the cloud account. Please refer to the description of the subsequent steps for details.

[0241] Among them, the key of the controlled device (also referred to as the third key) is also generated by the main chip of the controlled device. The third key can be a symmetric key or an asymmetric key. When the third key is an asymmetric key, it includes a first public key and a corresponding first private key. The third key can be used when the controlled device communicates temporarily with other devices, for example, when the controlled device authenticates with the master device to negotiate the subsequent non-temporary first key. For details, refer to the description of the subsequent steps.

[0242] The server key (also referred to as the second key) is generated by the server. The second key may be a symmetric key or an asymmetric key. When the second key is an asymmetric key, it specifically belongs to the public key in the asymmetric key. The second key may be used when the controlled device communicates with the server. For details, refer to the description of the subsequent steps.

[0243] Optionally, the second key issued by the server involved in this application can also be replaced by a certificate issued by the server, and the embodiment of this application is not limited to this.

[0244] S62, when the controlled device is turned on, the main processor of the controlled device completes registration on the server side based on the authentication code of the controlled device encrypted by the second key and the third key, and obtains the cloud account and the corresponding first password sent by the server.

[0245] Specifically, when the controlled device is in the turned-on state, the main processor of the controlled device is in the working state, and the main processor can request registration with the server to obtain a cloud account (also known as the first account) and a corresponding first password for subsequent establishment of a connection with the server, and based on the connection, receive remote control instructions of the main control device forwarded by the server.

[0246] In addition, in order to ensure secure communication between the controlled device and the server, the controlled device needs to use the second key issued by the server (specifically the server's public key) to encrypt the privacy information involved in the registration request, such as the authentication code and third key of the controlled device, to prevent illegal theft by other devices.

[0247] Optionally, when the controlled device is powered on, the main processor of the controlled device can also complete the registration on the server side based on the authentication information of the controlled device. That is, the main processor of the controlled device sends the authentication information of the controlled device to the server to request registration. The authentication information of the controlled device includes any one or more of the following: the authentication code of the controlled device, the identification of the controlled device or the public key of the controlled device. In addition, optionally, when the main processor of the controlled device sends the authentication information of the controlled device to the server to request registration, the authentication information of the controlled device can be encrypted using the public key of the server, and then the server uses the private key of the server to decrypt to obtain the authentication information of the controlled device, thereby completing the registration.

[0248] S63, when the controlled device is powered on, the main processor of the controlled device synchronizes information with the secondary processor, for example, including a cloud account and a corresponding first password.

[0249] Specifically, when the controlled device is in the turned-on state, the main processor of the controlled device needs to synchronize the cloud account and the corresponding first password to the sub-processor, so that when the main processor is in a non-working state (the controlled device is turned off, dormant, sleeping, or turned on but faulty), the controlled device can establish a connection with the server through the sub-processor, and based on the connection, receive the remote control instructions of the main control device forwarded by the server to remotely start the main processor.

[0250] In addition, the information synchronized from the main processor of the controlled device to the secondary processor also includes other more information, such as but not limited to the device configuration information and connection configuration information of the controlled device. Optionally, the device configuration information and connection configuration information can be synchronized in advance before the main processor synchronizes the cloud account and the corresponding first password to the secondary processor.

[0251] Among them, the device configuration information includes but is not limited to: device capability description file (also known as Profile file), network configuration information (such as WiFi account and corresponding password, etc.), device identification (such as serial number (SN), universally unique identifier (UUID)), authentication information (device account and corresponding password of the first device) and root key, etc.

[0252] The connection configuration information includes but is not limited to: server login information (i.e., cloud account and corresponding password), authentication code and third key of the controlled device, URL supported by the server for refreshing password, and second key or certificate issued by the server.

[0253] S64, when the controlled device is turned off, dormant, sleeping, or powered on but faulty, the sub-processor of the controlled device establishes a long connection with the server based on the cloud account and the corresponding first password.

[0254] Specifically, when the controlled device is in the state of being turned off, dormant, sleeping, or powered on but malfunctioning, the main processor of the controlled device is in a non-working state. Therefore, the sub-processor of the controlled device needs to establish a long connection (also called the first connection) with the server based on the cloud account (also called the first account) and the corresponding first password, which is used to subsequently receive the remote control instructions of the main control device forwarded by the server based on the long connection to remotely start the main processor.

[0255] S65, when the controlled device is powered off, dormant, sleeping, or powered on but in a faulty state, the sub-processor of the controlled device completes authentication of the master device through the server based on a long connection, and then negotiates a first key used by both the master device and the controlled device.

[0256] Specifically, after the server and the main processor of the controlled device complete registration, the server can send relevant information about the registration of the controlled device to the main control device, such as the authentication information of the controlled device. Then, the main control device sends an authentication request to the controlled device through the server, and the authentication request carries the first data encrypted by the public key (such as the third key) of the controlled device. The first data is calculated by a preset algorithm for any one or more of the following data: the authentication information of the controlled device, the random number of the controlled device, and the random number of the main control device; the main processor of the controlled device is in a non-working state, and the controlled device calculates the authentication information by the sub-processor using the private key of the controlled device and the preset algorithm to obtain the first data, and determines the first data as the first key based on the first data, or obtains the first key after processing the first data using the second algorithm. The preset algorithm is pre-stored for the main control device and the controlled device.

[0257] In the case where the first key is obtained by the controlled device after processing the first data using the second algorithm, the controlled device needs to further notify the main control device to process the first data using the second algorithm to obtain the first key.

[0258] It can be seen that the above authentication process is used for the controlled device and the master device to negotiate a first key, which is used for encryption when the master device subsequently sends instructions to the controlled device, and the first key is invisible to the server and other devices.

[0259] Optionally, before the controlled device executes S64, if the controlled device is in the on state, the controlled device can also establish a connection with the server based on the cloud account and the corresponding first password through the main processor (also called a second connection), and execute an authentication process such as that described in S65 based on the second connection, which will not be repeated here.

[0260] S66, when the controlled device is powered off, dormant, sleeping, or powered on but in a faulty state, the sub-processor of the controlled device receives an instruction encrypted with a first key from the main control device through the server based on a long connection, and starts the main processor in response to the decrypted instruction.

[0261] Specifically, when the controlled device is in the state of being turned off, dormant, sleeping, or powered on but failing, although the main processor of the controlled device is in a non-working state, the sub-processor of the controlled device is in a working state, and the controlled device can maintain a long connection with the server through the sub-processor, and then can receive information sent by the main control device forwarded by the server based on the long connection, where the information is, for example, an instruction encrypted with a first key, and the instruction can be, for example, an instruction for powering on, waking up from dormancy, waking up from sleep, or restarting. In response to the instruction, the server of the controlled device can start the main processor, so that the main processor switches to a working state, and then the controlled device switches to a powered-on state.

[0262] Optionally, the controlled device will also execute Fig. 9 The following operations are not shown in FIG.

[0263] Optionally, during the process of the controlled device executing the aforementioned S63, the main processor and the sub-processor of the controlled device may encrypt and store the privacy-related information to further ensure the security of the remote control.

[0264] Specifically, a root key is pre-stored in the main processor of the controlled device. The root key is used by the main processor or the sub-processor to encrypt and store information related to privacy. For example, other encryption methods are used to encrypt based on the root key. The information related to privacy includes but is not limited to connection configuration information, and network configuration information, authentication information and root key in the device configuration information. When the sub-processor encrypts / decrypts the above privacy information, it is encrypted / decrypted specifically through the encryption / decryption module in the remote connection management module. For the specific implementation of encrypted storage, please refer to the following text Fig.13 The interaction process will not be described here.

[0265] Optionally, after the controlled device executes the aforementioned S63, the controlled device will also request the server to update the password corresponding to the cloud account.

[0266] Specifically, before the main processor of the controlled device synchronizes information with the secondary processor, the main processor first obtains the information in the secondary processor and compares the obtained information with the information to be synchronized. If the comparison results are different, the information to be synchronized is sent to the secondary processor. This can reduce the write operation to the NVM of the secondary processor, thereby reducing the write loss to the NVM. For the specific implementation of synchronization information, please refer to the following Fig.13 The interaction process will not be described here.

[0267] Optionally, after the controlled device executes the aforementioned S63, the controlled device will also request the server to update the password corresponding to the cloud account.

[0268] Specifically, the server will periodically notify the controlled device that the password corresponding to the cloud account has expired. If the main processor of the controlled device is in a working state when the notification is received, the controlled device will request the server to update the password through the main processor, and then synchronize the updated password (for example, the second password) to the secondary processor; if the main processor of the controlled device is in a non-working state when the notification is received, the controlled device will request the server to update the password through the secondary processor, and after the secondary processor obtains the updated password, it will no longer synchronize the updated password to the main processor. Therefore, after the subsequent main processor switches to a working state, the main processor will still use the old password (that is, the aforementioned first password) to request to establish a connection with the server. If the server detects that the old password is not the new password previously updated to the secondary processor, the server will notify the main processor that the old password has expired, and the main processor will also request the server to update the password, and then synchronize the updated password (for example, the third password) to the secondary processor.

[0269] It can be seen that by regularly refreshing the password of the cloud account, the reliability of the connection between the controlled device and the server can be further improved, and the security of the remote control instructions can also be improved. For the specific implementation of refreshing the password, please refer to the following Fig.10 The interaction process will not be described here.

[0270] Optionally, when the controlled device executes the aforementioned S66, when receiving the instruction from the master device, the controlled device will also authenticate the instruction, so as to further improve the security of remote control.

[0271] Specifically, the controlled device stores authentication information (i.e., the device account of the controlled device and the corresponding password) and the device account, which can be set or updated by the user. Specifically, it can be used by the user through local power-on, sleep wake-up, sleep wake-up and restart operations. Therefore, when the user uses the master device for remote control, it is also necessary to enter the device account and password. The device account and password can be sent to the controlled device along with the instructions of the master device. The controlled device will only consider the authentication to be successful when it is determined that the received device account and password are consistent with the device account and password stored in the controlled device, and will then further process the instructions of the master device. For the specific implementation of the authentication instruction, please refer to the following text. Fig.10 The interaction process will not be described in detail here.

[0272] The information synchronized from the main processor to the sub-processor may be in the form of bus messages, which include but are not limited to the following types of messages: bus heartbeat messages, connection configuration messages (including the server login information introduced above), device configuration messages (including the device identification, network configuration information, and Profile files introduced above), etc. Optionally, some instructions sent by the main processor may also be sent to the sub-processor in the form of bus messages, such as instructions for logging into the server and instructions for restoring factory settings, which may be encapsulated in device configuration messages, and instructions for turning off the reconnection function and releasing resources, which may be encapsulated in device configuration messages.

[0273] In one possible implementation, after the main processor of the controlled device receives the input of shutdown, hibernation or sleep, the main processor will first send an instruction to log in to the server to the secondary processor, so that the secondary processor accesses the network based on the instruction and the network configuration information in the synchronization information, and then establishes a first connection with the server according to the server login information in the synchronization information (i.e., the long connection introduced above), and then the main processor cuts off power and stops working, thereby causing the controlled device to enter a shutdown, hibernation or sleep state.

[0274] In another possible implementation, after the main processor receives the input of shutdown, hibernation or sleep, if the main processor is powered off and stops working before sending the instruction to log in to the server to the sub-processor, the main processor cannot periodically send heartbeat messages to the sub-processor based on the bus connection. The sub-processor can then determine that the heartbeat between the main processor and the sub-processor is lost, thereby enabling the sub-processor to access the network according to the network configuration information in the synchronization information, and then establish a first connection with the server according to the server login information in the synchronization information.

[0275] In another possible implementation, when the controlled device is powered on but in a faulty state, the main processor is powered on but cannot work normally. The main processor cannot periodically send heartbeat messages to the secondary processor based on the bus connection. The secondary processor can then determine that the heartbeat between the secondary processor and the main processor is lost, thereby enabling the secondary processor to access the network according to the network configuration information in the synchronization information, and then establish a first connection with the server according to the server login information in the synchronization information.

[0276] In addition, the sub-processor of the controlled device is pre-configured with a reconnection function, that is, when the sub-processor establishes a connection with the server, the sub-processor automatically turns on the reconnection function by default. The reconnection function includes: after the sub-processor inactively disconnects the long connection with the server, the sub-processor can automatically re-request to establish the long connection with the server. For example, if the sub-processor suddenly disconnects and disconnects the long connection with the server, after power is restored, the sub-processor will automatically trigger a request to re-establish the long connection with the server.

[0277] When the controlled device is in the aforementioned shutdown, hibernation, sleep, or powered on but faulty state, if the controlled device receives a power on, hibernation wake-up, sleep wake-up, or restart input, the controlled device will establish a second connection with the server through the main processor and disconnect the first connection between the secondary processor and the server.

[0278] The following S63-1, S64-1, and S65-1 introduce the specific implementation of establishing a remote connection between the controlled device and the server through the main processor after triggering the controlled device to start up, wake up from sleep, wake up from sleep, or restart through local operations.

[0279] S63-1, the controlled device detects a local power-on, sleep wake-up, sleep wake-up, or restart operation.

[0280] Specifically, the local power-on, hibernation wake-up, sleep wake-up or restart operation can be an operation on a physical button or an operation on a virtual control in the display screen, and the above operations can be directly received by the main processor of the controlled device, or detected by the EC of the controlled device and then passed to the main processor. The embodiments of the present application do not limit this.

[0281] S64-1, the controlled device wakes up the main processor directly or through the EC, then controls the sub-processor to turn off the reconnection function through the main processor, and establishes a second connection with the server through the main processor.

[0282] Specifically, when the main processor of the controlled device directly receives a local power-on, sleep wake-up, sleep wake-up or restart operation, the controlled device directly starts the processor. When the controlled device detects a local power-on, sleep wake-up, sleep wake-up or restart operation through the EC, the controlled device starts the main processor through the EC. Starting the main processor includes powering on the main processor so that the main processor starts working. For example, the controlled device controls the sub-processor to turn off the reconnection function through the main processor, and establishes a second connection with the server after connecting to the network through the main processor. In addition, the main processor startup will also perform other initialization operations to switch the controlled device to the power-on, sleep wake-up, sleep wake-up or restart state.

[0283] Optionally, when the server login information (account) used by the main processor and the sub-processor is the same, when the main processor establishes a second connection with the server, the first connection between the sub-processor and the server will be automatically disconnected.

[0284] S65-1, the controlled device receives the remote control instruction of the master device forwarded by the server based on the second connection and responds.

[0285] Specifically, after the controlled device establishes a second connection with the server through the main processor, the controlled device can receive remote control instructions (such as shutdown, hibernation or sleep instructions, or instructions for playing music, etc.) sent by the main control device and forwarded by the server based on the second connection, thereby enabling the main processor to perform corresponding tasks according to the remote control instructions.

[0286] The following S63-2, S64-2, and S65-2 introduce the specific implementation of establishing a remote connection between the controlled device and the server through the main processor after the controlled device is triggered to power on, wake up from sleep, wake up from sleep, or restart through remote control instructions.

[0287] S63-2: The controlled device receives a power-on, sleep-wake-up, sleep-wake-up, or restart instruction based on the first connection.

[0288] Specifically, after the controlled device described in S62 establishes a first connection with the server through the sub-processor, the sub-processor of the controlled device can receive instructions from the main control device forwarded by the server based on the first connection, such as instructions for powering on, waking up from sleep, waking up from sleep, or restarting.

[0289] S64-2, the controlled device controls the EC through the secondary processor to start the main processor, and then controls the secondary processor through the main processor to turn off the reconnection function, and establishes a second connection with the server through the main processor.

[0290] Specifically, after the sub-processor of the controlled device receives the instruction of power on, dormancy wake-up, sleep wake-up or restart, the sub-processor can notify the EC to start the main processor based on the instruction. Starting the main processor includes powering on the main processor so that the main processor starts working. For example, the controlled device controls the sub-processor to turn off the reconnection function through the main processor, and establishes a second connection with the server after connecting to the network through the main processor. In addition, the main processor startup will also perform other initialization operations to switch the controlled device to the state after power on, dormancy wake-up, sleep wake-up or restart.

[0291] S65-2: The controlled device receives the remote control instruction of the master device forwarded by the server based on the second connection and responds.

[0292] Specifically, after the controlled device establishes a second connection with the server through the main processor, the controlled device can receive remote control instructions (such as shutdown, hibernation or sleep instructions, or instructions for playing music, etc.) sent by the main control device and forwarded by the server based on the second connection, thereby enabling the main processor to perform corresponding tasks according to the remote control instructions.

[0293] It can be seen that the implementation of the above Fig. 9 After the method shown, the following technical effects can be achieved:

[0294] First, more remote control scenarios are realized. That is, when the controlled device is powered off, asleep, dormant, or powered on but faulty, the controlled device can establish a long connection with the server through the secondary processor, and receive and respond to the remote control instructions of the main control device forwarded by the server based on the long connection; when the controlled device is powered on, the controlled device can establish a connection with the server through the main processor, and receive and respond to the remote control instructions of the main control device forwarded by the server based on the connection.

[0295] Secondly, the main and secondary processors are seamlessly connected to the server to ensure the stability of the remote connection.

[0296] On the one hand, when the controlled device is in the state of being powered off, dormant, sleeping, or powered on but malfunctioning, the sub-processor of the controlled device automatically turns on the reconnection function, so that when the sub-processor abnormally disconnects the long connection with the server, it can re-request to establish a long connection with the server, thereby ensuring the stability of the long connection between the sub-processor and the server. For example, when the sub-processor is powered off and then powered on again, the power off will cause the long connection to be disconnected, but after the power is turned on again, the sub-processor can automatically request to establish a long connection with the server.

[0297] On the other hand, when the controlled device is powered on, the main processor of the controlled device will first notify the sub-processor to turn off the reconnection function, and then establish a communication connection with the server through the main processor to automatically disconnect the connection between the sub-processor and the server. In this way, not only the communication connection between the main processor and the server is seamlessly connected to the long connection between the sub-processor and the server, ensuring that the controlled device and the server are always connected, so that the user does not perceive any abnormalities on the main control device side, but also the sub-processor's reconnection function is not triggered after being directly squeezed out by the main processor, resulting in the sub-processor competing with the sub-processor for the connection to the server.

[0298] Next, combine Fig.10 Hereinafter, the interaction process between devices involved in a remote control method provided by the present application will be introduced.

[0299] like Fig.10 As shown, the communication system involved in the remote control method includes: a main control device, a server and a controlled device, wherein the controlled device includes a main processor, a sub-processor and an EC. Fig.10 The EC is only taken as an independent chip in the controlled device as an example. In other achievable ways, the EC may also be integrated into the main processor, and this application does not impose any limitation on this.

[0300] Fig.10 The interaction process between the devices shown includes the following:

[0301] Phase 1 (S111-S117): The controlled device completes initialization, including completing registration with the server through the main processor and synchronizing information with the sub-processor.

[0302] S111, the main processor of the controlled device pre-stores information, including but not limited to: an authentication code, a third key and a second key.

[0303] Specifically, for the authentication code, the third key and the second key pre-stored in the main processor of the controlled device, reference may be made to the introduction in the aforementioned S61, which will not be described in detail here.

[0304] In addition, the main processor of the controlled device also stores other information, such as device configuration information and the URL for refreshing passwords supported by the server in the connection configuration information. For the introduction of device configuration information and connection configuration information, please refer to the description of S62 above, which will not be repeated here.

[0305] S112, the main processor of the controlled device receives a start input.

[0306] Specifically, after the user inputs a startup, hibernation wake-up, sleep wake-up or restart operation to the controlled device, the main processor can receive the startup input corresponding to the above operation, and the main processor switches to the working state, so that the controlled device switches to the startup state. The above operation can be directly received by the main processor, or received by the EC and then transmitted to the main processor, and the embodiment of the present application does not limit this.

[0307] S113: The main processor of the controlled device requests registration based on the authentication code, the third key and the second key.

[0308] Specifically, after the main processor of the controlled device is in working state, it can send a registration request to the server to obtain the cloud account and the corresponding password. Among them, the registration request may include the authentication information of the controlled device, and the authentication information includes any one or more of the following: the authentication code of the controlled device, the identification of the controlled device or the third key (specifically the public key of the controlled device). In order to ensure that the information contained in the registration request is securely sent to the server, the controlled device also needs to use the second key (specifically the public key of the server) to encrypt the above information. For example, when the third key is an asymmetric key, the main processor encrypts based on the public key in the asymmetric key.

[0309] In addition, the authentication code and the third key included in the registration request are used to be stored in the server, so that the server can subsequently refresh the password based on the password refresh request of the controlled device. And the server can also send it to the master device for the master device to subsequently perform security authentication with the controlled device, that is, negotiate a new key to transmit instructions.

[0310] S114, the server obtains and stores the authentication code and the third key based on the second key.

[0311] Specifically, after receiving the registration request, the server decrypts the authentication code and the third key based on the second key. For example, when the second key is an asymmetric key, the server decrypts the authentication code and the third key based on the private key in the asymmetric key and stores them.

[0312] S115, replying the cloud account and the first password based on the second key.

[0313] Specifically, after the server decrypts the authentication code and the third key, it is equivalent to completing the registration of the controlled device. A cloud account and a corresponding first password can be generated for the controlled device, and encrypted with the second key and sent to the main processor of the controlled device.

[0314] S116, the main processor of the controlled device synchronizes information with the secondary processor, including but not limited to: a cloud account and a first password.

[0315] Specifically, after the main processor of the controlled device receives the information returned by the server, it can use the second key to decrypt it to obtain the cloud account and the first password, and also synchronize the cloud account and the first password to the secondary processor, so that the secondary processor can establish a first connection with the server based on the information.

[0316] In addition, after the main processor of the controlled device obtains the cloud account and the first password, the main processor can also establish a second connection with the server based on the cloud account and the first password, and complete authentication with the main control device based on the second connection, etc., which will not be repeated here. For details, please refer to the examples of Stage 2-Stage 3 below, which establish a first connection with the server through the sub-processor, and complete authentication with the main control device based on the second connection.

[0317] In addition, the main processor of the controlled device can also synchronize other more information to the sub-processor, and can synchronize before or after synchronizing the above-mentioned cloud account and the first password, which is not limited in this embodiment of the present application.

[0318] The main processor of the controlled device can also synchronize other information to the secondary processor, including but not limited to device configuration information, and the URL for refreshing passwords supported by the server in the connection configuration information, etc. For the introduction of device configuration information and connection configuration information, please refer to the description of S62 above, which will not be repeated here.

[0319] S117, the sub-processor of the controlled device encrypts and stores the synchronized information.

[0320] Specifically, after the secondary processor of the controlled device receives the information synchronized by the main processor, it needs to be encrypted and stored. The encryption method can be based on the root key, profile file and device identification synchronized by the main processor to generate an encryption method, and then encrypt part or all of the synchronized information, for example, all information except the root key, profile file and device identification is encrypted and stored.

[0321] Phase 2 (S121-S122): After the controlled device is turned off, hibernated, asleep, or turned on but fails, a first connection is established with the server through the secondary processor.

[0322] S121, the secondary processor of the controlled device times out and fails to receive the bus heartbeat message from the primary processor or the server login instruction from the primary processor.

[0323] Specifically, when the main processor of the controlled device is in working state, the controlled device can establish a bus connection with the sub-processor through the main processor, and send bus messages, instructions for logging into the server, etc. based on the bus connection.

[0324] The type of bus connection may be any one or more types of connection modes such as USB, PCI, SPI, I2C, PCM, UART, MIPI, GPIO, etc. This application does not limit the type of bus connection.

[0325] The bus message includes but is not limited to a bus heartbeat message, a connection configuration message and a device configuration message. The connection configuration message and the device configuration message are a data form after encapsulating the synchronization information mentioned above. When the main processor of the controlled device does not receive the bus heartbeat message of the main processor or receives the login server instruction of the main processor after the timeout, the subsequent S122 is triggered.

[0326] S122, the sub-processor of the controlled device establishes a first connection based on the cloud account and the first password.

[0327] After the controlled device establishes a first connection with the server through the sub-processor, the controlled device can receive instructions from the master device forwarded by the server based on the first connection. However, in order to further ensure secure communication between the controlled device and the master device, the controlled device also needs to perform security authentication with the master device, that is, negotiate an encryption algorithm (such as a first key) so that the information of the master device and the controlled device can be shielded from the server and other devices for secure remote control.

[0328] Phase 3 (S131-S138): The controlled device performs security authentication on the master device based on the first connection.

[0329] S131, the main control device receives an input for turning on remote control.

[0330] Specifically, when the user inputs an operation to open the remote control application or the remote control interface to the main control device, the main control device can receive the corresponding input. For an introduction to the input operation, please refer to the above-mentioned UI embodiment. Figure 6A-6C Description.

[0331] S132, the master control device obtains the authentication code and the third key from the server.

[0332] S133, the master control device generates authentication information based on the authentication code and the third key.

[0333] Specifically, since there are security risks when the main control device directly forwards instructions through the server, the main control device can first generate authentication information based on the authentication code and the third key. After security authentication with the main control device, the negotiated encryption algorithm, i.e., the first key, is used to encrypt the instructions. In this way, the encrypted instructions can shield the server, i.e., other devices, to ensure the security of remote control.

[0334] S134-S135, the master control device sends authentication information for determining the first key to the controlled device through the server, and the authentication information does not include the first key.

[0335] Specifically, after the master device generates authentication information, it can request authentication from the controlled device through the server, and the authentication information does not contain the first key. The authentication information is only used by the master device and the controlled device to determine the encryption method, namely the first key, for subsequent treatment transmission.

[0336] S136, the controlled device verifies the authentication information through the authentication code synchronized by the main processor and the third key, and determines the first key.

[0337] S137-S138, the controlled device restores the authentication to the master device through the server.

[0338] Specifically, after the master control device receives the authentication pass information, it confirms that the encryption algorithm negotiated with the master control device, such as the first key, can be used to encrypt the remote control command when sending it subsequently.

[0339] Optionally, the main control device may also execute S132 first and then execute S131, which is not limited in this embodiment of the present application.

[0340] Optionally, the sub-stage S131-S134 in stage 3 may be executed at any step after the aforementioned S114, and is not limited to being executed after the second stage is executed.

[0341] Optionally, after S114 and before S132 , the master control device may also receive a notification sent by the server that a controlled device has registered.

[0342] Phase 4 (S141-S145): The controlled device receives and processes the instruction of the master device forwarded by the processing server based on the first connection.

[0343] S141, the main control device receives the input of the remote control.

[0344] Specifically, after the user inputs a remote operation for the controlled device to the master device, the master device can receive the corresponding input. The operation includes but is not limited to: power on, sleep wake-up, sleep wake-up or restart operation. For the introduction of the above operations, please refer to the above UI embodiment. Figure 6A-6E Description.

[0345] S142-S143, the master control device sends instructions to the controlled device through the server based on the first key and the authentication information.

[0346] Specifically, after receiving the remote control input, the master device can generate a corresponding instruction based on the input, encrypt the instruction with the first key, and carry authentication information, wherein the authentication information is the device account and password of the controlled device described above.

[0347] S144, the controlled device authenticates the instruction through the authentication information synchronized by the secondary processor and decrypts the instruction based on the first key.

[0348] Specifically, after the controlled device receives the instruction of the master device forwarded by the server, it needs to authenticate the instruction through authentication information, that is, compare the authentication information of the master device with the authentication information stored in the sub-processor to see if they are consistent. If they are, the authentication is considered to be successful, and the instruction needs to be decrypted using the first key.

[0349] Optionally, the authentication information and instructions in S144 may be sent separately, which is not limited in the embodiment of the present application.

[0350] S145, the controlled device starts the main processor through the secondary processor.

[0351] Specifically, after the sub-processor of the controlled device decrypts the instruction, if the instruction is power on, sleep wake-up, sleep wake-up or restart, then in response to the instruction, the sub-processor can start the main processor to switch to the working state, thereby switching the controlled device to the power on state. Specifically, the sub-processor can start the main processor through the EC.

[0352] In order to ensure secure communication between the controlled device and the server, the controlled device also needs to request a password refresh from the server, and use the refreshed password to establish a connection, thereby ensuring secure communication. For example, when the main processor of the controlled device is not in a working state, that is, during the aforementioned S122-S144, the controlled device can establish a first connection with the server through the secondary processor, and then request a password refresh from the server based on the first connection (see stage 5-1). For another example, when the main processor of the controlled device is in a working state, that is, after the aforementioned S145, the controlled device can establish a second connection with the server through the main processor, and then request a password refresh from the server based on the second connection and synchronize it to the secondary processor (see stage 5-2).

[0353] Phase 5-1: The controlled device requests a password refresh from the server through the secondary processor.

[0354] S151, the server determines that the first password has timed out.

[0355] Specifically, after the server completes the registration of the controlled device and generates the first password corresponding to the cloud account, the usage time of the first password is counted. When the preset time is exceeded, it is considered that the first password has timed out.

[0356] S152, the server notifies the sub-processor of the controlled device that the first password has timed out.

[0357] S153: Obtain signature verification information based on the second key.

[0358] Specifically, the extension information can be obtained by encrypting the device identification of the controlled device or the authentication code by using the second key.

[0359] When the second key is an asymmetric key, the second key used for encryption is a public key in the asymmetric key.

[0360] S154, the sub-processor of the controlled device requests the server to refresh the password, carrying the signature verification information.

[0361] Specifically, after obtaining the signature verification information, the controlled device requests the server to refresh the password, and carries the signature extension information, or also carries the device identification, etc.

[0362] S155: The server verifies the signature verification information based on the second key.

[0363] Specifically, the server can use the second key to decrypt the device identification or authentication code of the controlled device from the extension information, and then compare the decrypted device identification and authentication code with the device identification and authentication code stored on the local end. If the comparison results are consistent, the verification is considered to be passed.

[0364] When the second key is an asymmetric key, the second key used to decrypt the signature verification information is a private key in the asymmetric key.

[0365] S156, the server sends the second password corresponding to the cloud account to the sub-processor of the controlled device.

[0366] Specifically, after the server of the controlled device receives the second password, it uses the cloud account and the second password to re-establish the first connection with the server, and then receives the information sent by the server that the password has expired again, as well as the instructions of the master device forwarded by the server, etc. based on the first connection.

[0367] In addition, when the sub-processor of the controlled device receives the information sent by the server again to indicate that the second password updated last time has expired, the password refresh operation similar to the aforementioned S153-S156 is repeatedly performed.

[0368] In addition, the secondary processor usually does not have the authority to synchronize information with the main processor, so after the controlled device refreshes the password through the secondary processor, the main processor still uses the old password.

[0369] Phase 5-2: The controlled device requests a password refresh from the server through the main processor and synchronizes it to the secondary processor.

[0370] S161, establishing a second connection based on the cloud account and the first password request.

[0371] S162, determining that the first password has timed out or the first password is inconsistent with the second password corresponding to the cloud account.

[0372] Specifically, since the controlled device refreshes the password (second password) through the sub-processor, and the main processor does not know to use the refreshed password to establish a second connection, the server will notify the main processor to execute the subsequent password refresh process when it detects that the password used by the main processor is different from the latest password, in addition to detecting that the password has timed out.

[0373] S163-S167 correspond one-to-one to the aforementioned S152-S156, and will not be repeated here.

[0374] S168, the main processor synchronizes the third password to the sub-processor.

[0375] refer to Fig.11 , Next, the interactive process of how the controlled device involved in the aforementioned stage 2 establishes a remote connection with the server through the sub-processor to realize the remote function is described in detail.

[0376] Taking into account the possibility that the controlled device may fail, the present application provides an interactive process for realizing the remote control function through the sub-processor in the case of abnormal shutdown, hibernation, sleep, startup and failure (see stage 2-1), and also provides an interactive process for realizing the remote control function in the case of normal shutdown, hibernation and sleep (see stage 2-2).

[0377] Stage 2-1 (S1231-S1235): When the controlled device is abnormally shut down, hibernates, or sleeps, or is powered on and fails, a first connection is established with the server through the secondary processor to implement the remote control function.

[0378] S1231: The secondary processor of the controlled device times out and fails to receive the bus heartbeat message.

[0379] Specifically, the main processor of the controlled device can periodically send bus heartbeat messages to the secondary processor based on the bus connection. After receiving the bus heartbeat message, the secondary processor can parse the heartbeat status (including heartbeat establishment, heartbeat recovery and heartbeat loss, etc.) from it. When the secondary processor of the controlled device times out and fails to receive the bus heartbeat message, it is confirmed that the heartbeat status is heartbeat loss. Therefore, it can be considered that the controlled device is in an abnormal state, resulting in the main processor being unable to work normally and unable to send bus heartbeat messages to the secondary processor regularly.

[0380] The controlled device being in an abnormal state may be: the controlled device is abnormally shut down, hibernates or sleeps, causing the main processor to suddenly lose power and stop working; or the controlled device is in the powered-on state but a fault occurs, causing the main processor to be powered on but unable to work normally.

[0381] S1232: The sub-processor of the controlled device accesses the network based on the network configuration information in the device configuration message.

[0382] Specifically, after receiving the synchronized device configuration message described in S116, the sub-processor of the controlled device can parse out the network configuration information (such as WiFi account and password) therefrom, and then the sub-processor can access the network based on the network configuration information.

[0383] S1233: The sub-processor of the controlled device requests the server to establish a first connection based on the server login information (cloud account and first password) in the connection configuration message.

[0384] Specifically, after receiving the connection configuration message described in S116, the sub-processor of the controlled device can parse the server login information (such as the cloud account and the first password) therefrom, and then request the server to establish the first connection based on the server login information. In addition, when the sub-processor of the controlled device requests the server to establish the first connection, it also carries the identifier of the controlled device, which can be parsed in advance from the device configuration message.

[0385] S1234: The server sends a message to the sub-processor of the controlled device indicating that the first connection is successfully established.

[0386] Specifically, after the server receives the request to establish the first connection sent by the sub-processor of the controlled device, it can verify the cloud account and password. If the verification is successful, it will reply to the sub-processor with the indication information that the first connection is successfully established. The server will also store the identification of the controlled device for subsequent sending of the remote control command sent by the main control device to the corresponding controlled device based on the identification of the controlled device.

[0387] S1235: The secondary processor of the controlled device starts the reconnection function.

[0388] Specifically, the sub-processor of the controlled device is pre-configured with a reconnection function. After the sub-processor confirms the establishment of the first connection with the server, the reconnection function can be automatically enabled by default. The reconnection function includes: after the sub-processor inactively disconnects the long connection with the server, the sub-processor can automatically re-request to establish the long connection with the server. For example, if the sub-processor suddenly disconnects and disconnects the long connection with the server, after power is restored, the sub-processor will automatically trigger a request to re-establish the long connection with the server.

[0389] Stage 2-2 (S1241-S1246): After the controlled device is normally shut down, hibernates or sleeps, a first connection is established with the server through the sub-processor to implement the remote control function.

[0390] S1241, the main processor of the controlled device receives an input of shutdown, hibernation or sleep.

[0391] Specifically, when the controlled device is in a powered-on state, the main processor of the controlled device may receive an input of powering off, dormant or sleeping.

[0392] The aforementioned main processor receives the shutdown, hibernation or sleep input which may be triggered by an operation on a physical button of the controlled device, or by an operation on a virtual control displayed by the controlled device, which is not limited in this embodiment of the present application.

[0393] The aforementioned input may be directly detected by the main processor of the controlled device, or may be detected by the EC of the controlled device and transmitted to the main processor, which is not limited in this embodiment of the present application.

[0394] S1242, the main processor of the controlled device notifies the secondary processor to log in to the server.

[0395] Specifically, after the main processor of the controlled device receives the aforementioned shutdown, hibernation or sleep input, the main processor needs to send an instruction to the sub-processor before powering off and stopping working to notify the sub-processor to log in to the server. After the sub-processor logs in to the server, the main processor will power off and stop working, causing the controlled device to enter a shutdown, hibernation or sleep state.

[0396] The instruction to log in to the server may be sent in the form of a bus message, and may be specifically encapsulated in a device configuration message.

[0397] S1243-S1246 correspond one-to-one to S1232-S1235 in stage 2-1. For details, please refer to the previous description and will not be repeated here.

[0398] refer to Fig.12Next, the interactive process of how to wake up the controlled device involved in stage 4 when the controlled device is powered off, dormant or sleeping, and how to establish a remote connection with the server after the controlled device is woken up to realize the remote function will be described in detail.

[0399] Considering that the method for waking up the controlled device includes local wakeup in addition to the remote wakeup in the aforementioned stage 4, the present application provides an interactive process for realizing remote control through remote wakeup (see stage 4-1), and also provides an interactive process for realizing remote control through local wakeup (see stage 4-2).

[0400] Stage 4-1 (S1461-S1475): After the controlled device is remotely powered on, awakened from sleep or awakened from sleep, the controlled device disconnects the first connection and establishes a second connection with the server through the main processor to implement the remote control function.

[0401] S1461, the master control device sends a power-on, sleep wake-up, sleep wake-up or restart instruction to the server.

[0402] Specifically, the master device can log in to the server in advance and add the controlled device to the server to subscribe to the server for remote control of the controlled device. Afterwards, the master device can send a command for powering on, waking up from sleep, waking up from sleep, or restarting the controlled device to the server based on user operations.

[0403] S1462: The server sends a power-on, sleep-wake-up, sleep-wake-up, or restart instruction to the secondary processor of the controlled device based on the first connection.

[0404] Specifically, after receiving the remote control instruction for the controlled device sent by the main control device, the server can send a power-on, sleep wake-up, sleep wake-up or restart instruction to the secondary processor.

[0405] S1463: The sub-processor of the controlled device identifies the instruction based on the Profile file in the device configuration message.

[0406] Specifically, after receiving the device configuration message described in S112, the sub-processor of the controlled device can parse the Profile file therefrom and identify the meaning of the instruction based on the Profile file.

[0407] The Profile file describes the capabilities of the controlled device, that is, the commands that the controlled device can support. For example, a command can be represented by a command identifier (ID) and a specific value (value) under the command ID. For example, command ID = 1 indicates that the command belongs to a power-on / off type command, command value = 0 under command ID = 1 indicates a power-off command, and command value = 1 under command ID = 1 indicates a power-on command; command ID = 2 indicates that the command belongs to a sleep type command, command value = 0 under command ID = 2 indicates a sleep command, and command value = 1 under command ID = 2 indicates a sleep wake-up command.

[0408] Optional S1464, the sub-processor of the controlled device verifies the instruction based on the authentication information in the device configuration message.

[0409] After parsing the instruction, the controlled device can verify the instruction, that is, verify whether the instruction is sent by the authorized master device. S1464 is an optional operation.

[0410] S1465, the sub-processor of the controlled device sends a power-on, sleep wake-up, sleep wake-up or restart instruction to the EC based on the bus connection.

[0411] Specifically, the sub-processor of the controlled device establishes a bus connection with the EC in advance, and sends the parsed power-on, sleep wake-up, sleep wake-up or restart instruction to the EC based on the bus connection, so that the EC controls the main processor to perform the corresponding task.

[0412] S1466, the EC of the controlled device notifies the main processor to power on, wake up from hibernation, wake up from sleep, or restart.

[0413] S1467: The main processor of the controlled device starts, and the controlled device switches to the power-on state.

[0414] Specifically, the main processor of the controlled device is started under the control of the EC and starts to work, so that the controlled device is switched to the power-on state.

[0415] S1468, the main processor of the controlled device notifies the secondary processor to turn off the reconnection function.

[0416] Specifically, the main processor of the controlled device can establish a connection with the server after starting to work, but in order to prevent the secondary processor from preempting the first connection established with the server, the main processor needs to control the secondary processor to turn off the reconnection function first.

[0417] S1469: The secondary processor of the controlled device turns off the reconnection function.

[0418] S1470: The main processor of the controlled device requests the server to establish a second connection based on the server login information.

[0419] Specifically, the main processor of the controlled device pre-stores the server login information (cloud account and password) mentioned above, and the cloud account and password of the server can be the same as the cloud account and password used by the sub-processor mentioned above.

[0420] S1471, the server notifies the secondary processor of the controlled device that the first connection has been disconnected.

[0421] Specifically, since the same account can only be logged in by one client, after the server receives the information sent by the main processor of the controlled device requesting to establish a connection, the server will automatically exit the sub-processor of the controlled device that was previously logged in using the same cloud account, and notify the sub-processor of the controlled device that the first connection has been disconnected.

[0422] S1472, the server notifies the main processor of the controlled device that the second connection has been established.

[0423] Specifically, when the server disconnects the ground connection with the secondary processor of the controlled device, it establishes a second connection with the main processor of the controlled device and notifies the main processor of the controlled device that the second connection has been established.

[0424] It can be seen that the above S1471 and S1472 occur simultaneously, so the server and the controlled device establish a seamless connection, which can prevent the user using the main control device from feeling the controlled device being disconnected, thus bringing a smooth remote control experience to the user.

[0425] S1473, the main processor of the controlled device notifies the sub-processor to release resources.

[0426] Specifically, after the main processor of the controlled device successfully establishes a second connection with the server, the controlled device does not need to use a sub-processor to replace the main processor to implement the remote control function. Therefore, the main processor of the controlled device will notify the sub-processor to release resources, thereby reducing the work of the sub-processor and making the sub-processor enter a low-power working mode.

[0427] S1474, the sub-processor of the controlled device exits the network and clears the temporary data in the RAM.

[0428] Specifically, the sub-processor releases resources including but not limited to: exiting the network and clearing temporary data in the RAM. The exited network is the network connected based on the network configuration information in the configuration message described in S1232. The temporary data in the RAM includes but is not limited to data parsed from the bus heartbeat message to the memory, such as instruction information parsed from the profile file.

[0429] Stage 4-2 (S1481-S1489): After the controlled device is powered on locally, awakened from sleep or woken up from sleep, the controlled device disconnects the first connection and establishes a second connection with the server through the main processor to implement the remote control function.

[0430] S1481, the main processor of the controlled device receives input of power on, sleep wake-up or sleep wake-up.

[0431] Specifically, when the controlled device is in the shutdown state, the main processor of the controlled device can receive a power-on input; when the controlled device is in the sleep state, the main processor of the controlled device can receive a sleep wake-up input; when the controlled device is in the sleep state, the main processor of the controlled device can receive a sleep wake-up input; when the controlled device is in the power-on, sleep or sleep state, the main processor of the controlled device can also receive a restart input.

[0432] The aforementioned input may be triggered by operating a physical button of the controlled device, or by operating a virtual control displayed by the controlled device, which is not limited in the embodiments of the present application.

[0433] The aforementioned input may be directly detected by the main processor of the controlled device, or may be detected by the EC of the controlled device and transmitted to the main processor, which is not limited in this embodiment of the present application.

[0434] S1482-S1489 correspond one-to-one to S1467-S1474 in stage 4-1. Please refer to the previous description for details, which will not be repeated here.

[0435] Next, combined with the above Figure 3B The software architecture of the controlled device introduced, and Fig.10 The device interaction process shown in the figure is as follows. Next, the detailed module interaction process of each stage will be introduced in detail.

[0436] refer to Fig.13 , Fig.13 The interactive process of information synchronization between the primary and secondary processors is exemplified.

[0437] like Fig.13 As shown, the synchronization information stage is jointly executed by the main processor and the sub-processor of the controlled device, and specifically involves the following modules: the system manager and bus communication management module running on the main processor, the configuration management module, the wake-up management module and the remote connection management module running on the sub-processor. Among them, the configuration management module includes the bus communication management module, the bus heartbeat management module, the connection configuration management module and the device configuration management module. Among them, the wake-up management module includes the wake-up service management module, and the remote connection management module includes the networking management module. For the introduction of each module, please refer to the aforementioned Figure 3BDescription of the software architecture of the controlled device shown.

[0438] like Fig.13 As shown in the figure, the synchronization information phase includes the following interaction processes:

[0439] S711, the controlled device is powered on and the system manager is started.

[0440] S712: The system manager of the controlled device starts the bus communication management module in the main processor.

[0441] S713-S719 introduces the interaction process between the main processor and the sub-processor to maintain the bus connection through synchronous bus heartbeat messages.

[0442] S713, the bus communication management module in the main processor of the controlled device establishes a bus connection with the bus communication management module in the secondary processor.

[0443] S714, the bus communication management module in the main processor of the controlled device sends a bus heartbeat message to the bus communication management module in the secondary processor.

[0444] Specifically, the bus heartbeat message may be used to maintain the bus connection between the main processor and the secondary processor, so that the main processor sends a control command to the secondary processor through the bus connection.

[0445] S715, the bus communication management module in the sub-processor of the controlled device sends a bus heartbeat message to the bus heartbeat management module.

[0446] S716: The bus heartbeat management module in the sub-processor of the controlled device detects the bus heartbeat status.

[0447] Specifically, the bus heartbeat management module in the sub-processor of the controlled device detects the bus heartbeat state based on the received bus heartbeat message.

[0448] If the bus heartbeat message is parsed from the bus heartbeat management module and it is found that the bus heartbeat message is received for the first time during a boot process, it is considered that the bus heartbeat state at this time is heartbeat establishment;

[0449] If it is parsed from the bus heartbeat message of the bus heartbeat management module that the bus heartbeat message is not received during a startup process and after the heartbeat is established, it is considered that the bus heartbeat state at this time is heartbeat loss;

[0450] If it is parsed from the bus heartbeat message of the bus heartbeat management module that a bus heartbeat message is received again during a boot process and after the heartbeat is lost, it is considered that the bus heartbeat state at this time is heartbeat recovery.

[0451] S717, the bus heartbeat management module in the sub-processor of the controlled device sends the bus heartbeat status to the wake-up service management module.

[0452] Specifically, the bus heartbeat management module transmits the heartbeat status to the wake-up management module, so that the wake-up management module determines whether to start the remote connection management module according to the heartbeat status to execute establishment / disconnection of a long connection with the server.

[0453] S718-S719, the bus heartbeat management module in the secondary processor of the controlled device replies the bus heartbeat status to the bus communication management module in the main processor through the bus communication management module.

[0454] Specifically, the bus heartbeat management module in the sub-processor of the controlled device replies the bus heartbeat status to the bus communication management module in the main processor through the bus communication management module, which can be used by the main processor to know the online status of the sub-processor and maintain the bus connection for two-way communication.

[0455] The process of the synchronous bus heartbeat is that after the controlled device is turned on, the main processor periodically sends a bus heartbeat message to the secondary processor, and the secondary processor replies to the bus heartbeat message accordingly.

[0456] S721-S749 introduces the interactive process of synchronizing device configuration information to the secondary processor when the query comparison results of the main processor are different.

[0457] S721, the bus communication management module in the main processor of the controlled device queries the bus communication management module in the secondary processor for device identification, root key, and profile file.

[0458] Specifically, the device identifier, root key, and profile file are part of the device configuration information. The device configuration information can also be called a device configuration message. For an introduction to the device configuration information, please refer to the previous description, which will not be repeated here.

[0459] S722-S725, the bus communication management module in the secondary processor of the controlled device reads the device identification, root key, and profile file through the device configuration management module and returns them to the bus communication management module in the main processor.

[0460] Specifically, since the sub-processor of the controlled device stores the device identification, root key, and profile file synchronized by the main processor in the device configuration management module, before the main processor of the controlled device synchronizes the device configuration information to the sub-processor, it is necessary to query from the device configuration management module of the sub-processor whether there is information identical to the device configuration information to be synchronized. If there is identical information, synchronization will not be repeated. If it is different, synchronization will continue.

[0461] In addition, since the sub-processor of the controlled device may be restored to factory settings and thus clear the previously stored synchronized device configuration information, and the main processor does not know when the sub-processor is restored to factory settings, the main processor needs to query and compare with the sub-processor before synchronizing the information each time the controlled device is turned on.

[0462] S726, the bus communication management module of the main processor of the controlled device compares the device identification, the root key, and the profile file.

[0463] Specifically, the bus communication management module of the main processor of the controlled device compares the device identification, root key, and profile file to be synchronized with the device identification, root key, and profile file queried from the secondary processor. If the comparison results are the same, synchronization will not be repeated. If the comparison results are different, subsequent S727-S729 will be executed.

[0464] S727, the bus communication management module of the main processor of the controlled device sends the device identification, root key, and profile file to the bus communication management module of the secondary processor when the comparison results are different.

[0465] S728, the bus communication module of the secondary processor of the controlled device sends the device identification, the root key, and the profile file to the device configuration management module.

[0466] S729, the device configuration management module of the secondary processor of the controlled device stores the device identification, the root key, and the profile file.

[0467] Specifically, after receiving the device configuration information sent by the bus communication management module, the device configuration management module of the sub-processor of the controlled device stores it in the NVM to ensure that the device configuration information will not be lost after power failure, but the device configuration information will be cleared when the sub-processor restores the factory settings.

[0468] S730: The device configuration management module of the secondary processor of the controlled device sends the device identification, the root key, and the profile file to the remote connection management module.

[0469] Specifically, the remote connection management module receives the device identification, root key, and profile file and stores them. The device identification and root key are used by the remote connection management module to obtain an encryption algorithm based on the device identification, root key, etc., and then encrypt or decrypt the information that needs to be stored in the secondary processor later. The profile file is used to start the remote connection management module.

[0470] S741, the bus communication management module in the main processor of the controlled device queries the bus communication management module in the secondary processor for network configuration information and authentication information.

[0471] Specifically, the network configuration information and the authentication information belong to part of the device configuration information. The device configuration information may also be referred to as a device configuration message. For an introduction to the device configuration information, please refer to the previous description, which will not be repeated here.

[0472] S742, the bus communication management module in the sub-processor of the controlled device queries the device configuration management module for network configuration information and authentication information.

[0473] S743, the device configuration management module in the sub-processor of the controlled device obtains the decrypted network configuration information and authentication information from the remote connection management module.

[0474] Specifically, since the sub-processor of the controlled device encrypts the network configuration information and authentication information synchronized by the main processor through the remote connection management module and stores it in the device configuration management module, before the main processor of the controlled device synchronizes the device configuration information to the sub-processor, it is necessary to first obtain the encrypted information from the device configuration management module of the sub-processor, and then obtain the decrypted network configuration information and authentication information through the remote connection management module.

[0475] In addition, since the sub-processor of the controlled device may be restored to factory settings and thus clear the previously stored synchronized device configuration information, and the main processor does not know when the sub-processor is restored to factory settings, the main processor needs to query and compare with the sub-processor before synchronizing the information each time the controlled device is turned on.

[0476] S744-S745, the remote connection management module in the secondary processor of the controlled device sends the network configuration information and the authentication information to the bus communication management module of the main processor through the device configuration management module.

[0477] S746, the bus communication management module of the main processor of the controlled device compares the network configuration information and the authentication information.

[0478] Specifically, the bus communication management module of the main processor of the controlled device compares the network configuration information and authentication information to be synchronized with the network configuration information and authentication information queried from the secondary processor. If the comparison results are the same, synchronization will not be repeated. If the comparison results are different, subsequent S747-S749 will be executed.

[0479] S747, the bus communication management module of the main processor of the controlled device sends network configuration information and authentication information to the bus communication management module of the secondary processor when the comparison results are different.

[0480] S748, the bus communication module of the sub-processor of the controlled device sends the network configuration information and the authentication information to the device configuration management module.

[0481] S749, the device configuration management module of the sub-processor of the controlled device obtains the encrypted network configuration information and authentication information based on the device identification, root key, etc. from the remote management module and stores them.

[0482] Specifically, after receiving the device configuration information sent by the bus communication management module, the device configuration management module of the sub-processor of the controlled device first encrypts it through the remote management module, and then stores the encrypted ciphertext in the NVM to ensure that the information will not be lost after power failure, but the information will be cleared when the sub-processor is restored to factory settings.

[0483] S751-S773 introduces the interactive process of the main processor synchronizing the connection configuration information to the secondary processor when the query comparison results are different.

[0484] S751-S752, the bus communication management module in the main processor of the controlled device queries the device configuration management module for the key or certificate issued by the server through the bus communication management module in the secondary processor.

[0485] Specifically, the key or certificate issued by the server may be part of the connection configuration information, which may also be referred to as a connection configuration message. For an introduction to the connection configuration information, please refer to the previous description, which will not be repeated here. Alternatively, the key or certificate issued by the server may also be part of the device configuration information, which is not limited in this application.

[0486] S753, the device configuration management module in the sub-processor of the controlled device obtains the decrypted key or certificate issued by the server from the remote connection management module.

[0487] Specifically, since the sub-processor of the controlled device encrypts the key or certificate issued by the server synchronized with the main processor through the remote connection management module and then stores it in the device configuration management module, before the main processor of the controlled device synchronizes the device configuration information to the sub-processor, it is necessary to first obtain the encrypted information from the device configuration management module of the sub-processor, and then obtain the decrypted key or certificate issued by the server through the remote connection management module.

[0488] In addition, since the sub-processor of the controlled device may be restored to factory settings and thus clear the previously stored synchronized device connection information, and the main processor does not know when the sub-processor is restored to factory settings, the main processor needs to query and compare with the sub-processor before synchronizing the information each time the controlled device is turned on.

[0489] S754-S755, the remote connection management module in the secondary processor of the controlled device sends the key or certificate issued by the server to the bus communication management module of the main processor through the device configuration management module.

[0490] S756, the bus communication management module of the main processor of the controlled device compares the key or certificate issued by the server.

[0491] Specifically, the bus communication management module of the main processor of the controlled device compares the key or certificate issued by the server to be synchronized with the key or certificate issued by the server queried from the secondary processor. If the comparison results are the same, the synchronization will not be repeated. If the comparison results are different, the subsequent S747-S749 will be executed.

[0492] S757, the bus communication management module of the main processor of the controlled device sends the key or certificate issued by the server to the bus communication management module of the secondary processor when the comparison results are different.

[0493] S758, the bus communication module of the secondary processor of the controlled device sends the key or certificate issued by the server to the device configuration management module.

[0494] S759, the device configuration management module of the secondary processor of the controlled device obtains the encrypted key or certificate issued by the server based on the device identification, root key, etc. from the remote management module and stores it.

[0495] Specifically, after receiving the device configuration information sent by the bus communication management module, the device configuration management module of the sub-processor of the controlled device first encrypts it through the remote management module, and then stores the encrypted ciphertext in the NVM to ensure that the information will not be lost after power failure, but the information will be cleared when the sub-processor is restored to factory settings.

[0496] S761-S763, the bus communication management module in the main processor of the controlled device queries the remote connection management module for the cloud account and corresponding password, authentication code, third key and refresh password URL through the bus communication management module and connection configuration management module in the secondary processor.

[0497] Specifically, the cloud account and corresponding password, authentication code, third key and refresh password URL may be part of the connection configuration information. The connection configuration information may also be referred to as a connection configuration message. For an introduction to the connection configuration information, please refer to the previous description, which will not be repeated here.

[0498] S764, the remote connection management module in the main processor of the controlled device reads and decrypts the cloud account and the corresponding password, the authentication code, the third key and the URL of the refresh password.

[0499] Specifically, since the sub-processor of the controlled device encrypts the cloud account and the corresponding password, authentication code, third key and refresh password URL synchronized by the main processor through the remote connection management module and stores them in the remote connection management module, before the main processor of the controlled device synchronizes the device configuration information to the sub-processor, it is necessary to first obtain the encrypted information from the remote connection management module of the sub-processor, and then obtain the decrypted cloud account and the corresponding password, authentication code, third key and refresh password URL through the remote connection management module.

[0500] In addition, since the sub-processor of the controlled device may be restored to factory settings and thus clear the previously stored synchronized device connection information, and the main processor does not know when the sub-processor is restored to factory settings, the main processor needs to query and compare with the sub-processor before synchronizing the information each time the controlled device is turned on.

[0501] S765-S767, the remote connection management module in the secondary processor of the controlled device sends the cloud account and corresponding password, authentication code, third key and refresh password URL to the bus communication management module in the main processor through the connection configuration management module and the bus communication management module.

[0502] S768, the bus communication management module of the main processor of the controlled device compares the cloud account and the corresponding password, authentication code, third key and refresh password URL.

[0503] Specifically, the bus communication management module of the main processor of the controlled device compares the URL of the cloud account and the corresponding password, authentication code, third key and refresh password to be synchronized with the URL of the cloud account and the corresponding password, authentication code, third key and refresh password queried from the secondary processor. If the comparison results are the same, the synchronization connection configuration information will not be repeated. If the comparison results are different, the subsequent S769-S773 will be executed.

[0504] S769-S772, when the comparison results are different, the bus communication management module of the main processor of the controlled device sends the cloud account and corresponding password, authentication code, third key and refresh password URL to the remote connection management module through the bus communication management module and connection configuration management module of the secondary processor.

[0505] S773, the remote connection management module of the sub-processor of the controlled device obtains the URL of the cloud account and corresponding password, authentication code, third key and refresh password after encryption based on the device identification, root key, etc.

[0506] Specifically, after receiving the cloud account and the corresponding password, authentication code, third key and refresh password URL, the remote connection management module of the sub-processor of the controlled device encrypts them based on the device identification, root key, etc. and stores them in the NVM to ensure that the connection configuration information will not be lost after power failure, but the connection configuration information will be cleared when the sub-processor is restored to factory settings.

[0507] It is understandable that any of the above information can be queried and compared separately to determine whether they are synchronized, or any multiple types of information can be queried and compared simultaneously to determine whether they are synchronized. The embodiments of the present application do not limit this.

[0508] It can be seen that the above Fig.13 The interactive method of synchronization of information between the main and sub-processors shown not only reduces the write loss to the sub-processor and prolongs its service life, but also ensures that the privacy information can be stored safely. Specifically, due to the low specifications of the sub-processor, the number of writes to the NVM (such as flash memory) integrated in the sub-processor is limited. Therefore, before each information synchronization, the controlled device first needs to query the main processor whether the information in the sub-processor is consistent with the information to be synchronized. Only when there is inconsistency in the information, it will be written to the sub-processor.

[0509] It should be understood that each step in the above method embodiment provided by the present application can be completed by an integrated logic circuit of hardware in a processor or by instructions in the form of software. The method steps disclosed in the embodiments of the present application can be directly embodied as being executed by a hardware processor, or by a combination of hardware and software modules in a processor.

[0510] The present application also provides an electronic device, which may include: a memory and a processor, wherein the memory may be used to store a computer program; and the processor may be used to call the computer program in the memory so that the electronic device executes the method in any one of the above embodiments.

[0511] The present application also provides a chip system, which includes at least one processor for implementing the functions involved in the method executed by the electronic device in any of the above embodiments.

[0512] In one possible design, the chip system also includes a memory, which is used to store program instructions and data, and the memory is located inside or outside the processor.

[0513] The chip system may be composed of the chip, or may include the chip and other discrete devices.

[0514] Optionally, the processor in the chip system may be one or more. The processor may be implemented by hardware or by software. When implemented by hardware, the processor may be a logic circuit, an integrated circuit, etc. When implemented by software, the processor may be a general-purpose processor implemented by reading software code stored in a memory.

[0515] Optionally, the memory in the chip system may also be one or more. The memory may be integrated with the processor or may be separately arranged with the processor, which is not limited in the embodiments of the present application. Exemplarily, the memory may be a non-transient processor, such as a read-only memory ROM, which may be integrated with the processor on the same chip or may be arranged on different chips respectively. The embodiments of the present application do not specifically limit the type of memory and the arrangement of the memory and the processor.

[0516] Exemplarily, the chip system can be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a microcontroller unit (MCU), a programmable logic device (PLD) or other integrated chips.

[0517] The present application also provides a computer program product, which includes: a computer program (also referred to as code, or instruction), which enables a computer to execute the method executed by the electronic device in any of the above embodiments when the computer program is executed.

[0518] The present application also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program (also referred to as code or instruction). When the computer program is executed, the computer executes the method executed by the electronic device in any of the above embodiments.

[0519] The various implementation modes of the present application can be combined arbitrarily to achieve different technical effects.

[0520] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in this application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk SolidStateDisk), etc.

[0521] Those skilled in the art can understand that to implement all or part of the processes in the above-mentioned embodiments, the processes can be completed by computer programs to instruct related hardware, and the programs can be stored in computer-readable storage media. When the programs are executed, they can include the processes of the above-mentioned method embodiments. The aforementioned storage media include: ROM or random access memory RAM, magnetic disk or optical disk and other media that can store program codes.

[0522] In short, the above description is only an embodiment of the technical solution of the present invention, and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made according to the disclosure of the present invention shall be included in the protection scope of the present invention.

Claims

1. A remote control method, characterized in that: The method is applied to a communication system including a first device, a second device and a server, wherein the first device includes a main processor and a sub-processor, and the method includes: The main processor of the first device is in a working state, and the authentication information of the first device is sent to the server through the main processor; The server sends the authentication information of the first device to the second device; The second device sends an authentication request to the first device through the server, the authentication request carrying first data encrypted using the public key of the first device, the first data being calculated by a preset algorithm from any one or more of the following data: the authentication information, a random number of the first device, and a random number of the second device; The main processor of the first device is in a non-working state, and the secondary processor uses the private key of the first device and the preset algorithm to calculate the authentication information to obtain the first data, and determines the first key based on the first data; The second device sends a first instruction encrypted by the first key to the first device through the server; The first device uses the first key to decrypt the first instruction through the secondary processor, and starts the main processor through the secondary processor in response to the first instruction.

2. The method according to claim 1, characterized in that The authentication information of the first device includes any one or more of the following: an authentication code of the first device, an identifier of the first device, or a public key of the first device.

3. The method according to claim 1, characterized in that: The first device determines the first key based on the first data by the secondary processor specifically includes: The first device determines the first data as a first key through the secondary processor; Alternatively, the first device generates the first key through the first data and a second algorithm, and sends the second algorithm to the first device through the server.

4. The method according to any one of claims 1 to 3, characterized in that The first device sending the authentication information of the first device to the server through the main processor specifically includes: the first device sending a registration request to the server through the main processor, the registration request carrying the authentication information of the first device encrypted using the public key of the server.

5. The method according to claim 4, characterized in that The method further comprises: In response to the registration request, the server uses the private key of the server to decrypt and obtain the authentication information of the first device, and sends the first account number and the first password to the first device.

6. The method according to claim 5, characterized in that After the server sends the first account and the first password to the first device, the method further includes: When the main processor of the first device is in working state, the first device establishes a second connection with the server through the main processor using the first account and the first password; When the main processor of the first device is in a non-working state, the first device establishes a first connection with the server through the secondary processor using the first account and the first password.

7. The method according to claim 6, characterized in that The performance of the main processor is better than that of the secondary processor, and the first connection is disconnected when the second connection is established.

8. The method according to any one of claims 5 to 7, characterized in that: After the server sends the first account and the first password to the first device, the method further includes: When the usage time of the first password exceeds a preset time, the server updates the first password to a second password and sends the second password to the first device.

9. The method according to claim 8, characterized in that After the server sends the second password to the first device, the method further includes: The first device receives the second password through the main processor and sends the second password to the secondary processor; Alternatively, the first device receives the second password through the secondary processor.

10. The method according to claim 9, characterized in that After the first device receives the second password through the secondary processor, the method further includes: When the main processor of the first device is in working state, the first device requests the server to establish the second connection by using the first account and the first password through the main processor; The server determines that the first password is different from the second password; The server updates the second password to a third password, and sends the third password to the first device; The first device receives the third password through the main processor, and sends the third password to the secondary processor.

11. The method according to claim 9, characterized in that Before the server updates the first password to a second password, the method further includes: The first device sends signature verification information to the server, where the signature verification information is generated after being encrypted by the public key of the server; The server verifies the signature verification information using the private key of the server.

12. The method according to any one of claims 1 to 11, characterized in that Before the first device starts the main processor through the secondary processor, the method further includes: The first device also receives a device account and a password corresponding to the device account sent by the second device through the server; The first device determines that the device account is the same as the account of the first device, and the password corresponding to the device account is the same as the password corresponding to the account of the first device.

13. A remote control method, characterized in that: The method is applied to a first device including a main processor and a secondary processor, and the method includes: The main processor of the first device is in a working state, and the authentication information of the first device is sent to the server through the main processor, so that the server sends the authentication information of the first device to the second device; The first device receives, through the server, an authentication request sent by the second device, the authentication request carrying first data encrypted using a public key of the first device, the first data being calculated using a preset algorithm from any one or more of the following data: authentication information of the first device, a random number of the first device, and a random number of the second device; The main processor of the first device is in a non-working state, and the secondary processor uses the private key of the first device and the preset algorithm to calculate the authentication information to obtain the first data, and determines the first key based on the first data; The first device receives, through the server, a first instruction sent by the second device and encrypted using the first key; The first device uses the first key to decrypt the first instruction through the secondary processor, and starts the main processor through the secondary processor in response to the first instruction.

14. The method according to claim 13, characterized in that The authentication information of the first device includes any one or more of the following: an authentication code of the first device, an identification of the first device.

15. The method according to claim 13 or 14, characterized in that The first device determines a first key based on the first data by the secondary processor, specifically including: The first device determines the first data as a first key through the secondary processor; Alternatively, the first device generates the first key through the first data and a second algorithm, and sends the second algorithm to the first device through the server.

16. The method according to any one of claims 13 to 15, characterized in that The first device sending the authentication information of the first device to the server through the main processor specifically includes: the first device sending a registration request to the server through the main processor, the registration request carrying the authentication information of the first device encrypted using the public key of the server.

17. The method according to claim 16, characterized in that The method further comprises: When the authentication information of the first device is obtained by decrypting the private key of the server, the first device receives the first account number and the first password sent by the server.

18. The method according to claim 17, characterized in that The first device receives the first account and the first password sent by the server, specifically including: When the main processor of the first device is in working state, the first device establishes a second connection with the server through the main processor using the first account and the first password; When the main processor of the first device is in a non-working state, the first device establishes a first connection with the server through the secondary processor using the first account and the first password.

19. The method according to claim 18, characterized in that The performance of the main processor is better than that of the secondary processor, and the first connection is disconnected when the second connection is established.

20. The method according to any one of claims 17 to 19, characterized in that After the server sends the first account and the first password to the first device, the method further includes: When the usage time of the first password exceeds a preset time, the first device receives the second password sent by the server.

21. The method according to claim 20, characterized in that The first device receives the second password sent by the server, specifically including: The first device receives the second password sent by the server through the main processor, and sends the second password to the secondary processor; Alternatively, the first device receives the second password sent by the server through the secondary processor.

22. The method according to claim 21, characterized in that After the first device receives the second password sent by the server through the secondary processor, the method further includes: When the main processor of the first device is in working state, the first device requests the server to establish the second connection by using the first account and the first password through the main processor; The first device receives the third password sent by the server through the main processor, and sends the third password to the secondary processor.

23. The method according to claim 21 or 22, characterized in that Before the first device receives the second password sent by the server, the method further includes: The first device sends signature verification information to the server, where the signature verification information is generated after being encrypted by the public key of the server, and the signature verification information is used by the server to verify using the private key of the server.

24. The method according to any one of claims 13 to 23, characterized in that Before the first device starts the main processor through the secondary processor, the method further includes: The first device also receives a device account and a password corresponding to the device account sent by the second device through the server; The first device determines that the device account is the same as the account of the first device, and the password corresponding to the device account is the same as the password corresponding to the account of the first device.

25. The method according to any one of claims 13 to 24, characterized in that The first device encrypts and stores any one or more of the following: authentication information of the first device, the first key, a public key of the server, a public key of the first device, a private key of the first device, an account of the first device, or a password corresponding to the account of the first device.

26. A remote control method, characterized in that: The method is applied to a second device, and the method includes: The second device receives authentication information of the first device forwarded by the server; the first authentication information is that the server receives authentication information sent by the first device when the main processor of the first device is in a working state; The second device sends an authentication request to the first device through the server, the authentication request carrying first data encrypted using the public key of the first device, the first data being calculated by a preset algorithm from any one or more of the following data: the authentication information, a random number of the first device, and a random number of the second device; The authentication request is used when the main processor of the first device is in a non-working state, and the secondary processor uses the private key of the first device and the preset algorithm to calculate the authentication information to obtain the first data, and determines the first key based on the first data; The second device sends a first instruction encrypted with the first key to the first device through the server, so that the first device decrypts the first instruction using the first key through the secondary processor and starts the main processor through the secondary processor in response to the first instruction.

27. An electronic device, characterized in that: The electronic device includes multiple memories and multiple processors; the memory is coupled to the processor, the memory is used to store computer program code, the computer program code includes computer instructions, and the processor calls the computer instructions to enable the electronic device to execute the method as described in any one of claims 13-26.

28. A chip, wherein: The chip is applied to an electronic device, and the chip is used to call computer instructions so that the electronic device executes the method as described in any one of claims 13-26.

29. A computer-readable storage medium comprising instructions, characterized in that: When the instructions are executed on an electronic device, the electronic device is caused to execute the method as claimed in any one of claims 13 to 26.

30. A communication system, characterized in that: The communication system includes a first device, a second device and a server; the first device includes multiple memories and multiple processors; the memory is coupled to the processor, the memory is used to store computer program code, the computer program code includes computer instructions, and the processor calls the computer instructions so that the first device executes the method described in any one of claims 13-25; the second device includes multiple memories and multiple processors; the memory is coupled to the processor, the memory is used to store computer program code, the computer program code includes computer instructions, and the processor calls the computer instructions so that the second device executes the method described in claim 26.

Images