Equipment control method and related equipment

Abstract

The invention discloses an equipment control method and related equipment, and aims to realize alignment of outputs of multiple equipment as far as possible in a scene of multi-equipment linkage and improve user experience. For example, a host receives a play notification of an ith fragment of a first media sent by a first device, the ith fragment is one of n fragments of the first media, and both i and n are positive integers. The host responds to the received playing notification of the i-th fragment and controls the second device to present the i-th state, the i-th state corresponds to the i-th fragment, and the playing duration of the i-th fragment and the duration of the i-th state are both the first duration. And at the end of the first duration, if the host determines that the play notification of the (i + 1) th fragment of the first media sent by the first device is not received, controlling the second device to maintain the ith state or close the ith state.

Description

Technical Field

[0001] This application relates to the technical field of terminals, and in particular, to a device control method and related devices. Background Art

[0002] In some scenarios (such as birthday parties, home theaters), users expect different devices within the scenario to be联动 to enhance the atmosphere. For example, within the scenario, there are audio devices and lighting devices. During the playback of music by the audio device, the lighting device can produce a lighting effect, and the lighting effect can change with the change of the music rhythm, achieving the联动 of multiple devices. In such a scenario, the outputs between different devices need to be aligned. For example, the lighting effect needs to be aligned with the music rhythm. Otherwise, it will affect the user experience. Summary of the Invention

[0003] Embodiments of this application provide a device control method and related devices, which can achieve that the outputs between multiple devices are as aligned as possible in a scenario of联动 of multiple devices, improving the user experience.

[0004] In a first aspect, a device control method is provided, which is applied to a host. The method includes: the host receives a playback notification of the i-th segment of a first media sent by a first device, where the i-th segment is one of the n segments of the first media, and both i and n are positive integers; the host controls a second device to present the i-th state in response to receiving the playback notification of the i-th segment, where the i-th state corresponds to the i-th segment, and the playback duration of the i-th segment and the duration of the i-th state are both a first duration; at the end of the first duration, if the host determines that it has not received a playback notification of the (i + 1)-th segment of the first media sent by the first device, it controls the second device to maintain the i-th state or turn off the i-th state.

[0005] In this embodiment, the first device can report the playback progress of the first media to the host in real time, such as reporting which segment is about to be played. Therefore, the host can control the second device in real time to align the state of the second device with the playback progress of the first device. For example, if the first device is about to play the i-th segment, it sends a playback notification for the i-th segment to the host. Upon receiving the playback notification, the host controls the second device to enter the i-th state. This ensures that the i-th segment on the first device is aligned with the i-th state of the second device, providing finer granularity. Assuming the playback duration of the i-th segment is a first duration, if the first duration ends but the host does not receive a playback notification for the next segment (e.g., the first device experiences playback buffering due to network issues), the host controls the second device to remain in the previous state or close the previous state. This provides the user with the effect of the second device remaining in or closing the state when the media playback on the first device is buffering, thus improving the user experience.

[0006] In one possible design, the method further includes: the host receiving a playback notification for the (i+1)th segment sent by the first device; in response to receiving the playback notification for the (i+1)th segment, the host controlling the second device to present an (i+1)th state, the (i+1)th state corresponding to the (i+1)th segment, and the playback duration of the (i+1)th segment and the duration of the (i+1)th state both being a second duration.

[0007] In this embodiment, the playback duration of the i-th segment is a first duration. If the first duration ends but the host does not receive a playback notification for the next segment (e.g., the first device experiences playback interruption due to network anomalies), the host controls the second device to remain in the previous state or close the previous state. This provides the user with the effect that the second device remains in or closes its state when the first device experiences playback interruption, thus improving the user experience. After receiving a playback notification for the (i+1)-th segment (e.g., the first device resumes playback due to network recovery, thus continuing to report progress to the host), the host controls the second device to present the (i+1)-th state. This provides the user with the effect that the second device remains in or closes its state when the first device experiences playback interruption. Furthermore, after the first device resumes playback, the state of the second device continues to change with the media. Therefore, even when the first device experiences playback interruption, the playback progress of the first device can still be aligned with the state changes of the second device, improving the user experience.

[0008] In one possible design, before the host receives the playback notification for the i-th segment sent by the first device, the method further includes: the host acquiring a media stream of the first media; the host generating timing information of the first media based on the media stream of the first media, the timing information including the start time and end time corresponding to each of the n segments of the first media; the host sending the timing information to the first device, so that the first device sends a playback notification for each of the n segments to the host based on the timing information.

[0009] In this embodiment of the application, the host can generate timing information of the first media and provide the timing information to the first device. In this way, the first device can determine which segment it is playing based on the timing information, and report the playback progress to the host in real time so that the host can control the second device in real time.

[0010] In one possible design, the host, in response to receiving the playback notification of the i-th segment, controls the second device to present the i-th state, including: the host, in response to receiving the playback notification of the i-th segment, determines the i-th control information based on the i-th segment; the host controls the second device to present the i-th state based on the i-th control information.

[0011] In this embodiment, for each segment of the first media, the host can determine its corresponding control information and control the second device based on the control information so that the second device presents the corresponding state, thereby ensuring that the media playback progress of the first device and the state changes of the second device can be aligned, thus improving the user experience.

[0012] In one possible design, determining the i-th control information based on the i-th segment includes: the host determining the i-th control information based on the i-th segment and a first correspondence relationship, wherein the first correspondence relationship describes the correspondence between n segments of the first media and n control information, the n control information is used to control the second device to present n states, and the i-th control information is the control information corresponding to the i-th segment in the first correspondence relationship.

[0013] In this embodiment of the application, for each segment of the first media, the host can determine its corresponding control information according to the first correspondence, and then control the second device based on the control information so that the second device presents the corresponding state, thereby ensuring that the media playback progress of the first device and the state changes of the second device can be aligned, and improving the user experience.

[0014] In one possible design, the moment when the host receives the playback notification of the i-th segment is the first moment, the start time of the playback of the i-th segment is the second moment, the first moment is earlier than the second moment, and the duration between the first moment and the second moment is the third duration, which is greater than or equal to the duration required for the host to control the second device to present the i-th state.

[0015] In this embodiment of the application, before the first device starts playing the i-th segment, it can send a playback notification for the i-th segment to the host in advance. The advance amount can be greater than or equal to the duration required for the host to control the second device to present the i-th state, thereby ensuring that the playback of the i-th segment on the first device and the presentation of the i-th state on the second device can be synchronized as much as possible, thus improving the user experience.

[0016] In one possible design, before the host receives the playback notification for the i-th segment sent by the first device, the method further includes: the host sending second information to the first device, the second information indicating the third duration, so that the first device sends the playback notification for the i-th segment to the host in advance before the second time arrives, based on the third duration.

[0017] In this embodiment, the host can instruct the first device to send a third duration (e.g., how long in advance to send a segment playback notification) so that the first device sends a playback notification for the i-th segment to the host in advance before starting to play the i-th segment, thereby ensuring as much as possible that the playback of each segment of the first media on the first device can be aligned with the presentation of various states on the second device, thus improving the user experience.

[0018] In one possible design, the third duration includes the transmission delay between the host and the second device.

[0019] In this embodiment of the application, considering the transmission delay between the host and the second device, the first device can send a playback notification for the i-th segment to the host in advance before playing the i-th segment. The advance amount can be greater than or equal to the transmission delay between the host and the second device, thereby ensuring that the playback of the i-th segment on the first device and the presentation of the i-th state on the second device can be aligned as much as possible, thus improving the user experience.

[0020] In one possible design, before the host receives the playback notification for the i-th segment sent by the first device, the method further includes: the host receiving subscription information for a first service provided by the first device, wherein the first service is a service that notifies media playback progress; and the host subscribing to the first service based on user operation.

[0021] In this embodiment, after subscribing to the first service of the first device, the host can obtain real-time playback progress notifications of the media on the first device, thereby controlling the second device in real time. Users can decide whether to subscribe to the first service according to their needs, resulting in a better user experience.

[0022] In one possible design, the first device is an audio device.

[0023] It is understandable that the first device can be any device other than an audio device, such as a display device, without limitation.

[0024] In one possible design, the second device includes a lighting device, and the i-th state includes at least one of being lit, off, having a brightness of the i-th brightness, having a color of the i-th color, and having a color temperature of the i-th color temperature; or, the second device includes a display device, and the i-th state includes at least one of being turned on, off, displaying the i-th content, and having a brightness of the i-th brightness.

[0025] It is understandable that the second device can be other devices besides lighting and display devices, such as smart fireplaces and fresh air systems, without limitation.

[0026] In one possible design, the method further includes: in response to receiving a playback notification for the i-th segment, the host controls the third device to present a k-th state, the k-th state corresponding to the i-th segment, and the duration of the k-th state being the first duration; at the end of the first duration, if the host determines that it has not received a playback notification for the (i+1)-th segment sent by the first device, it controls the third device to maintain the k-th state or turn off the k-th state.

[0027] In this embodiment of the application, after the host receives the playback notification of the i-th segment sent by the first device, it can control multiple devices to present different states respectively. For example, the second device is in the i-th state and the third device is in the k-th state. For example, the second device is a lighting device and the third device is a display device (e.g., a television), thereby enhancing the atmosphere of the entire space environment.

[0028] Secondly, a device control method is also provided, applied to a first device. The method includes: the first device determining that the i-th segment of a first medium is about to be played, the first medium including n segments, the i-th segment being one of the n segments, where i and n are both positive integers; the first device sending a playback notification of the i-th segment to a host, so that the host controls a second device to present an i-th state, the i-th state corresponding to the i-th segment, and the playback duration of the i-th segment being the same as the duration of the i-th state; and the first device playing the i-th segment.

[0029] In this embodiment of the application, the first device (e.g., an audio device) can report the playback progress of the first media to the host in real time, such as which segment is about to be played, so that the host can control the second device in real time, thereby enabling the state of the second device to be aligned with the playback progress of the first device and improving the user experience.

[0030] In one possible design, the method further includes: when the i-th segment experiences a playback interruption, the first device determines the interruption duration of the i-th segment; the first device determines, based on the interruption duration, to delay the transmission of a playback notification for the (i+1)-th segment of the first media, and the delay duration is equal to the interruption duration.

[0031] In this embodiment of the application, when the i-th segment is interrupted on the first device, the first device may delay sending the playback notification of the next segment (i.e., the i+1-th segment). Correspondingly, the host also delays receiving the playback notification of the i+1-th segment, thereby delaying the control of the second device to present the i+1-th state, so as to ensure that the next segment and the next state of the second device can be aligned as much as possible.

[0032] In one possible design, the method further includes: when the transmission time of the (i+1)th segment is reached, the first device sends a playback notification of the (i+1)th segment to the host, so that the host controls the second device to present an (i+1)th state, the (i+1)th state corresponding to the (i+1)th segment, and the playback duration of the (i+1)th segment is the same as the duration of the (i+1)th state; the first device plays the (i+1)th segment.

[0033] In one possible design, before the first device determines that the i-th segment of the first media is about to be played, the method further includes: the first device acquiring timing information of the first media, the timing information including the start time and end time corresponding to each of the n segments of the first media; the first device determining that the i-th segment of the first media is about to be played includes: the first device determining that the i-th segment of the first media is about to be played based on the current playback progress of the first media and the timing information of the first media.

[0034] In one possible design, the first device acquires the timing information of the first media by: the first device receiving the timing information of the first media sent by the host.

[0035] In one possible design, the time when the playback notification of the i-th segment is sent is the first time, the time when the playback of the i-th segment starts is the second time, the first time is earlier than the second time, and the duration between the first time and the second time is the third duration, which is greater than or equal to the duration required for the host to control the second device to present the i-th state.

[0036] In one possible design, the method further includes: the first device receiving second information sent by the host, the second information indicating the third duration; and the first device sending a playback notification for the i-th segment to the host in advance before the second time arrives, based on the third duration.

[0037] In one possible design, the third duration includes the transmission delay between the host and the second device.

[0038] In one possible design, before the first device sends a playback notification for the i-th segment to the host, the method further includes: the first device providing the host with subscription information for a first service, wherein the first service is a service that notifies the media playback progress; and the first device determining that the host has subscribed to the first service.

[0039] In one possible design, the first device is an audio device.

[0040] In one possible design, the second device includes a lighting device, and the i-th state includes at least one of being lit, off, having a brightness of the i-th brightness, having a color of the i-th color, and having a color temperature of the i-th color temperature; or, the second device includes a display device, and the i-th state includes at least one of being turned on, off, displaying the i-th content, and having a brightness of the i-th brightness.

[0041] In one possible design, the method further includes: during the playback of the i-th segment, the first device receives a rewind instruction, the rewind instruction being used to instruct the first device to play the ij-th segment of the first media, where j is a positive integer; before the ij-th segment begins playback, the first device sends a playback notification for the ij-th segment to the host, so that the host controls the second device to present an ij-th state, the ij-th state corresponding to the ij-th segment, and the playback duration of the ij-th segment being the same as the duration of the ij-th state.

[0042] In one possible design, the method further includes: during the playback of the i-th segment, the first device receives a fast-forward instruction, the fast-forward instruction being used to instruct the first device to fast-forward to the (i+g)-th segment of the first media, where g is a positive integer; before the (i+g)-th segment begins playback, the first device sends a playback notification for the (i+g)-th segment to the host, so that the host controls the second device to present an (i+g)-th state, the (i+g)-th state corresponding to the (i+g)-th segment, and the playback duration of the (i+g)-th segment being the same as the duration of the (i+g)-th state.

[0043] Thirdly, a device control method is also provided, applied to a system, the system including a first device, a host, and a second device. The method includes: the host determining that the first device is about to play a first medium; the host generating timing information of the first medium based on the media stream of the first medium, the timing information including the start time and end time corresponding to each of the n segments of the first medium, where n is a positive integer; the host sending the timing information to the first device; the first device determining, based on the timing information, that the i-th segment of the first medium is about to be played, the i-th segment being one of the n segments, where i is a positive integer; the first device sending a playback notification for the i-th segment to the host; the first device playing the i-th segment; and the host, in response to receiving the playback notification for the i-th segment, controlling the second device to present an i-th state, the i-th state corresponding to the i-th segment, and the playback duration of the i-th segment and the duration of the i-th state both being a first duration.

[0044] In one possible design, the method further includes: at the end of the first duration, the host determines that it has not received a playback notification for the (i+1)th segment sent by the first device, and controls the second device to maintain the i-th state or turn off the i-th state.

[0045] In one possible design, the method further includes: when the i-th segment experiences a playback interruption, the first device determines the interruption duration of the i-th segment; the first device determines, based on the interruption duration, to delay the transmission of a playback notification for the (i+1)-th segment of the first media, where the delay duration is equal to the interruption duration.

[0046] In one possible design, the method further includes: when the transmission time of the (i+1)th segment is reached, the first device sends a playback notification for the (i+1)th segment to the host; the first device plays the (i+1)th segment; in response to receiving the playback notification for the (i+1)th segment, the host controls the second device to present an (i+1)th state, the (i+1)th state corresponding to the (i+1)th segment, and the playback duration of the (i+1)th segment is the same as the duration of the (i+1)th state.

[0047] In one possible design, the host, in response to receiving the playback notification of the i-th segment, controls the second device to present the i-th state, including: the host, in response to receiving the playback notification of the i-th segment, determines the i-th control information based on the i-th segment; the host controls the second device to present the i-th state based on the i-th control information.

[0048] In one possible design, determining the i-th control information based on the i-th segment includes: the host determining the i-th control information based on the i-th segment and a first correspondence relationship, wherein the first correspondence relationship describes the correspondence between n segments of the first media and n control information, the n control information is used to control the second device to present n states, and the i-th control information is the control information corresponding to the i-th segment in the first correspondence relationship.

[0049] In one possible design, the moment when the host receives the playback notification of the i-th segment is the first moment, the start time of the playback of the i-th segment is the second moment, the first moment is earlier than the second moment, and the duration between the first moment and the second moment is the third duration, which is greater than or equal to the duration required for the host to control the second device to present the i-th state.

[0050] In one possible design, before the host receives the playback notification for the i-th segment sent by the first device, the method further includes: the host sending second information to the first device, the second information indicating the third duration; and the first device sending the playback notification for the i-th segment to the host in advance before the second time arrives, based on the third duration.

[0051] In one possible design, the third duration includes the transmission delay between the host and the second device.

[0052] In one possible design, before the host receives the playback notification for the i-th segment sent by the first device, the method further includes: the host receiving subscription information for a first service provided by the first device, wherein the first service is a service that notifies media playback progress; and the host subscribing to the first service based on user operation.

[0053] Fourthly, an electronic device is also provided, comprising:

[0054] Processor, memory, and one or more programs;

[0055] The one or more programs are stored in the memory, and the one or more programs include instructions that, when executed by the processor, cause the electronic device to perform the method provided in the first or second aspect above.

[0056] Fifthly, a communication system is also provided, comprising: a host and a first device;

[0057] The host is used to perform the steps of the method described in the first aspect above;

[0058] A first device is used to perform the steps of the method described in the second aspect above.

[0059] In some embodiments, the communication system may further include a second device.

[0060] In a sixth aspect, a computer-readable storage medium is also provided for storing a computer program that, when run on a computer, causes the computer to perform the methods provided by any one of the first, second, or third aspects described above.

[0061] In a seventh aspect, a computer program product is also provided, comprising a computer program that, when run on a computer, causes the computer to perform the methods provided in any of the first, second, or third aspects described above.

[0062] Eighthly, a chip is also provided, which is coupled to a memory in an electronic device for calling a computer program stored in the memory and executing the technical solutions provided in the first or second aspect of the embodiments of this application. In the embodiments of this application, "coupling" means that two components are directly or indirectly combined with each other.

[0063] Ninthly, a chip system is also provided, the chip system comprising a first chip and a second chip.

[0064] The first chip is configured to perform the steps of the method described in the first aspect above;

[0065] The second chip is used to perform the steps of the method described in the second aspect above.

[0066] In one possible design, the first chip is located in the host and the second chip is located in the first device.

[0067] In one possible design, the chip system also includes a second chip, optionally located in a second device.

[0068] For the technical effects that can be achieved by the second to ninth aspects mentioned above, please refer to the description of the technical effects that can be achieved by the corresponding design schemes in the first aspect mentioned above. This application will not repeat them here. Attached Figure Description

[0069] Figure 1 A schematic diagram of a communication system provided in an embodiment of this application;

[0070] Figures 2A to 2C A schematic flowchart illustrating a device control method provided in an embodiment of this application;

[0071] Figures 3A to 3C Another schematic flowchart of a device control method provided in an embodiment of this application;

[0072] Figures 4A to 4C A schematic diagram of a device control process provided in an embodiment of this application;

[0073] Figure 5A Another schematic diagram of the device control process provided in an embodiment of this application;

[0074] Figures 6A to 6C Another schematic flowchart of a device control method provided in an embodiment of this application;

[0075] Figures 7A to 7B Another schematic flowchart of a device control method provided in an embodiment of this application;

[0076] Figure 8 A schematic diagram of an electronic device provided according to an embodiment of this application;

[0077] Figure 9 Another schematic diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0078] The following explanations of some terms used in the embodiments of this application are provided to facilitate understanding by those skilled in the art.

[0079] The embodiments of this application involve at least one, including one or more; where "multiple" means two or more. Furthermore, it should be understood that in the description of this specification, terms such as "first," "second," and "third" are used only for descriptive purposes and should not be construed as indicating relative importance or order. For example, "first device" and "second device" do not represent the degree of importance of the two or their order, but are merely for descriptive distinction. In the embodiments of this application, "and / or" merely describes an association relationship, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0080] The directional terms mentioned in the embodiments of this application, such as "up", "down", "left", "right", "inner", and "outer", are only for reference to the directions in the accompanying drawings. Therefore, the directional terms used are for better and clearer explanation and understanding of the embodiments of this application, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the embodiments of this application.

[0081] References to "one embodiment," "in some examples," or "some embodiments" as described in the embodiments of this application mean that one or more embodiments of this specification include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in some examples," "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0082] The technical solutions provided by the embodiments of this application are described below with reference to the accompanying drawings.

[0083] The technical solutions provided in this application can be applied to communication systems. For example, please refer to... Figure 1 This is a schematic diagram of a communication system provided in an embodiment of this application. Figure 1 The communication system comprises a smart host, a central device, and N subsystems. This communication system can be deployed in any location, such as a home, hotel, cinema, stage, or theater, without limitation. This article primarily uses a home setting as an example.

[0084] The intelligent host is responsible for processing various events in the communication system and can be understood as the decision-making center and command center of the communication system. Optionally, the intelligent host can be a mobile terminal or a server. Taking a mobile terminal as an example, the intelligent host can be a separate device independent of the central device and the N subsystems, or it can be integrated into the central device or into a device in one of the N subsystems, without limitation. For example, the intelligent host can be a simplified version (Easy Edition, EZ) or a smart version (Smart Home Edition, SZ) of Huawei's whole-house intelligent host. In addition, the deployment location of the intelligent host is not limited in this application embodiment; it can be built into the distribution box or located outside the distribution box and its position can be flexibly adjusted. In this application embodiment, the intelligent host can control one or more devices in the N subsystems.

[0085] For example, such as Figure 1The smart host includes a scene engine, which can be used to control one or more devices in N subsystems. For example, when the scene engine receives an input event, it controls one or more devices in the N subsystems. Optionally, the scene engine can receive input events in various ways, such as Method A and Method B. Method A: The smart host includes a first application, and the mobile terminal includes a second application. The first and second applications can be the same application (e.g., Huawei Smart Life application). The first and second applications can establish a connection in the cloud (e.g., the account of the first application is associated with the account of the second application). When a user triggers an operation within the second application on the mobile device, the mobile terminal can send the input event corresponding to the triggered operation to the cloud, which then sends the input event to the first application in the smart host. The first application then sends the input event to the scene engine. Therefore, in this way, the user can control the smart host through the mobile terminal. Optionally, the mobile terminal can include mobile phones, PCs, laptops, tablets, wearable devices, in-vehicle devices, etc. This application embodiment does not limit the type of mobile terminal. Method B: The smart host includes a first application, and the smart host is connected to a central control screen, which can display the first application. Therefore, users can open the interface of the first application on the central control screen and trigger operations within the interface of the first application. The first application then sends the input event corresponding to the triggered operation to the scene engine. Two methods for the scene engine to receive input events have been listed above; other methods are also possible, but not all will be listed here. After receiving an input event, the scene engine can control one or more devices in the N subsystems. The method of the scene engine controlling the devices will be explained later. Optionally, the scene engine can be a program or instruction. When the program or instruction is run, it controls one or more devices in the N subsystems. For example, the scene engine can be any form, such as an application or a plugin. Taking the scene engine as an application as an example, the application can be independent of the first application, or the scene engine can be built into the first application.

[0086] like Figure 1 The intelligent host also includes a parsing module, which determines the correspondence between media and control information, enabling the scene engine to align with media playback when controlling the device based on the control information. The specific implementation principle of the parsing module will be explained later. Optionally, the parsing module can be a program or instruction. For example, the parsing module can be any form, such as an application or plugin. Optionally, the parsing module can be independent of the scene engine and the first application, or it can be built into the scene engine or the first application. It should be noted that... Figure 1 Taking the parsing module located in the intelligent host as an example, the parsing module can also be located in the central device or in one of the N subsystems. In short, the deployment location of the parsing module is not limited.

[0087] A central device is responsible for the information management of all devices across the entire scenario. For example, it manages information about one or more devices across N subsystems. This information may include device location information, capability information, etc., as will be explained later. Optionally, the central device can be a separate device independent of the intelligent host and the N subsystems, or it can be integrated into the intelligent host or into one of the devices in the N subsystems; there are no limitations. Optionally, a central device may or may not be present. If there is no central device, the intelligent host can be responsible for the information management of all devices across the entire scenario. Figure 1 In the diagram, central equipment is represented by dashed lines, indicating that central equipment is optional.

[0088] N subsystems, such as Figure 1 The system comprises N subsystems, including lighting, shading, security, heating and ventilation, audio-visual, appliance, and furniture systems. A lighting system may include one or more lighting devices, such as chandeliers, ceiling lights, downlights, spotlights, floor lamps, wall lamps, table lamps, and nightlights. A shading system may include one or more shading devices, such as curtains, awnings, and skylights. A security system may include one or more security devices, such as smart locks, surveillance cameras, video doorbells, door and window alarms, smart peepholes, gas alarms, and audible and visual alarms. A heating and ventilation system may include one or more temperature control devices, such as air conditioners and fresh air systems. An audio-visual system may include one or more audio-visual devices, such as speakers and televisions. A appliance system may include one or more appliances, such as gas stoves, range hoods, robot vacuums, and smart toilets. A furniture system may include one or more furniture items, such as smart mattresses, massage chairs, and smart sofas. Figure 1 All N subsystems are connected to the intelligent host, so the intelligent host can control one or more devices in the N subsystems.

[0089] For ease of understanding, this article primarily uses the example of an intelligent host controlling the first and second devices in N subsystems. For example, as... Figure 1 The first device can be an audio device (e.g., a speaker) in an audio-visual system. The second device can be a lighting device in a lighting system. The following describes the control process of the intelligent host over the first and second devices.

[0090] In this embodiment, the smart host can control the first device and the second device in a coordinated manner. For example, while the first device is playing music, the smart host can control the state of the second device to change with the rhythm of the music. In this scenario, the rhythm of the music on the first device needs to be aligned with the lighting effects of the second device; otherwise, the user experience will be affected.

[0091] The embodiments of the present application provide a technical solution which can make the outputs between multiple devices as aligned as possible in the scenario of multi-device linkage. For example, it can make the music rhythm of the first device and the lighting effect of the second device as aligned as possible to improve the user experience. Optionally, the technical solutions provided by the embodiments of the present application can include the following two types.

[0092] The first solution

[0093] For example, please refer to Figure 2A , which is a schematic flowchart of the device control method provided by an embodiment of the present application. This process can be applicable to Figure 1 the communication system shown in Figure 2A . As

[0094] S201, the intelligent host determines that the first device is about to play the first media.

[0095] Exemplarily, S201 can be executed by the scene engine in the intelligent host. For the scene engine, please refer to the foregoing Figure 1 .

[0096] Exemplarily, the first device can be an audio device (such as a speaker), and the first media can be music, drama, play, sketch, cross talk, novel, etc., without limitation.

[0097] Optionally, a possible implementation of S201 is that the intelligent host determines that the first device is about to play the first media according to the user operation. The user operation can be an operation for triggering the first device to play the first media. A possible situation is that the first application in the central control screen receives the user operation and then sends the user operation to the intelligent host. Another possible situation is that the second application in the mobile terminal receives the user operation and then sends the user operation to the intelligent host. For the first application and the second application, please refer to the foregoing description.

[0098] Optionally, another possible implementation of S201 is that the intelligent host receives the indication information sent by the first device, and this indication information is used to indicate that the first device is about to play the first media. A possible situation is that after the first device receives the instruction (such as a sound signal) for indicating to play the first media, it sends the indication information to the intelligent host to indicate that the first device is about to play the first media. Another possible situation is that the first device automatically plays multiple pieces of music in sequence, and before playing the next piece of music, it sends the indication information to the intelligent host to indicate that the first device is about to play the next piece (i.e., the first media).

[0099] Optionally, another possible implementation of S201 is that when the smart host determines that the current time is about to reach the preset playback time of the first media, it determines that the first device is about to play the first media. For example, the preset time can be 20:00, 22:00, etc. Taking 22:00 as an example, when the smart host detects that the current time is about to reach 22:00, it determines that the first device is about to play the first media. Optionally, the preset time can be a system default or a user-set time, without limitation. Taking user setting as an example, there can be multiple setting methods, such as setting it within the interface of the first application on the central control screen, or setting it within the interface of the second application on the mobile terminal, etc. One possible scenario is that the user sets the first device to play "white noise" every day at 22:00 to help the user fall asleep; therefore, when the smart host determines that the current time is about to reach 22:00, it determines that the first device is about to play the first media, i.e., "white noise".

[0100] Optionally, another possible implementation of S201 is that the smart host determines the current environment to be a preset environment and determines that the first device is about to play the first media. Optionally, the environment may include weather, ambient brightness, temperature, humidity, etc. Taking a rainy day as an example, when the smart host detects that the current environment is rainy, it determines that the first device is about to play the first media. Optionally, the preset environment can be pre-configured by the system or set by the user, without limitation. One possible scenario is that the user sets the first device to play light music on rainy days; therefore, when the smart host determines that the current environment is rainy, it determines that the first device is about to play the first media, i.e., light music.

[0101] Optionally, another possible implementation of S201 is that when the smart host determines that the device state of one or more devices in the N subsystems has reached a preset state, it determines that the first device is about to play the first media. Optionally, the device state may include the device being turned on, off, etc. Taking one or more devices including a smart door lock as an example, assuming the preset state is that the smart door lock is open, then when the smart host detects that the smart door lock is open, it determines that the first device is about to play the first media.

[0102] S202, the intelligent host determines the control information of the second device based on the first media.

[0103] For example, S202 can be executed by the parsing module in the intelligent host; please refer to the previous text for information about the parsing module. Figure 1 .

[0104] Taking the second device as an example, the control information of the second device may include one or more functions such as turning on, turning off, adjusting brightness, and adjusting color temperature.

[0105] Optionally, one possible implementation of S202 is that the smart host determines the control information of the second device corresponding to the first media in the database. The database includes the control information of the second device corresponding to various media. Optionally, the database is located in the smart host or it may be located in the cloud. For ease of understanding, the database is illustrated in Table 1 below.

[0106] Table 1: Database

[0107]

[0108]

[0109] Therefore, the smart host can determine the control information of the second device corresponding to the first media from the database. Assuming the first media is music 1, the smart host determines the control information of the second device as control information 1 according to Table 1 above. Optionally, the database can be pre-configured or user-defined, without limitation. In some embodiments, the data in the database can be updated. For example, music 1 originally corresponds to control information 1, which can be updated to correspond to control information 3. The update method is not limited; for example, it can be manually updated by the user.

[0110] S203, the smart host sends a playback command to the first device to instruct the first device to play the first media.

[0111] S204, the intelligent host sends control information to the second device.

[0112] S205, the first device plays the first media.

[0113] S206, the second device adjusts its state based on the control information.

[0114] Continuing with the example of the first medium being music 1 in Table 1 above, the control information for the second device is control information 1. Therefore, based on control information 1, the second device adjusts its state, for example, by gradually decreasing the brightness and then increasing it again.

[0115] In some embodiments, S203 and S204 can be executed simultaneously. After receiving the playback instruction, the first device can immediately play the first media. After receiving the control information, the second device can immediately adjust the state of the second device based on the control information, which can, to a certain extent, make the playback of the first media aligned with the state of the second device.

[0116] In other embodiments, the playback command sent by the smart host to the first device carries time information, which indicates the time at which the first device begins playing the first media, for example, time T1. After receiving the playback command, the first device determines that the current time has reached time T1 and begins playing the first media. The control information sent by the smart host to the second device may also carry time information, indicating the time at which the second device begins adjusting its state based on the control information, for example, time T1. Optionally, the control information of the second device may also carry the time at which the second device stops adjusting its state based on the control information, for example, time T2. Time T2 may be the time at which the first media ends playback. After receiving the control information, the second device determines that the current time has reached time T1, begins adjusting its state based on the control information, and determines that the current time has reached time T2, stops adjusting its state based on the control information.

[0117] The first approach will be illustrated with an example below.

[0118] For example, see Figure 2B The first device starts playing the first media at time T1 and ends playing it at time T2. During the time interval from T1 to T2, the brightness of the second device changes with the rhythm of the first media. For example, at time T1, the second device lights up, then its brightness gradually decreases and then gradually increases again to match the rhythm of the music in the first media.

[0119] It should be noted that in practical applications, there is a possibility that the first device may experience a stutter while playing the first media, causing playback to be interrupted. During the period when playback of the first media is interrupted, the second device will still continue to adjust its state based on control information. For example, such as... Figure 2C The first device starts playing the first media at time T1, and the second device starts lighting up and adjusting its brightness at time T1. The state of the second device changes in real time with the rhythm of the first media. At time T3, the first device experiences a stutter, causing the first media playback to stop, and the interruption lasts from T3 to T4. However, during the period from T3 to T4, the state of the second device continues to adjust. Assume that at time T4, the first media resumes playback, but at this point, the state of the second device is no longer aligned with the rhythm of the first media. Figure 2C At time T2, the second device's status adjustment is complete, but the first media has not yet finished playing. At time T5, the first media finishes playing. Therefore, Figure 2C During the period from T4 to T5, after the first media resumes playback, the state of the second device is no longer aligned with the rhythm of the first media.

[0120] The second option

[0121] For example, see Figure 3AThis is a schematic flowchart illustrating another device control method provided in an embodiment of this application. This process can be applied to... Figure 1 The communication system shown. (As shown) Figure 3A The process may include:

[0122] S301, the smart host determines that the first device is about to play the first media.

[0123] For information on S301, please refer to [link / reference]. Figure 2A S201 in the document will not be repeated here.

[0124] S302, the intelligent host determines the timing information of the first medium and the control information of the second device based on the first medium. The timing information of the first medium includes the segment information of each of the n segments of the first medium. The control information of the second device includes n control information, which correspond one-to-one with the n segments of the first medium, where n is an integer greater than or equal to 2.

[0125] The timing information of the first medium and the control information of the second device are explained below.

[0126] I. Time-series information of primary media

[0127] In this embodiment, the timing information of the first media may include: segment information for each of the n segments of the first media, where the segment information may include the start and end times of the segment. For example, if the first media is a song with a playback duration of 1 minute, the song may contain n segments, each with a corresponding start and end time. For instance, if each segment lasts 5 seconds, then there are 12 segments. Optionally, the timing information of the first media may also include a segment identifier corresponding to each of the n segments. For example, the segment identifier may be a segment ID. Table 2 below illustrates an example of the timing information of the first media.

[0128] Table 2: Time Series Information of the First Media

[0129]

[0130] The above provides an example of the timing information of the first medium. It is understandable that before this, the intelligent host needs to determine the n segments contained in the first medium. One possible approach is for the intelligent host to determine the n segments based on the playback duration of the first medium. For example, every 10 seconds, 5 seconds, 3 seconds, etc., can be considered as a segment, thus determining n segments. Optionally, the playback duration of different segments can be the same or different, without limitation. Another possible approach is for the intelligent host to determine the n segments of the first medium based on the audio characteristics of the first medium, where the audio characteristics may include at least one of timbre, pitch, and loudness. For example, the intelligent host can determine all timbre transition positions in the first medium, defining the portion between two adjacent timbre transition positions as a segment, thus determining n segments. The above lists two methods for determining the n segments of the first medium; other methods are also possible, but not all are listed here.

[0131] II. Control information for the second device

[0132] In this embodiment of the application, the control information of the second device may include n control information items, each corresponding one-to-one with one of the n segments of the first medium. For example, Table 3 below shows an example of the control information of the second device.

[0133] Table 3: Correspondence between timing information of the first device and control information of the second device

[0134]

[0135] As shown in Table 3 above, the control information of the second device can include n control information items, each including an identifier, execution duration, and a control command. The identifier is used to uniquely identify the control information; optionally, the identifier can be the segment ID of the corresponding segment. The execution duration indicates how long the control command needs to be executed. The control command is used to indicate functions such as turning on / off the screen, adjusting brightness, and adjusting color temperature. Taking control information 1 as an example, control information 1 corresponds to segment 1, so the execution duration of control information 1 is equal to the playback duration of segment 1. Furthermore, control information 1 includes control command 1; when the second device executes control command 1, the state of the second device can match the rhythm of segment 1.

[0136] In some embodiments, the smart host can determine the control information of the second device based on the device capability information of the second device. The device capability information of the second device indicates what capabilities the second device possesses, such as audio playback capability, video playback capability, and image display capability. Optionally, the device capability information of the second device may include the device type, device model, supported communication protocols, and device operating parameters. Taking a lighting device as an example, the device operating parameters may include brightness range and color temperature range. Taking a display device as an example, the device operating parameters may include display brightness range. Optionally, the device capability information of the second device may also include the spatial location information of the second device, such as bedroom or living room. For example, if the second device is a lighting device, the smart host determines the control information of the second device based on the device capability information, including turning it on, turning it off, brightness value, and color temperature value.

[0137] The above describes the timing information of the first media and the control information of the second device. It is understandable that before determining the timing information of the first media and the control information of the second device, the smart host needs to acquire the first media, i.e., acquire its media stream. One possible approach is that the smart host can download the media stream of the first media itself. Another possible approach is that the smart host can receive the media stream of the first media sent by the first device. For example, after receiving an instruction (e.g., an audio signal) to play the first media, the first device sends instruction information to the smart host, indicating that it is about to play the first media, and also sends the media stream of the first media to the smart host, so that the smart host can determine the timing information of the first media and the control information of the second device based on the media stream. Optionally, when the first device sends the media stream of the first media to the smart host, it may send the entire media stream of the first media at once, or it may send it in segments, such as the first device downloading the media stream of the first media and sending each segment as it is downloaded. In short, the first device can provide the media stream of the first media to the smart host. After acquiring the media stream of the first medium, the intelligent host can determine the timing information of the first medium and the control information of the second device. One possible approach is for the intelligent host to determine the timing information of the first medium and the control information of the second device from a database based on the first medium. The database includes timing information for different media and the corresponding control information of the second device. Optionally, the database can be located within the intelligent host or in the cloud. For example, the database can be pre-configured.

[0138] S303, the intelligent host sends the timing information of the first media to the first device.

[0139] S304, the first device sends a playback notification for the i-th segment of the first media to the smart host. The i-th segment is one of the n segments of the first media. For example, the playback notification for the i-th segment of the first media may include: the segment ID of the i-th segment.

[0140] Optionally, the first device may send a playback notification for the i-th segment of the first media to the smart host at the same time as the i-th segment begins playback. Alternatively, the first device may send a playback notification for the i-th segment of the first media to the smart host before the i-th segment begins playback. These two methods will be explained later.

[0141] S305, the intelligent host determines the i-th control information based on the i-th segment. The i-th control information is one of the n control information of the second device and corresponds to the i-th segment.

[0142] For example, the intelligent host can determine the i-th control information corresponding to the i-th segment based on Table 3 above.

[0143] S306, the intelligent host controls the second device according to the i-th control information, so that the second device presents the i-th state.

[0144] One possible approach is that the intelligent host sends the i-th control information to the second device. The i-th control information includes the i-th control command and the execution duration of the i-th control command (see Table 3 above). After receiving the i-th control information, the second device executes the i-th control command to make the second device present the i-th state, and the duration of the i-th state is the duration stated above.

[0145] Another possible approach is that the smart host sends n control messages to the second device in advance. For example, while sending the timing information of the first media to the first device, the smart host sends n control messages to the second device before or afterward. Each control message includes a corresponding identifier, execution duration, and control command (see Table 3 above). In this case, since the n control messages have been sent to the second device in advance, the smart host can send the identifier of the i-th control message to the second device after receiving the playback notification of the i-th segment, without needing to send the i-th control message itself.

[0146] S307, the first device plays the i-th segment.

[0147] The second approach will be illustrated with an example below.

[0148] For example, see Figure 3B The first medium comprises three segments (taking n=3 as an example). The linkage control process between the first medium and the second device may include:

[0149] (1) At time T1, the first device sends a playback notification for the first segment to the intelligent host and begins playing the first segment. For example, the playback notification for the first segment may include the segment ID of the first segment. When the intelligent host receives the playback notification for the first segment, it determines the control information 1 corresponding to the first segment based on the correspondence (e.g., Table 3 above). Control information 1 includes control command 1 and the execution duration of control command 1. The intelligent host sends control information 1 or the identifier 1 of control information 1 to the second device (e.g., n control information messages have been sent to the second device in advance). When the second device executes control command 1, it presents the first state, for example, the brightness of the second device is brightness 1 and the color temperature is color temperature 1.

[0150] (2) At time T2, the first device sends a playback notification for the second segment to the intelligent host and begins playing the second segment. For example, the playback notification for the second segment may include the segment ID of the second segment. When the intelligent host receives the playback notification for the second segment, it determines the control information 2 corresponding to the second segment based on the correspondence (e.g., Table 3 above). The control information 2 includes the control command 2 and the execution duration of the control command 2. The intelligent host sends the control information 2 or the identifier 2 of the control information 2 to the second device (e.g., n control information messages have been sent to the second device in advance). When the second device executes the control command 2, it presents a second state, for example, the brightness is brightness 2 and the color temperature is color temperature 2.

[0151] (3) At time T3, the first device sends a playback notification for the third segment to the intelligent host and begins playing the third segment. For example, the playback notification for the third segment may include the segment ID of the third segment. When the intelligent host receives the playback notification for the third segment, it determines the control information 3 corresponding to the third segment based on the correspondence (e.g., Table 3 above). The control information 3 includes the control command 3 and the execution duration of the control command 3. The intelligent host sends the control information 3 or the identifier 3 of the control information 3 to the second device (e.g., n control information messages have been sent to the second device in advance). When the second device executes the control command 3, it presents a third state, such as brightness of 3 and color temperature of 3.

[0152] It should be noted that, Figure 3B Taking the first media as an example, which includes three segments, in practical applications, the first media may include more or fewer segments. The principle is the same, so I will not repeat it here.

[0153] Please see Figure 3C This is another schematic flowchart illustrating a device control method provided in an embodiment of this application. This flowchart can be understood as... Figure 3A The refinement, for example, such as Figure 3C The process includes:

[0154] S401, the scene engine sends a playback command for the first media to the first device.

[0155] Optionally, prior to S401, the following may also be included: the scene engine receives a user operation, which triggers the first device to play the first media. Alternatively, the intelligent host determines that the current time is about to reach the preset playback time of the first media. Alternatively, the intelligent host determines that the current environment is a preset environment. Alternatively, the intelligent host determines that the device state of one or more devices in N subsystems has reached a preset state. For more information on this, please refer to the preceding text. Figure 2A S201 in the middle.

[0156] S402, The first device acquires the media stream of the first media.

[0157] For example, the first device can download the media stream of the first media.

[0158] S403, the first device sends a parsing request for the first media to the parsing module.

[0159] For example, the parsing request for the first media may include information about the first media, such as its name, media stream, etc.

[0160] S404, the parsing module determines the timing information of the first medium and the control information of the second device.

[0161] For timing information about the first media and control information about the second device, please refer to the previous text. Figure 3A S302 in the middle.

[0162] In some embodiments, the intelligent host can determine the control information of the second device based on the device capability information of the second device. Please refer to the preceding description for details. Therefore, before S404, the parsing module needs to receive the device capability information of the second device. One possible scenario is that before S401, the second device sends its device capability information to the parsing module.

[0163] S405, the parsing module sends the timing information of the first media to the first device. For example, the timing information of the first media can be found in Table 3 above.

[0164] S406, the parsing module sends the control information of the second device to the scene engine. For example, the control information of the second device can be found in Table 3 above.

[0165] S407, the first device sends a playback notification for the i-th segment to the scene engine. Optionally, the playback notification may include the segment ID of the i-th segment.

[0166] S408, the first device plays the i-th segment.

[0167] S409, the scene engine responds to receiving the playback notification for the i-th segment and controls the second device to present the i-th state.

[0168] For example, based on the segment ID of the i-th segment, the scene engine determines the i-th control information corresponding to the i-th segment in the correspondence shown in Table 3 above, and controls the second device to present the i-th state based on the i-th control information.

[0169] In some examples, the scene engine may include an Event Condition Action (ECA). For example, an ECA may include three parts: Event, Condition, and Action. In this embodiment, the Event can be used to receive a playback notification of the i-th segment sent by the first device. One possible approach is that the playback notification of the i-th segment includes the start and end times of the i-th segment. The condition in the ECA can determine the corresponding i-th control information based on the start and end times of the i-th segment (e.g., determined according to Table 3 above). The Action in the ECA can control the second device to present the i-th state based on the i-th control information. Another possible approach is that the playback notification of the i-th segment may include the segment ID (also called the Event ID) of the i-th segment. The condition in the ECA can determine the corresponding i-th control information based on the Event ID (e.g., determined according to Table 3 above). The Action in the ECA can control the second device to present the i-th state based on the i-th control information.

[0170] In some examples, the condition in ECA can also be used to determine the delay, that is, to determine the actual time when the control message is sent, so that the Action sends the control message to the second device according to that time. For example, if the condition determines the delay to be a duration of 1 (e.g., 3 seconds), then the Action sends the i-th control message to the second device 1 duration (e.g., 3 seconds) before the i-th segment begins playback. The process of determining the delay will be discussed later. Figure 5A illustrate.

[0171] It is understandable that during the playback of the first media, you may encounter situations such as buffering, pausing, resuming playback, rewinding, and fast forwarding. Examples are given below.

[0172] 1. Lag or pause.

[0173] It's understandable that there could be various reasons for buffering, such as excessive load on the primary device or weak network signal. The pausing method could be user-instructed or automatic by the primary device. User-instructed pausing could include voice commands or button presses. Automatic pausing by the primary device could involve pausing playback of the primary media while performing other tasks. For example, if the primary device detects a user's voice prompt to "check the weather" while playing the primary media, it will check and play the weather information, pausing playback of the primary media during this process.

[0174] For example, see Figure 4A The first medium comprises three segments (taking n=3 as an example). The linkage control process between the first medium and the second device may include:

[0175] (1) At time T1, the first device sends a playback notification for the first segment to the intelligent host and begins playing the first segment. When the intelligent host receives the playback notification for the first segment, it determines the control information 1 corresponding to the first segment based on the correspondence (e.g., Table 3 above) and sends control information 1 or the identifier 1 of control information 1 to the second device, so that the second device is in the first state. Details of this part can be found above and will not be repeated here.

[0176] (2) At time T2, the first device sends a playback notification for the second segment to the intelligent host and begins playing the second segment. When the intelligent host receives the playback notification for the second segment, it determines the control information 2 corresponding to the second segment based on the correspondence (e.g., Table 3 above), and sends control information 2 or its identifier 2 to the second device, so that the second device is in the second state. Details of this part can be found above and will not be repeated here.

[0177] (3) During the playback of the second segment, a stutter or pause occurs, causing the first media to interrupt playback. For example, as shown in the image. Figure 4A At time Ta, the second segment's playback is interrupted, and playback resumes at time Tb. That is, the interruption duration is Ta-Tb, represented by a black box in the diagram. When the second segment's playback is interrupted, there are two possible handling methods. Method 1: The first device sends an interrupt command to the smart host to instruct the first media to stop playback. After receiving the interrupt command, the smart host can control the second device to pause. Method 2: The first device does not need to send an interrupt command to the smart host; that is, the smart host is unaware that the second segment's playback has been interrupted. In this case, the second device will continue to execute control command 2, for example, as... Figure 4A Control command 2 was executed at time T3.

[0178] For the first device, if playback had not been interrupted, the second segment should have finished playing at time T3. Due to the playback interruption, the second segment was not finished playing at time T3; for example, it was delayed until time T3+ΔT.

[0179] For the smart host, it should have received the playback notification for the third segment from the first device at time T3, and then controlled the second device based on control information 3. However, because the second segment on the first device was delayed, the smart host did not receive the playback notification for the third segment at time T3. Therefore, at time T3, the smart host could not control the second device based on control information 3.

[0180] For the second device, control command 2 had already been executed at time T3. It should have received control information 3 from the intelligent host at time T3 and then adjusted to the third state based on control information 3. However, due to the delay in the second segment on the first device, the second device did not receive control information 3 at time T3 and therefore could not present the third state.

[0181] In some embodiments, the second device can have multiple processing modes during the time period from T3 to T3+ΔT. Mode A: The second device maintains the second state. Mode B: The second device disables the second state.

[0182] Taking method A as an example, considering that the second state may be a fixed state (e.g., brightness, color temperature, etc. remain unchanged), or it may be a changing state (e.g., brightness and / or color temperature continuously change). If the second state is a fixed state, then the second device can maintain this fixed state during the time period from T3 to T3+ΔT. If the second state is a changing state, then during the time period from T3 to T3+ΔT, the second device can maintain the last state of the changing state, or the second device can cyclically display the changing states.

[0183] Taking method B as an example, the second state of the second device being turned off can be simply turning off the second device. For example, if the second device is a lighting device, turning off the second device can mean turning off the light. Alternatively, the second state of the second device being turned off can also be restoring to its initial state. Restoring to the initial state could include restoring the brightness to its initial value, restoring the color temperature to its initial value, etc.

[0184] (4) At time T3+ΔT, the first device sends a playback notification for the third segment to the intelligent host and starts playing the third segment. When the intelligent host receives the playback notification for the third segment, it determines the control information 3 corresponding to the third segment based on the correspondence (e.g., Table 3 above) and sends the control information 3 to the second device so that the second device presents the third state.

[0185] therefore, Figure 4A In the example, if playback is interrupted while the first device is playing a segment of the first media (e.g., the second segment), only that segment will not be aligned with the state of the second device; subsequent segments (e.g., the third segment) will still be aligned with the state of the second device. Please compare. Figure 2C and Figure 4A . Figure 2C In the process, when the first media playback is interrupted, the state of the second device continues to adjust until it ends, causing the state of the first media to be unable to align with that of the second device after playback resumes. Figure 4A In this system, when playback is interrupted in a segment, only that segment is affected; subsequent segments remain unaffected and can still be aligned with the state of the second device. Therefore, compared to... Figure 2C , Figure 4A The proposed solution is better.

[0186] 2. Rewind or fast forward.

[0187] In this embodiment, rewind can be understood as going back to a previous segment from the currently playing segment. Rewind can also be called "rewind" or other names. There are various ways to rewind, such as manual rewind by the user. Fast forward can be understood as going forward to a subsequent segment from the currently playing segment. Fast forward can also be called "forward" or other names. There are various ways to fast forward, such as manual fast forward by the user.

[0188] Taking rewinding as an example, please see [link to relevant documentation]. Figure 4B The first medium comprises three segments (taking n=3 as an example). The linkage control process between the first medium and the second device may include:

[0189] (1) At time T1, the first device sends a playback notification for the first segment to the intelligent host and begins playing the first segment. When the intelligent host receives the playback notification for the first segment, it determines the control information 1 corresponding to the first segment based on the correspondence (e.g., Table 3 above) and sends control information 1 or the identifier 1 of control information 1 to the second device, so that the second device is in the first state. Details of this part can be found above and will not be repeated here.

[0190] (2) At time T2, the first device sends a playback notification for the second segment to the intelligent host and begins playing the second segment. When the intelligent host receives the playback notification for the second segment, it determines the control information 2 corresponding to the second segment based on the correspondence (e.g., Table 3 above), and sends control information 2 or its identifier 2 to the second device, so that the second device is in the second state. Details of this part can be found above and will not be repeated here.

[0191] (3) During the playback of the second segment, the first device rewinds to the first segment, for example, to play the first media from the beginning.

[0192] For example, such as Figure 4B At time Ta, the playback rewinds to the first segment, starting from the beginning of the first media. Therefore, at time Ta, the first device sends a playback notification for the first segment to the intelligent host. Upon receiving this notification, the intelligent host, based on the correspondence (e.g., Table 3 above), determines that the first segment corresponds to control information 1 and sends control information 1 or its identifier 1 to the second device, causing the second device to enter the first state. It should be noted that, as... Figure 4B Originally, during the period from T2 to T3, the second device was in the second state. Due to the rewind, the second device is in the first state starting from time Ta, in order to keep synchronized with the first segment of the first device.

[0193] (4) At time Tb, the first device sends a playback notification for the second segment to the intelligent host and starts playing the second segment. When the intelligent host receives the playback notification for the second segment, it determines the control information 2 corresponding to the second segment based on the correspondence (e.g., Table 3 above) and sends control information 2 or the identifier 2 of control information 2 to the second device so that the second device is in the second state.

[0194] Therefore, through Figure 4B It can be seen that if the first device rewinds to a previous segment while playing the first media, it can still align with the state of the second device.

[0195] For example, to use fast forward, please see [link to fast forward section]. Figure 4C The first medium comprises three segments (taking n=3 as an example). The linkage control process between the first medium and the second device may include:

[0196] (1) At time T1, the first device sends a playback notification for the first segment to the intelligent host and begins playing the first segment. When the intelligent host receives the playback notification for the first segment, it determines the control information 1 corresponding to the first segment based on the correspondence (e.g., Table 3 above) and sends control information 1 or the identifier 1 of control information 1 to the second device, so that the second device is in the first state. Details of this part can be found above and will not be repeated here.

[0197] (2) At time T2, the first device sends a playback notification for the second segment to the intelligent host and begins playing the second segment. When the intelligent host receives the playback notification for the second segment, it determines the control information 2 corresponding to the second segment based on the correspondence (e.g., Table 3 above), and sends control information 2 or its identifier 2 to the second device, so that the second device is in the second state. Details of this part can be found above and will not be repeated here.

[0198] (3) During the playback of the second segment, the first device fast forwards to the third segment, that is, the third segment starts playing before the second segment has finished.

[0199] For example, such as Figure 4C At time Ta, the playback fast forwards to the third segment, meaning the third segment begins playing from time Ta. Therefore, at time Ta, the first device sends a playback notification for the third segment to the intelligent host. Upon receiving this notification, the intelligent host, based on the correspondence (e.g., Table 3 above), determines the corresponding control information 3 for the third segment and sends control information 3 or its identifier 3 to the second device, causing the second device to enter the third state. It should be noted that, as... Figure 4C Originally, during the period from T2 to T3, the second device was in the second state. Due to fast forward, starting from time Ta, the second device is in the third state to maintain synchronization with the third segment of the first device.

[0200] Therefore, through Figure 4C It can be seen that if the first device fast-forwards to a later segment while playing the first media, it can still align with the state of the second device.

[0201] The above embodiments are applicable to scenarios where there is no transmission latency between the first device, the second device, and the host. Figure 3B For example, at time T1, the first device sends a playback notification for the first segment to the intelligent host. Since there is no transmission delay, the intelligent host can control the second device to be in the first state at time T1. Understandably, in practical applications, there may be transmission delays between the first device, the second device, and the host. In this case, the first device can send a playback notification for the i-th segment to the intelligent host before the i-th segment begins playing.

[0202] For example, see Figure 5A The first medium comprises three segments (taking n=3 as an example). The linkage control process between the first medium and the second device may include:

[0203] (1) The first segment starts at time T1 and ends at time T2. At time Ta, before time T1, the first device sends a playback notification for the first segment to the intelligent host. Assume the duration between Ta and T1 is duration 1, which can be greater than or equal to the first transmission delay or the second transmission delay, or greater than or equal to the sum of the first and second transmission delays. The first transmission delay is the transmission delay between the first device and the host, and the second transmission delay is the transmission delay between the second device and the host.

[0204] Taking the time interval between Ta and T1 as 1, which is equal to the sum of the first and second transmission delays, as an example. For instance, as... Figure 5A At time Ta, the first device sends a playback notification for the first segment to the intelligent host. At time Tb, the intelligent host receives the playback notification for the first segment. The duration from Ta to Tb is the first transmission delay between the first device and the intelligent host. At time Tb, the intelligent host sends control information 1 to the second device. At time T1, the second device receives control information 1. The duration from Tb to T1 is the second transmission delay between the second device and the host. Since the duration 1 between Ta and T1 is equal to the sum of the first and second transmission delays, the first device sending the playback notification for the first segment to the intelligent host at time Ta ensures that the second device receives control information 1 at time T1. This ensures that when the first device starts playing the first segment, the second device starts executing control command 1, thereby aligning the first segment with the first state.

[0205] (2) The start time of the second segment is T2, and the end time is T3. At time Tc, before time T2, the first device sends a playback notification for the second segment to the smart host. Assume that the duration between Tc and T2 is duration 2, which can be greater than or equal to the first transmission delay or the second transmission delay, or greater than or equal to the sum of the first transmission delay and the second transmission delay.

[0206] Taking the time interval 2 between Tc and T2 as an example, which is equal to the sum of the first and second transmission delays. For example, as... Figure 5A At time Tc, the first device sends a playback notification for the second segment to the intelligent host. At time Td, the intelligent host receives the playback notification for the second segment. The duration from Tc to Td is the first transmission delay between the first device and the intelligent host. At time Td, the intelligent host sends control information 2 to the second device. At time T2, the second device receives control information 2. The duration from Td to T2 is the second transmission delay between the second device and the host. Since the duration 2 between Tc and T2 is equal to the sum of the first and second transmission delays, the first device sending the playback notification for the second segment to the intelligent host at time Tc ensures that the second device receives control information 2 at time T2. This ensures that when the first device starts playing the second segment, the second device starts executing control command 2, thereby aligning the second segment with the second state.

[0207] (3) The start time of the third segment is T3, and the end time is T4. At time Te, before time T3, the first device sends a playback notification for the third segment to the smart host. Assume that the duration between Te and T3 is 3, which can be greater than or equal to the first transmission delay or the second transmission delay, or greater than or equal to the sum of the first transmission delay and the second transmission delay.

[0208] Taking the time interval between Te and T3 as the sum of the first and second transmission delays as an example. For instance, as... Figure 5A At time Te, the first device sends a playback notification for the third segment to the intelligent host. At time Tf, the intelligent host receives the playback notification for the third segment. The duration from Te to Tf is the first transmission delay between the first device and the intelligent host. At time Tf, the intelligent host sends control information 3 to the second device. At time T3, the second device receives control information 3. The duration from Tf to T3 is the second transmission delay between the second device and the host. Since the duration between Te and T3 is equal to the sum of the first and second transmission delays, the first device sending the playback notification for the third segment to the intelligent host at time Te ensures that the second device receives control information 3 at time T2. This ensures that when the first device starts playing the third segment, the second device starts executing control command 3, thereby aligning the third segment with the third state.

[0209] exist Figure 5A In the example, the first device needs to obtain the first transmission delay and the second transmission delay to determine how far in advance to send the playback notification for the i-th segment. Taking the second transmission delay as an example, one possible way for the first device to obtain the second transmission delay is that the first device receives the second transmission delay sent by the smart host. Understandably, before this, the smart host needs to determine the second transmission delay. One possible scenario is that the second transmission delay is pre-configured; for example, the second device may have a pre-configured second transmission delay, allowing the second device to send the second transmission delay to the smart host in advance, so that the smart host can send the second transmission delay to the first device. Another possible approach is that the smart host can measure the second transmission delay. For example, the smart host can send a probe signal to the second device and record the sending time of the probe signal. Upon receiving a feedback signal from the second device, it records the receiving time of the feedback signal. The second transmission delay = (receiving time - sending time) / 2. Yet another possible approach is that the smart host determines the second transmission delay based on the communication protocol between it and the second device. It is understandable that different communication protocols result in different transmission delays when transmitting data. Therefore, the intelligent host can determine the second transmission delay based on the transmission protocol of the second device. For example, the intelligent host stores the correspondence between various communication protocols and transmission delays. Based on this correspondence, the second transmission delay corresponding to the communication protocol of the second device can be determined.

[0210] In some embodiments, when the smart host controls the second device, it needs to call an API interface to control the second device, for example, by calling the API interface to send corresponding control commands to the second device. It should be noted that if there are many second devices, such as multiple lighting devices (e.g., downlights, spotlights, etc.), and each lighting device has a different API interface, the smart host can merge the API interfaces corresponding to multiple lighting devices to achieve batch control in order to achieve unified management.

[0211] Understandably, the first transmission delay and / or the second transmission delay may vary dynamically; therefore, Figure 5A In this context, the duration 1 between Ta and T1, the duration 2 between Tc and T2, and the duration 3 between Te and T3 may be the same or different.

[0212] The above embodiments illustrate two schemes for the intelligent host to perform coordinated control of the first and second devices: the first scheme and the second scheme. One possible scenario is that the communication system only has the first scheme configured and not the second scheme; in this case, the first scheme is used. Alternatively, the communication system only has the second scheme configured and not the first scheme; in this case, the second scheme is used. Another possible scenario is that the communication system has both the first and second schemes configured; in this case, the intelligent host can choose one of the two schemes.

[0213] For example, see Figure 6A This is another schematic flowchart of a device control method provided in an embodiment of this application, which can be applied to... Figure 1 The communication system shown. (As shown) Figure 6A The process may include:

[0214] S601, the smart host determines that the first device is about to play the first media.

[0215] S602, the intelligent host determines whether the first condition is met. If yes, execute S603; otherwise, execute S604.

[0216] Optionally, the first condition may include at least one of the following:

[0217] (a) A first operation is received, which triggers the use of a second scheme. For example, the smart host provides a selection button that can be used to set either the first or the second scheme. The smart host can determine whether to use the first or the second scheme based on the user's operation on the selection button.

[0218] (b) Determine that the current remaining battery power of the smart host is higher than the preset battery power. Considering that the smart host does not need to control the second device for each segment in the first scheme, which saves power, while the second scheme consumes more power, the smart host can use the second scheme if it determines that the remaining battery power is relatively high; otherwise, the first scheme can be used.

[0219] (c) Determine that the current operating load of the intelligent host is lower than the preset load. Considering that the first scheme is more energy-efficient and the second scheme is more energy-efficient, the intelligent host can use the second scheme if it determines that the current load is low; otherwise, it can use the first scheme.

[0220] (d) Determine that the smart host has subscribed to the first service of the first device. The first service is the service provided by the first device to report media playback progress, and may also have other names, such as "rhythm notification service," "beat notification service," "tempo notification service," etc. One possible way for the smart host to subscribe to the first service of the first device is that the first device provides the smart host with subscription information for the first service, for example, through a QR code, barcode, etc., and the smart host subscribes to the first service based on user actions. After the smart host subscribes to the first service, it can use the second scheme; if it has not subscribed to the first service (including those who have subscribed and then canceled), it can use the first scheme.

[0221] The S603 intelligent host uses the second solution.

[0222] It should be understood that S603 may include Figure 3A S302 to S307 in the series.

[0223] S604, the intelligent host uses the first solution.

[0224] It should be understood that S604 may include Figure 2A S202 to S206 in the series.

[0225] For example, see Figure 6B This is another schematic flowchart of a device control method provided in an embodiment of this application, which can be applied to... Figure 1 The communication system shown. For example, such as... Figure 6B The process may include:

[0226] S601, The first device provides the intelligent host with subscription information for the first service.

[0227] For example, the subscription information can be displayed in the first application within the smart host (e.g., the Huawei Smart Life application). This can be done via QR codes, barcodes, or other similar formats.

[0228] S602, the smart host subscribes to the first service and sends a successful subscription indication message to the first device.

[0229] For example, a smart host subscribes to a first service based on user actions.

[0230] S603, the first device receives an instruction to play the first media.

[0231] For example, the first device receives an audio signal and, by analyzing the audio signal, determines that the audio signal contains the voice command "play the first media".

[0232] S604, the first device sends the media stream of the first media to the smart host.

[0233] Optionally, when the first device sends the media stream of the first media to the smart host, it can send the complete media stream at once or send it while preloading (i.e., send a portion at a time).

[0234] S605, the smart host has confirmed that it has subscribed to the first service.

[0235] S606, the intelligent host determines the timing information of the first media and the control information of the second device based on the first media. The timing information of the first media includes the segment information of each of the n segments of the first media. The control information of the second device includes n control information, which correspond one-to-one with the n segments of the first media, where n is an integer greater than or equal to 2.

[0236] Understandably, if the intelligent host determines that it has subscribed to the first service, it executes S606; if the intelligent host determines that it has not subscribed to the first service, it can execute... Figure 2A S202 to S206 in the series.

[0237] S607, the intelligent host sends the timing information of the first media to the first device.

[0238] S607, The first device confirms that the smart host has subscribed to the first service.

[0239] S608, the intelligent host sends the timing information of the first media to the first device.

[0240] S609, the first device sends a playback notification for the i-th segment of the first media to the smart host. The i-th segment is one of the n segments of the first media. For example, the playback notification for the i-th segment of the first media may include: the segment ID of the i-th segment.

[0241] S610, the intelligent host determines the i-th control information based on the i-th segment. The i-th control information is one of the n control information of the second device and corresponds to the i-th segment.

[0242] S611, the intelligent host controls the second device according to the i-th control information, so that the second device presents the i-th state.

[0243] S612, the first device plays the i-th segment.

[0244] about Figure 6B For the steps, please refer to Figure 3A The description will not be repeated here.

[0245] Please see Figure 6C This is a schematic flowchart of a device control method provided in an embodiment of this application. This method can be applied to... Figure 1 The communication system shown. (As shown) Figure 6C The process may include:

[0246] S600, the first device determines the designer's choice of the first media.

[0247] In some embodiments, S600 to S605 can be understood as a preparation phase. For example, the designer can be understood as the developer of the first service of the first device. The process of the first service has been described above. An exemplary scenario is that before the first service goes online, the designer can select various media, such as various songs, so that S601 to S605 are executed to prepare the timing information of each media and the corresponding control information of the second device. In this way, when the first device wants to play the first media, since the timing information of the first media and the corresponding control information of the second device have been determined in advance, there is no need to determine them again, thus improving efficiency.

[0248] S601, the first device acquires the media stream of the first media.

[0249] S602, the first device sends a parsing request for the first media to the parsing module.

[0250] S603, the parsing module determines the timing information of the first medium and the control information of the second device.

[0251] S604, the parsing module sends the timing information of the first media to the first device.

[0252] S605, the parsing module sends control information of the second device to the scene engine.

[0253] For the implementation principles of S601 to S605, please refer to the previous text. Figure 3C .

[0254] S606, the scene engine sends a playback command for the first media to the first device.

[0255] S607, the first device sends a playback notification for the i-th segment to the scene engine.

[0256] S608, the first device plays the i-th segment.

[0257] S609, the scene engine responds to receiving the playback notification for the i-th segment and controls the second device to present the i-th state.

[0258] For the implementation principles of S606 to S609, please refer to the previous text. Figure 3C .

[0259] In the above embodiments, taking a lighting device as an example, the linkage process between the second device and the first device is explained. Besides the second device, other devices can also be used, such as a third device. The linkage process between the third device, the second device, and the first device is described below. For example, the third device can be any device in N subsystems other than the first and second devices. For instance, the third device can be a device in a heating, cooling, and ventilation system, such as an air conditioner or a fresh air system; or, the third device can be a display device in an audio-visual system, such as a television; or, the third device can be a smart fireplace. The following explanation uses a television as an example of the third device.

[0260] For example, see Figure 7A This is a schematic flowchart of a device control method provided in an embodiment of this application. This flowchart can be applied to... Figure 1 The communication system shown. (As shown) Figure 7A The process may include:

[0261] S701, the smart host determines that the first device is about to play the first media.

[0262] For information on S701, please refer to [link / reference]. Figure 2A S201 in the document will not be repeated here.

[0263] S702, the intelligent host determines the timing information of the first medium, the control information of the second device, and the control information of the third device. The timing information of the first medium includes the segment information of each of the N segments of the first medium. The control information of the second device includes N control information 'a', which correspond to N segments. The control information of the third device includes N control information 'b', which correspond to N segments. N is an integer greater than or equal to 2.

[0264] Regarding the timing information of the first medium, please refer to the preceding text (e.g., Table 3), which will not be repeated here. In the embodiments of this application, the control information of the second device may include n control information a (e.g., control information a1, control information a2, etc.), which correspond one-to-one with the n segments of the first medium. The control information of the third device may include n control information b (e.g., control information b1, control information b2, etc.), which correspond one-to-one with the n segments of the first medium. For example, Table 4 below shows an example of the control information of the second device and the control information of the third device.

[0265] Table 4: Correspondence between timing information of the first device, control information of the second device, and control information of the third device

[0266]

[0267] S703, the intelligent host sends the timing information of the first media to the first device.

[0268] S704, the first device sends a playback notification for the i-th segment to the intelligent host. The i-th segment is one of the n segments of the first media.

[0269] S705, the intelligent host determines the i-th control information a of the second device and the i-th control information b of the third device based on the i-th segment. Both the i-th control information a and the i-th control information b correspond to the i-th segment.

[0270] S706, the intelligent host controls the second device according to the i-th control information a.

[0271] S707, the intelligent host controls the third device according to the i-th control information b.

[0272] S708, the first device plays the i-th segment.

[0273] The following example illustrates this.

[0274] For example, see Figure 7B The first medium comprises three segments (taking n=3 as an example). The linkage control process of the first medium, the second device, and the third device may include:

[0275] (1) At time T1, the first device sends a playback notification for the first segment to the intelligent host and begins playing the first segment. When the intelligent host receives the playback notification for the first segment, based on the correspondence (e.g., Table 4 above), it determines that the control information of the second device corresponding to the first segment is control information a1, and the control information of the third device corresponding to the first segment is control information b1. The intelligent host may send control information a1 or the identifier a1 of control information a1 (e.g., n control information a1s have been sent to the second device in advance) to cause the second device to execute control command a1. The intelligent host sends control information b1 or the identifier b1 of control information b1 (e.g., n control information b1s have been sent to the third device in advance) to cause the third device to execute control command b1.

[0276] (2) At time T2, the first device sends a playback notification for the second segment to the intelligent host and begins playing the second segment. When the intelligent host receives the playback notification for the second segment, based on the correspondence (e.g., Table 4 above), it determines that the control information for the second device corresponding to the second segment is control information a2, and the control information for the corresponding third device is control information b2. The intelligent host can send control information a2 or its identifier a2 to the second device to cause the second device to execute control command a2. The intelligent host can send control information b2 or its identifier b2 to the third device to cause the third device to execute control command b2.

[0277] (3) At time T3, the first device sends a playback notification for the third segment to the intelligent host and begins playing the third segment. When the intelligent host receives the playback notification for the third segment, based on the correspondence (e.g., Table 4 above), it determines that the control information for the second device corresponding to the third segment is control information a3, and the control information for the third device is control information b3. The intelligent host can send control information a3 or its identifier a3 to the second device to cause the second device to execute control command a3. The intelligent host can send control information b3 or its identifier b3 to the third device to cause the third device to execute control command b3.

[0278] Therefore, through Figure 7B It can be seen that the intelligent host can control the first, second and third devices to work together, and the outputs of the three devices can be aligned.

[0279] It is understandable that during the playback of the primary media, you may encounter situations such as buffering, pausing, resuming playback, rewinding, and fast forwarding. The processing principle is the same as that of the front end mentioned earlier, so it will not be repeated.

[0280] Please see Figure 8This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device can be one of the electronic devices listed above, such as a smart host, a first device, a second device, or a third device. Figure 8 As shown, the electronic device may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0281] Processor 110 may include one or more processing units, such as an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors. The controller may serve as the nerve center and command center of the electronic device. The controller can generate operation control signals based on instruction opcodes and timing signals to control instruction fetching and execution. Processor 110 may also include memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that processor 110 has just used or is repeatedly used. If processor 110 needs to reuse the instruction or data, it can directly retrieve it from the memory. This avoids repeated access, reduces the waiting time of processor 110, and thus improves system efficiency.

[0282] In some embodiments, the processor 110 may execute the device control method provided in the embodiments of this application.

[0283] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0284] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple I2C buses. The processor 110 can couple to the touch sensor 180K, charger, flash, camera 193, etc., through different I2C bus interfaces. For example, the processor 110 can couple to the touch sensor 180K through the I2C interface, enabling the processor 110 and the touch sensor 180K to communicate through the I2C bus interface, thereby realizing the touch function of the electronic device 100.

[0285] The I2S interface can be used for audio communication. In some embodiments, the processor 110 may include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to enable communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the I2S interface to enable the function of answering phone calls through a Bluetooth headset.

[0286] The PCM interface can also be used for audio communication, sampling, quantizing, and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled via the PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface, enabling the function of answering phone calls through a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0287] The UART interface is a universal serial data bus used for asynchronous communication. This bus can be a bidirectional communication bus. It converts the data to be transmitted between serial and parallel communication. In some embodiments, the UART interface is typically used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 via the UART interface to implement Bluetooth functionality. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the UART interface to enable music playback through Bluetooth headphones.

[0288] The MIPI interface can be used to connect the processor 110 to peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI) and a display serial interface (DSI). In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to enable the electronic device 100 to capture images. The processor 110 and the display screen 194 communicate via the DSI interface to enable the electronic device 100 to display images.

[0289] The GPIO interface can be configured via software. It can be configured as a control signal or a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 to a camera 193, a display screen 194, a wireless communication module 160, an audio module 170, a sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0290] USB port 130 is a USB standard compliant interface, specifically a Mini USB port, Micro USB port, USB Type-C port, etc. USB port 130 can be used to connect a charger to charge electronic device 100, and can also be used for data transfer between electronic device 100 and peripheral devices. It can also be used to connect headphones for audio playback. This interface can also be used to connect other electronic devices, such as AR devices.

[0291] It is understood that the interface connection relationships between the modules illustrated in the embodiments of the present invention are merely illustrative and do not constitute a structural limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0292] The wireless communication function of the electronic device can be implemented through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor, and baseband processor. Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device can be used to cover one or more communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in conjunction with a tuning switch.

[0293] The mobile communication module 150 can provide solutions for wireless communication applications including 2G / 3G / 4G / 5G in electronic devices. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 1. In some embodiments, at least some functional modules of the mobile communication module 150 may be housed in processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 may be housed in the same device.

[0294] The wireless communication module 160 can provide solutions for wireless communication applications in electronic devices, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0295] In some embodiments, antenna 1 of the electronic device is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, enabling the electronic device to communicate with networks and other devices via wireless communication technology.

[0296] The display screen 194 is used to display the application's interface, etc. The display screen 194 includes a display panel. In some embodiments, the electronic device may include one or N display screens 194, where N is a positive integer greater than 1.

[0297] The electronic device 100 can perform shooting functions through an ISP, a camera 193, a video codec, a GPU, a display 194, and an application processor. The ISP is used to process the data fed back by the camera 193.

[0298] Internal memory 121 can be used to store computer executable program code, which includes instructions. Processor 110 executes various functional applications and data processing of the electronic device by running the instructions stored in internal memory 121. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system and software code for at least one application program. The data storage area may store data generated during the use of the electronic device (e.g., images, videos, etc.). Furthermore, internal memory 121 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, general-purpose flash memory, etc.

[0299] The external storage interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device. The external memory card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, images, videos, and other files can be saved on the external memory card.

[0300] Electronic devices can implement audio functions such as music playback and recording through audio modules 170, speakers 170A, receivers 170B, microphones 170C, headphone jacks 170D, and application processors.

[0301] The audio module 170 is used to convert digital audio information into analog audio signals for output, and also to convert analog audio input into digital audio signals. The audio module 170 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 170 may be located in the processor 110, or some functional modules of the audio module 170 may be located in the processor 110.

[0302] The speaker 170A, also known as a "loudspeaker," is used to convert audio electrical signals into sound signals. The electronic device 100 can listen to music or listen to hands-free calls and other external playback scenarios through one or more speakers 170A.

[0303] The receiver 170B, also known as a "handpiece," can be one or more, and is used to convert audio electrical signals into sound signals. When the electronic device 100 answers a telephone call or voice message, the receiver 170B can be brought close to the ear to listen to the voice.

[0304] The microphone 170C, also known as a "microphone" or "voice transducer," is used to convert sound signals into electrical signals.

[0305] The 170D headphone jack is used to connect wired headphones.

[0306] The pressure sensor 180A is used to sense pressure signals and can convert the pressure signals into electrical signals. In some embodiments, the pressure sensor 180A may be disposed on the display screen 194.

[0307] The gyroscope sensor 180B can be used to determine the motion attitude of an electronic device. In some embodiments, the gyroscope sensor 180B can determine the angular velocity of the electronic device about three axes (i.e., the x, y, and z axes). The gyroscope sensor 180B can be used for image stabilization.

[0308] The barometric pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device calculates altitude using the air pressure value measured by the barometric pressure sensor 180C to assist in positioning and navigation.

[0309] The magnetic sensor 180D includes a Hall effect sensor. Electronic devices can use the magnetic sensor 180D to detect the opening and closing of a flip cover.

[0310] The 180E accelerometer can detect the magnitude of acceleration in various directions (typically three axes) of electronic devices. When the electronic device is stationary, it can detect the magnitude and direction of gravity.

[0311] The 180F distance sensor is used to measure distance. Electronic devices can measure distance using infrared or laser.

[0312] The proximity sensor 180G may include, for example, a light-emitting diode (LED) and a light detector, such as a photodiode. The LED may be an infrared LED. The electronic device emits infrared light outward through the LED. The electronic device uses the photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that an object is near the electronic device. When insufficient reflected light is detected, the electronic device can determine that no object is near the electronic device.

[0313] An ambient light sensor 180L is used to detect ambient light levels. Electronic devices can adaptively adjust the brightness of the display screen 194 based on the detected ambient light levels.

[0314] The fingerprint sensor 180H is used to collect fingerprints.

[0315] The 180J temperature sensor is used to detect temperature.

[0316] Touch sensor 180K, also known as a "touch panel," can be located on display screen 194. The touch sensor 180K and display screen 194 together form a touchscreen, also known as a "touch screen." Touch sensor 180K is used to detect touch operations applied to or near it. The touch sensor can then transmit the detected touch operation to the application processor to determine the type of touch event.

[0317] The bone conduction sensor 180M can acquire vibration signals. In some embodiments, the bone conduction sensor 180M can acquire vibration signals from the vibrating bone segments of the human vocal cords.

[0318] Buttons 190 include a power button, volume buttons, etc. Buttons 190 can be mechanical buttons or touch buttons. The electronic device can receive button inputs and generate key signal inputs related to user settings and function control. Motor 191 can generate vibration alerts. Motor 191 can be used for incoming call vibration alerts or for touch vibration feedback. Indicator 192 can be an indicator light, used to indicate charging status, battery level changes, messages, missed calls, notifications, etc. SIM card interface 195 is used to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface 195 to achieve contact and separation with the electronic device.

[0319] Understandable Figure 8 The components shown do not constitute a specific limitation on the electronic device. The electronic device in the embodiments of the present invention may include, but is not limited to, components that are more advanced than those shown. Figure 8 More or fewer parts. Furthermore, Figure 8 The combination / connection relationships between the components can also be adjusted and modified.

[0320] Figure 9 This is a schematic diagram of the structure of an electronic device 900 provided in an embodiment of this application. The electronic device 900 can be one of the electronic devices described above (e.g., a smart host, a first device, a second device, or a third device). Figure 9 As shown, the electronic device 900 may include: one or more processors 901; one or more memories 902; a communication interface 903; and one or more computer programs 904. These devices can be connected via one or more communication buses 905. The one or more computer programs 904 are stored in the memory 902 and configured to be executed by the one or more processors 901. The one or more computer programs 904 include instructions. For example, when the electronic device 900 is the intelligent host described above, the instructions can be used to perform the relevant steps of the intelligent host as in the corresponding embodiments above, such as executing... Figures 1 to 7BThe relevant steps of the intelligent host. For example, when electronic device 900 is the first device mentioned above, this instruction can be used to execute the relevant steps of the first device as in the corresponding embodiments above, such as executing... Figures 1 to 7B The relevant steps of the intelligent host. For example, when electronic device 900 is the second device mentioned above, this instruction can be used to execute the relevant steps of the second device as in the corresponding embodiments above, such as executing... Figures 1 to 7B The relevant steps of the intelligent host. For example, when electronic device 900 is the third device mentioned above, this instruction can be used to execute the relevant steps of the third device as in the corresponding embodiments above, such as executing... Figures 1 to 7B The relevant steps for the third device. Communication interface 903 is used to enable communication between electronic device 900 and other devices, such as a transceiver.

[0321] The methods provided in the embodiments of this application above are described from the perspective of an electronic device as the executing entity. To implement the functions of the methods provided in the embodiments of this application above, the electronic device may include hardware structures and / or software modules, implementing the above functions in the form of hardware structures, software modules, or a combination of hardware structures and software modules. Whether a particular function is executed in the form of hardware structures, software modules, or a combination of hardware structures and software modules depends on the specific application and design constraints of the technical solution.

[0322] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)). Where there is no conflict, the solutions in the above embodiments can be combined.

[0323] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0324] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0325] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0326] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0327] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the scope and intent of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application is also intended to include such modifications and variations.

Claims

1. A device control method, characterized in that, Applied to a host, the method includes: The host receives a playback notification for the i-th segment of the first media sent by the first device, wherein the i-th segment is one of the n segments of the first media, and i and n are both positive integers; In response to receiving the playback notification of the i-th segment, the host controls the second device to present the i-th state, the i-th state corresponds to the i-th segment, and the playback duration of the i-th segment and the duration of the i-th state are both the first duration; At the end of the first duration, if the host determines that it has not received a playback notification for the (i+1)th segment of the first media sent by the first device, it controls the second device to maintain the i-th state or turn off the i-th state.

2. The method according to claim 1, characterized in that, The method further includes: The host receives the playback notification for the (i+1)th segment sent by the first device; In response to receiving the playback notification of the (i+1)th segment, the host controls the second device to present the (i+1)th state, the (i+1)th state corresponds to the (i+1)th segment, and the playback duration of the (i+1)th segment and the duration of the (i+1)th state are both the second duration.

3. The method according to claim 1 or 2, characterized in that, Before the host receives the playback notification for the i-th segment sent by the first device, the method further includes: The host acquires the media stream of the first media; The host generates timing information of the first media based on the media stream of the first media. The timing information includes the start time and end time of each of the n segments of the first media. The host sends the timing information to the first device, so that the first device sends a playback notification for each of the n segments to the host according to the timing information.

4. The method according to any one of claims 1-3, characterized in that, In response to receiving the playback notification for the i-th segment, the host controls the second device to present the i-th state, including: In response to receiving the playback notification of the i-th segment, the host determines the i-th control information based on the i-th segment; The host computer controls the second device to present the i-th state according to the i-th control information.

5. The method according to claim 4, characterized in that, The step of determining the i-th control information based on the i-th segment includes: The host determines the i-th control information based on the i-th segment and the first correspondence relationship, wherein the first correspondence relationship is used to describe the correspondence between the n segments of the first media and the n control information, the n control information is used to control the second device to present n states, and the i-th control information is the control information corresponding to the i-th segment in the first correspondence relationship.

6. The method according to any one of claims 1-5, characterized in that, The moment when the host receives the playback notification of the i-th segment is the first moment, the start time of the playback of the i-th segment is the second moment, the first moment is earlier than the second moment, and the duration between the first moment and the second moment is the third duration, which is greater than or equal to the duration required for the host to control the second device to present the i-th state.

7. The method according to claim 6, characterized in that, Before the host receives the playback notification for the i-th segment sent by the first device, the method further includes: The host sends a second message to the first device, the second message indicating the third duration, so that the first device sends a playback notification for the i-th segment to the host in advance before the second time arrives, based on the third duration.

8. The method according to claim 6 or 7, characterized in that, The third duration includes the transmission delay between the host and the second device.

9. The method according to any one of claims 1-8, characterized in that, Before the host receives the playback notification for the i-th segment sent by the first device, the method further includes: The host receives subscription information for a first service provided by the first device, wherein the first service is a service that notifies the media playback progress. The host subscribes to the first service based on user actions.

10. The method according to any one of claims 1-9, characterized in that, The first device is an audio device.

11. The method according to any one of claims 1-10, characterized in that, The second device includes a lighting device, and the i-th state includes at least one of being on, off, having a brightness of the i-th brightness, having a color of the i-th color, and having a color temperature of the i-th color temperature; or, The second device includes a display device, and the i-th state includes at least one of the following: activating, deactivating, displaying i-th content, and having a brightness of i-th brightness.

12. A device control method, characterized in that, Applied to a first device, the method includes: The first device determines that the i-th segment of the first media is about to be played. The first media includes n segments, and the i-th segment is one of the n segments, where i and n are both positive integers. The first device sends a playback notification for the i-th segment to the host, so that the host controls the second device to present the i-th state, the i-th state corresponds to the i-th segment, and the playback duration of the i-th segment is the same as the duration of the i-th state; The first device plays the i-th segment.

13. The method according to claim 12, characterized in that, The method further includes: When the playback of the i-th segment is interrupted, the first device determines the duration of the interruption of the i-th segment; The first device determines that the playback notification for the (i+1)th segment of the first media will be delayed based on the interruption duration, and the delay duration is equal to the interruption duration.

14. The method according to claim 13, characterized in that, The method further includes: When the transmission time of the (i+1)th segment is reached, the first device sends a playback notification of the (i+1)th segment to the host, so that the host controls the second device to present the (i+1)th state, the (i+1)th state corresponds to the (i+1)th segment, and the playback duration of the (i+1)th segment is the same as the duration of the (i+1)th state. The first device plays the (i+1)th segment.

15. The method according to any one of claims 12-14, characterized in that, Before the first device determines that the i-th segment of the first media is about to be played, the method further includes: The first device acquires the timing information of the first media, the timing information including the start time and end time of each of the n segments of the first media; The first device determines that the i-th segment of the first media is about to be played, including: The first device determines the i-th segment of the first media to be played based on the current playback progress of the first media and the timing information of the first media.

16. The method according to claim 15, characterized in that, The first device acquires timing information of the first media, including: The first device receives the timing information of the first media sent by the host.

17. The method according to any one of claims 12-16, characterized in that, The sending time of the playback notification for the i-th segment is the first moment, the playback start time of the i-th segment is the second moment, the first moment is earlier than the second moment, and the duration between the first moment and the second moment is the third duration, which is greater than or equal to the duration required for the host to control the second device to present the i-th state.

18. The method according to claim 17, characterized in that, The method further includes: The first device receives second information sent by the host, the second information being used to indicate the third duration; The first device sends a playback notification for the i-th segment to the host in advance, before the second time arrives, based on the third duration.

19. The method according to claim 17 or 18, characterized in that, The third duration includes the transmission delay between the host and the second device.

20. The method according to any one of claims 12-19, characterized in that, Before the first device sends a playback notification for the i-th segment to the host, the method further includes: The first device provides the host with subscription information for a first service, wherein the first service is a service that notifies the media playback progress. The first device determines that the host has subscribed to the first service.

21. The method according to any one of claims 12-20, characterized in that, The first device is an audio device.

22. The method according to any one of claims 12-21, characterized in that, The second device includes a lighting device, and the i-th state includes at least one of being on, off, having a brightness of the i-th brightness, having a color of the i-th color, and having a color temperature of the i-th color temperature; or, The second device includes a display device, and the i-th state includes at least one of the following: activating, deactivating, displaying i-th content, and having a brightness of i-th brightness.

23. A device control method, characterized in that, Applied to a system, the system including a first device, a host and a second device, the method includes: The host determines that the first device is about to play the first media; The host generates timing information of the first media based on the media stream of the first media. The timing information includes the start time and end time of each of the n segments of the first media, where n is a positive integer. The host sends the timing information to the first device; The first device determines the i-th segment of the first media to be played based on the timing information, wherein the i-th segment is one of the n segments, and i is a positive integer; The first device sends a playback notification for the i-th segment to the host; The first device plays the i-th segment; In response to receiving the playback notification of the i-th segment, the host controls the second device to present the i-th state, the i-th state corresponds to the i-th segment, and the playback duration of the i-th segment and the duration of the i-th state are both the first duration.

24. The method according to claim 23, characterized in that, The method further includes: At the end of the first duration, if the host determines that it has not received a playback notification for the (i+1)th segment sent by the first device, it controls the second device to maintain the i-th state or turn off the i-th state.

25. The method according to claim 23, characterized in that, The method further includes: When the playback of the i-th segment is interrupted, the first device determines the duration of the interruption of the i-th segment; The first device determines that the playback notification for the (i+1)th segment of the first media will be delayed based on the interruption duration, and the delay duration is equal to the interruption duration.

26. An electronic device, characterized in that, include: Processor, memory, and one or more programs; The one or more programs are stored in the memory, and the one or more programs include instructions that, when executed by the processor, cause the electronic device to perform the steps of the method as described in any one of claims 1-22.

27. A system, characterized in that, include: The host and the first device; The host is used to perform the method as described in any one of claims 1-11; The first device is used to perform the method as described in any one of claims 12-22.

28. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a computer program that, when run on a computer, causes the computer to perform the method as described in any one of claims 1 to 25.

29. A computer program product, characterized in that, Includes a computer program that, when run on a computer, causes the computer to perform the method as described in any one of claims 1 to 25.

Images