Device control method and device

Abstract

The invention provides an equipment control method and equipment. A first device sends a first request to a first application process, the first application process corresponding to a first application. The first device sends a second request to the first server in response to the first request, the first application being associated with the second application. The first device receives the second application from the first server. The first application process sends a third request to a second application process, the second application process corresponds to the second application, and the first application process and the second application process are different processes. And the second application process sends a fourth request to a server corresponding to the second application. And the second application process receives the first resource from the server corresponding to the second application. And the second application process sends the first resource to the first application process. The first device performs a first response based on the first resource. Therefore, the storage and operation space of the applications in the equipment is reduced, and the docking cost between the equipment and the server corresponding to the second application is also reduced.

Description

[0001] This application is a divisional application. The application number of the original application is 202010617807.5, the original application date is June 30, 2020, and the name of the invention is “Device Control Method and Device”. The entire contents of the original application are incorporated into this application by reference. Technical Field

[0002] The present application relates to the field of communication technology, and in particular to a device control method and device. Background Art

[0003] With the continuous development of the Internet of Things, hardware equipment manufacturers need to not only do a good job of hardware, but also consider software. Taking speakers as an example, not only do they need to do a good job of hardware such as Bluetooth and Wi-Fi network connection, but they also need to consider software such as speaker playback resources and language interaction. For hardware equipment manufacturers with poor software development capabilities, the development costs of both the server and the client are relatively high. If the software capabilities of other manufacturers are integrated, the docking and debugging processes between manufacturers will be more complicated, increasing costs. Summary of the invention

[0004] The present application provides a device control method and device, through the dependency relationship between the first application and the second application, the first device can install the second application, so that the first application can obtain the resources for the device to perform a response operation from the server corresponding to the second application through the second application. Therefore, not only the storage space and running space of the application installed in the first device are reduced, the traffic consumption of the application download application is reduced, but also the docking cost of the device manufacturer and the application manufacturer is reduced.

[0005] In a first aspect, the present application provides a device control method, the method comprising: a first device sends a first request to a first application process, wherein the first application process corresponds to a first application, the first application is installed in the first device, and the first request is used to request the first device to execute a resource of a first response; the first device sends a second request to a first server in response to the first request, wherein the second request is used to request to obtain a second application, and the first application is associated with the second application; the first device receives the second application from the first server; the first application process sends a third request to the second application process, wherein the second application process corresponds to the second application, the first application process and the second application process are different processes, and the third request is used to request the first device to execute a resource of the first response; the second application process sends a fourth request to a server corresponding to the second application, wherein the fourth request is used to request the first device to execute a resource of the first response; the second application process receives a first resource from a server corresponding to the second application, wherein the first resource is used for the first device to execute the first response; the second application process sends the first resource to the first application process; the first device executes the first response based on the first resource.

[0006] In one possible design, before the first device sends the second request to the first server, the method also includes: the first device determines that the second application is not installed in the first device based on the absence of an identifier of the second application in a package management service (Bundle Manager Service, BMS), wherein the BMS is used to store identifiers of applications installed in the first device.

[0007] In a possible design, after the first device receives the second application from the first server, the method further includes: the first device installing the second application in the first device and storing an identifier of the second application in the BMS.

[0008] In a possible design, a configuration file of the first application is stored in the first device, and the configuration file of the first application includes an identifier of the second application; the second request carries the identifier of the second application.

[0009] In a possible design, the method further includes: the first device sends a fifth request to the first server in response to the first request, wherein the fifth request is used to request to obtain the third application, and the first application is associated with the third application; the first device receives the third application from the first server; the first application process sends a sixth request to the third application process, wherein the third application process corresponds to the third application, the first application process and the third application process are different processes, and the sixth request is used to request the first device to execute the first response resource; the third application process sends a seventh request to the server corresponding to the third application, wherein the seventh request is used to request the first device to execute the first response resource; the third application process receives the second resource from the server corresponding to the third application, wherein the second resource is used for the first device to execute the first response; the third application process sends the second resource to the first application process; the first device executes the first response based on the first resource and the second resource. Thus, an application in the first device can be associated with multiple applications, and more resources can be obtained by calling the second application, thereby providing a rich resource channel for the first device to perform operations.

[0010] In one possible design, a first device receives a first instruction from a second device, and the first device is communicatively connected to the second device; and the first device sends a first request to a first application process in response to the first instruction.

[0011] In one possible design, the first device sends a first request to the first application process in response to a first operation of the user.

[0012] In one possible design, the method also includes: a display in the first device displays an interface of the first application, the interface of the first application includes a first area, and the interface displayed in the first area is provided by the second application.

[0013] In a possible design, the method further includes: the first device releasing resources of the second application, thereby saving memory overhead of the first device.

[0014] In a possible design, the method further includes: the first application process stops calling the second application process, so that the second application process can be called by other applications, so that the second application can be effectively used.

[0015] In a possible design, the method further includes: the first device sends an eighth request to the fourth application process, wherein the fourth application process corresponds to the fourth application, the fourth application is installed in the first device, the fourth application is associated with the second application, and the eighth request is used to request the first device to execute the second response resource; the fourth application process sends a ninth request to the second application process, wherein the fourth application process and the second application process are different processes, and the ninth request is used to request the first device to execute the second response resource; the second application process sends a tenth request to the server corresponding to the second application, wherein the tenth request is used to request the first device to execute the second response resource; the second application process receives a third resource from the server corresponding to the second application, wherein the third resource is used for the first device to execute the second response; the second application process sends the third resource to the fourth application process; the first device executes the second response based on the third resource. Thus, multiple applications can be installed in the first device at the same time, and different applications can call the second application respectively, thereby obtaining the response resources respectively through the second application, so that the first device can perform different operations.

[0016] In a possible design, the first device is a speaker, the first application is a speaker FA application, the second device is a music FA application, and the first server is a FA cloud server; the method is specifically: the speaker requests to call the speaker FA process, wherein the speaker FA process corresponds to the speaker FA application, and the speaker FA application is installed in the speaker; the speaker determines whether the speaker FA process is running; when the speaker FA process is not running, the speaker calls the speaker FA process and sends the target song identifier to the speaker FA process; when the speaker FA process is running, the speaker sends the target song identifier to the speaker FA process; the speaker FA process requests The speaker determines whether the identifier of the music FA application exists in the BMS to determine whether the bundle of the music FA application exists in the speaker, wherein the music FA process corresponds to the music FA application, and the BMS stores the identifier of the FA application associated with the speaker FA application; if the identifier of the music FA application does not exist in the BMS, the operating system of the speaker sends a request to download the music FA application to the FA cloud server, wherein the request to download the music FA application is used to request the bundle of the music FA application, and the request to download the music FA application carries the music FA application. the operating system of the speaker receives the bundle of the music FA application from the FA cloud server; the operating system of the speaker installs the music FA application in the speaker by unpacking the bundle of the music FA application, stores the identifier of the music FA application in the BMS, and continues to determine whether the identifier of the music FA application exists in the BMS until the identifier of the music FA application exists in the BMS; if the identifier of the music FA application exists in the BMS, the speaker FA process calls the music FA process; the speaker FA process sends a first music resource request to the music FA process , wherein the first music resource request is used to request the music resource of the target song, and the first music resource request carries the identifier of the target song; the music FA process sends a second music resource request to the server corresponding to the music FA application, wherein the second music resource request is used to request the music resource of the target song, and the second music resource request carries the identifier of the target song; the music FA process receives the music resource of the target song from the server corresponding to the music FA application; the music FA process sends the music resource of the target song to the speaker FA process; the speaker plays the target song based on the music resource of the target song.

[0017] In a second aspect, the present application provides a first device, comprising: at least one memory and at least one processor; the memory is used to store program instructions; the processor is used to call the program instructions in the memory so that the electronic device executes the device control method in the first aspect and any possible design of the first aspect.

[0018] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor in an electronic device, implements the device control method in the first aspect and any possible design of the first aspect.

[0019] In a fourth aspect, the present application provides a computer program product, comprising: execution instructions, the execution instructions are stored in a readable storage medium, at least one processor of an electronic device can read the execution instructions from the readable storage medium, and at least one processor executes the execution instructions so that the electronic device implements the device control method in the first aspect and any possible design of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A schematic diagram of the architecture of a communication system provided in one embodiment of the present application;

[0021] Figure 2 A functional block diagram of a speaker provided in one embodiment of the present application;

[0022] Figure 3 A functional block diagram of a speaker provided in one embodiment of the present application;

[0023] Figure 4 A software structure block diagram of a speaker provided in one embodiment of the present application;

[0024] Figure 5A-Figure 5B A system architecture diagram of a device control method provided in one embodiment of the present application;

[0025] Figure 5C A signaling flow chart of a device control method provided in an embodiment of the present application;

[0026] Fig. 6A A schematic diagram of the process of integrating a music manufacturer's SDK into a speaker APP entity in the related art;

[0027] Figure 6B A schematic diagram of a flow chart of a virtual integrated music FA of a speaker FA provided in an embodiment of the present application;

[0028] Figure 7 A schematic diagram of a scenario of a device control method provided in an embodiment of the present application;

[0029] Figure 8A-8DA schematic diagram of a human-computer interaction interface provided in one embodiment of the present application;

[0030] Fig. 9 A schematic diagram of a scenario of a device control method provided in an embodiment of the present application;

[0031] Figures 10A-10D A schematic diagram of a human-computer interaction interface provided in one embodiment of the present application;

[0032] Fig.11 A schematic diagram of a scenario of a device control method provided in an embodiment of the present application;

[0033] Figure 12A-12B A schematic diagram of a human-computer interaction interface provided in one embodiment of the present application;

[0034] Figures 13A-13H A schematic diagram of a human-computer interaction interface provided in one embodiment of the present application;

[0035] Fig.14 A schematic diagram of a scenario software architecture provided for an embodiment of the present application;

[0036] Fig.15 A schematic diagram of a scenario software architecture provided for an embodiment of the present application;

[0037] Figure 16A-16B A schematic diagram of a human-computer interaction interface provided in one embodiment of the present application;

[0038] Fig.17 A schematic diagram of a scenario software architecture provided for an embodiment of the present application;

[0039] Figure 18A-18B A schematic diagram of a human-computer interaction interface provided in one embodiment of the present application;

[0040] Fig.19 A schematic diagram of a scenario software architecture provided for an embodiment of the present application. DETAILED DESCRIPTION

[0041] Figure 1 FIG. 1 is a schematic diagram showing the architecture of a communication system provided by an embodiment of the present application. Figure 1 As shown, the communication system may include: a first device 101 and a second server 102. The first device 101 may request service resources from the second server 102, and the second server 102 is used to provide service resources to the first device 101, so that the first device 101 provides corresponding services to the user.

[0042] The first device 101 and the second server 102 usually belong to different manufacturers. The first device 101 may include: speakers, mobile phones, tablet computers, wearable devices, vehicle-mounted devices, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPC), netbooks, personal digital assistants (PDA), TVs, smart screens, high-definition TVs, 4K TVs, projectors and other Internet of Things (IOT) devices. This application does not limit the specific type of the first device 101. For ease of explanation, this application takes the first device 101 as a speaker as an example, and the second server 102 may be a server that provides music resources to the speaker.

[0043] In some embodiments, the combination Figure 1 The communication system may further include: a third server 103. The speaker is connected to the third server 103, and the third server 103 is used to provide the speaker with conversion services such as voice and resource storage services, etc. The third server 103 and the speaker usually belong to the same manufacturer.

[0044] For example, when the speaker receives instruction 1 from the user in the form of voice operation, input operation, etc., the speaker can send instruction 1 to the third server 103, and the third server 103 converts instruction 1 into instruction 2 that the speaker can recognize, and the third server 103 sends instruction 2 to the speaker, so that the speaker performs a corresponding operation based on instruction 2. Of course, a conversion module such as one for voice recognition can also be installed inside the speaker. The speaker can use the conversion module to convert instruction 1 into instruction 2 that the speaker can recognize, so that the speaker performs a corresponding operation based on instruction 2.

[0045] In other embodiments, continue to combine Figure 1 The communication system may further include: a terminal 104. The terminal 104 may be loaded with an application (APP) capable of communicating with the speaker. The terminal 104 may communicate with the speaker directly or through the third server 103. The terminal 104 and the speaker may be connected by wire or wirelessly. For example, Wi-Fi direct connection, Bluetooth connection or ZigBee connection, etc., which is not limited in this application.

[0046] For example, when the user uses the terminal 104 to send an instruction 1 in the form of a voice operation, an input operation, etc. to the speaker, the terminal 104 can send the instruction 1 to the third server 103, and the third server 103 converts the instruction 1 into an instruction 2 that can be recognized by the speaker, and the third server 103 sends the instruction 2 to the speaker, so that the speaker performs a corresponding operation based on the instruction 2. For another example, the terminal 104 can send the instruction 1 to the speaker, and the speaker obtains the instruction 2 through its own conversion module or the third server 103, so that the speaker performs a corresponding operation based on the instruction 2. The specific implementation method of the music acquisition instruction 2 involved here can be found in the previous description, and will not be repeated here.

[0047] Figure 2 FIG. 1 shows a functional block diagram of a speaker provided by an embodiment of the present application. Figure 2 As shown, the speaker 200 may include one or more input devices 201, one or more output devices 202, and one or more processors 203. The input device 202 may detect various types of input signals (referred to as input), and the output device 202 may provide various types of output information (referred to as output). The processor 203 may receive input signals from one or more input devices 201, generate output information in response to the input signals, and output the information through one or more output devices 202.

[0048] In some embodiments, one or more input devices 201 can detect various types of inputs and provide signals corresponding to the detected inputs (e.g., input signals), and then one or more input devices 201 can provide the input signals to one or more processors 203. In some examples, one or more input devices 201 can include any component or assembly capable of detecting an input signal. For example, the input device 201 can include an audio sensor (e.g., one or more microphones), a distance sensor, an optical or visual sensor (e.g., a camera, a visible light sensor or an invisible light sensor), a proximity light sensor, a touch sensor, a pressure sensor, a mechanical device (e.g., a crown, a switch, a button or a key, etc.), a temperature sensor, a communication device (e.g., a wired or wireless communication device), etc., or the input device 201 can also be some combination of the above various components.

[0049] In some embodiments, one or more output devices 202 can provide various types of outputs. For example, one or more output devices 202 can receive one or more signals (e.g., output signals provided by one or more processors 203) and provide outputs corresponding to the signals. In some examples, output device 202 may include any suitable components or components for providing outputs. For example, output device 202 may include an audio output device (e.g., one or more speakers), a visual output device (e.g., one or more lights or displays), a tactile output device, a communication device (e.g., a wired or wireless communication device), etc., or the output device 202 may also be some combination of the above various components.

[0050] In some embodiments, one or more processors 203 may be coupled to the input device 201 and the output device 202. The processor 203 may communicate with the input device 201 and the output device 202. For example, the one or more processors 203 may receive an input signal (e.g., an input signal corresponding to an input detected by the input device 201) from the input device 201. The one or more processors 203 may parse the received input signal to determine whether to provide one or more corresponding outputs in response to the input signal. If so, the one or more processors 203 may send an output signal to the output device 202 to provide an output.

[0051] Understandably, Figure 2 The device structure shown in the figure does not constitute a specific limitation on the speaker. In other embodiments, the speaker may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0052] Figure 3 FIG. 1 shows a functional block diagram of a speaker provided by an embodiment of the present application. Figure 3 As shown, the sound box 300 may include: a processor 301, a memory 302, a communication interface 303, a microphone 304, a speaker 305, etc. These components may communicate via one or more communication buses or signal lines (not shown in the figure).

[0053] Combine the following Figure 3 The various components of the speaker 300 are introduced in detail:

[0054] The processor 301 is the control center of the speaker 300 , and uses various interfaces and lines to connect various parts of the speaker 300 . It executes various functions of the speaker 300 and processes data by running or executing applications stored in the memory 302 and calling data stored in the memory 302 .

[0055] In some embodiments, the processor 301 may include one or more processing units, for example, the processor 301 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). Among them, different processing units may be independent devices or integrated into one or more processors. Among them, the controller may be the nerve center and command center of the speaker 300. The controller may generate an operation control signal according to the instruction opcode and the timing signal to complete the control of fetching and executing instructions. In other embodiments, a memory may also be provided in the processor 301 for storing instructions and data. In some embodiments, the memory in the processor 301 is a high-speed cache memory. The memory may store instructions or data that the processor 301 has just used or circulated. If the processor 301 needs to use the instruction or data again, it may be directly called from the memory, avoiding repeated access, reducing the waiting time of the processor 301, and thus improving the efficiency of the system. The processor 301 can run the software code / module of the device control method provided in some embodiments of the present application to realize the function of controlling the speaker 300.

[0056] The memory 302 is used to store applications and data. The processor 301 executes various functions and data processing of the speaker 300 by running the applications and data stored in the memory 302. The memory 302 mainly includes a program storage area and a data storage area, wherein the program storage area can store an operating system (OS) and an application required for at least one function (such as a sound playback function, a voice collection function, etc.); the data storage area can store data created when using the speaker (such as audio data, etc.). In addition, the memory 302 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as a disk storage device, a flash memory device or other volatile solid-state storage device. In some embodiments, the memory 302 may store information such as a "wake-up word". In other embodiments, the memory 302 may also store audio information (such as songs, crosstalk, storytelling, etc.). In addition, the memory 302 may store various operating systems. The above-mentioned memory 302 may be independent and connected to the processor 301 through the above-mentioned communication bus; the memory 302 may also be integrated with the processor 301.

[0057] The communication interface 303 may be a wired interface (eg, an Ethernet interface) or a wireless interface (eg, a cellular network interface or a wireless local area network interface). For example, the communication interface 303 may be specifically used to communicate with Figure 1 The second server 102 is used to communicate with other devices such as mobile phones and televisions, and is also used to communicate with Figure 1 Communicate with the third server 103 in.

[0058] In some embodiments, the communication interface 303 may also provide an audio circuit between the user and the speaker. On the one hand, the audio circuit may convert the received audio signal into an audio electrical signal and transmit it to the speaker 305, which converts it into a sound signal for output. On the other hand, the microphone 304 converts the collected sound signal (such as the voice of the user) into an electrical signal, which is received by the audio circuit and converted into audio data (or voice data), and then the audio data is output, for example, the speaker sends the audio data to the Figure 1 The third server 103 or Figure 1 or outputs the voice data to the memory 302 for further processing.

[0059] Microphone 304, also called "microphone" or "microphone", is used to collect sound signals (such as the sound emitted by the user) and convert the sound signals into electrical signals. In some embodiments, one or more microphones 304, such as a microphone array, may be provided on the speaker 300. In other embodiments, in addition to collecting sound signals, microphone 304 may also realize the function of noise reduction of sound signals, or may also identify the source of sound signals, realize directional recording functions, etc.

[0060] The speaker 305 , also called a “speaker”, is used to convert the audio electrical signal into a sound signal. The speaker 300 can play sound signals such as music through the speaker 305 .

[0061] In some embodiments, the microphone 304 and the speaker 305 are coupled to the processor 301. For example, after receiving a sound signal, the microphone 304 sends the sound signal or an audio electrical signal converted from the sound signal to the processor 301. The processor 301 determines whether to respond to the sound signal or the audio electrical signal, and if so, outputs a corresponding output signal, such as playing music through the speaker 305.

[0062] In addition, the speaker 300 may include a display 306 (or display screen), or may not include the display 306. The display 306 may be used to display the display interface of the application, such as the currently playing song, etc. The display 306 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active matrix organic light-emitting diode or an active matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light emitting diode (QLED), etc. In some embodiments, a touch sensor may be provided in the display 306 to form a touch screen, which is not limited in the present application. The touch sensor is used to detect a touch operation acting on or near it. The touch sensor may transmit the detected touch operation to the processor 301 to determine the type of touch event. The processor 301 may provide a visual output related to the touch operation through the display 306.

[0063] In addition, the speaker 300 may also include a power supply device 307 (such as a battery and a power management chip) for supplying power to each component. The battery can be logically connected to the processor 301 through the power management chip, so that the power supply device 307 can manage functions such as charging, discharging, and power consumption management.

[0064] In addition, the speaker 300 may further include a sensor module 308, which may include an air pressure sensor, a temperature sensor, etc. In practical applications, the speaker 300 may further include more or fewer sensors, or use other sensors with the same or similar functions to replace the above-listed sensors, etc., which is not limited in this application.

[0065] Air pressure sensor, used to measure air pressure. In some embodiments, the processor 301 may be coupled to the air pressure sensor, and the air pressure value measured by the air pressure sensor may be used to assist in calculations, such as calculating the attenuation coefficient of sound.

[0066] Temperature sensor, used to detect temperature. In some embodiments, the processor 301 may be coupled to the temperature sensor, and the temperature value measured by the temperature sensor may be used to assist in calculations, such as calculating the attenuation coefficient of sound.

[0067] Understandably, Figure 3 The device structure shown in the figure does not constitute a specific limitation on the speaker. In other embodiments, the speaker may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0068] The software system of the speaker can adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present invention takes the Android system of the layered architecture as an example to illustrate the software structure of the speaker.

[0069] Figure 4 A software structure block diagram of a speaker provided in an embodiment of the present application is shown.

[0070] The layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the Android runtime and system library, and the kernel layer.

[0071] The application layer can include a series of application packages.

[0072] like Figure 4As shown, the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, game, shopping, travel, instant communication (such as short message), smart home, device control, etc.

[0073] Among them, smart home applications can be used to control or manage home devices with networking functions. For example, home devices can include electric lights, televisions, and air conditioners. For another example, home devices can also include anti-theft door locks, speakers, sweeping robots, sockets, body fat scales, desk lamps, air purifiers, refrigerators, washing machines, water heaters, microwave ovens, rice cookers, curtains, fans, televisions, set-top boxes, doors and windows, etc. Device control applications are used to control or manage a single device.

[0074] In addition, the application package may also include: main screen (i.e. desktop), negative one screen, control center, notification center, card application, card service application and other system applications.

[0075] Among them, the negative one screen, also known as the "-1 screen", refers to sliding the screen to the right on the main screen of the electronic device until sliding to the user interface (UI) of the leftmost split screen. For example, the negative one screen can be used to place some quick service functions and notification messages, such as global search, quick entry to a certain page of an application (payment code, WeChat, etc.), instant messages and reminders (express information, expenditure information, commuting conditions, taxi travel information, schedule information, etc.) and attention dynamics (football stands, basketball stands, stock information, etc.). The control center is the pull-down message notification bar of the electronic device, that is, the user interface displayed by the electronic device when the user performs a downward operation on the electronic device. The notification center is the pull-up message notification bar of the electronic device, that is, the user interface displayed by the electronic device when the user performs an upward operation on the electronic device. The card service application is used to provide card services to support the UI display of cards, the sharing of cards between various applications in the same device, and the sharing of cards across devices.

[0076] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0077] like Figure 4 As shown, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like.

[0078] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0079] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0080] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0081] The phone manager is used to provide communication functions for the speaker, such as the management of call status (including answering, hanging up, etc.).

[0082] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0083] The notification manager allows applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is displayed in the status bar, a prompt sound is emitted, the speaker vibrates, the indicator light flashes, etc.

[0084] Android Runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system.

[0085] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0086] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0087] The system library may include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0088] The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications.

[0089] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0090] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0091] A 2D graphics engine is a drawing engine for 2D drawings.

[0092] The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver, camera driver, audio driver, and sensor driver.

[0093] The following is an example of the workflow of the speaker software and hardware, combined with the capture and photo scene.

[0094] When the touch sensor 180K receives a touch operation, the corresponding hardware interrupt is sent to the kernel layer. The kernel layer processes the touch operation into a raw input event (including touch coordinates, timestamp of the touch operation, and other information). The raw input event is stored in the kernel layer. The application framework layer obtains the raw input event from the kernel layer and identifies the control corresponding to the input event. For example, if the touch operation is a touch single-click operation and the control corresponding to the single-click operation is the control of the camera application icon, the camera application calls the interface of the application framework layer to start the camera application, and then starts the camera driver by calling the kernel layer to capture static images or videos through the camera 193.

[0095] Below, some terms in this application are explained to facilitate understanding by those skilled in the art.

[0096] 1. Atomic ability (AA): Developed by developers, it is a program entity that implements a single function without a user interface (UI). AA has a dependency relationship with system services, but not with each other. System services may include, but are not limited to: clock service (RTC), connection manager (WIFI, GSM modem), positioning (GPS), power supply (PMIC), sensor, communication (GSM modem, WIFI, Bluetooth).

[0097] 2. Feature ability (FA): It is a program entity that implements user feature functions by calling AA and / or FA.

[0098] 3. Business FA (also known as Parent FA), and service FA (also known as Child FA or service AA (also known as Child AA): both are program entities.

[0099] Business FA can virtually integrate service FA and / or service AA, so that data communication can be realized between business FA and service FA and / or service AA. Among them, business FA adopts a dual-thread model, namely, rendering thread (Render thread) and transaction logic processing thread (Worker thread). Render thread is mainly responsible for the rendering function of UI, and Worker thread is mainly responsible for the data business logic processing and data request communication displayed or operated in the UI.

[0100] 4. Virtual integration (VI): When a developer develops a service FA, he needs to register the service FA and / or service AA that the service FA needs to rely on in the virtual integration management module configuration file (referred to as the configuration file) of the service FA. For example, the identifier (such as the package name) of the storage service FA.

[0101] When the service FA needs to be run on the device, the first device searches the package management service of the device through the configuration file to see whether there is a running service FA and / or service AA. If not, the first device can download and start the service FA and / or service AA from the FA cloud server (such as the FA application market). Among them, the FA cloud server mentioned in this application is the first server.

[0102] 5. Cards and search boxes are both ways for business FA and service FA to present UI. Cards can display information and have a jump function. For example, when a user clicks on song A in the player card, the user can jump to the interface that displays the lyrics of song A. Cards can also display information and have business functions triggered by click events. For example, when a user clicks on the next song operation button in the player card, the song switching function can be realized. In addition, cards can be developed by developers themselves, or existing cards can be downloaded from the FA application server.

[0103] 6. Ability manager service (AMS): Mainly responsible for the operation and management of ability (i.e. FA and / or AA), which may include the management of ability instance stack, scheduling of ability life cycle, and management interaction with window manager.

[0104] 7. Bundle manager service (BMS): Mainly responsible for the installation, uninstallation, optimization and query management of various resource packages (such as bundles).

[0105] 8. Process: refers to an independent memory working space allocated by the operating system to each application.

[0106] Next, combine Figures 5A-10C , an example is given of the specific implementation process of the speaker playing songs through the speaker FA.

[0107] Figure 5A-Figure 5B A system architecture diagram of a device control method provided in an embodiment of the present application is shown. Figure 5C A signaling flow chart of a device control method provided in an embodiment of the present application is shown.

[0108] like Figure 5A and Figure 5B As shown, the music manufacturer develops music FA (music FA as a service FA and / or service AA) based on the defined FA / AA development specification (or interface specification), and uploads the music FA to the FA cloud server 505 (corresponding to Figure 5A Step 1) in the above code.

[0109] Among them, this application does not limit the specific implementation of the FA / AA development specification. In addition, music FA is uploaded to Figure 5A The information of the FA cloud server 505 may include but is not limited to: the identification of the music FA (such as the package name), the FA / AA version number, whether it can be virtually integrated, the authorization method of virtual integration, the user permissions required for AA, API and functional interface, whether data synchronization linkage is supported and the data synchronization linkage interface, and whether a fee is required, etc. The data synchronization linkage here means that the data corresponding to the speaker FA (such as page logic data, page display data, picture resource data, etc.) changes with the change of the data corresponding to the music FA, and the data corresponding to the music FA changes with the change of the data corresponding to the speaker FA, that is, the data corresponding to the speaker FA is synchronized with the data corresponding to the music FA.

[0110] It should be noted that the music FA involved in this application is the music FA application.

[0111] The speaker manufacturer negotiates with the music manufacturer and reaches a business cooperation. Based on the music FA uploaded to the FA cloud server 505, the speaker manufacturer virtually integrates the music FA into the speaker FA (the speaker FA is the business FA.)

[0112] It should be noted that the speaker FA involved in the present application is the speaker FA application.

[0113] The specific implementation process of the virtual integration of the speaker FA with the music FA can be understood as follows: the developer registers the music FA that the speaker FA needs to depend on in the configuration file of the speaker FA, such as storing the identifier of the music FA (such as the package name). That is, the developer pre-configures the dependency relationship between the speaker FA and the music FA, rather than integrating the entity of the music FA into the speaker FA.

[0114] Next, combine Fig. 6A and Figure 6B , the specific implementation process of the speaker APP physical integrated music manufacturer's SDK in the related technology and the speaker FA virtual integrated music FA in this application is introduced.

[0115] like Fig. 6A As shown, in the related art, the specific implementation process from the creation to the operation of the speaker APP may include: step 11-step 15.

[0116] Step 11. During the integrated development environment (IDE) development phase, download the music manufacturer's SDK from the music manufacturer's SDK server (such as the music manufacturer's SDK official website), and write the APP business code and SDK dependency configuration file of the speaker APP.

[0117] Step 12: During the IDE compilation and packaging stage, compile and package the APP business code, SDK dependency configuration file, and the music manufacturer's SDK to obtain the speaker APP.

[0118] Step 13: In the application listing stage, the speaker APP is listed on the speaker manufacturer's APP cloud server (such as the APP application market). In addition, the speaker manufacturer's APP application market can also include: chat APP ( Fig. 6A Wechat APP is used as an example) and payment APP ( Fig. 6A Take alipay APP as an example) etc.

[0119] Step 14: During the application download and use phase, the speaker downloads the speaker APP from the speaker manufacturer's APP application market. In addition, the speaker can also download WeChat APP and Alipay APP from the speaker manufacturer's APP application market. It should be noted that downloading the speaker APP from the speaker APP does not depend on downloading WeChat APP and Alipay APP from the speaker APP. Downloading WeChat APP and Alipay APP from the speaker APP is used to indicate that the speaker can also download other APPs besides the speaker APP.

[0120] Step 15. From the process perspective of the speaker APP running stage, the APP business code, SDK dependency configuration file and the music manufacturer's SDK are all running in the speaker APP process.

[0121] In summary, in the relevant technology, speaker manufacturers need to independently develop their own speaker APPs, and need to physically integrate the music manufacturer's SDK, and install the speaker APP in the speaker, which results in a larger data packet of the speaker APP, occupying more storage and running space. Since the music manufacturer's SDK and the speaker APP process belong to the same process, the music manufacturer's SDK will occupy storage space regardless of whether it is used or not. In addition, the music manufacturer's SDK has many functional interfaces, and the docking process and time cost are high. In addition, the music manufacturer's SDK cannot run independently.

[0122] like Figure 6B As shown, in the related art, the specific implementation process from creation to operation of the speaker FA may include: step 21-step 25.

[0123] Step 21, in the IDE development stage, download the music FA from the FA cloud server 505 (such as the FA application market), and write the FA business code and AA / FA dependency configuration file of the speaker FA, so that the developer of the speaker FA can preview the interface displayed by the speaker FA, know and confirm the specific functions implemented by the music FA in the music FA, etc., by debugging the various interfaces and functions of the music FA. Among them, the identifier of the music FA is registered in the AA / FA dependency configuration file.

[0124] Step 22: In the IDE compilation and packaging stage, the FA business code and AA / FA dependency configuration files of the music FA are compiled and packaged to obtain the speaker FA.

[0125] Step 23: In the FA listing phase, the speaker FA is listed on the FA application market. In addition, the FA application market may also include: chat FA ( Figure 6B Wechat FA is used as an example), payment FA ( Figure 6B Alipay FA is used as an example) and scanning FA ( Figure 6B (Scan FA is used as an example for illustration) etc.

[0126] Step 24, during the FA download and use phase, the speaker downloads the speaker FA from the FA application market. In addition, the speaker can also download wechat FA, alipay FA and scan FA from the FA application market. It should be noted that downloading the speaker FA from the speaker APP does not depend on downloading wechat FA, alipay FA and scan FA from the speaker APP. Downloading wechat FA, alipay FA and scan FA from the speaker APP is used to indicate that the speaker can also download other FAs besides the speaker FA.

[0127] Step 25, from the process perspective of the FA running stage, the FA business code and AA / FA dependency configuration file of the speaker FA are both running in the speaker FA process, the music FA is running in the music FA process, and the speaker FA process and the music FA process are two different processes. Among them, if the speaker does not contain the music FA, the speaker FA depends on downloading the music FA from the FA application market based on the identifier of the music FA registered in the AA / FA dependency configuration file.

[0128] In summary, in this application, the speaker manufacturer stores the identifier of the music FA in the speaker FA based on the uploaded music FA, realizes the virtual integration of the speaker FA with the music FA, and installs the speaker FA in the speaker, so that the data packet of the speaker FA is smaller, occupies less storage and running space, and reduces the traffic consumption when the user downloads the speaker FA.

[0129] When the speaker FA requests music resources from a music manufacturer, the speaker FA can obtain the music resources (such as song resources) from the music manufacturer's server (i.e., the second server 502) through the virtually integrated music FA, so that the speaker can play songs based on the music resources. The data synchronization linkage between the speaker FA and the music FA is also guaranteed, and the data synchronization linkage between the speaker FA and the music FA is much lower than the development and docking cost of the music manufacturer's SDK.

[0130] Since the speaker FA process and the music FA process are two different processes, the music FA will occupy storage space when the speaker FA needs to call the music FA, and will not occupy storage space when the speaker FA does not need to call the music FA, which reduces the storage memory and running memory overhead when the speaker FA is running, and is conducive to improving the utilization rate of storage space.

[0131] After the music FA is completed, the operating system of the speaker can control and kill the music FA according to factors such as memory resources (corresponding to Figure 5BStep 7) in the figure, that is, not only can the operating status and other parameters of the music FA be monitored, and thus timely know whether the music FA can be called, but also the music FA process can be released when the music FA is not needed, and related resources such as the music FA downloaded from the FA application market can be released, thereby saving the memory overhead of the speaker.

[0132] In addition, the music FA can be run independently, and can also be virtually integrated in other FAs except the speaker FA, so that the speaker FA and other FAs can call the music FA respectively. It should be noted that the speaker FA and other FAs generally cannot call the music FA at the same time, and the order in which the speaker FA calls the music FA and other FAs call the music FA can be determined based on the calling strategy.

[0133] Based on the above description, the speaker manufacturer installs the virtual speaker FA integrated with the music FA in the speaker 501 (corresponding to Figure 5A Step 2) in the above description, so that the speaker 501 does not need to physically integrate the music manufacturer's SDK, but can virtually integrate the music FA through the speaker FA and realize the function of music playback.

[0134] Next, continue to combine Figure 5A-Figure 5B ,like Figure 5C As shown, the specific implementation process of the device control method of the present application may include: steps S101-S115.

[0135] S101, the speaker 501 responds to the user's input (such as calling the startAbility() function) to request to start the speaker FA process. The user's input is used to indicate that the user requests the speaker 501 to play the song Love Story. It should be noted that the meaning of start and call mentioned in this application is the same.

[0136] The speaker FA process corresponds to the speaker FA application, which is used to obtain music resources from the second server 502 through music FA, and to play songs through the speaker 501 based on the music resources.

[0137] In general, each song has a unique identifier, i.e., an identity document (ID), such as a song name or a song number. Therefore, in some embodiments, the user's input may carry the identifier of the song Love Story. The user's input of the present application may include multiple feasible implementations, so that the user may request the speaker 501 to play the song Love Story in multiple ways.

[0138] Below, combining three scenarios, the specific implementation process of a user requesting a speaker to play the song Love Story is given as an example.

[0139] In some embodiments, the terminal 504 is equipped with a speaker FA that communicates with the speaker 501. The terminal 504 can send the user's input to the speaker FA through the connection with the speaker FA, and can also send the user's input to the speaker FA through the connection with the third server 503, which is not limited in this application.

[0140] Thus, the speaker FA application can determine the user's needs in response to the user's input. The user's input can be represented by at least one operation such as clicking, sliding, and long pressing, or can be represented by voice, picture, and other inputs, which are not limited in this application.

[0141] Figure 7 A schematic diagram of a scenario of a device control method provided in an embodiment of the present application is shown. Figure 8A-8D A schematic diagram of a human-computer interaction interface provided by an embodiment of the present application is shown. Figure 7-8D In the figure, a mobile phone as the terminal and a Bluetooth connection between the terminal and a speaker are taken as an example for illustration.

[0142] like Figure 7 As shown, the user requests the speaker FA in the mobile phone to play the song Love Story, such as the mobile phone sends the identification of the song Love Story to the speaker. The icon of the speaker FA 801 can be located on the desktop of the mobile phone or in the notification center of the mobile phone. The present application does not limit the specific position, shape, size and other parameters of the icon of the speaker FA 801.

[0143] Take the speaker FA801 icon located on the mobile phone desktop as an example. The mobile phone display shows Fig. 8A The interface shown in FIG. 1 shows an icon of the speaker FA 801. The user can Fig. 8A Click the icon of speaker FA801 on the interface shown.

[0144] Among them, the desktop application responds to the click operation and calls the startAbility() function to start the speaker FA process. The specific process may include: step 31-step 37.

[0145] Step 31, the Launcher process initiates a startAbility request to the system_server process through IPC communication.

[0146] Step 32: After receiving the request, the system_server process sends a create process request to the zygote process.

[0147] Step 33, the zygote process forks a new child process, namely the speaker FA process.

[0148] Step 34, the speaker FA process initiates an attachApplication request to the system_server process.

[0149] Step 35, after receiving the request, the system_server process performs a series of preparations and then sends a startAbility request to the speaker FA process through IPC communication.

[0150] Step 36: After receiving the request, the speaker FA process sends a startup message to the main thread in the speaker FA process.

[0151] Step 37. After receiving the startup message, the main thread in the speaker FA process creates the target ability and calls back the lifecycle method of the ability.

[0152] At this point, the speaker FA process is officially started, that is, the speaker FA 801 is running. At this time, in some embodiments, the display of the mobile phone changes from Fig. 8A The interface shown changes to Figure 8B The interface of the speaker FA shown is used to display the speaker FA.

[0153] The interface of the speaker FA can use the interface of the music FA, or can use the interface developed by the developer of the speaker FA, or can use the above two implementation methods, which is not limited in this application. Figure 8B In the example, the interface of speaker FA displays the name of speaker FA “AI Huawei Speaker”, a search box 802, card A 803 and card B 804.

[0154] The search box 802 is used to receive text, pictures, voice, etc. input by the user and display the corresponding search results using a self-developed interface. For example, Figure 8B In the search box 802, a search icon, the word "search" and an object for indicating a search resource are displayed, such as the word "QQ Music and NetEase Cloud Music".

[0155] Card A803 is used to display the song rankings to the user using the ranking interface of QQ Music. For example, card A803 includes the QQ Music icon, the text "rankings", three song titles and the name of the singer of each song (not limited to the name of the singer of each song). Figure 8B song title and artist name shown).

[0156] Card B 804 is used to display the playing status to the user using the recently played interface of NetEase Cloud Music. For example, card B 804 includes the NetEase Cloud Music icon, the text "recently played", the song name Dangerously, the singer name of the song Dangerously Charlie Puth, the operation button of the previous song icon, the operation button of the play / pause icon and the operation button of the next song icon, the operation button of adding to favorites, etc.

[0157] Users in Figure 8B Click the search box 802 of the speaker FA on the interface shown in the figure and enter LoveStory in the search box 802. The display of the mobile phone will display the following: Figure 8C The interface shown in FIG. 8 includes a search box 802 drop-down menu for providing the user with search results of Love Story (such as the source and quality of Love Story). Figure 8C The interface shown displays search results such as QQ Music Love Story standard quality, QQ Music Love Story HQ high quality, NetEase Cloud Music Love Story standard quality, and NetEase Cloud Music Love Story higher quality.

[0158] Users can Figure 8C Clicking the search result "Love Story HQ High Quality" on the interface shown in the figure causes the mobile phone to send a request to the speaker to play the song "Love Story" from QQ Music HQ High Quality. Thus, the speaker can know the user's request for the speaker to play the song Love Story, and display the following Fig.8D shown.

[0159] In other embodiments, the speaker 501 includes a display 5011. The speaker FA is installed in the speaker 501, and the speaker FA can determine the user's needs in response to the user's input. The user's input can be represented by at least one operation such as clicking, sliding, and long pressing, or can be represented by input such as voice and pictures, which is not limited in this application.

[0160] Fig. 9 A schematic diagram of a scenario of a device control method provided in an embodiment of the present application is shown. Figures 10A-10D A schematic diagram of a human-computer interaction interface provided by an embodiment of the present application is shown.

[0161] like Fig. 9As shown, the user requests the speaker to play the song Love Story through the speaker FA application in the speaker, and the speaker receives the identification of the song Love Story. Take the speaker FA1001 icon located above the speaker desktop as an example. The speaker display shows Fig. 10A The interface shown in FIG. 1 shows an icon of the speaker FA 1001. The user can Fig. 10A Click the icon of speaker FA 1001 on the interface shown.

[0162] The speaker display is as follows Fig. 10A The interface shown changes to Fig. 10B The interface of the speaker FA shown in the figure. The interface of the speaker FA on the display of the speaker may be the same as or different from the interface of the speaker FA on the display of the mobile phone, and this application does not limit this. For the sake of convenience, Fig. 10B The interface of the speaker FA shown is similar to Figure 8B The interface contents of the speakers FA shown are the same, but the interface layouts are different.

[0163] Users in Fig. 10B Click the search box of the speaker FA on the interface shown and enter LoveStory in the search box. The speaker display will show Fig. 10C The interface shown in FIG. 1 includes a search box drop-down menu for providing the user with search results of Love Story (such as the source and quality of Love Story). Fig. 10C The interface shown displays search results such as QQ Music Love Story standard quality, QQ Music Love Story HQ high quality, NetEase Cloud Music Love Story standard quality, and NetEase Cloud Music Love Story higher quality.

[0164] Users can Fig. 10C Clicking the search result of QQ Music Love Story HQ high quality on the interface shown in the figure enables the speaker to know the user's request for the speaker to play the HQ high quality song "Love Story" from QQ Music. Thus, the speaker can know the user's request for the speaker to play the song Love Story.

[0165] In other embodiments, the speaker 501 has a voice recognition function and can recognize the user's voice input to determine the user's needs. The present application does not limit the specific implementation of the voice input. For example, the voice input includes a keyword to wake up the speaker and a specific content requesting the speaker 501 to perform an operation.

[0166] Among them, this application does not limit the specific implementation method of the speaker 501 to realize voice recognition.

[0167] For example, the speaker 501 may integrate a conversion module for speech recognition in the speaker 501 , and the speaker 501 itself performs speech recognition on the user's speech input through the conversion module for speech recognition.

[0168] For another example, speaker FA in speaker 501 is connected to a third server 503. Speaker FA can send the user's voice input to the third server 503, and the third server 503 recognizes the voice input using voice recognition technology to obtain a voice recognition result, which is then transmitted to speaker FA.

[0169] Fig.11 FIG. 1 is a schematic diagram showing a scenario of a device control method provided by an embodiment of the present application. Fig.11 As shown, the user says to the speaker: "Xiaoyi Xiaoyi", so that the speaker is awakened. Among them, "Xiaoyi Xiaoyi" is the keyword for waking up the speaker. Generally, the speaker can be in a low power consumption state before being awakened. After the speaker is awakened, the speaker says: "I am here", so that the user confirms that the speaker has been awakened. The user then says to the speaker: "Play the song Love Story". Among them, "Play the song Love Story" is the specific content of the request for music to perform an operation, that is, notifying the speaker to perform the playback operation and notifying the speaker to play the song Love Story. Therefore, the speaker uses voice recognition technology to obtain the user's request for the speaker to play the song Love Story based on the user's voice input, such as the speaker recognizes the song Love Story.

[0170] In addition, when the speaker includes a display, the display of the speaker is displayed as follows Fig. 12A The interface shown, for example, displays the text "Play the song Love Story" on the interface, so that the speaker prompts the user of the voice recognition result of the speaker through the interface, allowing the user to confirm whether the voice recognition result is accurate and to make timely modifications.

[0171] It should be noted that the speaker 501 can obtain the user input in more than the above ways. In addition, the process of the speaker 501 application and other mobile phone applications starting the speaker FA process can be specifically referred to the description of the mobile phone desktop application starting the speaker FA process, which will not be repeated here.

[0172] S102, the operating system of the speaker 501 determines whether the speaker FA process is running.

[0173] Those skilled in the art will appreciate that when any process is running, the operating system of the device will register the identifier of the process in the process list. Therefore, the operating system of the speaker 501 can query whether there is an identifier of the speaker FA process in the process list to determine whether the speaker FA process is running.

[0174] If the speaker FA process is not running, then execute step S103. If the speaker FA process is running, then execute step S104.

[0175] S103, the speaker 501 calls the startAbility() function to start the speaker FA process and sends the song identifier to the speaker FA process.

[0176] In some embodiments, the specific implementation process of the speaker 501 starting the speaker FA process may include: step 41-step 47.

[0177] Step 41, the operating system process of the speaker 501 initiates a startAbility request to the system_server process through IPC communication;

[0178] Step 42, after receiving the request, the system_server process sends a request to create a process to the zygote process;

[0179] Step 43, the zygote process forks a new child process, namely the speaker FA process;

[0180] Step 44, the speaker FA process initiates an attachApplication request to the system_server process;

[0181] Step 45, after receiving the request, the system_server process performs a series of preparations and then sends a startAbility request to the speaker FA process through IPC communication;

[0182] Step 46, after receiving the request, the speaker FA process sends a start message to the main thread in the speaker FA process;

[0183] Step 47: After receiving the startup message, the main thread in the speaker FA process creates the target ability and calls back the lifecycle method of the ability.

[0184] At this point, the speaker FA process is officially started.

[0185] S104, the speaker 501 sends the song identifier to the speaker FA process.

[0186] S105, the speaker FA process calls the startAbility() function to request to start the music FA process.

[0187] It should be noted that the operating system of the speaker 501 will detect the running status of each process in real time. Therefore, the operating system of the speaker 501 can detect that the speaker FA process requests to start the music FA process (corresponding to Figure 5B Then, the operating system of the speaker 501 may execute step S106 of querying the identification of the music FA in the BMS.

[0188] S106, the operating system of the speaker 501 determines whether there is a music FA identifier in the BMS (corresponding to Figure 5B In step 2), the speaker 501 can query whether there is a bundle of music FA in the local speaker (such as the ROM, RAM, etc. of the speaker 501). When the service FA or service AA is installed in the speaker 501, it will be registered in the BMS, such as storing the identifier of the service FA or the identifier of the service AA. Therefore, the operating system can determine whether the music FA is installed in the speaker 501 through the BMS.

[0189] If the identification of the music FA does not exist in the BMS, then steps S107 to S109 are executed. If the identification of the music FA exists in the BMS, then steps S110 to S115 are executed.

[0190] S107, the operating system of the speaker 501 sends a music download FA request to the FA cloud server 505 (corresponding to Figure 5A Step 3 in , and corresponding to Figure 5B Step 3 in the above description). The music FA download request is used to instruct the FA cloud server 505 to send the music FA bundle to the speaker 501. In some embodiments, the music FA download request includes the identifier of the music FA.

[0191] S108, FA cloud server 505 sends music FA bundle (corresponding to Figure 5A Step 4 in , and corresponding to Figure 5B 4 in the previous step).

[0192] S109, the operating system of the speaker 501 unpacks the bundle of the music FA, installs the music FA in the local speaker, and registers the music FA in the BMS (such as storing the identifier of the music FA) (corresponding to Figure 5BStep 5 in the above), and continue to execute S106 until the identification of music FA exists in the BMS (i.e., the bundle of music FA exists locally in the speaker). Since the operating system of the speaker 501 may fail to install music FA, resulting in music FA not being registered in the BMS, the operating system of the speaker 501 needs to continue to execute S106 to determine whether music FA is installed in music 501.

[0193] S110, the speaker FA process calls the startAbility() function to start the music FA process (corresponding to Figure 5B Step 6 in the above). Since the bundle of the music FA process exists locally in the speaker 501, the speaker FA process can successfully start the music FA process. In addition, the operating system of the speaker 501 can also notify the speaker FA process that the bundle of the music FA exists locally in the speaker 501, so that the speaker FA process starts the music FA process.

[0194] In some embodiments, the specific implementation process of the speaker FA process starting the music FA process may include: steps 51 to 57.

[0195] Step 51, the speaker FA process initiates a startAbility request to the system_server process through IPC communication;

[0196] Step 52, after receiving the startAbility request, the system_server process sends a create process request to the zygote process;

[0197] Step 53: The zygote process forks a new child process, namely the music FA process.

[0198] Step 54, the music FA process initiates an attachApplication request to the system_server process;

[0199] Step 55, after receiving the attachApplication request, the system_server process performs a series of preparations and then sends a startAbility request to the music FA process through IPC communication;

[0200] Step 56, after receiving the request, the music FA process sends a start message to the main thread in the music FA process;

[0201] Step 57: After receiving the start message, the main thread in the music FA process creates the target ability and calls back the lifecycle method of the ability.

[0202] At this point, the music FA process officially starts.

[0203] S111, after the music FA process is started, the speaker FA process sends a first music resource request to the music FA process (corresponding to Figure 5A In step 5), the first music resource request is used to request the music resource of the song Love Story from the music FA. In some embodiments, the first music resource request includes an identifier of the song Love Story.

[0204] S112, the music FA process sends a second music resource request (corresponding to the second server 501) (i.e., the server corresponding to the music FA) to the second server 501. Figure 5A In step 6 of the above, the second music resource request is used to request the music resource of the song Love Story from the music FA. In some embodiments, the second music resource request includes an identifier of the song Love Story.

[0205] It should be noted that the essential contents of the first music resource request and the second music resource request are the same, both of which are used to request the music resource of the song Love Story from the music FA. The representation forms of the first music resource request and the second music resource request may be the same or different, and this application does not limit this.

[0206] S113, the second server 501 sends the music resource (corresponding to the music FA process) Figure 5A In step 7 of the method, in some embodiments, the music resource is a resource of a song corresponding to the identifier of the song.

[0207] S114, the music FA process sends the music resource (corresponding to the speaker FA process) Figure 5A 8 in the previous step).

[0208] S115, the speaker FA process plays the song Love Story based on the music resource.

[0209] Continue to combine Figure 7 In the scenario, while the speaker is playing the song Love Story, the speaker FA can return a song playing notification to the terminal. The song playing notification is used to inform the terminal that the speaker is playing the song Love Story and the playing progress and other information. As a result, the interface of the speaker FA on the display of the mobile phone is changed from Figure 8C As shown Fig.8D As shown. Among them, Figure 8C and Fig.8D In contrast, in the card B 804 of the speaker FA, Figure 8C The song title Dangerously changes to Fig.8DThe song title is Love Story. Figure 8C The singer's name in the song Dangerously was changed to Charlie Puth Fig.8D Taylor Swift Figure 8C The pause icon in the action button changes to Fig.8D The operation button of the play icon is removed, and the rest of the interface of the speaker FA remains unchanged.

[0210] Continue to combine Fig. 9 In the scenario where the speaker includes a display, the interface of the speaker FA on the speaker display changes from Fig. 10C As shown Fig. 10D As shown. Among them, Fig. 10C and Fig. 10D In contrast, in card B of speaker FA, Fig. 10C The song title Dangerously changes to Fig. 10D The song title is Love Story. Fig. 10C The singer's name in the song Dangerously was changed to Charlie Puth Fig. 10D Taylor Swift Fig. 10C The pause icon in the action button changes to Fig. 10D The operation button of the play icon is removed, and the rest of the interface of the speaker FA remains unchanged.

[0211] Continue to combine Fig.11 In the scenario where the speaker includes a display, the speaker display changes from Fig. 12A The interface shown changes to Fig. 12B The interface shown in the figure may include but is not limited to: the song title of the song Love Story, the singer name of the song Love Story Taylor Swift, the lyrics of the song, the operation button of the previous song icon, the operation button of the play / pause icon and the operation button of the next song icon, the operation button of the single loop icon, the operation button of the playlist icon, the operation button of the song playing progress bar, the song start time, the song end time, etc.

[0212] In summary, by means of the virtual integration of speaker FA and music FA (i.e., the dependency between speaker FA and music FA), speaker manufacturers can obtain music resources from the server corresponding to music FA and implement the function of song playback based on the music resources.

[0213] Based on the above description, the speaker FA may have no interface or may have an interface. If the speaker FA has an interface, the developer or user may use the interface of the music FA to create the speaker FA, or may develop a new interface to display the speaker FA, or may use the above two methods to create the speaker FA, and this application does not limit this.

[0214] Below, the interface of the speaker FA is Figure 8B Take the interface shown as an example, combined with Figures 13A-13H , giving examples of specific implementation methods for developers or users to create speaker FA on mobile phones.

[0215] Figures 13A-13H A schematic diagram of a human-computer interaction interface provided by an embodiment of the present application is shown. Fig.13A The mobile phone desktop interface is shown. When a developer or user slides right on the mobile phone desktop interface, the mobile phone display shows Fig. 13B The interface of the negative first screen of the mobile phone is shown in FIG. The developer or user clicks on the card of creating a small program on the negative first screen interface, and the display of the mobile phone displays the following Fig. 13C The interface shown in the figure. Developers or users Fig. 13C Click the card of the music applet on the interface shown, and the phone display will show Fig.13D The interface shown.

[0216] Among them, the interface displays the text "Mini Program Name" and an input box for the text "Please enter the mini program card name" prompt, which is convenient for developers or users to name the interface of the created speaker FA.

[0217] The interface also includes the words "Add music card", the words "All cards" and option 1301. The cards in option 1301 are used to display music FA to the user. The music FA here can be the music FA that the mobile phone has downloaded from the FA cloud server (such as the FA application market), that is, the downloaded music FA, or the music FA that the mobile phone can download from the FA cloud server (such as the FA application market), that is, the downloadable music FA. Usually, when the music FAs corresponding to different cards belong to the same music manufacturer, the different cards belonging to the same music manufacturer will be displayed together, such as arranged in the same rounded rectangular frame; when the music FAs corresponding to different cards belong to different music manufacturers, the different cards will be displayed separately, such as arranged in different rounded rectangular frames.

[0218] For example, there are two rounded rectangular boxes in option 1301, one of which includes card 1, card 2, card 3, and card 4, and the other rounded rectangular box includes card 5, card 6, and card 7.

[0219] Among them, card 1 displays the card name "QQ Music My Collection", the QQ Music icon, and an operation button 1302 for adding card 1. Card 2 displays the card name "QQ Music Recently Played", the QQ Music icon, and an operation button 1302 for adding card 2. Card 3 displays the card name "QQ Music Ranking", the QQ Music icon, and an operation button 1302 for adding card 3. Card 4 displays the card name "QQ Music Search Box", the QQ Music icon, and an operation button 1302 for adding card 4. Card 5 displays the card name "NetEase Cloud Music Today's Recommendation", the NetEase Cloud Music icon, and an operation button 1302 for adding card 5, and card 6 displays the card name "NetEase Cloud Music Recently Played", the NetEase Cloud Music icon, and an operation button 1302 for adding card 6. Card 7 displays the card name "NetEase Cloud Music Search Box", the NetEase Cloud Music icon, and an operation button 1302 for adding card 7. The operation buttons 1302 in each of Card 1, Card 2, Card 3, Card 4, Card 5, Card 6 and Card 7 are illustrated by taking an icon including a plus sign in a circle as an example.

[0220] In addition, the interface also displays an operation button 1303 for adding a new card, which is illustrated by taking an icon including a plus sign in a circle as an example. If there is a newly downloaded card, the newly downloaded card will be added to option 1301.

[0221] Developers or users can Fig.13D On the interface shown, select a downloaded card, such as at least one of Card 1, Card 2, Card 3, Card 4, Card 5, Card 6 and Card 7, or select a newly downloaded card from the FA cloud server, or select a downloaded card and a newly downloaded card from the FA cloud server.

[0222] If the developer or user selects Card 3, Card 4, Card 6, and Card 7, the phone display will show Fig.13E The interface shown in the figure. Fig.13D The interface shown is different in that the text "Cards Added" and option 1304 are newly added, the cards in option 1304 are all the cards selected by the developer or user (i.e. Card 3, Card 4, Card 6 and Card 7), and the cards in option 1301 have the selected cards removed (i.e. Card 3, Card 4, Card 6 and Card 7 are removed, and Card 1, Card 2 and Card 5 are retained).

[0223] Developers or users can Fig.13E Click the check mark icon on the interface shown, and the phone display will show Fig.13F The interface shown is used to provide a preview of the interface of the speaker FA to the developer or user.

[0224] For example, Fig.13E In the interface of the speaker FA shown, the search box is displayed using a self-developed interface, and the search results of the search box depend on the servers of the QQ Music manufacturer and the NetEase Cloud Music manufacturer. Card A corresponds to card 3, and card A is displayed using the QQ Music ranking interface. Card B corresponds to card 6, and card B is displayed using the recently played interface of NetEase Cloud Music.

[0225] Thus, the developer or user can timely understand the interface of the speaker FA, and also enable the developer or user to timely modify the interface of the speaker FA. For example, if the developer or user clicks the left arrow icon on the interface, the display of the mobile phone returns to the following. Fig.13E The interface shown makes it easy for developers or users to reselect cards.

[0226] Developers or users can Fig.13F Click the check mark icon on the interface shown, and the interface of the speaker FA can be displayed on the negative one screen of the mobile phone, such as Figure 13G The interface of the speaker FA shown is displayed after the payment method card. Figure 13G If the interface of the speaker FA shown cannot be fully displayed, the developer or user may Figure 13G Slide down on the interface shown in the figure, so that the interface of the speaker FA is as follows Fig.13H The display is shown in full on the phone.

[0227] In other embodiments, a developer or user can drag a first card embedded in a first application in a first device to a second application in the first device. The interfaces of the first card displayed by the first application and the second application can be the same or different, and this application does not limit this. For example, a user drags a speaker FA displayed in the negative first screen of a mobile phone to the desktop of the mobile phone, so that the desktop can also display the speaker FA. Thus, the speaker FA has a rich display channel.

[0228] In summary, developers can develop various types of speaker FA interfaces, providing users with a rich selection of interfaces. Users can also develop various types of speaker FA interfaces, realizing the development of personalized interfaces. In addition, when users need to use speaker FA, they can also download speaker FA from the FA cloud server (such as the FA application market) and install the speaker FA in the terminal or speaker. Fig. 8A The icon of the speaker FA801 shown or Fig. 10A The icon of the speaker FA1001 is shown.

[0229] Based on the above description, continue to combine Fig.13H, the following introduces the specific implementation process of the search results of the search box of speaker FA relying on the server of QQ music manufacturer and the server of NetEase Cloud Music manufacturer.

[0230] Fig.14 A schematic diagram of a scenario software architecture provided by an embodiment of the present application is shown. When the search box of the speaker FA has user input (such as Love Story), the speaker FA process sends a search request to the search box FA process of QQ Music and the search box FA process of NetEase Cloud Music based on the user input (corresponding to Fig.14 Step 1) in the above code.

[0231] Among them, the search box FA process of QQ Music corresponds to the search box FA of QQ Music. The search box FA process of NetEase Cloud Music corresponds to the search box FA of NetEase Cloud Music. And the search box FA of QQ Music is a service FA distributed by QQ Music manufacturers to the FA cloud server (such as the FA application market). The search box FA of NetEase Cloud Music is a service FA distributed by NetEase Cloud Music manufacturers to the FA cloud server (such as the FA application market).

[0232] The Worker thread in the FA process of the search box of QQ Music parses the search request and sends a data request to the server of the QQ Music manufacturer (corresponding to Fig.14 Step 21 in the above). The QQ Music manufacturer's server returns the search resource to the Worker thread in the FA process of the QQ Music search box (corresponding to Fig.14 Step 22 in the search box FA process of QQ Music. The Worker thread in the search box FA process of QQ Music performs logical processing on the search resources to obtain corresponding search results, such as QQ Music Love Story standard quality and QQ Music Love Story HQ high quality. The Worker thread in the search box FA process of QQ Music returns the search results to the speaker FA (corresponding to Fig.14 23 in step 23).

[0233] Similarly, the Worker thread in the FA process of the search box of NetEase Cloud Music parses the search request and sends a data request to the server of the NetEase Cloud Music manufacturer (corresponding to Fig.14 The server of the NetEase Cloud Music manufacturer returns the search resource to the Worker thread in the FA process of the search box of NetEase Cloud Music (corresponding to Fig.14 The Worker thread in the FA process of the search box of NetEase Cloud Music performs logical processing on the search resources to obtain corresponding search results, such as the standard quality of NetEase Cloud Music Love Story and the higher quality of NetEase Cloud Music Love Story. The Worker thread in the FA process of the search box of NetEase Cloud Music returns the search results to the speaker FA (corresponding to Fig.14 33 in step 33).

[0234] The Worker thread in the speaker FA process sends search data to the Render thread FA in the speaker FA process based on the search results returned by the QQ Music search box FA process and the search results returned by the NetEase Cloud Music search box FA process. The Render thread in the speaker FA process renders the search data so that the mobile phone display shows the following: Figure 8C The interface shown. Among them, this application does not limit the arrangement, position layout, display size and other parameters of the search results returned by different music manufacturers on this interface.

[0235] It should be noted that the mobile phone display shows Fig. 10C The specific implementation of the interface can be found in the display of the mobile phone. Figure 8C In addition, since the user input will cause the interface of the speaker FA to change without the interface of the speaker FA jumping, the search box FA of QQ Music and the search box FA of NetEase Cloud Music can be a service FA with UI, a service FA without UI, or a service AA without UI, and this application does not limit this.

[0236] In summary, the search box of speaker FA relies on the search resources provided by the servers of QQ Music and NetEase Cloud Music to display the search results. In addition to relying on the search resources provided by the servers of QQ Music and NetEase Cloud Music, the search box of speaker FA can also rely on the search resources provided by the servers of speaker manufacturers.

[0237] Based on the above description, the content of card B can change on the interface of speaker FA, such as the content of card B changes from Figure 8C Change to Fig.8D , or from Fig. 10C Change to Fig. 10D The following is the interface of the speaker FA. Fig.13H Taking the interface shown as an example, the specific implementation process of interface jump is introduced.

[0238] Fig.15 A schematic diagram of a scenario software architecture provided by an embodiment of the present application is shown. When the operation button on the card B of the speaker FA has a user input (such as a click operation), the speaker FA process sends an interface display request (corresponding to the recently played FA process of NetEase Cloud Music) based on the user input. Fig.15 Step 1) in the above code.

[0239] The Worker thread in the FA process of the recent play of NetEase Cloud Music sends a data request to the server of the NetEase Cloud Music manufacturer by parsing the interface display request (corresponding to Fig.15 NetEase Cloud Music's server returns the requested resource to the Worker thread in the recently played FA process of NetEase Cloud Music (corresponding to Fig.15 Step 3 in ). The Worker thread in the FA process of the recent play of NetEase Cloud Music will perform logical processing on the requested resources to obtain the corresponding display data. The Worker thread in the FA process of the recent play of NetEase Cloud Music returns the display data to the Worker thread in the FA process of the speaker (corresponding to Fig.15 Step 4 in the speaker FA process). The Worker thread in the speaker FA process returns the display data to the Render thread in the speaker FA process (corresponding to Fig.15 Step 5 in the figure). The Render thread in the speaker FA renders the display data so that the display of the mobile phone displays Fig.8D or Fig. 10D The interface shown.

[0240] It should be noted that, since the user's input requires the interface of the music FA to jump to the interface of the card B, the recently played FA of NetEase Cloud Music is a service FA with a UI.

[0241] Since the user's input will cause the interface of the speaker FA to change without the interface of the speaker FA jumping, the search box FA of QQ Music and the search box FA of NetEase Cloud Music can be a service FA with UI, a service FA without UI, or a service AA without UI. This application does not limit this.

[0242] It should be noted that since the user's input will cause the interface of the speaker FA to change without the need for the interface of the speaker FA to jump, the recently played FA of NetEase Cloud Music can be a service FA (that is, with UI) or a service AA (that is, without UI), and this application does not limit this.

[0243] In summary, the content of card B on the interface of speaker FA depends on the resources provided by the recently played FA of NetEase Cloud Music, which can realize the change of the content of card B on the interface of speaker FA.

[0244] Based on the above description, the interface of the speaker FA can jump to the interface of the card A. Fig.13H Taking the interface shown as an example, the specific implementation process of interface jump is introduced.

[0245] Figure 16A-16B A schematic diagram of a human-computer interaction interface provided by an embodiment of the present application is shown. Fig.16A Click on the blank area of ​​card A of speaker FA on the interface shown, and the display of the mobile phone jumps to the following Fig. 16B The interface of card A shown in FIG. 1 is used to provide users with music resources from the QQ music rankings. For example, in Fig. 16B The interface shown shows the words "This Week's Ranking", "Selected Weekly Hot Ranking", a play all operation button, the words "Total 40 Songs", and a song list (including the song title and singer name of each song), etc. Thus, the user can browse the songs in the QQ Music ranking.

[0246] Fig.17 A schematic diagram of a scenario software architecture provided by an embodiment of the present application is shown. When there is user input in the area other than the operation button on the card A of the speaker FA, the speaker FA process sends an interface jump request (corresponding to the ranking FA process of QQ Music) based on the user input. Fig.17 Step 1 in the above. The FA process of the QQ music ranking list corresponds to the FA of the QQ music ranking list. The FA of the QQ music ranking list is a service FA distributed by the QQ music manufacturer to the FA cloud server (such as the FA application market).

[0247] The Worker thread in the FA process of QQ Music's rankings parses the interface jump request and sends a data request to the server of the QQ Music manufacturer (corresponding to Fig.17 The server of the QQ music manufacturer returns the requested resource to the Worker thread in the FA process of the QQ music ranking list (corresponding to step 3 in FIG. 16 ). The Worker thread in the FA process of the QQ music ranking list performs logical processing on the requested resource to obtain the corresponding display data, such as Fig. 16B The Worker thread in the FA process of QQ Music's ranking returns the display data to the Render thread in the FA process of QQ Music's ranking (corresponding to Fig.17 Step 4 in the FA of the QQ music ranking list. The Render thread in the FA renders the display data, so that the display of the mobile phone jumps to the following Fig. 16B The interface of card A is shown.

[0248] It should be noted that, since the user's input requires the interface of the speaker FA to jump to the interface of the card A, the ranking list FA of QQ Music is a service FA with UI.

[0249] In summary, the interface of card A of speaker FA depends on the resources provided by QQ music's ranking list FA, and can realize the jump from the interface of speaker FA to the interface of card A.

[0250] Based on the above description, the interface of the speaker FA can jump to the interface of the card B. The following takes the interface of the speaker FA as an example. Fig.13H Taking the interface shown as an example, the specific implementation process of interface jump is introduced.

[0251] Figure 18A-18B A schematic diagram of a human-computer interaction interface provided by an embodiment of the present application is shown. Fig.18A Click the blank area of ​​card B of speaker FA on the interface shown, and the display of the mobile phone jumps to the following Fig.18B The interface of card B shown in FIG. 1 is used to provide users with recently played music resources from NetEase Cloud Music. For example, in Fig.18B The interface shown displays the song title of the song Love Story, the name of the singer of the song Love Story, the lyrics of the song Love Story, the operation button of the previous song icon, the operation button of the play / pause icon and the operation button of the next song icon, the operation button of the play mode, the operation button of the recently played list, etc. Thus, the user can browse the recently played songs of NetEase Cloud Music.

[0252] Fig.19 A schematic diagram of a scenario software architecture provided by an embodiment of the present application is shown. When there is user input in the area other than the operation button on the card B of the speaker FA, the speaker FA process sends an interface jump request (corresponding to the recently played FA process of NetEase Cloud Music) based on the user input. Fig.19 Step 1 in the above description). The recently played FA process of NetEase Cloud Music corresponds to the recently played FA of NetEase Cloud Music. The recently played FA of NetEase Cloud Music is a service FA distributed by the NetEase Cloud Music manufacturer to the FA cloud server (such as the FA application market).

[0253] The Worker thread in the FA process of the recent play of NetEase Cloud Music parses the interface jump request and sends a data request to the server of the NetEase Cloud Music manufacturer (corresponding to Fig.19 NetEase Cloud Music's server returns the requested resource to the Worker thread in the recently played FA process of NetEase Cloud Music (corresponding to Fig.19 Step 3 in ). The Worker thread in the recently played FA process of NetEase Cloud Music will request resources for logical processing and obtain the corresponding display data, such as Fig.18B The Worker thread in the FA process of the recent play of NetEase Cloud Music returns the display data to the Render thread in the FA process of the recent play of NetEase Cloud Music (corresponding to Fig.19 Step 4 in ). The Render thread in the recently played FA of NetEase Cloud Music renders the display data, causing the mobile phone display to jump to the following Fig.18B The interface of card B is shown.

[0254] It should be noted that, since the user's input requires the interface of the music FA to jump to the interface of the card B, the recently played FA of NetEase Cloud Music is a service FA with a UI.

[0255] In summary, the interface of card B of speaker FA relies on the resources provided by the recently played FA of NetEase Cloud Music, and can realize the jump from the interface of speaker FA to the interface of card B.

[0256] Based on some of the aforementioned embodiments, the device control method provided by the present application is introduced below.

[0257] Exemplarily, the present application provides a device control method.

[0258] The device control method of the present application may include: step S201 to step S208.

[0259] S201. A first device sends a first request to a first application process, wherein the first application process corresponds to a first application, the first application is installed in the first device, and the first request is used to request the first device to execute a first response resource.

[0260] In this application, step S201 can refer to the above Figure 5C The description of step S103 or S104 shown in the figure, the first device refers to the description above, the first application refers to the description of the speaker FA mentioned above, the first application refers to the description of the speaker FA process mentioned above, and the first application refers to the description of the speaker FA process mentioned above, which will not be repeated here. Among them, this application does not limit the specific implementation method of the first request. In some embodiments, the first request may include an identifier of the second application.

[0261] S202. The first device sends a second request to the first server in response to the first request, wherein the second request is used to request to obtain a second application, and the first application is associated with the second application.

[0262] In this application, step S202 can refer to the above Figure 5C For the description of step S107 shown, the second application refers to the description of the music FA process mentioned above, the second application refers to the description of the music FA process mentioned above, the second request refers to the description of the download music FA request mentioned above, the association between the first application and the second application can refer to the description of the dependency relationship between the speaker FA and the music FA or the virtual integration of the speaker FA with the music FA mentioned above, which will not be repeated here.

[0263] S203. The first device receives the second application from the first server.

[0264] In the present application, the first device receiving the second application from the first server can be understood as: the first device receiving the bundle of the second application, so that the first device installs the second application into the first device based on the bundle of the second application. Figure 5C The description of steps S108 and S109 shown in the figure is not repeated here.

[0265] S204. The first application process sends a third request to the second application process, wherein the second application process corresponds to the second application, the first application process and the second application process are different processes, and the third request is used to request the first device to execute resources of the first response.

[0266] In this application, step S204 can refer to the above Figure 5C The description of steps S110 and S111 shown in the figure, the third request refers to the description of the first music resource request mentioned above, and the second application process refers to the description of the music FA process mentioned above, which will not be repeated here. It should be noted that the substantive content of the second request and the third request are the same, both of which are used to request the first device to execute the resources of the first response, but the representation form can be the same or different, and this application does not limit this.

[0267] S205. The second application process sends a fourth request to the server corresponding to the second application, wherein the fourth request is used to request the first device to execute resources of the first response.

[0268] In this application, step S205 can refer to the above Figure 5C For the description of step S112 shown, the fourth request refers to the description of the second music resource request mentioned above, and the server corresponding to the second application refers to the description of the second server 501 mentioned above, which will not be repeated here.

[0269] S206. The second application process receives a first resource from a server corresponding to the second application, wherein the first resource is used by the first device to execute a first response.

[0270] In this application, step S206 can refer to the above Figure 5C The description of step S113 shown in the figure, the first resource refers to the description of the music resource mentioned above, which will not be repeated here.

[0271] S207: The second application process sends the first resource to the first application process.

[0272] In this application, step S207 can refer to the above Figure 5C The description of step S114 shown is not repeated here.

[0273] S208. The first device executes a first response based on the first resource.

[0274] In this application, step S208 can refer to the above Figure 5C For the description of step S114 shown, the first response refers to the description of playing the song Love Story mentioned above, which will not be repeated here.

[0275] In some embodiments, before the first device sends the second request to the first server, the device control method of the present application may also include: the first device determines that the second application is not installed in the first device based on the fact that the second application does not exist in the bundle management service (Bundle Manager Service, BMS), wherein the BMS is used to store the identifiers of the applications installed in the first device. Thus, the first device may need to download and install the second application so that the first application obtains the resources for executing the first response with the help of the second application. The above implementation process can be specifically referred to in the previous text. Figure 5C The description of steps S106-S109 shown is not repeated here.

[0276] In some embodiments, after the first device receives the second application from the first server, the first device may also install the second application in the first device and store the identifier of the second application in the BMS. Thus, the first device can confirm that the second device has been installed in the first device, which facilitates the first application to obtain resources for executing the first response with the help of the second application. Figure 5C The description of step S106 shown is not repeated here.

[0277] In some embodiments, the first device stores a configuration file of the first application, and the configuration file of the first application includes an identifier of the second application. The second request carries the identifier of the second application. In this application, the above implementation process can be specifically referred to the above description, which will not be repeated here.

[0278] In some embodiments, the device control method of the present application may further include: steps S301-S307.

[0279] S301. The first device sends a fifth request to the first server in response to the first request, wherein the fifth request is used to request to obtain a third application, and the first application is associated with the third application.

[0280] S302. The first device receives a third application from the first server.

[0281] S303. The first application process sends a sixth request to the third application process, wherein the third application process corresponds to a third application, the first application process and the third application process are different processes, and the sixth request is used to request the first device to execute resources of the first response.

[0282] S304. The third application process sends a seventh request to the server corresponding to the third application, wherein the seventh request is used to request the first device to execute resources of the first response.

[0283] S305. The third application process receives a second resource from a server corresponding to the third application, wherein the second resource is used by the first device to execute a first response.

[0284] S306: The third application process sends the second resource to the first application process.

[0285] S307: The first device executes a first response based on the first resource and the second resource.

[0286] In the present application, the first application in the first device can simultaneously virtually integrate the second application and the third application. That is, the process of the first application obtaining resources through the third application in the above steps S301-S307 can be specifically referred to the description of the first application obtaining resources through the second application. The second resource can be referred to the description of the music resource mentioned above, which will not be repeated here.

[0287] In some embodiments, the first device receives a first instruction from the second device, and the first device is in communication with the second device. In response to the first instruction, the first device sends a first request to the first application process. Figure 7-8D As shown in the description, the first instruction can refer to the description of the user's input operation mentioned above, which will not be repeated here.

[0288] In some embodiments, the first device sends a first request to the first application process in response to the first operation of the user. Figure 9-10D ,as well as Figure 11-12B As shown in the description, the user's first operation can refer to the description of the user's input operation mentioned above, which will not be repeated here.

[0289] In some embodiments, the device control method of the present application may further include: the display in the first device displays the interface of the first application, the interface of the first application includes a first area, and the interface displayed in the first area is provided by the second application. Figure 13A-13G The description shown is not repeated here.

[0290] In some embodiments, the device control method of the present application may further include: the first device releases the resources of the second application when a preset condition is met, that is, the first device may terminate the second application process and release the related resources of the second application in the memory of the first device. The first preset may include but is not limited to: whether the proportion of space occupied by the second application exceeds the preset proportion, whether the remaining space of the first device is less than the preset size, whether the number of applications in the second application exceeds the preset number, whether the running time of the second application when not called exceeds the preset time, etc.

[0291] In some embodiments, the device control method of the present application may further include: the first application process stops calling the second application process, that is, the first application process no longer starts the second application process, so that the second application process can be called by other processes. For example, the first device can set a flag for the first application process to stop calling the second application process. When the value on the flag is a preset value, the first device can determine that the first application process has stopped calling the second application process.

[0292] In some embodiments, the device control method of the present application may further include: steps S401-S407.

[0293] S401. The first device sends an eighth request to a fourth application process, wherein the fourth application process corresponds to a fourth application, the fourth application is installed in the first device, the fourth application is associated with the second application, and the eighth request is used to request the first device to execute resources for a second response.

[0294] S402. The fourth application process sends a ninth request to the second application process, wherein the fourth application process and the second application process are different processes, and the ninth request is used to request the first device to execute resources of the second response.

[0295] S403. The second application process sends a tenth request to the server corresponding to the second application, wherein the tenth request is used to request the first device to execute resources of the second response.

[0296] S404. The second application process receives a third resource from a server corresponding to the second application, wherein the third resource is used by the first device to execute a second response.

[0297] S405. The second application process sends the third resource to the fourth application process.

[0298] S406. The first device executes a second response based on the third resource.

[0299] In this application, in addition to installing the first application, the first device may also be installed with a fourth application, and the second application and the fourth application may respectively call the second application. The specific implementation process of the fourth application in steps S401-S407 implementing the first device to execute the second response through the second application can refer to the description of the specific implementation process of the first application implementing the first device to execute the first response through the second application in the above text, which will not be repeated here.

[0300] In some embodiments, when the first device is a speaker, the first application is a speaker FA application, the second device is a music FA application, and the first server is a FA cloud server, the device control method of the present application may include: steps S501-S513.

[0301] S501. The speaker requests to call the speaker FA process, wherein the speaker FA process corresponds to the speaker FA application, and the speaker FA application is installed in the speaker.

[0302] S502: The speaker determines whether the speaker FA process is running.

[0303] S503. When the speaker FA process is not running, the speaker calls the speaker FA process and sends the identifier of the target song to the speaker FA process.

[0304] S504: When the speaker FA process is running, the speaker sends the identifier of the target song to the speaker FA process.

[0305] S505, the speaker FA process requests to call the music FA process.

[0306] S506. The speaker determines whether there is an identifier of the music FA application in the BMS to determine whether there is a bundle of the music FA application in the speaker, wherein the music FA process corresponds to the music FA application, and the BMS stores an identifier of the FA application associated with the speaker FA application.

[0307] S507. If the identifier of the music FA application does not exist in the BMS, the operating system of the speaker sends a request to download the music FA application to the FA cloud server, wherein the request to download the music FA application is used to request the bundle of the music FA application, and the request to download the music FA application carries the identifier of the music FA application. The operating system of the speaker receives the bundle of the music FA application from the FA cloud server. The operating system of the speaker unpacks the bundle of the music FA application, installs the music FA application in the speaker, stores the identifier of the music FA application in the BMS, and continues to determine whether the identifier of the music FA application exists in the BMS until the identifier of the music FA application exists in the BMS.

[0308] S508: If the identifier of the music FA application exists in the BMS, the speaker FA process calls the music FA process.

[0309] S509. The speaker FA process sends a first music resource request to the music FA process, wherein the first music resource request is used to request the music resource of the target song, and the first music resource request carries an identifier of the target song.

[0310] S510. The music FA process sends a second music resource request to the server corresponding to the music FA application, wherein the second music resource request is used to request the music resource of the target song, and the second music resource request carries an identifier of the target song.

[0311] S511. The music FA process receives the music resources of the target song from the server corresponding to the music FA application.

[0312] S512, the music FA process sends the music resources of the target song to the speaker FA process.

[0313] S513: The speaker plays the target song based on the music resource of the target song.

[0314] In the present application, the specific implementation process of steps S501-S513 can be found in the previous description and will not be repeated here.

[0315] Exemplarily, the present application also provides an electronic device. The electronic device may include: at least one memory and at least one processor; the memory is used to store program instructions; the processor is used to call the program instructions in the memory so that the first device executes the device control method of the above embodiment.

[0316] Exemplarily, the present application also provides a computer program product, including: execution instructions, the execution instructions are stored in a readable storage medium, at least one processor of the electronic device can read the execution instructions from the readable storage medium, and at least one processor executes the execution instructions so that the electronic device implements the device control method of the previous embodiment.

[0317] Exemplarily, the present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor in an electronic device, the device control method of the foregoing embodiment is implemented.

[0318] In the above embodiments, all or part of the functions can be implemented by software, hardware, or a combination of software and hardware. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loading and executing computer program instructions on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or a data center that includes one or more available media integrations. Available media can be magnetic media, (e.g., floppy disks, hard disks, tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid state drives (SSDs)), etc.

[0319] Those skilled in the art can understand that to implement all or part of the processes in the above-mentioned embodiments, the processes can be completed by computer programs to instruct related hardware, and the programs can be stored in computer-readable storage media. When the programs are executed, they can include the processes of the above-mentioned method embodiments. The aforementioned storage media include: ROM or random access memory RAM, magnetic disk or optical disk and other media that can store program codes.

Claims

1. A device control method, characterized in that: The method comprises: A first device sends a first request to a first application process, wherein the first application is installed in the first device, and the first request is used to request the first device to execute a resource of a first response; The first device sends a second request to the first server in response to the first request, wherein the second request is used to request to obtain a second application, and the first application is associated with the second application; The first device receives the second application from the first server; The first application process sends a third request to the second application process, wherein the first application process and the second application process are different processes, and the third request is used to request the first device to execute the resource of the first response; The second application process sends a fourth request to a server corresponding to the second application, wherein the fourth request is used to request the first device to execute resources of the first response; The second application process receives a first resource from a server corresponding to the second application, wherein the first resource is used by the first device to execute the first response; The second application process sends the first resource to the first application process; The first device executes the first response based on the first resource.

2. The method according to claim 1, characterized in that Before the first device sends the second request to the first server, the method further includes: The first device determines that the second application is not installed in the first device based on the absence of an identifier of the second application in a bundle management service (Bundle Manager Service, BMS), wherein the BMS is used to store identifiers of applications installed in the first device.

3. The method according to claim 2, characterized in that After the first device receives the second application from the first server, the method further includes: The first device installs the second application in the first device, and stores an identifier of the second application in the BMS.

4. The method according to any one of claims 1 to 3, characterized in that: The first device stores a configuration file of the first application, and the configuration file of the first application includes an identifier of the second application; the second request carries the identifier of the second application.

5. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: The first device sends a fifth request to the first server in response to the first request, wherein the fifth request is used to request to obtain a third application, and the first application is associated with the third application; The first device receives the third application from the first server; The first application process sends a sixth request to a third application process, wherein the third application process corresponds to the third application, the first application process and the third application process are different processes, and the sixth request is used to request the first device to execute resources of the first response; The third application process sends a seventh request to the server corresponding to the third application, wherein the seventh request is used to request the first device to execute the resource of the first response; The third application process receives a second resource from a server corresponding to the third application, wherein the second resource is used by the first device to execute the first response; The third application process sends the second resource to the first application process; The first device performs the first response based on the first resource and the second resource.

6. The method according to any one of claims 1 to 5, characterized in that: The first device receives a first instruction from a second device, and the first device is in communication connection with the second device; The first device sends the first request to the first application process in response to the first instruction.

7. The method according to any one of claims 1 to 5, characterized in that: The first device sends the first request to the first application process in response to a first operation of a user.

8. The method according to any one of claims 1 to 7, characterized in that: The method further comprises: The display in the first device displays the interface of the first application, the interface of the first application includes a first area, and the interface displayed in the first area is provided by the second application.

9. The method according to any one of claims 1 to 8, characterized in that: The method further comprises: The first device releases resources of the second application.

10. The method according to any one of claims 1 to 9, characterized in that: The method further comprises: The first application process stops calling the second application process.

11. The method according to claim 10, characterized in that The method further comprises: The first device sends an eighth request to a fourth application process, wherein the fourth application process corresponds to a fourth application, the fourth application is installed in the first device, the fourth application is associated with the second application, and the eighth request is used to request the first device to execute a second response resource; The fourth application process sends a ninth request to the second application process, wherein the fourth application process and the second application process are different processes, and the ninth request is used to request the first device to execute resources of the second response; The second application process sends a tenth request to the server corresponding to the second application, wherein the tenth request is used to request the first device to execute the resource of the second response; The second application process receives a third resource from a server corresponding to the second application, wherein the third resource is used by the first device to execute the second response; The second application process sends the third resource to the fourth application process; The first device executes the second response based on the third resource.

12. The method according to any one of claims 1 to 11, characterized in that: The first device is a speaker, the first application is a speaker FA application, the second device is a music FA application, and the first server is a FA cloud server; the method is specifically: The speaker requests to call the speaker FA process, wherein the speaker FA process corresponds to the speaker FA application, and the speaker FA application is installed in the speaker; The speaker determines whether the speaker FA process is running; When the speaker FA process is not running, the speaker calls the speaker FA process and sends the identifier of the target song to the speaker FA process; When the speaker FA process is running, the speaker sends the identification of the target song to the speaker FA process; The speaker FA process requests to call the music FA process; The speaker determines whether there is an identifier of the music FA application in the BMS to determine whether there is a bundle of the music FA application in the speaker, wherein the music FA process corresponds to the music FA application, and the BMS stores an identifier of the FA application associated with the speaker FA application; If the identifier of the music FA application does not exist in the BMS, the operating system of the speaker sends a request to download the music FA application to the FA cloud server, wherein the request to download the music FA application is used to request the bundle of the music FA application, and the request to download the music FA application carries the identifier of the music FA application; the operating system of the speaker receives the bundle of the music FA application from the FA cloud server; the operating system of the speaker installs the music FA application in the speaker by unpacking the bundle of the music FA application, and stores the identifier of the music FA application in the BMS, and continues to determine whether the identifier of the music FA application exists in the BMS until the identifier of the music FA application exists in the BMS; If the identifier of the music FA application exists in the BMS, the speaker FA process calls the music FA process; The speaker FA process sends a first music resource request to the music FA process, wherein the first music resource request is used to request the music resource of the target song, and the first music resource request carries an identifier of the target song; The music FA process sends a second music resource request to the server corresponding to the music FA application, wherein the second music resource request is used to request the music resource of the target song, and the second music resource request carries the identifier of the target song; The music FA process receives the music resource of the target song from the server corresponding to the music FA application; The music FA process sends the music resource of the target song to the speaker FA process; The speaker plays the target song based on the music resource of the target song.

13. A first device, characterized in that: include: at least one memory and at least one processor; The memory is used to store program instructions; The processor is used to call the program instructions in the memory so that the electronic device executes the device control method according to any one of claims 1-12.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: The computer program is executed by the processor so that the electronic device implements the device control method according to any one of claims 1 to 12.

Images