Audio and video playing method and electronic equipment

Abstract

The invention provides an audio and video playing method and electronic equipment. According to the method, after the multimedia layer in the software architecture supported by the audio and video device is started, the multiple signalings included in the instruction sent by the controller are divided into the audio and video playing signalings and the non-audio and video playing signalings in the multimedia layer; the method comprises the following steps: starting an application layer, processing an audio and video playing signaling used for indicating to play a target audio and video in a multimedia layer to play the target audio and video, temporarily caching a non-audio and video playing signaling included in an instruction in the multimedia layer, and processing the cached non-audio and video playing signaling in the application layer after the application layer is started. Visibly, the video device in the electronic equipment is used for replacing a controller originally used for audio and video playing to perform audio and video playing, and the audio and video device has relatively large storage space and relatively high processing performance, so that the audio and video playing quality can be greatly improved.

Description

Technical Field

[0001] This application relates to the technical field of audio - video playback, and particularly relates to an audio - video playback method and an electronic device. Background Art

[0002] For some electronic devices, such as intelligent security devices like smart locks and smart doorbells, it is usually necessary to implement the interaction between the user and the device by playing prompt voices or interactive videos, etc. Currently, the electronic devices that can achieve preset audio - video playback usually use a controller in the electronic device, such as a microcontroller unit (MCU: Microcontroller Unit), that is, store the audio - video locally in the controller. In the case of needing to play, directly play the corresponding audio - video stored locally by the controller. However, due to the limited storage space in the controller, the quality of the audio - video played by the controller is relatively low. Summary of the Invention

[0003] In view of this, this application provides an audio - video playback method and an electronic device to improve the quality of audio - video playback.

[0004] The technical solutions provided by this application are as follows:

[0005] According to an embodiment of the first aspect of this application, an audio - video playback method is provided. This method is applied to an audio - video device in an electronic device, and the electronic device further includes a controller. The method includes:

[0006] When the audio - video device is powered on, start the multimedia layer and the application layer in the software architecture supported by the audio - video device; the startup duration of the multimedia layer is less than that of the application layer; the software architecture is designed based on the separation of multimedia services and application logic; when the multimedia layer finishes starting, it indicates that the audio - video device has the audio - video playback ability;

[0007] After the multimedia layer finishes starting, divide the signaling sent by the controller into audio - video playback signaling and non - audio - video playback signaling in the multimedia layer; the audio - video playback signaling refers to the signaling used to indicate playing the target audio - video;

[0008] Process the audio - video playback signaling in the multimedia layer to play the target audio - video, and cache the non - audio - video playback signaling in the multimedia layer;

[0009] After the application layer finishes starting, read the non - audio - video playback signaling cached in the multimedia layer to the application layer as needed, so as to process the non - audio - video playback signaling in the application layer to perform the operations indicated by the non - audio - video playback signaling.

[0010] Optionally, after processing the audio-video playback signaling in the multimedia layer to play the target audio-video, the method further includes:

[0011] After the target audio-video playback is completed, detect whether the application layer has been started;

[0012] If so, forward the first response from the multimedia layer to the application layer, so that the application layer forwards the first response to the controller; the first response is used to indicate that the target audio-video playback is completed;

[0013] If not, forward the first response from the multimedia layer to the controller.

[0014] Optionally, after processing the non-audio-video playback signaling in the application layer to perform the operation indicated by the non-audio-video playback signaling, the method further includes:

[0015] Determine whether a first sleep condition is satisfied;

[0016] When the first sleep condition is satisfied, control the application layer to start pre-sleep, so as to prohibit the application layer from processing the unprocessed non-audio-video playback signaling, and store the processing results of the audio-video playback signaling processed in the multimedia layer and the non-audio-video playback signaling processed in the application layer; the unprocessed non-audio-video playback signaling refers to the non-audio-video playback signaling obtained by the controller after the application layer starts pre-sleep;

[0017] Send a pre-sleep request to the controller, so that after receiving the pre-sleep request, the controller divides the obtained signaling into unprocessed audio-video playback signaling and unprocessed non-audio-video playback signaling, caches the unprocessed non-audio-video playback signaling, and sends the unprocessed audio-video playback signaling to the multimedia layer to process the unprocessed audio-video playback signaling in the multimedia layer and play the target audio-video indicated by the unprocessed audio-video playback signaling; the unprocessed audio-video playback signaling refers to the audio-video playback signaling obtained by the controller after the application layer starts pre-sleep.

[0018] Optionally, the satisfaction of the first sleep condition includes:

[0019] The application layer does not receive non-audio-video playback signaling within a specified duration, and there is no non-audio-video playback signaling that needs to be processed by the application layer in the controller.

[0020] Optionally, after the application layer starts pre-sleep, the method further includes:

[0021] After pre-sleep is completed at the application layer, it is determined whether the second sleep condition is satisfied; the completion of pre-sleep at the application layer means that the application layer has completed the storage operation of the processing results of the audio-video playback signaling processed in the multimedia layer and the non-audio-video playback signaling processed in the application layer;

[0022] When the second sleep condition is satisfied, the controller is notified to power down the audio-video device to prohibit the processing of all signaling at the application layer and the multimedia layer.

[0023] Optionally, the satisfaction of the second sleep condition includes:

[0024] The current state of the multimedia layer is the idle state, and there is no pending audio-video playback signaling in the controller; the idle state is used to indicate that there is no audio-video playback signaling being processed in the multimedia layer currently.

[0025] Optionally, each signaling sent by the controller is configured with a type identifier; the division of the signaling sent by the controller into audio-video playback signaling and non-audio-video playback signaling in the multimedia layer includes:

[0026] The signaling sent by the controller is parsed by the multimedia layer, and the signaling sent by the controller is divided into audio-video playback signaling and non-audio-video playback signaling according to the type identifier carried by each signaling.

[0027] Optionally, the electronic device is a low-power device, the audio-video device is the audio-video device in the low-power device, and the audio-video device is powered on when there is signaling that needs to be processed by the audio-video device.

[0028] According to an embodiment of the second aspect of the present application, an electronic device is provided, including:

[0029] A controller, configured to send signaling that needs to be processed by the audio-video device to the audio-video device after the multimedia layer in the audio-video device is started;

[0030] An audio-video device, the software architecture supported by the audio-video device includes a multimedia layer and an application layer, the multimedia layer and the application layer are started when the audio-video device is powered on, the start-up duration of the multimedia layer is less than the start-up duration of the application layer, and the software architecture is designed based on the separation of multimedia services and application logic; the completion of the start-up of the multimedia layer indicates that the audio-video device has the voice playback ability; the multimedia layer is configured with a controller preprocessing unit, an audio-video processing unit, and a virtual controller unit;

[0031] Among them, the controller preprocessing unit is used to divide the signaling sent by the controller into audio / video playback signaling and non-audio / video playback signaling. The audio / video playback signaling refers to the signaling used to indicate playing the target audio / video.

[0032] The audio / video processing unit is used to process the audio / video playback signaling to play the target audio / video.

[0033] The virtual controller component is used to cache the non-audio / video playback signaling.

[0034] After the application layer is started, it is used to read the cached non-audio / video playback signaling in the virtual controller component to the application layer as needed, and process the non-audio / video playback signaling in the application layer to execute the operations indicated by the non-audio / video playback signaling.

[0035] Optionally, after the application layer processes the non-audio / video playback signaling to execute the operations indicated by the non-audio / video playback signaling, the application layer is further used to:

[0036] Judge whether the first sleep condition is satisfied. The satisfaction of the first sleep condition includes: the application layer does not receive non-audio / video playback signaling within a specified duration, and there is no non-audio / video playback signaling that needs to be processed by the application layer in the controller.

[0037] When the first sleep condition is satisfied, start pre-sleep, prohibit the application layer from processing the non-audio / video playback signaling to be processed, and store the processing results of the audio / video playback signaling processed by the audio / video processing unit and the non-audio / video playback signaling processed by this application layer. The non-audio / video playback signaling to be processed refers to the non-audio / video playback signaling obtained by the controller after the application layer starts pre-sleep.

[0038] Send a pre-sleep request to the controller, so that after the controller receives the pre-sleep request, it divides the obtained signaling into the audio / video playback signaling to be processed and the non-audio / video playback signaling to be processed, caches the non-audio / video playback signaling to be processed, and sends the audio / video playback signaling to be processed to the controller preprocessing unit, so that the controller preprocessing unit forwards the audio / video playback signaling to be processed to the audio / video processing unit, and the audio / video processing unit processes the audio / video playback signaling to be processed to play the target audio / video indicated by the audio / video playback signaling to be processed. The audio / video playback signaling to be processed refers to the audio / video playback signaling obtained by the controller after the application layer starts pre-sleep.

[0039] And / or, after the application layer starts pre-sleep, the application layer is further used to:

[0040] After completing pre-sleep, it is determined whether the second sleep condition is satisfied; the completion of pre-sleep means that the application layer has completed the storage operation of the processing results of the audio-video playback signaling processed in the multimedia layer and the non-audio-video playback signaling processed in the application layer; the satisfaction of the second sleep condition includes: the current state of the multimedia layer is the idle state, and there is no pending audio-video playback signaling in the controller, and the idle state is used to indicate that there is no audio-video playback signaling being processed in the multimedia layer currently; the pending audio-video playback signaling refers to the audio-video playback signaling obtained by the controller after the application layer starts pre-sleep.

[0041] When the second sleep condition is satisfied, the controller is notified to control the power-down of the audio-video device to prohibit the application layer and the multimedia layer from processing all signaling.

[0042] As can be seen from the above technical solutions, after the multimedia layer in the software architecture supported by the audio-video device of the present application is started, the multiple signaling included in the instruction sent by the controller in the multimedia layer are divided into audio-video playback signaling and non-audio-video playback signaling, and the audio-video playback signaling for indicating the playback of the target audio-video is processed in the multimedia layer to play the target audio-video, and the non-audio-video playback signaling included in the instruction is temporarily cached in the multimedia layer. After the application layer is started, the cached non-audio-video playback signaling is processed in the application layer. It can be seen that the present application uses the video device in the electronic device to replace the original controller for audio-video playback, and performs audio-video playback. Since the audio-video device itself has a large storage space and strong processing performance, it can greatly improve the quality of audio-video playback.

[0043] Furthermore, the solution proposed in the present application does not need to wait for the application layer in the software architecture supported by the audio-video device to be started, but can perform audio-video playback in the multimedia layer after the multimedia layer is started, alleviating the problem of playback delay caused by audio-video playback through the audio-video device. Description of the Drawings

[0044] The drawings here are incorporated into the specification and form a part of the specification, showing the embodiments in line with the present application, and are used together with the specification to explain the principles of the present application.

[0045] Figure 1 It is a schematic diagram of the audio-video playback scenario in the intelligent door lock scenario provided by the embodiment of the present application;

[0046] Figure 2 It is a flow block diagram of an audio-video playback method provided by the embodiment of the present application;

[0047] Figure 3Schematic diagram of the software architecture supported by the audio - video device proposed in the embodiments of the present application;

[0048] Figure 4 Interaction schematic diagram of the power - on startup process provided by the embodiments of the present application;

[0049] Figure 5 Interaction schematic diagram of the power - off sleep process provided by the embodiments of the present application;

[0050] Figure 6 Schematic diagram of the structure of an electronic device provided by the embodiments of the present application. Detailed implementation manners

[0051] In order to enable those skilled in the art to better understand the technical solutions provided by the embodiments of the present application, and to make the above - mentioned objects, features, and advantages of the embodiments of the present application more obvious and understandable, the technical solutions in the embodiments of the present application will be further described in detail below with reference to the accompanying drawings.

[0052] For some electronic devices, such as intelligent security devices like smart locks and smart doorbells, it is usually necessary to implement the interaction between the user and the device by playing prompt voices or interactive videos. For example, taking a smart lock as an example, when the user completes the door - opening operation, a preset audio such as "The door has been unlocked" can be played, or a preset video such as a video showing changing words like "Welcome home" can be played.

[0053] Currently, electronic devices usually implement audio - video playback through a controller in the electronic device, such as a microcontroller unit (MCU: Microcontroller Unit).

[0054] Specifically, the audio - video to be played can be pre - stored locally in the controller. When audio - video playback is required, the corresponding audio - video is directly played from the local by the controller. However, due to the limited performance and storage space of the controller, the audio - video played through the controller usually has low quality and can only store a small amount of audio - video content.

[0055] Based on the above problems, there are currently two solutions: one is to expand the external memory Flash of the controller (such as an MCU), store the audio - video files in the external memory Flash of the controller, and directly control the playback by the controller; the other is to store the audio - video files in an audio - video device dedicated to playing audio - video. After the audio - video device is started, the controller controls the audio - video device to achieve audio - video playback by sending signaling.

[0056] However, the above solution for expanding the external Flash memory of the controller requires replacing the hardware device, and the actual operation is rather cumbersome; in the solution of controlling the audio-video device to play audio-video by sending signaling through the controller, the controller needs to send instructions to the audio-video device, so that the application layer in the audio-video device can parse the instructions and forward the audio-video playback signaling that needs to be processed by the multimedia layer in the instructions to the multimedia layer for processing. However, during the startup and sleep processes of the audio-video device, the application layer cannot handle services, resulting in the inability to respond to the audio-video playback requests of the controller during this period, leading to a large response delay and reducing the user experience.

[0057] Specifically, please refer to Figure 1 , Figure 1 which is a schematic diagram of the audio-video playback scenario in the intelligent door lock scenario provided by the embodiment of the present application.

[0058] As Figure 1 shown, the intelligent door lock device includes a sensor, a controller, a communication module, an audio-video device, and an identity recognition module.

[0059] Among them, the sensor is used to collect sensor information to determine whether a preset event occurs through the sensor information, such as detecting whether someone passes by through an infrared sensor, etc.

[0060] The controller is used to receive the sensor information and start the corresponding module to complete the processing of the corresponding event. Among them, the communication module is usually in a continuously powered-on low-power mode to maintain the networking and activity perception capabilities of the intelligent door lock, while the audio-video device and the identity recognition module are usually in a powered-off state and are powered on only when there are corresponding tasks to be processed.

[0061] In Figure 1 , the process of controlling the audio-video device to play audio-video by the controller can be that the controller sends an instruction to the application layer of the audio-video device. After receiving the instruction, the application layer parses the instruction and forwards the signaling included in the instruction for instructing the audio-video device to play the target audio-video to the multimedia layer, and the multimedia layer processes the signaling to play the target audio-video.

[0062] Since the audio-video device is powered on only when in use, the current power-on startup time of the audio-video device is about 2 s, and the power-off sleep time is about 3 s. Moreover, after startup is completed, due to the complex application services and diverse scenarios, there will be a time delay ranging from 2.5 s to 5 s between when the audio-video device is powered on and when it starts playing audio and video. During the power-off sleep process, the application layer cannot process any services either. Thus, it can be seen that when playing audio and video through the video processing SOC, during the power-on process, the delay in playing audio and video is 2.5 s to 5 s, and during the power-off sleep process, the delay in playing audio and video is 3 s + (2.5 s to 5 s), that is, 3 s during which it is impossible to play during the power-off process, and then plus the power-on playing time delay of 2.5 s to 5 s.

[0063] Obviously, the current solution for playing audio and video through the audio-video device cannot meet the real-time requirements for audio and video playback. If it is necessary to play audio and video through the audio-video device, the problem of long time delay in playing audio and video during the power-on and power-off processes of the audio-video device must be solved.

[0064] Based on this, the present application proposes an audio-video playback method, which can improve the audio-video playback quality while also taking into account the response speed of audio-video playback.

[0065] Please refer to Figure 2 , Figure 2 which is the flowchart of the audio-video playback provided by the embodiment of the present application.

[0066] In this embodiment, the method can be applied to the audio-video device in an electronic device, and the electronic device further includes a controller. The audio-video device can be a device for playing audio and / or video.

[0067] In this embodiment, the electronic device can be a low-power device, and the audio-video device can be the audio-video device in the low-power device. The audio-video device is powered on when there is a signaling that needs to be processed by the audio-video device, that is, the audio-video device has the characteristic of being powered on when in use.

[0068] As an embodiment, the electronic device can be an intelligent security device such as an intelligent door lock or an intelligent door eye, etc., which needs to achieve the interaction between the user and the device through playing prompt voices or video interactions, etc. The present application does not limit this.

[0069] As an embodiment, the controller can be an MCU, a microprocessor unit (MPU: Microprocessor Unit), a field programmable gate array (FPGA: Field-Programmable Gate Array), etc. The present application does not limit this.

[0070] As Figure 2 shown, the method can include the following steps:

[0071] Step 201: When the audio - video device is powered on, start the multimedia layer and the application layer in the software architecture supported by the audio - video device.

[0072] In this embodiment, the audio - video device is in a powered - off state when not performing tasks. When receiving a power - on instruction sent by the controller, the audio - video device is powered on. The power - on instruction can be a control signal sent by the controller to the General - Purpose Input / Output (GPIO) connected to the audio - video device when the controller determines that there is a task that needs to be processed by the audio - video device, so as to start the power supply and power on the audio - video device. This application does not limit this.

[0073] Among them, the timing of sending the power - on instruction to the controller can be that the user triggers the controller to send a power - on instruction to the audio - video device (the user operates the electronic device, such as entering a password, etc.), or the controller sends a power - on instruction to the audio - video device when detecting a specified event (such as detecting that the battery power of the electronic device is lower than a preset threshold, etc.). This application does not limit this.

[0074] In this embodiment, when the audio - video device is powered on, the multimedia layer and the application layer in the software architecture supported by the audio - video device can be started. In this embodiment, the startup duration of the multimedia layer is less than that of the application layer. Since the startup of the multimedia layer usually does not depend on complex operating system services and only requires basic hardware drivers and algorithm support, the startup speed of the multimedia layer is relatively fast, about 500 ms, and the user can hardly feel its startup delay. While the application layer usually involves the startup of multiple background services and processes, and its startup speed is slow, usually taking 2 s to 5 s.

[0075] Regarding the software architecture supported by the audio - video device, it will be described in detail below in combination with Figure 3 and will not be elaborated here.

[0076] So far, the description of step 201 ends. Next, execute step 202.

[0077] Step 202: After the multimedia layer is started, divide the signaling sent by the controller into audio - video playback signaling and non - audio - video playback signaling in the multimedia layer.

[0078] In this embodiment, since the audio - video device is powered on only when it needs to process services, when the audio - video device is powered on, it usually receives an instruction sent by the controller to instruct the audio - video device to perform related services.

[0079] Usually, the instruction sent by the controller includes multiple signaling, and each signaling is used to instruct the audio - video device to perform the related service indicated by the signaling.

[0080] In this embodiment, when the multimedia layer finishes starting up, it indicates that the audio-video device already has the audio-video playback capability, and at this time, the signaling sent by the controller can be processed.

[0081] As an embodiment, the specific method for the multimedia layer to divide the signaling sent by the controller into audio-video playback signaling and non-audio-video playback signaling may include:

[0082] The multimedia layer parses the signaling sent by the controller, and divides the signaling sent by the controller into audio-video playback signaling and non-audio-video playback signaling according to the type identifier carried by each signaling.

[0083] In this embodiment, each signaling included in the instruction sent by the controller is configured with a type identifier, which is used to identify whether the signaling is audio-video playback signaling. After receiving the instruction sent by the controller, the audio-video device can parse the instruction in the multimedia layer, and determine whether each signaling belongs to audio-video playback signaling or non-audio-video playback signaling according to the type identifier carried by each signaling in the instruction. Among them, the audio-video playback signaling refers to the signaling used to indicate playing the target audio-video.

[0084] In this embodiment, the multimedia service and application logic in the software architecture supported by the audio-video device are split, and its software architecture may include three layers, namely the system layer, the multimedia layer, and the application layer.

[0085] Among them, the system layer is responsible for the management of underlying hardware and resource allocation. In this embodiment, the system layer is not improved, and the functions of the system layer will not be elaborated later; in related technologies, the multimedia layer is mainly used to process audio-video processing tasks such as encoding, decoding, compression, transmission, and playback of multimedia data to ensure that different types of multimedia content (such as audio, video, etc.) can be effectively processed and displayed; in related technologies, the application layer is mainly used to parse instructions from users or external systems, and determine operations for processing or forwarding the signaling included in the instructions according to these instructions.

[0086] In this embodiment, considering that the multimedia layer can operate independently without relying on the application layer to separately execute audio-video playback tasks, and its startup speed is relatively fast, plugins can be added to the multimedia layer so that the multimedia layer can directly communicate with the controller without passing through the application layer for forwarding.

[0087] In this embodiment, in addition to being configured with an audio-video processing unit for implementing its own audio-video processing function, the multimedia layer is also configured with a controller preprocessing unit and a virtual controller unit.

[0088] Among them, the controller preprocessing unit is used to divide the signaling sent by the controller into audio / video playback signaling and non-audio / video playback signaling;

[0089] The audio / video processing unit is used to process the audio / video playback signaling to play the target audio / video;

[0090] The virtual controller unit is used to cache the non-audio / video playback signaling.

[0091] In this embodiment, the controller preprocessing unit and the controller can be connected through a serial port. Based on this connection, the controller can communicate directly with the multimedia layer across the application layer, so as to play the audio / video through the controller preprocessing component in the multimedia layer that has been started before the application layer is started.

[0092] At the same time, considering that non-audio / video playback signaling that needs to be processed by the application layer may also be received before the application layer is started, such as identity authentication signaling, firmware upgrade signaling, etc., in order to avoid that these signaling that need to be processed by the application layer cannot be responded to, a virtual controller unit is also newly added in the multimedia layer in this embodiment, which is used to temporarily cache the non-audio / video playback signaling that needs to be processed by the application layer when the application layer is not started.

[0093] In this embodiment, since the audio / video playback signaling can be processed separately in the multimedia layer, and the power-on speed of the multimedia layer is relatively fast during the power-on process of the audio / video device, about 500 ms, the multimedia layer can be powered on first to process the audio / video playback signaling in the multimedia layer to achieve fast audio / video playback response.

[0094] For the non-audio / video playback signaling included in the instruction, since the non-audio / video playback signaling needs to be processed by the application layer, and the power-on process of the application layer is relatively slow, usually about 2 s, and the current application layer has not completed the power-on startup, so at this time, these non-audio / video playback signaling can be cached in the multimedia layer first, and specifically can be stored in the newly added virtual processor unit mentioned above.

[0095] As an embodiment, please refer to Figure 3 , Figure 3 which is a schematic diagram of the software architecture supported by the audio / video device proposed in the embodiment of the present application.

[0096] As Figure 3 shown, the software architecture supported by the audio / video device can include three layers, namely the system layer, the multimedia layer, and the application layer.

[0097] In this embodiment, a virtual controller unit and a controller preprocessing unit are newly added to the multimedia layer. The virtual controller unit is responsible for caching the non-audio / video playback signaling that interacts between the controller and the application layer. The controller preprocessing unit is responsible for parsing the instructions directly sent by the controller to the multimedia layer, forwarding the audio / video playback signaling included in the instructions to the audio / video processing unit for processing, and forwarding the non-audio / video playback signaling to the virtual controller unit for caching.

[0098] In Figure 3 the audio / video service that interacts between the multimedia layer and the application layer means that after the multimedia layer completes the audio / video playback through the audio / video processing unit, it further performs operations such as storing the playback result, which needs to be processed through the application layer. This process is a common method in the related art and will not be elaborated here.

[0099] The non-audio / video service that interacts between the multimedia layer and the application layer means that after the application layer is started, it reads the cached non-audio / video playback signaling from the virtual controller unit to execute the non-audio / video services that need to be executed as indicated by the non-audio / video playback signaling through the application layer, such as identity recognition, firmware upgrade, and other services.

[0100] The other services that interact between the system layer and the application layer refer to the services executed by the audio / video device itself and unrelated to the controller instructions, such as periodically collecting images.

[0101] So far, the description of Figure 3 ends.

[0102] It should be noted that the newly added virtual controller unit and controller preprocessing unit in this embodiment can be virtual structures at the software level or physical structures at the hardware level, and this application does not limit this.

[0103] The solution proposed in this embodiment can, in addition to processing audio / video processing tasks such as encoding, decoding, compression, transmission, and playback of multimedia data in the multimedia layer, also receive the instructions sent by the controller, parse the instructions, and determine the audio / video playback signaling and non-audio / video playback signaling included in the instructions; while in the application layer, there is no need to perform the operation of parsing instructions, and only the non-audio / video playback signaling that needs to be processed by the application layer needs to be processed.

[0104] So far, the description of step 202 ends, and then step 203 is executed.

[0105] Step 203: Process the audio / video playback signaling in the multimedia layer to play the target audio / video, and cache the non-audio / video playback signaling in the multimedia layer.

[0106] In this embodiment, after determining the audio-video playback signaling and non-audio-video playback signaling through step 202, the audio-video playback signaling can be processed at the multimedia layer. Specifically, the audio-video playback signaling can be processed by the audio-video processing unit in the multimedia layer.

[0107] Meanwhile, the non-audio-video playback signaling can be cached in the multimedia layer. Specifically, the non-audio-video playback signaling can be cached in the virtual controller component.

[0108] In this embodiment, after processing the audio-video playback signaling at the multimedia layer to play the target audio-video, the method can further include:

[0109] After the target audio-video is played, detect whether the application layer has completed startup;

[0110] If so, forward the first response to the application layer through the multimedia layer, so that the application layer forwards the first response to the controller; the first response is used to indicate that the target audio-video has been played;

[0111] If not, forward the first response to the controller through the multimedia layer.

[0112] In this embodiment, after successfully processing the audio-video playback signaling at the multimedia layer, that is, after completing the playback of the target audio-video, a first response indicating successful processing can be returned to the controller. Specifically, there are two return paths:

[0113] If the current application layer has completed startup, the first response can be forwarded to the application layer in the normal response return manner, and then the application layer forwards the first response to the controller to ensure the accuracy of data timing.

[0114] If the current application layer has not been started yet, the audio-video playback signaling can be directly forwarded to the controller by the multimedia layer.

[0115] Specifically, the forwarding process of the first response will be described in detail below in combination with Figure 4 Details are not described here.

[0116] So far, the description of step 203 is completed. Next, execute step 204.

[0117] Step 204, after the application layer is started, read the cached non-audio-video playback signaling in the multimedia layer to the application layer as needed, so as to process the non-audio-video playback signaling in the application layer to perform the operations indicated by the non-audio-video playback signaling.

[0118] In this embodiment, after the application layer is started, the cached non-audio-video playback signaling in the multimedia layer can be read, and the non-audio-video playback signaling can be further processed in the application layer.

[0119] Specifically, the cached non-audio / video playback signaling can be read from the virtual controller unit in the multimedia layer to the application layer, and the non-audio / video playback signaling is processed in the application layer to perform the operations indicated by the non-audio / video playback signaling, such as authentication, firmware upgrade, etc.

[0120] In this embodiment, reading the cached non-audio / video playback signaling from the multimedia layer can be performed according to actual requirements, such as reading all the cached non-audio / video playback signaling, or reading part of the cached non-audio / video playback signaling. The present application does not limit this.

[0121] Thus, the description of step 204 ends.

[0122] It should be noted that the above solution solves the problem that audio / video playback cannot be performed during the power-on startup process of the audio / video device. The following describes the audio / video playback method of the audio / video device during the power-off sleep process.

[0123] After processing the non-audio / video playback signaling in the application layer to perform the operations indicated by the non-audio / video playback signaling, the method may further include:

[0124] Determine whether a first sleep condition is satisfied;

[0125] When the first sleep condition is satisfied, control the application layer to start pre-sleep, so as to prohibit the application layer from processing the to-be-processed non-audio / video playback signaling, and store the processing results of the audio / video playback signaling processed in the multimedia layer and the non-audio / video playback signaling processed in the application layer; the to-be-processed non-audio / video playback signaling refers to the non-audio / video playback signaling obtained by the controller after the application layer starts pre-sleep.

[0126] Send a pre-sleep request to the controller, so that after receiving the pre-sleep request, the controller divides the obtained signaling into to-be-processed audio / video playback signaling and to-be-processed non-audio / video playback signaling, caches the to-be-processed non-audio / video playback signaling, and sends the to-be-processed audio / video playback signaling to the multimedia layer to process the to-be-processed audio / video playback signaling in the multimedia layer and play the target audio / video indicated by the to-be-processed audio / video playback signaling; the to-be-processed audio / video playback signaling refers to the audio / video playback signaling obtained by the controller after the application layer starts pre-sleep.

[0127] In this embodiment, satisfying the first sleep condition may specifically include:

[0128] The application layer does not receive non-audio / video playback signaling within a specified duration, and there is no non-audio / video playback signaling that needs to be processed by the application layer in the controller.

[0129] It is easy to understand that if the application layer does not receive non-target instructions within the specified duration, it indicates that there has been no business processing for some time. At this time, the audio-video device can request the controller to query whether there are still non-target instructions that need to be processed by the application layer. If not, it indicates that the first sleep condition is met, and the pre-sleep of the application layer can be started.

[0130] In this embodiment, starting the pre-sleep of the application layer means prohibiting the application layer from processing non-audio-video playback signaling to be processed. The application layer will not process new non-audio-video playback signaling in the pre-sleep state and only needs to perform preparatory work before power-off, such as storing system data. Specifically, it can be storing the processing results of the audio-video playback signaling processed by the multimedia layer and the non-audio-video playback signaling processed by the application layer.

[0131] Further, after the first sleep condition is met, the audio-video device can send a pre-sleep request to the controller. After receiving the pre-sleep request, the controller divides the multiple signaling included in the obtained instructions into audio-video playback signaling to be processed and non-audio-video playback signaling to be processed, and caches the non-audio-video playback signaling to be processed in the controller. The non-audio-video playback signaling to be processed refers to the non-audio-video playback signaling included in the instructions obtained by the controller after controlling the application layer to start pre-sleep.

[0132] It should be noted that in this pre-sleep process, the application layer does not process new signaling. Different from the power-on process, it is not the multimedia layer that parses the instructions to determine the audio-video playback signaling and non-audio-video playback signaling, but the controller parses the instructions to be processed. After determining the signaling type included in the instructions to be processed, the controller does not send the non-audio-video playback signaling to be processed to the application layer, but stores it locally in the controller. The multimedia layer remains in the startup state. At this time, if there is audio-video playback signaling to be processed, the controller can still send the audio-video playback signaling to be processed to the multimedia layer for processing.

[0133] As an embodiment, after controlling the application layer to start pre-sleep, the method may further include:

[0134] After the application layer completes pre-sleep, it is judged whether the second sleep condition is met; the application layer completing pre-sleep means that the application layer completes the storage of the processing results of the audio-video playback signaling processed by the multimedia layer and the non-audio-video playback signaling processed by the application layer;

[0135] In the case where the second sleep condition is met, the controller is notified to control the power-off of the audio-video device to prohibit the application layer and the multimedia layer from processing all signaling.

[0136] In this embodiment, the satisfaction of the second sleep condition may specifically include:

[0137] The current state of the multimedia layer is the idle state, and there is currently no audio-video playback signaling to be processed in the controller; the idle state is used to indicate that there is currently no audio-video playback signaling being processed in the multimedia layer.

[0138] In this embodiment, after the application layer completes pre-sleep, that is, the preparation work before power-off, the current state of the multimedia layer can be obtained;

[0139] If the current state of the multimedia layer is the idle state, a second query message is further sent to the controller, so that the controller determines whether there is currently any audio-video playback signaling that needs to be processed by the multimedia layer after receiving the second query message; the idle state is used to indicate that the multimedia layer is not currently processing audio-video playback signaling;

[0140] If the second response sent by the controller is received, an instruction to allow power-off is sent to the controller, so that the controller controls the audio-video device to power off; the second response is used to indicate that there is currently no audio-video playback signaling that needs to be processed by the multimedia layer in the controller.

[0141] Specifically, after the application layer completes the sleep preparation, that is, after the processes such as storing data before power-off are completed, the current state of the multimedia layer can be obtained, and when it is determined that the multimedia layer is not currently processing audio-video playback signaling, it is further queried whether there is any audio-video playback signaling that needs to be processed by the multimedia layer in the controller.

[0142] If the current state of the multimedia layer is the idle state, that is, there is currently no audio-video playback signaling being processed, and there is no audio-video playback signaling that needs to be processed by the multimedia layer in the controller, it indicates that there is no service being processed in the current system. At this time, the controller can be notified to control the audio-video device to power off.

[0143] If the current state of the multimedia layer is a non-idle state or the third response sent by the controller is received, it indicates that there is an audio-video service that needs to be processed in the current system. At this time, the step of obtaining the current state of the multimedia layer can be returned. The third response is used to indicate that there is currently an audio-video playback signaling that needs to be processed by the multimedia layer in the controller, that is, the audio-video playback signaling to be processed.

[0144] It is easy to understand that, in order to solve the delay in playing voice during the power-off sleep process, a pre-sleep process is newly added during the power-off process in this solution. During this pre-sleep process, the controller can continue to send audio-visual play signaling (such as voice play signaling) processed by the multimedia layer to the audio-visual device, but will not send non-audio-visual play signaling to the application layer. During the pre-sleep process, the multimedia layer is not affected and can continue to execute services (such as playing voice). During the pre-sleep process, the application layer no longer processes new non-audio-visual play signaling, but completes the preparation process before power-off. After completing the preparation process before power-off, the power-off of the audio-visual device is controlled according to whether there is audio-visual play signaling being processed in the controller and the multimedia layer. Obviously, such a power-off process solves the problem that audio-visual play cannot be performed through the audio-visual device during the power-off process of the audio-visual device.

[0145] So far, the description of the Figure 2 flowchart of the audio-visual play method ends.

[0146] After the multimedia layer in the audio-visual device of this application is started, the multiple signaling included in the instructions sent by the controller are divided into audio-visual play signaling and non-audio-visual play signaling through the multimedia layer, and the audio-visual play signaling used to indicate the playback of the target audio-visual is processed through the multimedia layer to play the target audio-visual. The non-audio-visual play signaling included in the instructions is temporarily cached in the multimedia layer. After the application layer is started, the cached non-audio-visual play signaling is processed through the application layer. It can be seen that this application uses the video device in the electronic device to replace the original controller for audio-visual play. Since the audio-visual device itself has a large storage space and strong processing performance, it can greatly improve the quality of audio-visual play.

[0147] Furthermore, the solution proposed in this application does not need to wait for the application layer in the audio-visual device to be started, but can perform audio-visual play after the multimedia layer is started, alleviating the problem of play delay caused by audio-visual play through the audio-visual device.

[0148] Specifically, the solution proposed in the embodiment of this application adds a controller preprocessing unit and a virtual controller unit in the multimedia layer of the audio-visual device, enabling the controller to directly communicate with the multimedia layer. As long as the multimedia layer is started, without waiting for the slower-starting application layer to be started, audio-visual play can be directly performed through the multimedia layer, which can take into account both the quality of audio-visual play and the audio-visual response speed, greatly improving the user experience.

[0149] Next, according to Figure 4 and Figure 5 the audio-visual play method proposed in the embodiment of this application will be described.

[0150] Please refer to Figure 4 , Figure 4 the interactive schematic diagram of the power-on startup process provided by the embodiments of the present application.

[0151] As Figure 4 shown, the leftmost column in the figure is the controller, and the two columns on the right in the figure are the multimedia layer and the application layer included in the audio-video device. The solid arrows indicate the transmission process of signaling, and the dashed arrows indicate the transmission process of the response returned to the controller.

[0152] In this embodiment, when the audio-video device is powered on, the multimedia layer and the application layer start to start. After the multimedia layer starts successfully, the instructions sent by the controller to the audio-video device are read through the controller preprocessing unit.

[0153] After the instructions sent by the controller are read, a plurality of signaling included in the instructions can be parsed through the controller preprocessing unit, and the plurality of signaling included in the instructions are divided into audio-video playback signaling and non-audio-video playback signaling.

[0154] Furthermore, the audio-video playback signaling that needs to be processed by the multimedia layer is forwarded to the audio-video processing unit in the multimedia layer for processing to execute the specified service (such as playing audio and video); at the same time, the non-audio-video playback signaling that needs to be processed by the application layer is cached in the virtual controller unit of the multimedia layer.

[0155] After the audio-video playback is completed, a first response indicating successful execution can be sent back to the controller. If the application layer has been started, the first response can be forwarded to the application layer, and the first response is sent to the virtual controller unit through the application layer, and further forwarded to the controller through the virtual controller unit.

[0156] It should be noted that the purpose of forwarding the first response to the application layer at this time, then sending the first response to the virtual controller unit through the application layer, and further forwarding the first response to the controller through the virtual controller unit is to uniformly process all the responses returned to the controller through the application layer and then forward them to the controller in sequence through the virtual controller unit, so that the signaling in the audio-video device is processed according to the correct timing relationship to avoid disorder of the response messages.

[0157] If the application layer has not been started, the first response can be directly forwarded back to the controller through the controller preprocessing unit in the multimedia layer.

[0158] After the application layer is started, cached non-audio / video playback signaling can be read from the virtual controller unit, and the non-audio / video playback signaling can be parsed and processed. After the processing is completed, a successful execution response can also be sent back to the controller, that is, the response information is sent to the virtual controller unit so that the virtual controller unit can forward the response information to the controller.

[0159] Meanwhile, for the controller response information actively triggered by the application layer, such as tasks executed by the audio / video device itself rather than controlled by instructions sent by the controller, such as the task of periodically capturing images, the controller response information can be actively triggered after one image capture is completed. The specific process of sending back the response information is the same as above and will not be elaborated here.

[0160] Thus, the description of Figure 4 the power-on startup process above is ended.

[0161] Please refer to Figure 5 , Figure 5 the interaction schematic diagram of the power-off sleep process provided by the embodiment of this application.

[0162] As Figure 5 shown, the leftmost column in the figure is the controller, and the right two columns in the figure are the multimedia layer and the application layer in the audio / video device.

[0163] The application layer detects whether the first sleep condition is met, that is, when the application layer detects that no non-audio / video playback signaling has been received within a specified duration, it queries the controller to check whether there is any non-audio / video playback signaling that needs to be processed by the application layer in the controller.

[0164] In the case where the controller returns that there is no non-audio / video playback signaling that needs to be processed by the application layer, the application layer can perform pre-sleep, prohibit processing the non-audio / video playback signaling to be processed, and start preparing for data storage before power-off.

[0165] During the pre-sleep process of the application layer, if the controller obtains an instruction to be processed, the controller can parse the multiple signaling included in the instruction, send the audio / video playback signaling included in the instruction to the multimedia layer for processing, cache the non-audio / video playback signaling locally in the controller, and after the audio / video device is powered on again next time, send the non-audio / video playback signaling cached locally in the controller to the multimedia layer and cache it in the virtual controller unit in the multimedia layer for reading and processing after the application layer is started.

[0166] After the application layer completes the preparation before power-off, that is, after completing pre-sleep, it can query the status of the multimedia layer, that is, determine whether the multimedia layer is currently processing audio / video playback signaling.

[0167] If it is determined that there is currently no audio - video playback signaling being processed in the multimedia layer, that is, the multimedia layer is in an idle state, then further obtain the status of the controller to determine whether there is any pending audio - video playback signaling in the controller. If there is also no pending audio - video playback signaling in the controller, then the controller can be notified to power down the audio - video device.

[0168] If the multimedia layer is not in an idle state, or there is pending audio - video playback signaling in the controller, then return to the step of obtaining the status of the multimedia layer until there is currently no audio - video playback signaling being processed in the multimedia layer and there is also no pending audio - video playback signaling in the controller. At this time, the audio - video device can be controlled to power down.

[0169] Thus far, the description of the Figure 5 power - down and sleep process ends.

[0170] In addition, this application also provides an electronic device.

[0171] Please refer to Figure 6 , Figure 6 which is a schematic structural diagram of an electronic device provided by an embodiment of this application.

[0172] As Figure 6 shown, the electronic device may include:

[0173] A controller, configured to send instructions to the audio - video device after the multimedia layer in the audio - video device is started. The instructions include multiple signaling.

[0174] An audio - video device, which includes a multimedia layer and an application layer. The multimedia layer and the application layer are started when the audio - video device is powered on. The start - up duration of the multimedia layer is less than that of the application layer. The multimedia layer includes a controller pre - processing unit, an audio - video processing unit, and a virtual controller unit.

[0175] Among them, the controller pre - processing unit is configured to divide the multiple signaling included in the instructions sent by the controller into audio - video playback signaling and non - audio - video playback signaling. The audio - video playback signaling refers to the signaling used to indicate playing the target audio - video.

[0176] The audio - video processing unit is configured to process the audio - video playback signaling to play the target audio - video.

[0177] The virtual controller component is configured to cache the non - audio - video playback signaling.

[0178] The application layer is configured to, after completion of start - up, read the non - audio - video playback signaling cached in the virtual controller component and process the non - audio - video playback signaling to perform the operations indicated by the non - audio - video playback signaling.

[0179] As an embodiment,

[0180] After processing non-audio / video playback signaling at the application layer to perform the operations indicated by the non-audio / video playback signaling, the application layer is further configured to:

[0181] Determine whether a first sleep condition is satisfied; satisfying the first sleep condition includes: the application layer has not received non-audio / video playback signaling within a specified duration, and there is no non-audio / video playback signaling in the controller that requires processing by the application layer;

[0182] When the first sleep condition is satisfied, start pre-sleep, prohibit the application layer from processing pending non-audio / video playback signaling, and store the processing results of the audio / video playback signaling processed by the audio / video processing unit and the non-audio / video playback signaling processed by this application layer; the pending non-audio / video playback signaling refers to the non-audio / video playback signaling obtained by the controller after the application layer starts pre-sleep;

[0183] Send a pre-sleep request to the controller, so that after receiving the pre-sleep request, the controller divides the obtained signaling into pending audio / video playback signaling and pending non-audio / video playback signaling, caches the pending non-audio / video playback signaling, and sends the pending audio / video playback signaling to the controller preprocessing unit, so that the controller preprocessing unit forwards the pending audio / video playback signaling to the audio / video processing unit, and the audio / video processing unit processes the pending audio / video playback signaling to play the target audio / video indicated by the pending audio / video playback signaling; the pending audio / video playback signaling refers to the audio / video playback signaling obtained by the controller after the application layer starts pre-sleep;

[0184] And / or, after the application layer starts pre-sleep, the application layer is further configured to:

[0185] After completing pre-sleep, determine whether a second sleep condition is satisfied; completing pre-sleep means that the application layer has completed the storage operation of the processing results of the audio / video playback signaling processed in the multimedia layer and the non-audio / video playback signaling processed in the application layer; satisfying the second sleep condition includes: the current state of the multimedia layer is the idle state, and there is no pending audio / video playback signaling in the controller, and the idle state is used to indicate that there is no audio / video playback signaling being processed in the multimedia layer currently; the pending audio / video playback signaling refers to the audio / video playback signaling obtained by the controller after the application layer starts pre-sleep;

[0186] When the second sleep condition is satisfied, notify the controller to control the power-down of the audio / video device to prohibit all signaling from being processed at the application layer and the multimedia layer.

[0187] In this embodiment, the functions of each device and unit in the electronic device have been described in detail above, and will not be elaborated here.

[0188] So far, the description of Figure 6 ends here.

[0189] Correspondingly, an embodiment of the present application further provides a computer-readable storage medium. A number of computer instructions are stored on the computer-readable storage medium. When the computer instructions are executed, the methods disclosed in the above examples of the present application can be implemented.

[0190] Exemplarily, the above computer-readable storage medium can be any electronic, magnetic, optical or other physical storage device that can contain or store information such as executable instructions, data, etc. For example, the computer-readable storage medium can be: RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, storage drives (such as hard disk drives), solid state drives, any type of storage disk (such as optical discs, DVDs, etc.), or similar storage media, or a combination thereof.

[0191] The above are only the preferred embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the scope of protection of the present application.

Claims

1. An audio-video playing method, characterized in that, The method is applied to an audio - video device in an electronic device, and the electronic device further includes a controller. The method includes: When the audio - video device is powered on, starting a multimedia layer and an application layer in a software architecture supported by the audio - video device; the startup duration of the multimedia layer is less than that of the application layer; the software architecture is designed based on the separation of multimedia services and application logic; when the multimedia layer finishes starting up, it indicates that the audio - video device has the audio - video playback ability; After the multimedia layer finishes starting up, dividing the signaling sent by the controller by the multimedia layer into audio - video playback signaling and non - audio - video playback signaling; the audio - video playback signaling refers to the signaling used to indicate playing a target audio - video; Processing the audio - video playback signaling in the multimedia layer to play the target audio - video, and caching the non - audio - video playback signaling in the multimedia layer; After the application layer finishes starting up, reading the non - audio - video playback signaling cached in the multimedia layer to the application layer as needed, so as to process the non - audio - video playback signaling in the application layer to perform the operations indicated by the non - audio - video playback signaling.

2. The method according to claim 1, wherein After processing the audio - video playback signaling in the multimedia layer to play the target audio - video, the method further includes: After the target audio - video finishes playing, detecting whether the application layer has finished starting up; If so, forwarding a first response from the multimedia layer to the application layer, so that the application layer forwards the first response to the controller; the first response is used to indicate that the target audio - video has finished playing; If not, forwarding the first response from the multimedia layer to the controller.

3. The method according to claim 1, characterized in that, After processing the non - audio - video playback signaling in the application layer to perform the operations indicated by the non - audio - video playback signaling, the method further includes: Judging whether a first sleep condition is satisfied; When the first sleep condition is satisfied, controlling the application layer to start pre - sleeping, so as to prohibit processing the unprocessed non - audio - video playback signaling in the application layer, and storing the processing results of the audio - video playback signaling processed in the multimedia layer and the non - audio - video playback signaling processed in the application layer; the unprocessed non - audio - video playback signaling refers to the non - audio - video playback signaling obtained by the controller after the application layer starts pre - sleeping; Sending a pre - sleep request to the controller, so that after receiving the pre - sleep request, the controller divides the obtained signaling into unprocessed audio - video playback signaling and unprocessed non - audio - video playback signaling, caches the unprocessed non - audio - video playback signaling, and sends the unprocessed audio - video playback signaling to the multimedia layer to process the unprocessed audio - video playback signaling in the multimedia layer and play the target audio - video indicated by the unprocessed audio - video playback signaling; the unprocessed audio - video playback signaling refers to the audio - video playback signaling obtained by the controller after the application layer starts pre - sleeping.

4. The method according to claim 3, characterized in that, The satisfaction of the first sleep condition includes: The application layer has not received non-audio / video playback signaling within a specified duration, and there is currently no non-audio / video playback signaling in the controller that requires processing by the application layer.

5. The method according to claim 3, wherein After the application layer starts pre-sleeping, the method further includes: After the application layer completes pre-sleeping, determine whether a second sleep condition is satisfied; the application layer completing pre-sleeping means that the application layer has completed the storage operation of the processing results of the audio / video playback signaling processed in the multimedia layer and the non-audio / video playback signaling processed in the application layer. When the second sleep condition is satisfied, notify the controller to power down the audio / video device to prohibit all signaling from being processed in the application layer and the multimedia layer.

6. The method according to claim 5, wherein The satisfaction of the second sleep condition includes: The current state of the multimedia layer is the idle state, and there is currently no pending audio / video playback signaling in the controller; the idle state is used to indicate that there is currently no audio / video playback signaling being processed in the multimedia layer.

7. The method according to claim 1, wherein Each signaling sent by the controller is configured with a type identifier; the dividing of the signaling sent by the controller into audio / video playback signaling and non-audio / video playback signaling in the multimedia layer includes: Parsing the signaling sent by the controller through the multimedia layer, and dividing the signaling sent by the controller into audio / video playback signaling and non-audio / video playback signaling according to the type identifier carried by each signaling.

8. The method according to claim 1, wherein The electronic device is a low-power device, the audio / video device is the audio / video device in the low-power device, and the audio / video device is powered on when there is signaling that requires processing by the audio / video device.

9. An electronic device, characterized in that, Including: A controller for sending signaling that requires processing by the audio / video device to the audio / video device after the multimedia layer in the audio / video device completes startup. An audio / video device, the software architecture supported by the audio / video device includes a multimedia layer and an application layer. The multimedia layer and the application layer start when the audio / video device is powered on. The startup duration of the multimedia layer is less than the startup duration of the application layer. The software architecture is designed based on the separation of multimedia services and application logic; the completion of the startup of the multimedia layer indicates that the audio / video device has the voice playback ability; the multimedia layer is configured with a controller preprocessing unit, an audio / video processing unit, and a virtual controller unit. Wherein, the controller preprocessing unit is used to divide the signaling sent by the controller into audio / video playback signaling and non-audio / video playback signaling. The audio / video playback signaling is the signaling used to indicate playing the target audio / video. The audio / video processing unit is used to process the audio / video playback signaling to play the target audio / video. The virtual controller component is used to cache the non-audio / video playback signaling. The application layer is used to, after completing startup, read the cached non-audio / video playback signaling in the virtual controller component to the application layer as needed, and process the non-audio / video playback signaling in the application layer to perform the operations indicated by the non-audio / video playback signaling.

10. The electronic device according to claim 9, wherein After processing the non-audio / video playback signaling at the application layer to perform the operations indicated by the non-audio / video playback signaling, the application layer is further configured to: Determine whether a first sleep condition is met; the satisfaction of the first sleep condition includes: the application layer has not received non-audio / video playback signaling within a specified duration, and there is currently no non-audio / video playback signaling in the controller that requires processing by the application layer; In the case where the first sleep condition is met, start pre-sleep, prohibit the application layer from processing the non-audio / video playback signaling to be processed, and store the processing results of the audio / video playback signaling already processed by the audio / video processing unit and the non-audio / video playback signaling already processed by this application layer; the non-audio / video playback signaling to be processed refers to the non-audio / video playback signaling obtained by the controller after the application layer starts pre-sleep; Send a pre-sleep request to the controller, so that after receiving the pre-sleep request, the controller divides the obtained signaling into the audio / video playback signaling to be processed and the non-audio / video playback signaling to be processed, caches the non-audio / video playback signaling to be processed, and sends the audio / video playback signaling to be processed to the controller preprocessing unit, so that the controller preprocessing unit forwards the audio / video playback signaling to be processed to the audio / video processing unit, and the audio / video processing unit processes the audio / video playback signaling to be processed to play the target audio / video indicated by the audio / video playback signaling to be processed; the audio / video playback signaling to be processed refers to the audio / video playback signaling obtained by the controller after the application layer starts pre-sleep; And / or, after the application layer starts pre-sleep, the application layer is further configured to: After completing pre-sleep, determine whether a second sleep condition is met; the completion of pre-sleep means that the application layer has completed the storage operation of the processing results of the audio / video playback signaling already processed in the multimedia layer and the non-audio / video playback signaling already processed in the application layer; the satisfaction of the second sleep condition includes: the current state of the multimedia layer is the idle state, and there is currently no audio / video playback signaling to be processed in the controller, and the idle state is used to indicate that there is currently no audio / video playback signaling being processed in the multimedia layer; the audio / video playback signaling to be processed refers to the audio / video playback signaling obtained by the controller after the application layer starts pre-sleep; In the case where the second sleep condition is met, notify the controller to control the power-down of the audio / video device to prohibit the application layer and the multimedia layer from processing all signaling.

Images