Media file playing method and electronic equipment

Abstract

The invention discloses a media file playing method and electronic equipment. The electronic equipment can provide the simple playing function interface for each application of the application layer, and the application can directly call the playing function interface to obtain the playing function provided by the operating system of the electronic equipment. According to the scheme, the operating system unifies the playing rule of the media file, and convenient and safe media file playing experience can be provided.

Description

Technical Field

[0001] This application relates to the technical field of terminals, and in particular, to a method for playing media files and an electronic device. Background Art

[0002] The playback of media files such as audio and video is a key function of electronic devices such as mobile phones, tablet computers, and laptop computers. How to provide high-quality media file playback services and give users a good experience is the current research direction. Summary of the Invention

[0003] This application provides a method for playing media files and an electronic device, which can provide a convenient and secure media file playback experience.

[0004] In a first aspect, a method for playing a media file is provided. The method is applied to a first electronic device on which a first operating system is running. The method may include: running a first application on the first operating system, where the first application is installed on the first electronic device. The first operating system provides: a first interface and a media data interface; the first application is a third-party application, and the first application includes program code for calling the first interface. The first interface is used to call the media data interface, and the media data interface includes the ability to perform one or more of the following processes on a first media file: a first process, a second process, and a third process; where the first process includes one or more of the following processes: setting the data format of the first media file, managing the audio session; the audio session is used to regulate the behavior of the playback function for playing the first media file; the second process includes one or more of the following processes: setting a policy for playing the first media file on the first electronic device, setting a policy for playing the first media file on a device other than the first electronic device; the third process includes one or more of the following processes: setting a policy for playing multiple media files, setting a policy for playing the first media file in a multi-device collaboration manner, where the multiple media files include the first media file; detecting a user operation to start the playback function of the first application, where the playback function is used to trigger the playback of the first media file; the first electronic device plays the first media file, or triggers a second electronic device to play the first media file.

[0005] Through the method of the first aspect, the electronic device can provide a simple first interface to each application in the application layer. The application can directly call the first interface to obtain the playback function provided by the operating system of the electronic device. In this way, the operating system of the electronic device provides unified playback rules for media files, which can ensure consistent media file playback effects for each application.

[0006] In combination with the first aspect, in some embodiments, when the first electronic device plays the first media, the first application may first call the first interface, and then the first interface calls the media data interface. The media data interface then selects the capabilities provided by one or more of the first processing, second processing, and third processing to play the first media file.

[0007] In combination with the first aspect, in some embodiments, when the first electronic device triggers the second electronic device to play the first media file, the first application in the first electronic device first calls the first interface, then the first interface calls the media data interface, and the media data interface then calls the media data interface of the second electronic device. Then, the media data interface of the second electronic device calls the capabilities provided by one or more of the first processing, second processing, and third processing in the second electronic device, so that the second electronic device plays the first media file.

[0008] In combination with the first aspect, in some embodiments, the parameters passed into the first interface may include: the device name of the first electronic device, the application name of the first application, and the parameters passed in by the first application. Among them, the parameters passed in by the first application are in the form of key-value pairs. The key represents the parameter for playing the first media file, and the value represents the content of the parameter for playing the first media file.

[0009] In combination with the first aspect, in some embodiments, the parameters passed into the first interface further include: the first device identifier. When the first device identifier indicates the first electronic device, the first electronic device plays the first media file; when the first device identifier indicates the second electronic device, the first electronic device triggers the second electronic device to play the first media file. Equivalently, the first application can pass into the first interface the identifier of the device used to play the first media file.

[0010] In combination with the first aspect, in some embodiments, the parameters passed into the first interface further include: the first application identifier. The first electronic device plays the first media file through the application indicated by the first application identifier, or triggers the second electronic device to play the first media file through the application indicated by the first application identifier. Equivalently, the first application can pass into the first interface the identifier of the application used to play the first media file.

[0011] In combination with the first aspect, in some embodiments, the method for playing a media file provided in this application can achieve playback within the same application or across applications in the same electronic device, and cross-device playback. For example, the first media file belongs to the first application, and the first application identifier indicates the first application; or, the first media file belongs to the second application, and the first application identifier indicates the first application; or, the first media file belongs to the first application, and the first application identifier indicates the second application.

[0012] In combination with the first aspect, in some embodiments, before playing the first media file, the first electronic device may convert the first media file into a first data format, or, before triggering the second electronic device to play the first media file, the first electronic device converts the first media file into the first data format and sends the first media file in the first data format to the second electronic device. The first data format is a unified data format provided for unified access management. This can unify the format of media files and ensure the successful playback of media files.

[0013] In combination with the first aspect, in some embodiments, the first electronic device may create a first audio session, where the first audio session is a structure for saving the state of the first media file; set parameters of the first audio session, and the parameters of the first audio session include one or more of the following for the first media file: the title, author, content, playing duration, playing volume, playing mode, the name of the device playing the first media file, the name of the application playing the first media file, the playing mode after the device screen is turned off, and the playing mode when the application is moved to the background; play the first media file according to the first audio session.

[0014] In combination with the first aspect, in some embodiments, the first electronic device may also manage the lifecycle of the first audio session. For example, when the number of applications using the first audio session is 0, destroy the first audio session; when the survival time of the first audio session reaches a first duration, destroy the first audio session; when the first audio session is not in use, destroy the first audio session. The first duration can be preset and is not limited here.

[0015] In combination with the first aspect, in some embodiments, the playback of the first media file follows the 0-layer rules in unified area management.

[0016] For example, the second electronic device is the device with the highest priority among the devices connected to the first electronic device, or the device with the highest priority that is newly connected to the first electronic device; or, the first media file is telephone audio based on a circuit-switched network or a mobile communication network; or, after triggering the second electronic device to play the first media file, the first electronic device may detect a user operation of adjusting the volume and send an instruction to adjust the volume to the second electronic device.

[0017] In some embodiments, the device priorities from high to low are: the device selected by the user, hearing aid, wearing-detection type Bluetooth headset, open earphone, wired earphone, Bluetooth headset / Bluetooth car stereo / Bluetooth speaker, PC / PAD / smart screen / HiCar device, external speaker of earpiece / speaker / watch.

[0018] For another example, when triggering the second electronic device to play the first media file, the first electronic device also plays the second media file. In some embodiments, the media sound and TTS in the screen mirroring scenario are played on the screen mirroring device, and the remaining audio is played locally on the electronic device. For example, the first media file is a media file or TTS, and the second media file is a media file other than the media file or TTS.

[0019] In combination with the first aspect, in some embodiments, the playback of the first media file follows the layer 1 rule in the unified area management. For example, when triggering the second electronic device to play the first media file, the second electronic device is a device that cannot connect to the network; or, the first media file is a ringtone or alarm sound, and the first electronic device also plays the first media file. In some embodiments, the first media file is a media file or TTS, and the second media file is a media file other than the media file or TTS.

[0020] In combination with the first aspect, in some embodiments, the playback of the first media file follows the focus management rule in the unified device management. One is the preemption strategy of short focus. For example, after the first electronic device plays the first media file, it starts to play the third media file and pauses the playback of the first media file; after finishing playing the third media file, it continues to play the first media file. One is the preemption strategy of long focus. For example, after the first electronic device plays the first media file, it starts to play the third media file and stops the playback of the first media file; after finishing playing the third media file, it remains stopped from playing the first media file. One is the occupation of temporary focus. For example, after the first electronic device plays the first media file, it plays the third media file and plays the first media file simultaneously; after finishing playing the third media file, it continues to play the first media file.

[0021] In combination with the first aspect, in some embodiments, the playback of the first media file follows the concurrency strategy in the unified device management. For example, when triggering the second electronic device to play the first media file, the first electronic device can also play the second media file; the first electronic device stops playing the second media file; then, the first electronic device instructs the second electronic device to pause the playback of the first media file, and the first electronic device continues to play the first media file.

[0022] In a second aspect, an electronic device is provided, including: a memory, a processor, and a computer program stored on the memory. The computer program includes a first operating system. A first application is installed on the first electronic device. The first operating system provides: a first interface, a media data interface; the first application is a third-party application, and the first application includes program code for calling the first interface. The first interface is used to call the media data interface. The media data interface includes the ability to perform one or more of the following processes on a first media file: a first process, a second process, a third process; wherein, the first process includes one or more of the following processes: setting the data format of the first media file, managing an audio session; the audio session is used to regulate the behavior of the playback function for playing the first media file; the second process includes one or more of the following processes: setting a policy for playing the first media file on the first electronic device, setting a policy for playing the first media file on a device other than the first electronic device; the third process includes one or more of the following processes: setting a policy for playing multiple media files, setting a policy for playing the first media file in multi-device collaboration, and the multiple media files include the first media file; the processor executes the computer program to implement the method provided in the first aspect or any one of the implementation manners of the first aspect.

[0023] In a third aspect, a computer-readable storage medium is provided, on which a computer program is stored. The computer program includes a first operating system. A first application is installed on the first electronic device. The first operating system provides: a first interface, a media data interface; the first application is a third-party application, and the first application includes program code for calling the first interface. The first interface is used to call the media data interface. The media data interface includes the ability to perform one or more of the following processes on a first media file: a first process, a second process, a third process; wherein, the first process includes one or more of the following processes: setting the data format of the first media file, managing an audio session; the audio session is used to regulate the behavior of the playback function for playing the first media file; the second process includes one or more of the following processes: setting a policy for playing the first media file on the first electronic device, setting a policy for playing the first media file on a device other than the first electronic device; the third process includes one or more of the following processes: setting a policy for playing multiple media files, setting a policy for playing the first media file in multi-device collaboration, and the multiple media files include the first media file; when the computer program is executed by the processor, the method provided in the first aspect or any one of the implementation manners of the first aspect is implemented.

[0024] Fourthly, a computer program product is provided. The computer program product includes a computer program which includes a first operating system. A first electronic device is installed with a first application. The first operating system provides: a first interface, a media data interface, a first capability, a second capability, and a third capability. The first application is a third-party application, and the first application includes program code for calling the first interface. The first interface is used to call the media data interface. The media data interface includes the capability to perform one or more of the following processes on a first media file: a first process, a second process, and a third process. Among them, the first process includes one or more of the following processes: setting the data format of the first media file, managing the audio session. The audio session is used to regulate the behavior of playing the first media file by the playback function. The second process includes one or more of the following processes: setting the policy for playing the first media file on the first electronic device, setting the policy for playing the first media file on a device other than the first electronic device. The third process includes one or more of the following processes: setting the policy for playing multiple media files, setting the policy for playing the first media file in multi-device collaboration. The multiple media files include the first media file. When the computer program is executed by a processor, it implements the method provided in the first aspect or any implementation manner of the first aspect. Description of the Drawings

[0025] Figure 1A It is a software architecture diagram of the electronic device provided in the embodiment of the present application;

[0026] Figure 1B It is a specific example of a playback application calling a function interface provided by a unified media framework in the embodiment of the present application;

[0027] Figure 2 It is a set of user interfaces for controlling audio playback through the playback control center of the electronic device provided in the embodiment of the present application;

[0028] Figure 3 It is the architecture related to audio playback in the software structure of the electronic device provided in the embodiment of the present application;

[0029] Figure 4 It is an example of an audio playback scenario provided in the embodiment of the present application;

[0030] Figure 5 It is an example of audio migration with the user in a distributed collaboration scenario provided in the embodiment of the present application;

[0031] Figure 6 It is an example of local peripherals and collaborative devices provided in the embodiment of the present application;

[0032] Figure 7 It is the priority order when selecting an audio output device provided in the embodiment of the present application;

[0033] Figure 8 Rules for each device provided by the embodiments of the present application to output audio;

[0034] Figure 9A Audio playback routing rules for screen mirroring provided by the embodiments of the present application;

[0035] Figure 9B Audio playback routing rules in a collaborative scenario provided by the embodiments of the present application;

[0036] Figure 10 Audio playback rules in different scenarios provided by the embodiments of the present application;

[0037] Figure 11 Examples of applying for a long focus provided by the embodiments of the present application;

[0038] Figure 12 Examples of applying for a short focus provided by the embodiments of the present application;

[0039] Figure 13 Examples of applying for a temporary focus provided by the embodiments of the present application;

[0040] Figure 14 Single-task type migration scenarios provided by the embodiments of the present application;

[0041] Figure 15 Screen sharing type migration scenarios provided by the embodiments of the present application;

[0042] Figure 16 Flowchart of a method for playing a media file provided by the embodiments of the present application;

[0043] Figure 17 Hardware structure block diagram of an electronic device provided by the embodiments of the present application. Detailed implementation manners

[0044] Electronic devices such as mobile phones, tablet computers, and laptop computers all provide a file playback function. Files can also be referred to as media files. Media files can be divided into multiple categories, for example, they may include audio files, video files, audio-video files, picture files, text files, etc. Audio files may include audio data, video files include multiple frames of image data, audio-video files include both audio data and image data at the same time, picture files may include picture data, and text files may include text data.

[0045] An application (APP) in an electronic device can control the audio playback rules of the files it plays. Other APPs or applets embedded in the APP can also control the audio playback rules of the files they play. This can make it difficult for the electronic device to control the playback of each APP. For example, some third-party applications may keep themselves alive by playing media files with a volume of 0 in the background, resulting in uncontrollable playback of third-party applications, and users cannot even know which applications are playing files currently.

[0046] In addition, when multiple devices work together, and when an electronic device plays multiple audio files, there may be various application scenarios and various audio playback rules. These audio playback rules are diverse and not unified, which is not user-friendly.

[0047] This application provides a method for playing media files. The operating system (OS) of the electronic device can provide unified audio playback rules for each application in the electronic device, giving users a consistent audio usage experience.

[0048] The file playback method provided by this application can be applied to any file playback scenario, mainly applicable to the following file playback scenarios:

[0049] 1. File playback scenarios within an application, such as scenarios where an application plays files on the first-layer interface, on the second-layer interface, and on deeper-layer interfaces. The first-layer interface is usually the interface of the application itself, and the second-layer interface and deeper-layer interfaces may be the interfaces of other applications embedded in the application. Using the method provided by this application, the audio playback rules of the above-mentioned each layer of interface are no longer controlled by each application itself, but are uniformly controlled by the OS of the electronic device.

[0050] 2. File playback scenarios between applications, such as scenarios where two applications play files simultaneously. When two or more applications play files simultaneously, their audio playback rules are uniformly controlled by the OS of the electronic device.

[0051] 3. File playback scenarios between devices, such as file playback scenarios after an electronic device is connected to other electronic devices. For example, after a mobile phone is connected to devices such as a Bluetooth headset, a Bluetooth speaker, and an intelligent cockpit, the OS of the mobile phone can uniformly control the audio playback rules of the files on the mobile phone, such as which connected device emits sound, etc.

[0052] This application provides a software architecture for an electronic device, which can avoid the above deficiencies.

[0053] The electronic device provided by the embodiments of this application can be configured with an operating system (OS), and the OS can be one of etc.

[0054] Figure 1A It shows the software structure of the OS of the electronic device provided by the embodiment of the present application, and this software structure includes a unified media framework.

[0055] As Figure 1A shown, the electronic device can include the following four layers from top to bottom: application layer, framework layer, service layer, and kernel layer. The higher the layer, the more interactions with the user; the lower the layer, the more system capabilities it represents.

[0056] The application layer includes a series of user-oriented applications (application, APP), for example, it can include a recorder, a camera, a gallery, a video application, a music playback application, a broadcast application, etc. Each of the above applications can provide a user interface for the user to view and input user operations, so that the electronic device can respond to these user operations to generate new files or view existing files. For example, an audio playback application can be used to play audio files, a video application can be used to play video files, a recorder can be used to collect audio data and generate audio files, and a camera application can be used to collect audio and video data and generate audio and video files.

[0057] The applications included in the application layer can be system applications or third-party applications. System applications include applications configured to make the electronic device work properly and maintain the OS, and third-party applications can include applications developed by other developers outside the electronic device manufacturer.

[0058] The framework layer can also be called the program framework layer, which is used to provide application programming interfaces (application programming interface, API) and programming frameworks for the applications in the application layer. The program framework layer can include some predefined functions. The framework layer can also include modules such as a window manager, a content provider, a view system, a telephone manager, a resource manager, etc., which will not be introduced in detail here. The framework layer is used to connect the application layer and the service layer.

[0059] As Figure 1AAs shown in the figure, the framework layer may include, from top to bottom: a playback function interface (PlayMode). The playback function interface can be directly called by an application in the application layer. The playback function interface is used to provide a playback function. The playback function interface can be an interface (PlayModeKit) or a selector (PlayModePicker). Among them, multiple playback function interfaces can be encapsulated in PlayModePicker. One playback function interface can be used to start one playback function, and this playback function can be one of an audio playback function, a video playback function, an audio-video playback function, etc. PlayModePicker can select the corresponding interface from multiple playback function interfaces according to the parameters passed in by the application. When different playback function interfaces are called, the application can pass different parameters to the corresponding interfaces.

[0060] For example, when the electronic device 100 starts an audio-video playback function through an application with a playback function, such as a video application, the video application can call PlayModePicker and pass the parameters passed in by the application. Then, through this parameter, PlayModePicker can select the audio-video playback function interface to implement the playback function of the video application.

[0061] It can be seen that the framework layer can provide a unified playback interface for multiple applications in the application layer, enabling these multiple applications to implement the same function. For example, both the system music application and the third-party music application can call PlayMode to implement the playback function. Compared with different applications calling different interfaces, it solves the problem of chaotic interfaces called during the operation of the application.

[0062] As Figure 1A shown in the figure, the application framework layer may further include: a media data interface (MediaDataKit). The media data interface can be used to be called by the PlayMode interface to control the playback of media files.

[0063] As Figure 1AAs shown in the figure, the application framework layer may further include the following capabilities: interfaces such as unified access management, unified area management, unified device management, and high-performance framework management. Among them, the unified access management is used to set the data format of the first media file and manage the audio session; the unified area management is used to set the policy for playing the first media file on the first electronic device and the policy for playing the first media file on devices other than the first electronic device; the unified device management is used to set the policy for playing multiple media files and the policy for playing the first media file in a multi-device collaboration. The high-performance framework management provides the overall system management capabilities for the above three management modules to support the personalized capabilities of the above three management modules, and the high-performance framework management can also improve performance and reduce power consumption. The specific functions of the above management modules will be introduced in detail later.

[0064] This application also provides a unified media framework, which may include the above-mentioned several modules in the framework layer and is used to provide unified file playback capabilities for applications in the application layer. The unified media framework of this application can be implemented as a system capability, or can be implemented as a software development kit (SDK), a resident service, a binary shared object (SO) file, etc. Among them, the resident service can be regarded as a resident process and is always on after the electronic device is powered on. The unified media framework can provide corresponding interfaces for each application (such as a third-party application) to call, so that these applications can use the functions provided by the unified media framework and achieve unified management of file playback in each application and each device through the unified media framework. The OS provides a unified PlayMode interface, and the application directly calls the PlayMode interface without caring about the calling method of the underlying hardware and / or algorithm. The OS can directly help the application implement the required playback function. Such a calling method is more friendly to the application, shields the complex underlying hardware and algorithms, and can simplify the programming operations of the application developer.

[0065] As Figure 1A As shown in the figure, the service layer provides one or more service layer interfaces, and the service layer interfaces can encapsulate one or more kernel layer interfaces provided by the kernel layer. Specifically, the service layer may include the following several modules: docking with the unified access management upward to provide an access management service for the business logic of APP access; docking with the unified area management upward to provide an area management service for the business logic of file playback in different areas; docking with the unified device management upward to provide a device management service for the business logic of file playback between different devices; docking with the high-performance framework management upward to provide a performance optimization service for the business logic of performance optimization. The above several service modules can be regarded as service layer interfaces. In some embodiments, the above several modules of the service layer may also be moved up to the framework layer.

[0066] The kernel layer is the layer between hardware and software. As Figure 1A shown, the kernel layer may include a microphone driver, a speaker driver, and other categories of audio drivers, and may also include some underlying algorithms related to file playback. These algorithms may include, for example, but are not limited to, decoding algorithms, demultiplexing algorithms, audio rendering algorithms, video rendering algorithms, audio-video synchronization algorithms, etc.

[0067] As the connection layer between software and hardware, the kernel layer can be compatible with different hardware devices. The models or types of processing chips used in different electronic devices may be different, that is, there are differences in hardware. In the electronic device OS provided by the embodiments of the present application, the kernel layer may provide one or more kernel layer interfaces, and these one or more kernel layer interfaces encapsulate underlying capabilities, including playback capabilities. These underlying capabilities specifically include two types of capabilities: basic capabilities and differentiated capabilities. Basic capabilities refer to capabilities common to most platforms, and differentiated capabilities refer to differentiated capabilities unique to each platform rather than shared. Here, the platform refers to hardware devices such as chips. The kernel layer can provide driver programs and hardware access interfaces for invoking basic capabilities, and can also provide driver programs and hardware access interfaces for invoking differentiated capabilities, etc. In this way, no matter which chips the electronic device loads, the OS of the electronic device is ready to invoke the capabilities of these chips, achieving compatibility. The underlying capabilities encapsulated by the above-mentioned kernel layer interfaces include playback capabilities. The kernel layer implements a compatible southbound ecological interface design, provides southbound interfaces for docking hardware devices, and simplifies the docking cost.

[0068] Figure 1A The functions of the provided modules have been clearly introduced above, and their names do not constitute a limitation. The above-mentioned modules can also be called other names.

[0069] Figure 1A The software architecture of the electronic device shown may also include more or fewer modules, which are not limited here. For example, the application layer may also include playback applications for text data such as notepads and readers.

[0070] Figure 1A The software architecture of the electronic device shown is a new playback architecture, realizing a four-layer architecture implementation of the playback function.

[0071] Based on Figure 1A the playback architecture shown, the specific implementation of the playback function in the electronic device may include:

[0072] Taking an audio playback application as an example, after the electronic device starts the audio playback application, it can receive a user operation input by the user to play a certain piece of music. Then, the audio playback application at the application layer will issue an instruction to the framework layer, the framework layer will issue the instruction to the service layer, and the service layer will issue the instruction to the kernel layer, triggering the kernel layer to start the audio driver, thereby starting the corresponding audio device, such as speakers and devices such as headphones and speakers connected to the electronic device.

[0073] Figure 1B An exemplary specific example of a playback application calling a function interface provided by the unified media framework is shown.

[0074] Such as Figure 1B As shown, after the playback application receives a user operation to play a certain media file, it calls the PlayMode interface, and the PlayMode interface then calls MediaDataKit. MediaDataKit determines that the operation to be performed on the media data is "play" according to PlayMode. After that, MediaDataKit selects and calls one or more of unified access management, unified area management, and unified device management according to the parameters passed in by the playback application to determine the playback rules for the media file, including on which device to play and through which application to play. After that, the electronic device will play the media file according to the determined playback rules.

[0075] Among them, the form of the PlayMode interface is the PlayMode(deviceName, appName, key, value, extend) interface. The PlayMode interface includes the following parameters: the name of the device that calls the PlayMode interface (deviceName), the name of the application that calls the PlayMode interface (appName), the indication information of the parameters for playing the media file (key), the content of the parameters for playing the media file (value), and the extended parameter (extend).

[0076] The device that calls the PlayMode interface can be the local device or a remote device connected to the local device. The application that calls the PlayMode interface can be an application on the local device or an application on the remote device. The parameters for playing a media file may include the type, size, duration, and playing position (such as playing from the beginning or from the middle) of the playing content, etc. The indication information of the parameters for playing a media file may include parameter names, identifiers, numbers, indexes, etc., and the parameter content may refer to specific parameter values. A key and a value form a key-value pair, and a PlayMode interface may include one or more key-value pairs. For example, multiple keys and values can indicate that the music name (musicName) is the song "My Motherland", the music type, size is 70M, duration is 3 min 55 s, and play from 0 (indicating playing from the beginning), etc.

[0077] In some embodiments, deviceName, appName, key, value, etc. can be transmitted from an application in the application layer to the corresponding PlayMode interface in the framework layer. Among them, deviceName and appName can be determined by user operations or by the application that calls the PlayMode interface in the electronic device, and key and value can be determined by the application that calls the PlayMode interface. In other embodiments, if the application in the application layer does not transmit parameters such as deviceName, appName, key, value, etc., the OS of the electronic device can also default to filling deviceName with the name of the local end of the electronic device, filling appName with the name of the application that calls the PlayMode interface, and using default keys and values to fill into the PlayMode interface to implement the playback function corresponding to the PlayMode interface.

[0078] The parameters transmitted from the application in the application layer to the PlayMode interface may further include: the identifier of the device for playing the media file. If the device for playing the media file is the local electronic device, the electronic device plays the media file locally. If the device for playing the media file is a device other than the local electronic device, after the MediaDataKit interface is called, the parameters it receives are transmitted to the MediaDataKit interface of the other electronic device, and the MediaDataKit interface of the other device then continues to call its own modules layer by layer, such as one or more of unified access management, unified area management, unified device management, and high-performance framework management, to meet the audio playback policy provided by the OS. After that, the MediaDataKit interface in the framework layer of the other electronic device can continue to call the service layer interface and the kernel layer interface to implement the playback function. In other embodiments, the device for playing the media file may not be determined by the application, but may also be determined after the MediaDataKit calls several management capabilities provided by the framework layer.

[0079] The parameters transmitted from the application in the application layer to the PlayMode interface may further include: the identifier of the application for playing the media file. The application for playing the media file may be the first application or other applications.

[0080] The media data interface (MediaDataKit) can be implemented as MediaDataKit(type, key1, value1, key2, value2, extend). It includes the following parameters:

[0081] Type (type), indicating the type to which the operation currently performed by the electronic device belongs. The type may include, but is not limited to, the following: general type (normal), acquisition, encoding, storage, sharing, decoding, playback, other, etc. The type can be determined according to the previous interface that calls the MediaDataKit interface. For example, if the previous interface is the PhotoMode interface, the electronic device can determine that encoding needs to be performed currently, so it is regarded as the corresponding encoding type; for example, if the previous interface is the EditMode interface, the electronic device can determine that decoding needs to be performed currently, so it is regarded as the corresponding decoding type; for example, if the previous interface is the PlayMode interface, the electronic device can determine that a playback action needs to be performed currently, so it is regarded as the corresponding playback type.

[0082] The indication parameter (key) of the function point is used to indicate the function point to be launched. Multiple different function points can be included under each type. For example, for the playback type, there can be different playback methods, etc. The indication parameter (key) of the function point is the key in the previous interface for calling the MediaDataKit interface. For example, if the previous interface is the PlayMode interface, the indication parameter (key) of the function point is the key in the PlayMode interface.

[0083] The value (value) required for the function point is used to indicate the parameter value required to implement the function point. The value (value) required for the function point is the value value of the previous interface (such as PlayMode) for calling the MediaDataKit interface.

[0084] The indication parameter (key) of the function point and the value (value) required for the function point are in a corresponding relationship, which can be one pair or multiple pairs. The value (value) corresponding to the indication parameter (key) of a function point can be one or include multiple.

[0085] The extension field (extend) can be used for the construction of pre-buried capabilities or differentiated capabilities.

[0086] In some embodiments, after the PlayMode interface is called, it can bypass the MediaDataKit and directly call the capabilities of other framework layers below the MediaDataKit to implement the playback function of the application.

[0087] In some embodiments, the PlayMode interface can be a kit or a selector (picker). When the PlayMode interface is implemented as a picker, the MediaDataKit can also be correspondingly divided into multiple interfaces with more detailed functions, so that the PlayMode interface can select a suitable interface from these multiple interfaces for calling.

[0088] After the capabilities such as unified access management, unified area management, and unified device management in the framework layer are called, the service layer interface and the kernel layer interface will be called further down to implement the corresponding encoding and decoding functions.

[0089] Based on the unified media framework provided in the embodiments of the present application, after an application in an electronic device accesses the unified media framework, the electronic device can uniformly control the file playback rules (such as audio playback rules) in multiple scenarios, including file playback within an application, file playback between applications, and file playback between devices. Among them, file playback within an application includes playback on the first-layer interface and playback on the second-layer interface. The first-layer interface usually includes the interface directly provided by the application (such as the main page), and the second-layer page usually includes other interfaces entered from the first-layer interface, such as the interface of other applications embedded in the application, etc. File playback between applications refers to cross-application file playback, such as migrating the audio of Huawei Music to QQ Music for playback. File playback between devices refers to cross-device file playback. For example, migrating the audio in a mobile phone to a Bluetooth speaker or a car intelligent cockpit for playback.

[0090] The following Figure 1A will introduce in detail several management modules of the unified media framework in the framework layer.

[0091] The following introduction takes audio files as an example. In specific implementation, it may also include other types of files, which are not limited here. For other types of files, the processing logics of several modules in the unified media framework are basically the same.

[0092] From the perspective of application scenarios, the OS of the electronic device in the embodiments of the present application provides the following several management modules:

[0093] I. Unified access management

[0094] Unified access management is used to set the data format of the first media file and manage the audio session. The audio session is used to standardize the behavior of playing media files. The management of the audio session includes actions such as creating, adjusting, and destroying the audio session.

[0095] Unified access management mainly includes the following aspects:

[0096] 1. Access management

[0097] Access management refers to managing the data formats of various applications to ensure that the formats of all data meet the data formats specified by the unified media framework. If the data format in an application does not conform to the data format specified by the unified media framework, these application data can be converted into the data format specified by the unified media framework according to the rule of converting as much as possible, so as to be able to interact with other applications or other devices later.

[0098] 2. Session management

[0099] A session refers to an audio session, which is an object or structure created inside the electronic device to save the data state. The session is used to set the audio context environment of the application and express the intention of the audio behavior in the application program to the system.

[0100] Session management involves several stages, such as creating a session, setting or managing session information, and controlling session behavior. After the user triggers file playback, the electronic device creates a corresponding session according to the type of the file, such as an audio session, a video session, a picture session, etc. After creating the session, session information can be managed, such as setting session information, adjusting session information, closing the session, etc. Session information, that is, session parameters, may include one or more of the following: file information (such as title, author, and content), playback duration, name of the playback application, name of the device for playback (such as the local device or a remote device), playback volume, playback mode (such as loop playback, loop playback by list, etc.), how to respond after the screen is turned off, how to respond when the application goes to the background, etc. With clear session information, the file can be played according to the session information. After that, in response to the user's playback control behavior, the electronic device can also control the session behavior, that is, control the file playback process. For example, when the user adjusts the volume to exceed the maximum volume value of the playback application, the electronic device can adjust the volume to the optimal volume value (such as 75% of the maximum volume value of the playback application). In some embodiments, a session can be created in advance. After the user inputs a playback operation to the electronic device, the electronic device then sets the corresponding session information. If the user does not input a playback operation, the session is an empty session without session information.

[0101] In some embodiments, when the electronic device creates a session, it can simultaneously set the session information. Then the above three processes can be simplified into two steps: creating a session and setting session information, and controlling session behavior. When creating a session and setting session information are carried out synchronously, the instantaneity is higher, which is more suitable for short-term existing session-on-demand sessions, can simplify operations, and improve efficiency. If it is a long-term existing session, the above three steps can be executed separately. First, create a session, and then set session parameters as needed when they are required, which can ensure the reasonable use of resources. When the electronic device creates a session, it can preset the session as a long-term session or a short-term session. The session type can be determined according to the application for which the session is applied. Generally, foreground sessions belong to short-term sessions, and background sessions belong to long-term sessions.

[0102] In the embodiments of the present application, session management can ensure the experience consistency of file playback within an application, between applications, and between devices.

[0103] The electronic device also provides a unified playback control center, which is convenient for users to control file playback within an application, between applications, and between devices. The playback control center is an application or service in the electronic device used to control file playback. Refer to Figure 2 , Figure 2 Exemplarily shows a set of user interfaces for controlling audio playback through the playback control center of the electronic device. AsFigure 2 As shown in a, the electronic device displays a user interface provided by the broadcast control center, and a call card is displayed in the upper left corner of the user interface, and the call card indicates that the electronic device is currently in a call. Figure 2 The user interface shown in a also displays information about other devices connected to the electronic device, such as the name, picture, volume, etc. of FreeBuds3, HUAWEI Watch, etc. When the electronic device detects a user operation on the call card, the electronic device can display the following information: Figure 2 The user interface shown in b, in which the user interface is expanded to display Figure 2 The call card shown in a of the figure is displayed, and a list of devices that can be used for calls is displayed below the call card, such as speakers, mobile phone handsets, FreeBuds3, HUAWEI Eyewear, etc. If there is more than one audio session in the electronic device, including other audio sessions besides calls, the user can also Figure 2 Input a left or right sliding operation on the call card in b to trigger the electronic device to switch to the card corresponding to the other audio session. After receiving the left sliding operation on the call card, the electronic device can display the following Figure 2 The playback card of the music app shown in c shows a list of devices that can be used to play the audio in the music app. The priority of various conversations can be set in the electronic device. For example, the priority of calls can be set higher than the priority of audio in the music app. Since FreeBuds3 is occupied by the call audio, the FreeBuds3 in the device list below the playback card of the music app can be clicked by the user, but the audio of the music app cannot be switched to FreeBuds3 for playback.

[0104] 3. Lifecycle Management

[0105] Lifecycle management plays a role in the entire process of access management and session management, including the creation, use and destruction of sessions.

[0106] The electronic device can query the session status. If the session is not used for a long time, the session can be directly destroyed. In the specific implementation, the destruction mechanism can be executed according to the status marked in the session, which may include the following methods:

[0107] (1) Counting method: When the number of applications using a session increases by one, the session count increases by 1; when an application using the session is destroyed or closed, the session count decreases by 1. The session usage status is determined according to the count value. When the session count is 0, the session can be destroyed; when the session count is not 0, the session is maintained.

[0108] (2) Timing type: The electronic device sets the survival time of the session in advance, and destroys the session when it times out.

[0109] (3) Query type: The electronic device can query the session status and attempt to close it. If it is found that the current session is not in use, the session can be closed; if it is found that the current session is still in use, the session cannot be closed and can be queried again after a period of time.

[0110] II. Unified area management

[0111] Unified area management is used to set the policy for playing media files locally on the electronic device and the policy for playing media files on devices other than the electronic device. Unified area management mainly includes the following aspects:

[0112] 1. Local area management

[0113] Local area management is mainly responsible for managing the file playback of the playback devices within the application and on the local electronic device. The processing efficiency of local files is high, and no other applications or playback devices of the electronic device need to intervene. The playback devices can include display screens for playing text and pictures, speakers for playing audio, and other devices.

[0114] Local area management defines the following items:

[0115] (1) Original routing policy, which refers to which local application and which local playback device the file is routed to. If an application plays its own file internally, it can be played directly without complex routing.

[0116] (2) Focus management, which refers to managing the focus of multiple open files. For example, if the focus is managed in a first-in, first-out manner and the user opens a new file, the new file will overwrite other previously opened files and occupy the playback device to start playing directly.

[0117] (3) Specification management, which refers to regulating the specifications of files. For example, the file specifications can be matched according to the playing application and specific device. For example, for video files, parameters such as resolution, volume, time, picture quality, and sound quality can be set according to the current playing application and specific device.

[0118] 2. Remote area management

[0119] Remote area management is mainly responsible for managing file playback outside the application or on remote devices. File playback outside the application means that the file of one application in the electronic device is played by another application.

[0120] Remote area management includes the following parts:

[0121] (1) Lifecycle management refers to confirming whether a file can be played outside the application or on a remote device, and whether there are conflicts, etc. If the file is enabled, playable, and not timed out, then the file is still within its lifecycle and can be played directly.

[0122] (2) Multi-routing management refers to managing to which other applications outside the application to which the file belongs or which device the file is to be played. Specifically, it can include the following situations: playing local file(s) (one or more) to multiple remote devices or multiple other applications, and playing remote file(s) (one or more) to the local device or local application of the electronic device.

[0123] (3) Remote specification management refers to file management when playing across applications or devices. In one case, if there is a conflict with the specification management in the local area management, the unified policy introduced later is used as the main method to manage the file specification. In another case, automatic matching can be performed according to the specification of the currently playing application or the device itself.

[0124] Reference Figure 3 , Figure 3 Exemplarily shows the architecture related to audio file processing in the software structure of the electronic device.

[0125] As Figure 3 shown, the electronic device may include an audio policy framework, and the audio policy framework may include audio types, local audio zones, remote audio zones, and output devices. Among them, the audio types may include different categories of audio differentiated by services, such as media audio, voice over internet protocol (VOIP) based on internet protocol (IP), dual tone multi frequency (DTMF), navigation audio, etc. The local audio zone corresponds to the local area management mentioned above, and the remote audio zone corresponds to the remote area management mentioned above. The output devices may include various audio playback devices local and remote to the electronic device, such as speakers, earpieces, wired headphones, Bluetooth devices, etc. The electronic device may also include a screen mirroring service, etc. The screen mirroring service may include services such as digital living network alliance (DLNA), Miracast, CAST+, etc. The latter two services can use the internal recording device in the audio policy framework. The screen mirroring service may also include a device management service development platform (DMSDP), which can use the virtual audio in the audio policy framework.

[0126] 3. Unified Policy

[0127] The unified policy defines the basic principles for file playback and control, which are applicable to the management of both the local area and the remote area. The unified policy mainly includes the following three parts:

[0128] (1) Policy definition

[0129] The policy definition includes the following three parts:

[0130] Routing rules, which refer to the rules for selecting the device on which to play the file;

[0131] Concurrent rules, which refer to how to select devices when two or more files are concurrent;

[0132] Adjustment rules, which refer to how to adjust the file specifications of the local electronic device and other connected devices, such as volume.

[0133] For example, referring to Figure 4 , Figure 4 shows an example of an audio playback scenario.

[0134] As Figure 4 shown in the left flowchart, in a single device, music starts to be played through the speaker first. When a new audio (such as a phone call tone, etc.) arrives, the electronic device will pause playing music and start playing the new audio. At this time, if the user presses the volume adjustment key, the electronic device will adjust the volume of the new audio (such as the phone call tone).

[0135] As Figure 4 shown in the right flowchart, in a multi-device collaboration scenario, the mobile phone can play music locally first. When a Bluetooth headset is connected, it switches to playing music through the Bluetooth headset. After that, when a new audio (such as a phone call tone, etc.) arrives, the user can switch the music from the mobile phone's playback control center to the speaker for playback. Then, the electronic device can play the new audio (such as the phone call tone) through the Bluetooth headset. At this time, whether the music played by the speaker needs to be paused, and, if the user presses the volume adjustment key, which device's volume the electronic device will adjust, require unified rules to define. The following layer 0 rules and layer 1 rules solve this problem.

[0136] (2) Layer 0 rules

[0137] The layer 0 rules define the rules for each device to play audio, and the rules for which device's volume to specifically adjust when the user adjusts the volume.

[0138] The layer 0 rules are divided into the following two parts:

[0139] (a) Playback rules for the device carried with the user

[0140] The device carried with the user may include the speaker and earpiece of the local electronic device, and may also include connected portable devices such as Bluetooth headsets and wired earphones.

[0141] The playback rules of the device's portable devices may include the following points:

[0142] Automatically switch according to device priority, and give priority to using the device with a higher priority to play files; among devices with the same priority, play files according to the rule of "last in, first out".

[0143] When there are multiple concurrent audio instances, the audio file of the phone type has the highest priority and can interrupt other audio.

[0144] When the user inputs a volume adjustment operation to the electronic device, adjust the volume of the electronic device; if the electronic device is connected to other portable devices, adjust the volume of other portable devices.

[0145] (b) Playback rules in a distributed collaboration scenario

[0146] The distributed collaboration scenario refers to a scenario where the electronic device collaborates with the connected remote devices. The remote devices may include, for example, devices such as tablets, Bluetooth speakers, and car radios.

[0147] The playback rules in the distributed collaboration scenario may include the following points:

[0148] Based on the single-device audio strategy, automatically migrate the audio to another device that the user is concerned about according to the user's focus. Here, the user's focus can refer to the device where the user's line of sight is located, or the device that the user is operating. For example, if the user clicks screen mirroring on the mobile phone, the focus will shift from the mobile phone to the large screen, and the audio on the mobile phone will also be automatically migrated to the large screen. On the other hand, the user's focus can also refer to the audio that the user is concerned about. If the electronic device itself contains multiple audio, the audio that the user is concerned about can be migrated to another device to ensure that the audio on the device itself and the remote audio do not interfere with each other. For example, the electronic device (i.e., the source device) plays Music A and Music B at the same time. The user can input an operation to screen mirror Music A to the large screen, and then Music B will still be played locally on the electronic device.

[0149] Reference Figure 5 , Figure 5 Exemplarily shows an example of audio migration with the user in a distributed collaboration scenario.

[0150] At the same time, the electronic device also provides a selection entry to facilitate the user to manually select which device to play the audio on.

[0151] (3) Layer 1 rules

[0152] The layer 1 rules define how the electronic device selects an audio output device for different audio types.

[0153] Such as Figure 6As shown in the figure, audio output devices can be divided into two categories: local peripherals and collaborative devices. Among them, local peripherals usually need to rely on electronic devices such as mobile phones to play audio, and generally cannot be directly connected to the network and play audio independently. For example, they may include hearing aids, Bluetooth headsets with wearing detection, Bluetooth glasses, wired earphones, ordinary Bluetooth headsets, car radios, Bluetooth speakers, receivers, speakers, smart bracelets, smart watches that cannot be independently connected to the network, etc. Bluetooth headsets with wearing detection refer to Bluetooth headsets with wearing detection function, and ordinary Bluetooth headsets refer to Bluetooth headsets without wearing detection function. Collaborative devices can usually be directly connected to the network and play audio independently. For example, they may include personal computers (PCs), PADs, smart screens, smart speakers, monitors, etc. Local peripherals and the portable devices mentioned above are the same concept.

[0154] When selecting an audio output device, it is usually selected in order of decreasing priority. The priority of each device is usually the highest for manually selected devices, followed by the rest. Among other devices, the priority of devices using the Bluetooth protocol to communicate with electronic devices is higher than that of devices using the Wi-Fi p2p protocol to communicate with electronic devices. Refer to Figure 7 , Figure 7 Exemplarily shows the order of high and low priorities. As Figure 7 shown, the priority of the device selected when the user manually switches is the highest. For example, the user can manually switch the audio output device through the playback control center, as Figure 2 shown. Another example is that the electronic device can also manually switch the audio output device in the user interface provided by the collaborative capsule when collaborating with each device. Refer to Figure 7 , except for manually switching devices, when each device automatically switches, the priorities of the following devices decrease in turn: hearing aids, Bluetooth headsets with wearing detection, open earphones, wired earphones, (Bluetooth headsets, Bluetooth car radios, Bluetooth speakers), collaborative devices (PCs, PADs, smart screens, HiCar devices), (receivers, speakers, external speakers of watches). The priorities of the devices within the same parentheses above are the same. Open earphones are a type of earphone that can be worn without inserting into the ear.

[0155] Refer to Figure 8 , Figure 8 Exemplarily shows the rules for each device to output audio.

[0156] As Figure 8 shown, it shows various types of audio that may appear in the electronic device.

[0157] Among them, media sound refers to the sound of normal media data playback, system sound refers to the prompt sound for system restart and shutdown, alarm sound refers to the prompt sound when the electronic device cannot operate or operates incorrectly, dual-tone multi-frequency (DTMF) signal refers to the sound during a phone call, forced sound refers to some necessary completed sounds, barrier-free sound refers to the screen reading sound, intelligent voice refers to the voice generated by the voice assistant, and preface text to speech (TTS) refers to the voice converted from text.

[0158] In local peripherals, all sounds are played on the portable device connected to the electronic device; incoming call ringtones and alarm sounds are played simultaneously on the local speaker of the electronic device and the connected portable device; when the electronic device is connected to a Bluetooth device, the audio is played on the device where the user inputs an operation (such as a click operation). For example, when the user brings the mobile phone into the car and the mobile phone is connected to Bluetooth glasses, the vehicle intelligent cockpit opens the Bluetooth music channel at this time, plays music on the mobile phone side, and the sound is played from the Bluetooth glasses; clicking on the music card on the intelligent cockpit makes a sound on the intelligent cockpit.

[0159] In the wireless screen mirroring scenario, the audio playback rules are the same as those during mirror screen mirroring, and both the screen mirroring sound and TTS are played on the device to which the electronic device is screen mirrored. Refer to Figure 9A , Figure 9A Exemplarily shows the audio playback routing rules for wireless screen mirroring. As Figure 9A shown, when the mobile phone uses wireless screen mirroring technologies such as Cast+, Miracast, digital video (DV) camera, etc. to screen mirror to a large screen, the media sound and TTS are played on the large screen, and other audio is played on the mobile phone.

[0160] In the scenario of mobile phone and PC collaboration or mobile phone and PAD collaboration, all audio except for the system forced sound can be transferred to the PC or PAD for playback, and the incoming call ringtone and alarm sound are also played locally on the electronic device (such as the mobile phone). For example, when the mobile phone and the computer are collaborating, the user operation interface of the mobile phone is switched to be displayed on the computer. At this time, the mobile phone serves as the data content source, and all the mobile phone-side sounds will be switched to the computer-side to make a sound. The routing rules in the collaboration scenario may include: after collaboration, the sound is default switched to the collaborative device side to make a sound; call types can be answered locally on the electronic device or on the collaborative device side. When receiving a call, the call sound is played on the device where the user presses to answer; provide a manual switching option for the audio playback device. Refer to Figure 9B , Figure 9B Exemplarily shows the audio playback routing rules in the collaboration scenario.

[0161] In the scenario where a mobile phone is connected to a car stereo, all audio is transferred to the car stereo for playback. After an electronic device (such as a mobile phone) is Bluetooth-connected to the car stereo, media sound, call sound, and incoming call ringtones can be played by the electronic device (such as a mobile phone) itself. The routing rules for this scenario may include: The audio is played on the device where the user clicks to play the audio.

[0162] For the task center, in the scenario of single-application screen mirroring (such as application migration in a distributed task center, single-application screen mirroring, etc.), the experience for the user during single-application migration is that the single application migrates to run on the peer device. Therefore, only the sound of that single application goes to the peer device, and the other sounds default to be emitted on the local device. Additionally, for the volume management and playback control status display of the migrated application, they are separated from other applications on this device. The routing rules for single-application migration may include: The migrated application emits sound on the peer device, and other applications emit sound on the local device.

[0163] For the scenario of projecting audio to a speaker, i.e., the pure audio projection scenario, the user can select a pure audio application to emit sound on the speaker through the playback control center. The routing rules for this scenario may include: Only media sound and call sound are transferred to the speaker for playback, and the remaining sounds are played on this device.

[0164] III. Unified Device Management

[0165] Unified area management is used to set the policies for playing multiple media files and the policies for playing media files during multi-device collaboration.

[0166] Unified device management mainly includes the following aspects:

[0167] 1. Device Routing Management

[0168] Device routing management mainly includes device classification and device collaboration.

[0169] (1) Device Classification

[0170] The requirements for applications on different desktops of different devices are different. Therefore, various problems may occur when playing audio. To solve these possible problems, this application provides unified audio playback rules.

[0171] Refer to Figure 10 , Figure 10 which exemplarily shows the classification of devices and the audio playback rules in different scenarios.

[0172] Such as Figure 10As shown, for screen sharing services such as wireless screen mirroring, PC collaboration, and PAD collaboration, the audio output device will change with the connection or disconnection of local peripherals. For example, when there are no local peripherals, the audio is output through the peripherals of the remote device; when the local device first connects to the local peripheral and then to the remote device or vice versa, the audio is output through the local peripheral. For service extension services such as HiCar, the audio output device will change with the connection or disconnection of local peripherals. For single-task migration scenarios, the audio output device does not change with the connection or disconnection of local peripherals. Single-task migration scenarios may include, for example, scenarios where only audio data is migrated. For pure audio projection scenarios, the audio output device does not change with the connection or disconnection of local peripherals, and the playback control center of the electronic device provides the ability to switch the playback device. Figure 10 The audio playback rules shown above are Figure 8 consistent with the audio playback rules shown above.

[0173] (2) Device collaboration

[0174] Device collaboration defines the audio playback rules in the scenario of multi-device collaboration.

[0175] Illustrated with two specific examples:

[0176] Example 1: The user wears Bluetooth glasses and gets in the car. A Bluetooth music card is launched in the intelligent cockpit, and the mobile phone is connected to the super desktop launched in the intelligent cockpit. This example may include the following specific usage scenarios:

[0177] (a) In the scenario of playing music, the music is played on the device where the user clicks. If the user clicks to play music on the glasses, the glasses will output the audio; if the user clicks to play music on the intelligent cockpit, the intelligent cockpit will output the audio.

[0178] (b) In the call scenario, the phone call audio is played on the device where the user clicks to answer the call. This call scenario can be a cellular call scenario or a VoIP call scenario.

[0179] (c) When opening an app in the super desktop of the intelligent cockpit, the audio of the app is played through the intelligent cockpit. When opening other apps on the mobile phone, the sound of other apps is played through the glasses.

[0180] When a Bluetooth headset or Bluetooth glasses support two-way physical connections, that is, they can be connected to two electronic devices simultaneously (such as device A and device B), the audio output rules between the two electronic devices may include the following:

[0181] (a) Media audio does not override media audio: After device A opens a media file, when device B opens a media file, the Bluetooth headset / glasses still hear the media audio of device A;

[0182] (b) Media audio does not override call audio: After device A makes a call and device B plays a media file, the call audio of device A is heard through the Bluetooth headset / glasses.

[0183] (c) Call audio overrides media audio: After device A plays a media file and device B makes a call, the call audio of device B is heard through the Bluetooth headset / glasses.

[0184] (d) Call audio does not override call audio: After device A makes a call and device B makes a call, the call audio of device A is heard through the Bluetooth headset / glasses, and the user can switch devices through the device selection interface in the call application.

[0185] The media audio mentioned above can refer to audio such as music being played.

[0186] 2. Type and policy coordination

[0187] Type and policy coordination mainly includes focus management and device coordination.

[0188] (1) Focus management

[0189] Focus management refers to strategies such as playing multiple audio instances together in a mixed audio, reducing the volume of a certain audio instance in the mix, or pausing the playback of a certain audio instance when two or more audio instances are concurrent. For the currently played audio, it can be considered that the audio occupies the audio focus.

[0190] Audio focus is divided into the following categories:

[0191] Long focus: Continuously obtains the focus, and other applications pause playback. The long focus can be preempted.

[0192] Short focus: Temporarily obtains the focus, and other applications pause playback. The short focus can be preempted. After the electronic device finishes playing the audio that obtains the short focus, the short focus is released immediately.

[0193] Temporary focus (duck): Temporarily obtains the focus, and other applications continue to play but need to reduce the volume, or other applications pause playback.

[0194] (a) The ways to apply for focus include the following:

[0195] AUDIOFOCUS_GAIN, obtain the long focus;

[0196] AUDIOFOCUS_GAIN_TRANSIENT, obtain the short focus;

[0197] AUDIOFOCUS_GAIN_TRANSIENT_MAY_DUCK, obtain the temporary focus, and other applications reduce the volume;

[0198] AUDIOFOCUS_GAIN_TRANSIENT_EXCLUSIVE, obtain temporary focus, and other applications pause playback.

[0199] (b) The ways to lose focus include the following:

[0200] AUDIOFOCUS_LOSS, permanently lose focus;

[0201] AUDIOFOCUS_LOSS_TRANSIENT, temporarily lose focus;

[0202] AUDIOFOCUS_LOSS_TRANSIENT_CAN_DUCK, temporarily lose focus and can remain unmuted.

[0203] There is usually only one long focus, and there can be multiple short foci and temporary foci at different time points.

[0204] Each device has an independent audio focus stack. The following are several examples to introduce how each application applies for focus.

[0205] Reference Figure 11 , Figure 11 Exemplarily shows an example of applying for a long focus. As Figure 11 shown, Music A first applies for a long focus and joins the stack; then the alarm clock reaches the reminder time, and the alarm clock applies for a short focus and joins the stack. At this time, Music A temporarily loses focus, and Music A pauses playback but does not leave the stack; then, Music B applies for a long focus. At this time, both Music A and the alarm clock leave the stack, and only Music B remains in the stack, and the electronic device plays the audio of Music B.

[0206] Reference Figure 12 , Figure 12 Exemplarily shows an example of applying for a short focus. In Figure 12 , Music A first applies for a long focus and joins the stack, then the alarm clock and the timer apply for short foci successively. After the alarm clock loses focus, it actively releases the focus, and then the timer is turned off by the user, and the timer releases the focus. Music A regains focus and plays.

[0207] Reference Figure 13 , Figure 13 Exemplarily shows an example of applying for a temporary focus. In Figure 13 , Music A first applies for a long focus and joins the stack, then the alarm clock applies for a short focus, and then the navigation application applies for a temporary focus. At this time, the alarm clock continues to play. After the navigation application finishes broadcasting, it actively releases the focus. At this time, the alarm clock continues to play, and then the alarm clock is manually turned off by the user, and the alarm clock releases the focus. Music A regains focus and plays.

[0208] (2) Concurrency strategy

[0209] The concurrent strategies mainly include the following points:

[0210] (a) Each device has its own independent audio focus stack, and the rules of each stack are processed concurrently according to three types of audio focus (long focus, segment focus, and temporary focus).

[0211] (b) After the audio migrates between devices, the audio focus stacks of each device need to be updated dynamically, such as adding or deleting applications in the stack.

[0212] (c) After the audio focus stacks of the local device and the remote device are updated, the two devices need to synchronize the stack status with each other so that both sides can know the stack status of the other side, thereby maintaining the concurrent rules of each device. For example, the local device first plays Huawei Music. If the user opens QQ Music, QQ Music can be migrated to the remote device for playback. The remote device continues to play QQ Music. If the local device then stops playing Huawei Music, QQ Music will return to the local device for playback.

[0213] The following introduces two different audio migration scenarios:

[0214] Scenario 1. Single-task type migration (i.e., only the migration of audio data). In this scenario, the local audio focus stack is updated, following the audio concurrency rules of a single machine. After the audio migrates, the local device plays other audio and does not interrupt the remote device from playing the migrated audio. As Figure 14 shown, initially, the audio focus stack 1 of the local device can include two audios, B and A, from top to bottom, and the audio focus stack 2 of the remote device can include C from top to bottom; then, audio B of the local device migrates to the remote device, so only audio A remains in the local device's audio focus stack. The local device plays audio A, and the top of the remote device's audio focus stack 2 becomes audio B'. The remote device plays audio B'. Audio B' is the migrated audio B on the remote device.

[0215] Scenario 2. Screen sharing type migration (i.e., the migration that includes both audio and interface). In this scenario, following the local audio concurrency rules, all local audio streams are stacked as a complete instance on the peer device; if the remote device is a Windows computer, according to the Windows audio rules, multiple windows with audio playback will sound simultaneously. That is, all local audio streams are migrated to the remote device and implemented according to the playback principle of the remote device. If the remote device can only play one audio stream, it plays the audio at the top of the stack. If the remote device can play multiple audio streams, it plays multiple audios at the top of the stack. As Figure 15As shown in the figure, initially, the audio focus stack 1 of the local device may include audio and video provided by Douyin and QQ Music from top to bottom, and the audio focus stack 2 of the remote device may include audio and video provided by Huawei Video from top to bottom. After that, the Douyin content of the local device is migrated to the remote device, so only the content of QQ Music remains in the audio focus stack of the local device. The local device plays QQ Music, and the top of the audio focus stack 2 of the remote device becomes the Douyin content. The remote device plays the audio of Douyin and displays the interface of Douyin.

[0216] IV. High-performance Management Framework

[0217] The high-performance management framework realizes the mixing of multiple low-latency client objects in the service layer through a unified interface, and realizes low power consumption and low latency through time slicing control. In addition, the unified media framework combines with the chip to rectify the spatial audio architecture, sinking the spatial audio effect to the kernel, reducing the load of the CPU, reducing power consumption, and improving the front-to-back performance of the headphone sound field.

[0218] All the management rules of the electronic device introduced above take audio as an example. In specific implementations, other types of files can also be controlled by the OS in the electronic device, which will not be elaborated here one by one.

[0219] In the embodiments of the present application, after each application in the electronic device is connected to the unified media framework, it can be controlled by the unified media framework. The file playback rules of each application are restricted by the unified media framework. Each application does not need to care about the specific implementation logic at the application layer. During its operation, it can play various files according to the regulations of the unified media framework, unifying file playback within the application, between applications, and between devices. By uniformly controlling the file playback rules through the unified media framework, problems such as abnormal audio playback in the background, unclear video picture quality, and audio-video asynchronization can be avoided. In addition, problems during audio-video collaboration between devices can be avoided in the case of an increasing number of peripheral collaborative devices, enabling users to obtain a consistent playback experience.

[0220] All the embodiments introduced above in the embodiments of the present application can be combined and implemented.

[0221] Figure 16 It is a flowchart of the method for playing media files provided by the embodiments of the present application. As Figure 16 shown, the method may include the following steps:

[0222] S1601, the first electronic device runs the first application on the first operating system.

[0223] The first electronic device may be the electronic device mentioned above, and the first operating system is the OS mentioned above, which can be referred to Figure 1A and related descriptions.

[0224] A first operating system runs on a first electronic device. The first operating system is used to run a first application and provides: a first interface (PlayMode) and a media data interface. The first application is a third-party application and includes program code that calls the first interface. The first interface is used to start the playback function provided by the first application and is used to call the media data interface. The media data interface includes the ability to perform one or more of the following processes on a first media file: a first process, a second process, and a third process.

[0225] Among them, the ability of the first process can be Figure 1A the ability provided by the unified access management in []. The first process includes one or more of the following processes: setting the data format of the first media file and managing the audio session, where the audio session is used to regulate the behavior of the playback function for playing the first media file. The ability of the second process can be Figure 1A the ability provided by the unified area management in []. The second process includes one or more of the following processes: setting the policy for playing the first media file on the first electronic device and setting the policy for playing the first media file on a device other than the first electronic device. The ability of the third process can be Figure 1A the ability provided by the unified device management in []. The third process includes one or more of the following processes: setting the policy for playing multiple media files and setting the policy for playing the first media file in multi-device collaboration, where the multiple media files include the first media file.

[0226] S1602. The first electronic device detects a user operation to start the playback function of the first application, and the playback function is used to trigger the playback of the first media file.

[0227] This user operation can be, for example, an operation to play audio, an operation to play video, etc. The first media file can be an audio file, a video file, an audio-video file, etc.

[0228] S1603. The first electronic device plays the first media file, or triggers the second electronic device to play the first media file.

[0229] There can be a communication connection between the first electronic device and the second electronic device. This communication connection can be a Bluetooth connection, a Wi-Fi connection, or other connections. The first electronic device can trigger the second electronic device to play the first media file based on this communication connection.

[0230] When the first electronic device plays the first media file, it can call the interfaces and capabilities of the OS layer by layer from the first application layer of the application layer to implement the playback function. For specific reference, please refer to the relevant descriptions in the previous text Figure 1A and Figure 1B above.

[0231] When the first electronic device triggers the second electronic device to play the first media file, the MediaDataKit interface of the second electronic device can obtain the incoming parameters from the first electronic device and call the capabilities of the second electronic device layer by layer to implement the playback function. For specific details, please refer to the relevant description in the previous text. Figure 1A and Figure 1B the relevant description.

[0232] When the first application at the application layer calls the first interface, the parameters passed to the first interface may include: the device name of the first electronic device, the application name of the first application, and the parameters passed by the first application. Among them, the parameters passed by the first application are in the form of key-value pairs, where the key represents the parameter for playing the first media file, and the value represents the content of the parameter for playing the first media file.

[0233] In some embodiments, the parameters passed to the first interface further include: the first device identifier. When the first device identifier indicates the first electronic device, the first electronic device plays the first media file; when the first device identifier indicates the second electronic device, the first electronic device triggers the second electronic device to play the first media file. Equivalently, the first application can pass the identifier of the device for playing the first media file to the first interface.

[0234] In some embodiments, the parameters passed to the first interface further include: the first application identifier. The first electronic device plays the first media file through the application indicated by the first application identifier, or triggers the second electronic device to play the first media file through the application indicated by the first application identifier. Equivalently, the first application can pass the identifier of the application for playing the first media file to the first interface.

[0235] In some embodiments, the method for playing a media file provided in this application can implement playback within the same application or across applications on the same electronic device, as well as cross-device playback. For example, the first media file belongs to the first application, and the first application identifier indicates the first application; or, the first media file belongs to the second application, and the first application identifier indicates the first application; or, the first media file belongs to the first application, and the first application identifier indicates the second application.

[0236] In some embodiments, before the first electronic device plays the first media file, the first media file can be converted into the first data format, or, before the first electronic device triggers the second electronic device to play the first media file, the first media file is converted into the first data format, and the first media file in the first data format is sent to the second electronic device. Among them, the first data format is a unified data format provided for unified access management. This can unify the format of media files and ensure the successful playback of media files.

[0237] In some embodiments, the first electronic device may create a first audio session, which is a structure for saving the state of the first media file; set parameters of the first audio session, where the parameters of the first audio session include one or more of the following for the first media file: the title, author, content, playing duration, playing volume, playing mode, name of the device for playing the first media file, name of the application for playing the first media file, playing mode after the device screen is turned off, and playing mode after the application is moved to the background; and play the first media file according to the first audio session.

[0238] In some embodiments, the first electronic device may also manage the life cycle of the first audio session. For example, when the number of applications using the first audio session is 0, destroy the first audio session; when the survival time of the first audio session reaches the first duration (the first duration can be preset and is not limited here), destroy the first audio session; when the first audio session is not in use, destroy the first audio session.

[0239] In some embodiments, the playing of the first media file follows the 0-layer rule in the unified area management described above.

[0240] For example, the second electronic device is the device with the highest priority among the devices connected to the first electronic device, or the device with the highest priority that was most recently connected to the first electronic device; or the first media file is telephone audio based on a circuit-switched network or a mobile communication network; or after triggering the second electronic device to play the first media file, the first electronic device can detect a user operation for adjusting the volume and send a volume adjustment instruction to the second electronic device. The device priority can refer to the 1-layer rule in the unified area management described above.

[0241] For another example, in the case of triggering the second electronic device to play the first media file, the first electronic device also plays a second media file.

[0242] In some embodiments, the media sound and TTS in the screen mirroring scenario are played on the screen mirroring device, and the remaining audio is played locally on the electronic device. For example, the first media file is a media file or TTS, and the second media file is a media file other than a media file or TTS.

[0243] In some embodiments, the playing of the first media file follows the 1-layer rule in the unified area management described above. For example, when triggering the second electronic device to play the first media file, the second electronic device is a device that cannot connect to the network; or the first media file is an incoming call ringtone or an alarm tone, and the first electronic device also plays the first media file.

[0244] In some embodiments, the playback of the first media file follows the focus management rules in the aforementioned unified device management. One is the preemption strategy for short focus. For example, after the first electronic device plays the first media file, it starts playing the third media file and pauses the playback of the first media file; after finishing playing the third media file, it resumes playing the first media file. Another is the preemption strategy for long focus. For example, after the first electronic device plays the first media file, it starts playing the third media file and stops the playback of the first media file; after finishing playing the third media file, it remains stopped from playing the first media file. Another is the occupation of temporary focus. For example, after the first electronic device plays the first media file, it plays the third media file and plays the first media file simultaneously; after finishing playing the third media file, it continues playing the first media file.

[0245] In some embodiments, the playback of the first media file follows the concurrency strategy in the aforementioned unified device management. For example, when triggering the second electronic device to play the first media file, the first electronic device can also play the second media file; the first electronic device stops playing the second media file; afterwards, the first electronic device instructs the second electronic device to pause playing the first media file, and the first electronic device continues to play the first media file.

[0246] Reference Figure 17 , Figure 17 is the hardware structure diagram of the electronic device 100 provided by the embodiments of the present application. The electronic device 100 can be the electronic device and the first electronic device mentioned above. The operating system introduced above runs on the electronic device 100.

[0247] As Figure 17 shown, the electronic device may include a processor 110, an external memory interface 120, an internal memory 121, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a camera 193, a display screen 194, etc.

[0248] It can be understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than those shown in the figure, or combine certain components, or split certain components, or have different component arrangements. The illustrated components can be implemented in hardware, software, or a combination of software and hardware.

[0249] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors.

[0250] The controller can generate operation control signals according to the instruction operation code and timing signals to complete the control of fetching and executing instructions.

[0251] A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can save the instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to use the instruction or data again, it can directly call it from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0252] The wireless communication function of the electronic device 100 can be implemented by antenna 1, antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor, and the baseband processor, etc.

[0253] Antenna 1 and antenna 2 are used for transmitting and receiving electromagnetic wave signals. Each antenna in the electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization rate of the antennas. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0254] The mobile communication module 150 may provide a solution for wireless communication such as 2G / 3G / 4G / 5G applied to the electronic device 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 may receive electromagnetic waves through the antenna 1, filter and amplify the received electromagnetic waves, and then transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 may also amplify the signal modulated by the modulation and demodulation processor and convert it into electromagnetic waves through the antenna 1 for radiation. In some embodiments, at least some functional modules of the mobile communication module 150 may be provided in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 may be provided in the same device.

[0255] The modulation and demodulation processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. Subsequently, the demodulator transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After being processed by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs a sound signal through an audio device (not limited to the speaker 170A, receiver 170B, etc.), or displays an image or video through the display screen 194. In some embodiments, the modulation and demodulation processor may be an independent device. In other embodiments, the modulation and demodulation processor may be independent of the processor 110 and provided in the same device as the mobile communication module 150 or other functional modules.

[0256] The wireless communication module 160 may provide solutions for wireless communications applied to the electronic device 100, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite systems (GNSSs), frequency modulation (FM), near field communication (NFC), infrared (IR), etc. The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, demodulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 may also receive signals to be sent from the processor 110, frequency-modulate and amplify them, and convert them into electromagnetic waves through the antenna 2 for radiation.

[0257] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, such that electronic device 100 can communicate with a network and other devices through wireless communication technologies. The wireless communication technologies may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology, etc. GNSS may include global positioning system (GPS), global navigation satellite system (GLONASS), beidou navigation satellite system (BDS), quasi-zenith satellite system (QZSS), and / or satellite based augmentation systems (SBAS).

[0258] Electronic device 100 implements a display function through a GPU, display screen 194, and an application processor, etc. The GPU is a microprocessor for image processing, and is connected to display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or change display information.

[0259] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD). The display panel can also be made of an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a mini-LED, a micro-LED, a micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device may include one or N display screens 194, where N is a positive integer greater than 1.

[0260] The electronic device 100 can implement the shooting function through the ISP, the camera 193, the video codec, the GPU, the display screen 194, and the application processor, etc.

[0261] The ISP is used to process the data fed back by the camera 193. For example, when taking a photo, the shutter is opened, and the light passes through the lens and is transmitted to the camera photosensitive element. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye. The ISP can also optimize the noise and brightness of the image through algorithms. The ISP can also optimize parameters such as the exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 193.

[0262] The camera 193 is used to capture static images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in standard RGB, YUV, etc. formats. In some embodiments, the electronic device 100 may include one or N cameras 193, where N is a positive integer greater than 1.

[0263] The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy, etc.

[0264] The video codec is used to compress or decompress digital videos. The electronic device 100 can support one or more video codecs. In this way, the electronic device 100 can play or record videos in multiple encoding formats, such as: Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0265] The NPU is a neural-network (NN) computing processor. By drawing on the structure of the biological neural network, for example, by referring to the transmission mode between human brain neurons, it can quickly process the input information and can also continuously self-learn. Through the NPU, applications such as intelligent cognition of the electronic device 100 can be realized, such as: image recognition, face recognition, speech recognition, text understanding, etc.

[0266] The internal memory 121 can include one or more random access memories (RAM) and one or more non-volatile memories (NVM).

[0267] The random access memory can include static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM, for example, the fifth generation of DDR SDRAM is generally called DDR5 SDRAM), etc.; the non-volatile memory can include disk storage devices, flash memory.

[0268] Flash memory can be classified into NOR Flash, NAND Flash, 3D NAND Flash, etc. according to the operating principle, and can be classified into single-level cell (SLC), multi-level cell (MLC), triple-level cell (TLC), quad-level cell (QLC), etc. according to the number of potential levels of storage cells. According to the storage specification, it can be classified into universal flash storage (UFS), embedded multi media Card (eMMC), etc.

[0269] The random access memory can be directly read and written by the processor 110, and can be used to store the operating system or executable programs (such as machine instructions) of other running programs, and can also be used to store data of users and application programs, etc.

[0270] The non-volatile memory can also store executable programs and data of users and application programs, etc., and can be pre-loaded into the random access memory for the processor 110 to directly read and write.

[0271] The external memory interface 120 can be used to connect to an external non-volatile memory to expand the storage capacity of the electronic device 100. The external non-volatile memory communicates with the processor 110 through the external memory interface 120 to implement the data storage function. For example, files such as music and videos are saved in the external non-volatile memory.

[0272] The electronic device 100 can implement audio functions through the audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor, etc. For example, music playback, recording, etc.

[0273] The audio module 170 is used to convert digital audio information into an analog audio signal for output, and is also used to convert an analog audio input into a digital audio signal. The audio module 170 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 170 can be disposed in the processor 110, or some functional modules of the audio module 170 can be disposed in the processor 110.

[0274] The speaker 170A, also known as the "loudspeaker", is used to convert an audio electrical signal into a sound signal. The electronic device 100 can listen to music or hands-free calls through the speaker 170A.

[0275] The receiver 170B, also known as the "earpiece", is used to convert audio electrical signals into sound signals. When the electronic device 100 answers a call or a voice message, the receiver 170B can be placed close to the human ear to receive the voice.

[0276] The microphone 170C, also known as the "microphone" or "transmitter", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak close to the microphone 170C to input the sound signal into the microphone 170C. The electronic device 100 can be provided with at least one microphone 170C. In some other embodiments, the electronic device 100 can be provided with two microphones 170C, which can not only collect sound signals but also implement a noise reduction function. In some other embodiments, the electronic device 100 can also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify the sound source, and implement functions such as directional recording.

[0277] The headphone jack 170D is used to connect a wired headphone. The headphone jack 170D can be a USB interface or a 3.5mm open mobile terminal platform (OMTP) standard interface, or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0278] In the embodiments of the present application, the internal memory 121 is used to store a computer program for implementing the media file playback method provided in the embodiments of the present application, and the processor 110 is used to execute the computer program to implement the media file playback method provided in the embodiments of the present application. The antenna 2 and the wireless communication module 160 can be used for the electronic device 100 to communicate with other devices (such as Bluetooth headsets, car radios, PCs, PADs, etc.).

[0279] It should be understood that the steps in the above method embodiments can be completed by the integrated logic circuit of the hardware in the processor or the instructions in the form of software. The method steps disclosed in combination with the embodiments of the present application can be directly implemented by the execution of the hardware processor, or implemented by the combination of the hardware and software modules in the processor.

[0280] The present application also provides an electronic device, which may include a memory, a processor, and a computer program stored on the memory, and the processor executes the computer program to implement the method executed by the electronic device in any one of the above embodiments.

[0281] The present application also provides a chip system, including a processing circuit and an interface circuit. The interface circuit is configured to receive computer instructions and transmit them to the processing circuit, and the processing circuit is configured to execute the computer instructions to implement the method executed by the electronic device in any of the foregoing embodiments.

[0282] The present application also provides a chip system, which includes at least one processor configured to implement the method executed by the electronic device in any of the foregoing embodiments. In a possible design, the chip system further includes a memory configured to store program instructions and data, and the memory is located inside or outside the processor.

[0283] The chip system may be composed of chips or may include chips and other discrete devices.

[0284] Optionally, the processor in the chip system may be one or more. The processor may be implemented by hardware or by software. When implemented by hardware, the processor may be a logic circuit, an integrated circuit, etc. When implemented by software, the processor may be a general-purpose processor that implements by reading software code stored in the memory.

[0285] Optionally, the memory in the chip system may also be one or more. The memory may be integrated with the processor or may be separately provided from the processor, and the embodiments of the present application do not limit this. Exemplarily, the memory may be a non-transitory processor, such as a read-only memory (ROM), which may be integrated with the processor on the same chip or may be separately provided on different chips. The embodiments of the present application do not specifically limit the type of the memory and the setting manner of the memory and the processor.

[0286] Exemplarily, the chip system may be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processing unit (CPU), a network processor (NP), a digital signal processing circuit (DSP), a micro controller unit (MCU), a programmable logic device (PLD), or other integrated chips.

[0287] The present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method executed by the electronic device in any of the above embodiments is implemented.

[0288] The present application also provides a computer program product, including a computer program. When the computer program is executed by a processor, the method executed by the electronic device in any of the above embodiments is implemented.

[0289] The various embodiments of the present application can be combined arbitrarily to achieve different technical effects.

[0290] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions according to the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire (such as coaxial cable, optical fiber, digital subscriber line) or wirelessly (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer, or a data storage device such as a server or data center that includes one or more integrated available media. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk (SSD)), etc.

[0291] Those of ordinary skill in the art can understand all or part of the processes in the methods of the above embodiments. The processes can be completed by instructing relevant hardware with a computer program. The program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the above method embodiments. The aforementioned storage medium includes: ROM or random access memory RAM, magnetic disks, optical disks, and other media that can store program codes.

[0292] In the description of the embodiments of the present application, unless otherwise specified, " / " means "or". For example, A / B may mean A or B; "and / or" in the text is only a description of the relationship between associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in the description of the embodiments of the present application, "a plurality" means two or more than two.

[0293] The terms "first" and "second" are only used for descriptive purposes and cannot be construed as implying or suggesting relative importance or implicitly indicating the quantity of the indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the embodiments of the present application, unless otherwise specified, the meaning of "a plurality" is two or more.

[0294] In summary, the above are only embodiments of the technical solution of the present application and are not intended to limit the protection scope of the present application. Any modifications, equivalent replacements, improvements, etc. made in accordance with the disclosure of the present application shall be included within the protection scope of the present application.

Claims

1. A method for playing a media file, characterized in that, The method is applied to a first electronic device on which a first operating system runs. The method includes: Running a first application on the first operating system. The first application is installed on the first electronic device. The first operating system provides: a first interface and a media data interface. The first application is a third-party application and includes program code for calling the first interface. The first interface is used to call the media data interface, and the media data interface includes the ability to perform one or more of the following processes on a first media file: a first process, a second process, and a third process. Wherein, the first process includes one or more of the following processes: setting the data format of the first media file and managing an audio session. The audio session is used to regulate the behavior of playing the first media file by the playback function. The second process includes one or more of the following processes: setting a policy for playing the first media file on the first electronic device and setting a policy for playing the first media file on a device other than the first electronic device. The third process includes one or more of the following processes: setting a policy for playing multiple media files, including the first media file, and setting a policy for playing the first media file in a multi-device collaboration scenario. Detecting a user operation to start the playback function of the first application, where the playback function is used to trigger the playback of the first media file. The first electronic device plays the first media file, or triggers the second electronic device to play the first media file.

2. The method according to claim 1, characterized in that The parameters passed into the first interface include: the device name of the first electronic device, the application name of the first application, and the parameters passed in by the first application. Among them, the parameters passed in by the first application are in the form of key-value pairs, where the key represents the parameter for playing the first media file, and the value represents the content of the parameter for playing the first media file.

3. The method according to claim 1 or 2, characterized in that, The parameters passed into the first interface further include: a first device identifier. When the first device identifier indicates the first electronic device, the first electronic device plays the first media file. When the first device identifier indicates the second electronic device, the first electronic device triggers the second electronic device to play the first media file.

4. The method according to any one of claims 1 to 3, characterized in that, The parameters passed into the first interface further include: a first application identifier. The first electronic device plays the first media file through the application indicated by the first application identifier, or triggers the second electronic device to play the first media file through the application indicated by the first application identifier.

5. The method according to claim 4, wherein The first media file belongs to the first application, and the first application identifier indicates the first application; Or, the first media file belongs to a second application, and the first application identifier indicates the first application; Or, the first media file belongs to the first application, and the first application identifier indicates the second application.

6. The method according to any one of claims 1-5, wherein Before the first electronic device plays the first media file, the method further includes: converting the first media file into a first data format; Before triggering the second electronic device to play the first media file, the method further includes: converting the first media file into a first data format, and sending the first media file in the first data format to the second electronic device.

7. The method according to any one of claims 1-6, characterized in that, The first electronic device plays the first media file, specifically including: Creating a first audio session, where the first audio session is a structure for saving the state of the first media file; Setting parameters of the first audio session, where the parameters of the first audio session include one or more of the following for the first media file: the title, author, content, playing duration, playing volume, playing mode, name of the device playing the first media file, name of the application playing the first media file, playing mode after the device screen is turned off, playing mode after the application goes to the background; Playing the first media file according to the first audio session.

8. The method according to claim 7, characterized in that, The method further includes: When the number of applications using the first audio session is 0, destroying the first audio session; When the survival time of the first audio session reaches a first duration, destroying the first audio session; When the first audio session is not in use, destroying the first audio session.

9. The method according to any one of claims 1-8, characterized in that The second electronic device is the device with the highest priority among the devices connected to the first electronic device, or the device with the highest priority that was most recently connected to the first electronic device; Or, the first media file is telephone audio based on a circuit-switched network or a mobile communication network; Or, after triggering the second electronic device to play the first media file, the method further includes: detecting a user operation of adjusting the volume, and sending an instruction to adjust the volume to the second electronic device.

10. The method according to any one of claims 1-9, characterized in that, The method further includes: in the case of triggering the second electronic device to play the first media file, the first electronic device also plays a second media file.

11. The method according to claim 10, characterized in that, The device priorities from high to low are: the device selected by the user, hearing aid, wearing-detection Bluetooth headset, open-ear headset, wired headset, Bluetooth headset / Bluetooth car kit / Bluetooth speaker, PC / PAD / smart screen / HiCar device, external speaker of earpiece / speaker / watch.

12. The method according to any one of claims 1-11, characterized in that, Triggering the second electronic device to play the first media file, The second electronic device is a device that cannot connect to the network; Or, the first media file is a ringtone or an alarm tone, and the first electronic device also plays the first media file.

13. The method according to claim 10, wherein The first media file is a media file or TTS, and the second media file is a media file other than the media file or the TTS.

14. The method according to any one of claims 1 to 13, characterized in that, After the first electronic device plays the first media file, the method further includes: Starting to play a third media file and pausing the play of the first media file; Ending the play of the third media file and continuing to play the first media file.

15. The method according to any one of claims 1 to 13, characterized in that, After the first electronic device plays the first media file, the method further includes: Start playing the third media file and stop playing the first media file; End playing the third media file and keep the first media file stopped.

16. The method according to any one of claims 1-13, characterized in that, After the first electronic device plays the first media file, the method further includes: Play the third media file and simultaneously play the first media file; End playing the third media file and continue playing the first media file.

17. The method according to claim 10, wherein The method further includes: The first electronic device stops playing the second media file; The first electronic device instructs the second electronic device to pause playing the first media file, and the first electronic device continues to play the first media file.

18. An electronic device, characterized in that, The electronic device includes: a memory, a processor, and a computer program stored on the memory. The computer program includes a first operating system. The first electronic device is installed with a first application. The first operating system provides: a first interface, a media data interface; the first application is a third-party application, and the first application includes program code for calling the first interface. The first interface is used to call the media data interface. The media data interface includes the ability to perform one or more of the following processes on the first media file: a first process, a second process, a third process; Wherein, the first process includes one or more of the following processes: setting the data format of the first media file, managing the audio session; the audio session is used to regulate the behavior of playing the first media file by the playback function; the second process includes one or more of the following processes: setting the policy for playing the first media file on the first electronic device, setting the policy for playing the first media file on a device other than the first electronic device; the third process includes one or more of the following processes: setting the policy for playing multiple media files, setting the policy for playing the first media file in multi-device collaboration, and the multiple media files include the first media file; The processor executes the computer program to implement the method according to any one of claims 1-17.

19. A computer-readable storage medium, characterized in that, Stored thereon is a computer program. The computer program includes a first operating system. The first electronic device is installed with a first application. The first operating system provides: a first interface, a media data interface; the first application is a third-party application, and the first application includes program code for calling the first interface. The first interface is used to call the media data interface. The media data interface includes the ability to perform one or more of the following processes on the first media file: a first process, a second process, a third process; Wherein, the first processing includes one or more of the following processings: setting the data format of the first media file, managing an audio session; the audio session is used to regulate the behavior of the playback function for playing the first media file; the second processing includes one or more of the following processings: setting a policy for playing the first media file on the first electronic device, setting a policy for playing the first media file on a device other than the first electronic device; the third processing includes one or more of the following processings: setting a policy for playing multiple media files, setting a policy for playing the first media file in multi-device cooperation, the multiple media files including the first media file; When the computer program is executed by a processor, it implements the method according to any one of claims 1-17.

20. A computer program product, characterized in that, The computer program product includes a computer program, the computer program includes a first operating system, the first electronic device is installed with a first application, and the first operating system provides: a first interface, a media data interface, a first capability, a second capability, a third capability; the first application is a third-party application, and the first application includes program code for calling the first interface, the first interface is used to call the media data interface, and the media data interface includes the capability of performing one or more of the following processings on the first media file: the first processing, the second processing, the third processing; Wherein, the first processing includes one or more of the following processings: setting the data format of the first media file, managing an audio session; the audio session is used to regulate the behavior of the playback function for playing the first media file; the second processing includes one or more of the following processings: setting a policy for playing the first media file on the first electronic device, setting a policy for playing the first media file on a device other than the first electronic device; the third processing includes one or more of the following processings: setting a policy for playing multiple media files, setting a policy for playing the first media file in multi-device cooperation, the multiple media files including the first media file; When the computer program is executed by a processor, it implements the method according to any one of claims 1-17.

Images