Audio processing method and apparatus

Abstract

Embodiments of the present application provide an audio processing method and device, and relate to the technical field of terminals. The method comprises: a first electronic device creating a first player in response to a request for creating the first player; the first electronic device binding the first player with a first thread, the first thread corresponding to a first audio type; the first electronic device creating a second player in response to a request for creating the second player; and the first electronic device binding the second player with a second thread, the second thread corresponding to a second audio type. It can be understood that the first electronic device can realize the shunting of different audio types by creating different threads for different audio types and creating different players. The classification of audio types enables audio data under different audio types to be played in different forms, thereby improving the flexibility of audio data playing.

Description

[0001] This application is a divisional application. The original application has the application number 202311867805.1 and the original application date is December 29, 2023. The entire contents of the original application are incorporated herein by reference. Technical Field

[0002] This application relates to the field of terminal technology, and in particular to an audio processing method and apparatus. Background Technology

[0003] With the rapid development of electronic technology, various electronic technologies are gradually being applied to vehicles, making them increasingly intelligent, and various types of interaction methods are being integrated into vehicles. For example, before driving, users can connect their electronic devices to the vehicle's infotainment system. After the connection is successfully established, users can view navigation or music provided by the electronic devices on the vehicle's screen. Alternatively, some vehicle infotainment systems can also download audio applications (APPs) or navigation apps to their own devices, enabling the presentation of navigation or music content and providing a safer and more convenient in-car experience while driving.

[0004] However, the in-vehicle infotainment system offers only a limited range of audio playback options, which negatively impacts the user experience. Summary of the Invention

[0005] This application provides an audio processing method and apparatus. Electronic devices can set different audio types for audio data from different applications and different audio types. Vehicle systems can set different playback modes according to the audio type to improve the flexibility of audio data playback.

[0006] In a first aspect, embodiments of this application provide an audio processing method, the method comprising: a first electronic device receiving a device connection event of a second electronic device, creating a first thread, creating a second thread; the first electronic device receiving a first operation applied to a first application; after receiving the first operation, the first electronic device receiving a request to create a first player, the request to create the first player carrying: a first media type and an identifier of the first application; the first electronic device responding to the request to create the first player creating the first player, the first electronic device binding the first player to a first thread, the first thread corresponding to a first audio type; the first electronic device receiving a second operation applied to a second application; after receiving the second operation, the first electronic device receiving a request to create a second player, the request to create the second player carrying: a first media type and an identifier of the second application; the first electronic device responding to the request to create the second player creating the second player, the first electronic device binding the second player to a second thread, the second thread corresponding to a second audio type.

[0007] The first electronic device can be the electronic device described in the embodiments of this application (such as a mobile phone), and the second electronic device can be the vehicle system described in the embodiments of this application.

[0008] The first thread can be playback thread 1 as described in the embodiments of this application. The first application can be the media application as described in the embodiments of this application. The first operation can be the operation of playing music audio data in the media application. The first player can be audio player 1 as described in the embodiments of this application. The first media type can be the media type as described in the embodiments of this application. The identifier of the first application can be the package name or UID of the media application. The first audio type can be the music streaming type as described in the embodiments of this application.

[0009] The second thread can be playback thread 2 as described in the embodiments of this application. The second application can be the navigation application as described in the embodiments of this application. The second operation can be the operation of playing navigation audio data in the navigation application. The second player can be audio player 2 as described in the embodiments of this application. The identifier of the second application can be the package name or UID of the navigation application. The second audio type can be the navigation streaming type as described in the embodiments of this application.

[0010] Understandably, the first electronic device can achieve audio type splitting by creating different lines and different players for different audio types. Classifying audio types allows audio data of different types to be played in different ways, thus improving the flexibility of audio data playback.

[0011] In one possible implementation, before the first electronic device binds the first player to the first thread, the method further includes: the first electronic device determining that the first application corresponds to the first audio type based on the correspondence between the first media type, the identifier of the first application, and the first audio type; before the first electronic device binds the second player to the second thread, the method further includes: the first electronic device determining that the second application corresponds to the second audio type based on the correspondence between the first media type, the identifier of the second application, and the second audio type.

[0012] Based on this, electronic devices can differentiate audio data from different applications that share the same media type by using media type, application identifiers, and the correspondence between audio types. This refines existing media types, making the categorized audio types more aligned with the playback habits of electronic devices. For example, a first or second electronic device can subsequently determine different playback formats based on the first and second audio types.

[0013] For example, when the first media type is a media type and the identifier of the first application is the identifier of a media application, the first audio type can be a music streaming type. Alternatively, when the first media type is a media type and the identifier of the first application is the identifier of a navigation application, the second audio type can be a navigation streaming type. The first electronic device distinguishes audio data for different applications by refining the classification of media types.

[0014] In one possible implementation, during the creation of the first thread, the first electronic device creates a first buffer and a second buffer; during the creation of the first player, the first electronic device creates a third buffer.

[0015] The first buffer can be buffer1 as described in the embodiments of this application, the second buffer can be buffer2 as described in the embodiments of this application, and the third buffer can be buffer3 as described in the embodiments of this application.

[0016] The first buffer is used to receive audio data from the third buffer; the second buffer is used to receive audio data from the first buffer; and the third buffer is used to receive audio data from the first application.

[0017] The first electronic device can pre-create three buffers before playing audio data to prevent audio data from stuttering during playback.

[0018] In one possible implementation, after the first player is successfully created, the first electronic device receives the first audio data from the first application and caches the first audio data in a third cache area; the first electronic device caches the first audio data in the third cache area into the first cache area; the first electronic device caches the first audio data in the first cache area into a second cache area.

[0019] The first electronic device can establish an audio path through three pre-created buffers. This allows audio data to be transmitted to the appropriate location through these three buffers, ensuring that the audio data is transmitted according to the pre-defined audio path.

[0020] This application embodiment can create three buffers corresponding to any type of audio data before playing different types of audio data, so that audio data of any audio type can be transmitted according to the three created buffers, thereby realizing the splitting of audio data.

[0021] In one possible implementation, creating a first buffer includes: determining a first storage space for the first buffer, and creating a first buffer that satisfies the first storage space, wherein the first storage space is determined based on one or more of the following: maximum data format, maximum number of channels, maximum sampling rate, preset frame length, or preset buffer multiplier; creating a second buffer includes: determining a second storage space for the second buffer, and creating a second buffer that satisfies the second storage space, wherein the second storage space is determined based on one or more of the following: data format of a first audio path, sampling rate of the first audio path, number of channels supported by the first audio path, or preset frame length; creating a third buffer includes: determining a third storage space for the third buffer, and creating a third buffer that satisfies the third storage space, wherein the third storage space is determined based on one or more of the following: first data format, first number of channels, first sampling rate, preset frame length, or preset buffer multiplier.

[0022] Understandably, the maximum data format, maximum number of channels, and maximum sampling rate can be preset. Since the first buffer can be used to cache audio data copied from the media application, and the first electronic device cannot determine the size of the audio data copied by the media application in advance, in order to ensure that the first buffer can cache audio data of any size, the first electronic device can determine the first buffer that can cache more data based on the maximum data format, maximum number of channels, and maximum sampling rate.

[0023] The first electronic device can set the size of the storage area and create the storage area according to the size of the storage area, so that audio data can be transmitted in a sufficient storage area to ensure the stability of data transmission.

[0024] In one possible implementation, before the first electronic device receives a device connection event from the second electronic device, the method further includes: the first electronic device loading a configuration file in response to an operation to start the first electronic device; the first electronic device obtaining a first configuration parameter in the configuration file, wherein the first configuration parameter corresponds to a first audio type, and the first configuration parameter includes one or more of the following: the name of a first audio channel corresponding to the first audio type, the data format of the first audio channel, the correspondence between the first audio channel and the second electronic device, the sampling rate of the first audio channel, or the number of channels supported by the first audio channel.

[0025] The first audio path can be the music audio path described in the embodiments of this application. The first configuration parameter can be the configuration parameter corresponding to the music audio path.

[0026] The first electronic device can obtain the first configuration parameters by parsing the configuration file, and then configure the first audio path based on the first configuration parameters. The creation process of the first audio path may include the creation of the first thread and the creation of the first player as described above.

[0027] In one possible implementation, the first electronic device includes an audio strategy module and an audio engine module. The first electronic device receives a device connection event from the second electronic device and creates a first thread, including: the audio strategy module receiving the device connection event; in response to receiving the device connection event, the audio strategy module sending a request to the audio engine module to create a first playback thread; and in response to the request to create the first playback thread, the audio engine module creating the first playback thread.

[0028] In this way, the audio strategy module can create the first thread upon receiving a device connection event. This allows subsequent audio data playback to be based on the first thread, ensuring stable audio data transmission.

[0029] In one possible implementation, the first electronic device further includes a DMSDP audio processing module. The first electronic device creates a first buffer and creates a second buffer, including: the audio engine module creating the first buffer; the audio engine module sending a message for creating an audio path to the DMSDP audio processing module; and in response to receiving the message for creating an audio path, the DMSDP audio processing module creating the second buffer.

[0030] In this way, the first electronic device can create buffers in different modules, allowing audio data to flow from the audio engine module to the DMSDP audio processing module. This ensures that audio data can be transmitted according to the pre-defined audio path.

[0031] In one possible implementation, the first electronic device, in response to a request to create a first player, creates a first player, including: an audio engine module receiving a request to create a first player; in response to receiving the request to create a first player, the audio engine module sending a message to an audio strategy module for selecting an audio type; in response to receiving the message for selecting an audio type, the audio strategy module determining a first audio type and a first thread; the audio strategy module notifying the audio engine module to create the first player in the first thread; and the audio engine module creating the first player.

[0032] In this way, the audio strategy module can predetermine the audio type corresponding to the current application and media type. When the audio engine module creates the first player, it can create a player belonging to the first audio type. This first player can transmit audio data under the first audio type and realize the splitting of audio data.

[0033] In one possible implementation, the first electronic device further includes: a smart mobility application; an audio strategy module determines a first audio type, including: in response to receiving a message for selecting an audio type, the audio strategy module sends a message for selecting an audio type to an audio service module; in response to receiving the message for selecting an audio type, the audio service module obtains the first audio type from the smart mobility application; the smart mobility application includes: a first media type, an identifier of a first application, and a correspondence between the first audio type; the audio service module sends the first audio type to the audio strategy module; and the audio strategy module determines the first audio type.

[0034] The embodiments of this application describe a first correspondence relationship, which may include the correspondence between a first media type, the identifier of a first application, and a first audio type.

[0035] In smart mobility applications, a first correspondence can be pre-set, and when the audio service module determines that an audio type needs to be selected, the first correspondence can be used to determine the current audio type that meets the first audio type requirement.

[0036] Understandably, this first correspondence setting can meet users' needs for distributing media types across different applications. For example, the first correspondence can distinguish between media types in navigation applications and media types in media applications, allowing a first electronic device or a second electronic device to play navigation audio data or music audio data through different playback formats.

[0037] In one possible implementation, the first electronic device creates a third buffer, including: an audio engine module creating a third buffer; the first electronic device receives first audio data from a first application and caches the first audio data in the third buffer, including: the audio engine module receives the first audio data from the first application and caches the first audio data in the third buffer; the first electronic device caches the first audio data in the third buffer into a first buffer, including: the audio engine module caching the first audio data in the third buffer into the first buffer; the first electronic device caches the first audio data in the first buffer into a second buffer, including: the audio engine module sending the first audio data in the first buffer to a DMSDP audio processing module, and the DMSDP audio processing module caching the first audio data in the first buffer into the second buffer.

[0038] The first electronic device can create buffers in different modules, allowing the audio engine module to receive first audio data from the first application, and the first audio data to flow from the audio engine module to the DMSDP audio processing module. This ensures that audio data can be transmitted according to a pre-defined audio path.

[0039] In one possible implementation, the method further includes: a first electronic device sending first audio data and a first audio type to a second electronic device, the first audio data corresponding to the first audio type; and the first electronic device sending second audio data and a second audio type to the second electronic device, the second audio data corresponding to the second audio type.

[0040] The first electronic device can send audio data and audio type to the second electronic device, enabling the second electronic device to determine different playback methods based on the audio type. These different playback methods may include: controlling different speakers to play audio data of different audio types, or playing audio data of different audio types at different volumes, etc., but this embodiment does not limit the specific implementation.

[0041] In one possible implementation, the method further includes: the first electronic device receiving a device disconnection event from the second electronic device, destroying the first thread, and destroying the second thread.

[0042] The first electronic device can promptly destroy both the first and second threads upon detecting a device disconnection event, thereby reducing the memory usage of the first and second threads.

[0043] Secondly, embodiments of this application provide an audio processing method, the method comprising: a second electronic device receiving first audio data and a first audio type from a first electronic device, the first audio data corresponding to the first audio type; the second electronic device playing the first audio data through a first speaker, the first speaker corresponding to the first audio type; the second electronic device receiving second audio data and a second audio type from the first electronic device, the second audio data corresponding to the second audio type; and the second electronic device playing the second audio data through a second speaker, the second speaker corresponding to the second audio type, wherein the first speaker and the second speaker are different.

[0044] This allows the second electronic device to determine different playback methods based on the audio type, improving the flexibility of audio data playback.

[0045] In one possible implementation, the method further includes: when the second electronic device receives both the first audio data and the second audio data simultaneously, the second electronic device plays the second audio data, wherein the priority of the second audio type is higher than the priority of the first audio type.

[0046] This allows the second electronic device to prioritize audio data of higher priority audio types based on audio type priority, thus avoiding conflicts caused by multiple audio data being played in the same speaker.

[0047] Fourthly, embodiments of this application provide an audio processing apparatus. The audio processing apparatus may be a first electronic device, or a chip or chip system within the first electronic device. Alternatively, the audio processing apparatus may be a second electronic device, or a chip or chip system within the second electronic device. The audio processing apparatus may include a communication unit and a processing unit.

[0048] Fifthly, embodiments of this application provide an electronic device including a processor and a memory, the memory for storing code instructions, and the processor for running the code instructions to perform the method described in the first aspect or any possible implementation of the first aspect of the first electronic device, or to perform the method described in the second aspect or any possible implementation of the second aspect of the second electronic device.

[0049] Sixthly, embodiments of this application provide a computer-readable storage medium storing a computer program or instructions that, when executed on an electronic device, cause the electronic device to perform the method described in the first aspect or any possible implementation of the first aspect, or to perform the method described in the second aspect or any possible implementation of the second aspect by a second electronic device.

[0050] In a seventh aspect, embodiments of this application provide a computer program product including a computer program, which, when run on an electronic device, causes the electronic device to perform the method described in the first aspect or any possible implementation of the first aspect in the form of a first electronic device, or to perform the method described in the second aspect or any possible implementation of the second aspect in the form of a second electronic device.

[0051] Eighthly, this application provides a chip or chip system including at least one processor and a communication interface, the communication interface and the at least one processor being interconnected via a circuit, the at least one processor being used to run computer programs or instructions to perform the methods described in the first aspect or any possible implementation thereof. The communication interface in the chip may be an input / output interface, pins, or circuits, etc.

[0052] In one possible implementation, the chip or chip system described above in this application further includes at least one memory storing instructions. The memory can be an internal storage unit of the chip, such as a register or cache, or it can be a storage unit of the chip itself (e.g., read-only memory, random access memory, etc.).

[0053] It should be understood that the third to eighth aspects of this application correspond to the technical solutions of the first aspect of this application, and the beneficial effects achieved by each aspect and the corresponding feasible implementation are similar, and will not be repeated here. Attached Figure Description

[0054] Figure 1 A scenario diagram provided for an embodiment of this application;

[0055] Figure 2 A schematic diagram of the speaker distribution in a vehicle infotainment system provided in an embodiment of this application;

[0056] Figure 3 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application;

[0057] Figure 4 A schematic diagram of the software structure of an electronic device provided in an embodiment of this application;

[0058] Figure 5 A schematic diagram of module interaction for an audio processing method provided in an embodiment of this application;

[0059] Figure 6 A schematic diagram illustrating a process of playing music audio data based on a music audio path, provided in an embodiment of this application;

[0060] Figure 7 A schematic diagram illustrating a process of playing navigation audio data based on a navigation audio path, provided in an embodiment of this application;

[0061] Figure 8 This application provides a schematic diagram of a process for playing ringtone audio data based on other audio paths, as illustrated in an embodiment of the present application.

[0062] Figure 9 This application provides a schematic diagram illustrating the steps for releasing an audio path according to an embodiment of the present application;

[0063] Figure 10 A schematic diagram of an interface for setting up a speaker, provided as an embodiment of this application;

[0064] Figure 11 A schematic diagram of another interface for setting up a speaker provided in an embodiment of this application;

[0065] Figure 12 A flowchart illustrating an audio processing method provided in an embodiment of this application;

[0066] Figure 13 This is a schematic diagram of the hardware structure of another electronic device provided in an embodiment of this application. Detailed Implementation

[0067] To facilitate a clear description of the technical solutions in the embodiments of this application, the terms "first" and "second" are used in the embodiments of this application to distinguish identical or similar items with essentially the same function and effect. For example, the first value and the second value are only used to distinguish different values ​​and do not limit their order. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and the terms "first" and "second" are not necessarily different.

[0068] It should be noted that, in this application, the terms "exemplary" or "for example" are used to indicate that something is being described as an example, illustration, or illustration. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0069] In this application, "at least one" means one or more, and "more than one" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can mean: a, b, c, a and b, a and c, b and c, or a, b, and c, where a, b, and c can be single or multiple.

[0070] For example, Figure 1 This is a schematic diagram of a scenario provided for an embodiment of this application. Figure 1 In the corresponding embodiments, a mobile phone is used as an example for illustration. This example does not constitute a limitation on the embodiments of this application.

[0071] like Figure 1 As shown, this scenario may include: electronic devices and the vehicle's display screen, with the electronic devices establishing a communication connection with the vehicle's display screen.

[0072] Electronic devices can display something like this when they detect that a user has opened the playback interface of a media application. Figure 1 The audio playback interface shown in 'a' can display: the song title of audio 100, the artist information of audio 100, the song cover of audio 100, the playback progress of audio 100, the play button 101, the button for viewing the previous song, and the button for viewing the next song, etc.

[0073] When an electronic device establishes a connection with the vehicle's infotainment system, the system can display the following when it detects that the user has opened the mobile-vehicle connectivity application from within the system: Figure 1 The interface shown in b is a mobile phone-vehicle interconnection application interface, which can display the corresponding interface, such as window 106 and window 107.

[0074] Window 106 can display: navigation application icon 102, vehicle-to-everything (V2X) application and SMS application icon 103, phone application icon 104, and function icon 105 for opening more functions, etc.

[0075] Window 107 may display one or more of the following: window 108 corresponding to the navigation application, control 109 for opening the navigation route home or to the office, weather information 110, and audio playback information 111.

[0076] The navigation application window 108 can display: a map, and information such as signs indicating the location of the electronic device; the weather information 110 can display: 27°, XX city, sunny, good air quality, 32° / 22°, etc.; the audio playback information 111 can display: the song name of audio 100, artist information, play button 112, a button to view the previous song, and a button to view the next song, etc.

[0077] Mobile phone-vehicle connectivity applications can be understood as a type of intelligent interconnection product between electronic devices and vehicles. These applications allow users to enjoy convenient and safe internet ecosystem services within the vehicle. For example, users can access navigation, SMS, music, and phone applications on their electronic devices through these applications.

[0078] In a possible implementation, in response to a user's trigger action on icon 102, the vehicle's infotainment system can display something like this. Figure 1 The interface shown in c has a window 107 that can display specific content from the navigation application, such as a map.

[0079] For example, in response to user requests Figure 1 As shown in the interface (a), by operating the play button 101, the mobile phone can send a command to the vehicle's infotainment system to play audio. The infotainment system can then control its speakers to play the audio data (or simply music) from the audio playback application. Alternatively, in response to user requests... Figure 1 As shown in b of the interface, the operation of the play button 112 allows the vehicle system to send a command to the mobile phone to request the playback of audio. When the mobile phone agrees to play the audio data, it sends the audio playback command and music to the vehicle system, which can then control the speakers on the vehicle system to play music.

[0080] For example, in response to a user setting a navigation route and triggering navigation in an electronic device, the mobile phone can send a command to the vehicle's infotainment system to play audio data. The infotainment system can then control its speakers to play the navigation audio data (or simply navigation tone). Alternatively, in response to a user setting a navigation route and triggering navigation in the infotainment system, the infotainment system can send a command to the mobile phone to request audio playback. Upon agreeing to play the audio data, the mobile phone can send the audio playback command and the navigation tone to the infotainment system. After receiving the audio playback command and the navigation tone, the infotainment system can control its speakers to play the navigation tone.

[0081] For example, Figure 2 This is a schematic diagram illustrating the speaker distribution in a vehicle infotainment system, provided as an embodiment of this application. Figure 2 As shown, the vehicle infotainment system may include one or more of the following speakers: driver speaker 1 located around the driver's seat, passenger speaker 1 located around the front passenger seat, rear speaker 1 located around the rear seats 1, rear speaker 2 located around the rear seats 2, or rear speaker 3 located around the rear seats 3, etc.

[0082] It is understood that this application embodiment does not specifically limit the number of rows of seats in the vehicle system, the number of seats in any row, or the number of speakers around the seats. For example, the vehicle system may include three rows of seats, any row may include two or three seats, and any seat may be surrounded by two or three speakers, etc.

[0083] When the vehicle's infotainment system controls the speakers to play music and navigation audio simultaneously, the driver's speaker 1 can play both simultaneously. For the driver, the music playing from the driver's speaker 1 may interfere with the simultaneously playing navigation audio, making it difficult for the driver to hear the navigation accurately. Similarly, the rear speakers 3 can play both music and navigation audio simultaneously. For passengers sitting in the rear seats 3, the navigation audio playing from the rear speakers 3 may also interfere with the simultaneously playing music, preventing them from fully immersing themselves in the music experience. Therefore, the vehicle's infotainment system offers a relatively limited range of audio playback options.

[0084] In view of this, embodiments of this application provide an audio processing method. An electronic device can set a navigation streaming type for audio data from a navigation application that is of media type, and set a music streaming type for audio data from a media application that is of media type. Then, the vehicle system can control different speakers to play audio corresponding to different streaming types based on the correspondence between the streaming type and the speakers. For example, it can control the driver's speaker 1 to play audio data under the navigation streaming type, or control one or more of the passenger speaker 1, rear speaker 1, rear speaker 2, and rear speaker 3 to play audio data under the music streaming type, achieving flexible playback of audio data and improving the user's auditory experience.

[0085] In some possible implementations, the vehicle's infotainment system could also control music playback or navigation sounds without using the aforementioned mobile-vehicle connectivity application (e.g., Figure 1 (The scenario shown). For example, when an electronic device establishes a connection with the vehicle's infotainment system, the system can also control music playback or navigation audio playback based on other applications or other control interfaces. This application does not limit this aspect.

[0086] It is understood that the aforementioned electronic devices can also be referred to as terminals, user equipment (UE), mobile stations (MS), mobile terminals (MT), etc. Electronic devices can include mobile phones with touchscreens, smart TVs, wearable devices, tablets, computers with wireless transceiver capabilities, virtual reality (VR) electronic devices, augmented reality (AR) electronic devices, and so on. The embodiments of this application do not limit the specific technologies or device forms used in the electronic devices.

[0087] It is understood that the audio processing method provided in this application embodiment is not limited to the scenario of establishing a connection between an electronic device and a vehicle system, but can also be applied to electronic devices and other electronic devices with multiple speakers. This application embodiment does not limit this.

[0088] To better understand the embodiments of this application, the structure of the electronic device according to the embodiments of this application is described below. For example, Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0089] The electronic device may include a processor 110, internal memory 121, a universal serial bus (USB) interface 130, an antenna 2, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, and a display screen 194, etc.

[0090] The sensor module 180 may include sensors such as a touch sensor. The touch sensor may be located on the display screen 194, and the touch sensor and the display screen 194 together form a touchscreen. The touch sensor is used to receive user trigger operations on the touchscreen.

[0091] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device. In other embodiments of this application, the electronic device may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0092] The processor 110 may include one or more processing units. These processing units may be independent devices or integrated within one or more processors. The processor 110 may also include a memory for storing instructions and data. For example, the processor 110 may store instructions and data related to an audio processing method provided in an embodiment of this application.

[0093] USB interface 130 is an interface that conforms to the USB standard specification, specifically it can be a Mini USB interface, Micro USB interface, USB Type C interface, etc. USB interface 130 can be used to realize data transmission between electronic devices and peripheral devices. For example, USB interface 130 can realize wired connection between vehicle system and electronic device in this embodiment of the application.

[0094] The wireless communication module 160 can provide functions such as wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks) or Bluetooth (BT) for use in electronic devices.

[0095] For example, the wireless communication module 160 can realize wireless connections such as WIFI or Bluetooth between the vehicle system and electronic devices, and realize data transmission between the vehicle system and electronic devices based on the WIFI or Bluetooth connection. In this embodiment, the specific connection method between devices is not limited.

[0096] Electronic devices utilize GPUs, displays (194), and application processors to achieve display functions. The GPU is a microprocessor for image processing, connecting the displays (194) and the application processor.

[0097] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. For example, display screen 194 can realize [something related to displaying images, videos, etc.]. Figure 1 The interface shown as 'a' in the diagram, or display screen 194, can also be used to configure playback settings for the speakers in the vehicle's infotainment system (see [link]). Figures 10-11 (Corresponding embodiments), etc., are not limited in this application embodiment.

[0098] The software systems of electronic devices can adopt layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture, etc., which will not be elaborated here.

[0099] For example, Figure 4 This is a schematic diagram of the software structure of an electronic device provided in an embodiment of this application.

[0100] like Figure 4 As shown, the layered architecture divides the software into several layers, each with a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into multiple layers, from top to bottom: the application (APP) layer, the application framework layer, the hardware abstraction layer (HAL), and the kernel layer, etc. This application does not limit the specific layers.

[0101] The application layer may include a series of application packages. The application layer may include one or more of the following: media applications, navigation applications, smart mobility applications (or smart vehicle connectivity applications), or telephone applications, etc. Media applications are applications that allow the playback of audio data; for example, media applications may include audio playback applications and / or video playback applications, etc., but this application embodiment does not limit this.

[0102] Media applications or navigation applications can generate audio data. For example, media applications can generate music audio data, navigation applications can generate navigation audio data, and phone applications can be used to generate ringtone audio data (or simply ringtones).

[0103] The smart mobility application is used to notify the audio service module to register (or deregister) callback interfaces based on the vehicle-to-vehicle connection (or vehicle-to-vehicle disconnection) status, and to notify the audio service module of device connection events (or device disconnection events).

[0104] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes several predefined interfaces. It may include one or more of the following: an audio strategy module, an audio engine module, or an audio service module.

[0105] The audio strategy module is used to load and store configuration parameters according to the configuration file, and also to configure different audio paths according to the configuration parameters. Each audio path can correspond to a stream type. For details of the configuration file and configuration parameters, please refer to the description in S501, which will not be repeated here.

[0106] The audio engine module is used to create playback threads according to the instructions issued by the audio strategy module, and also to issue instructions to the audio processing module of the distributed mobile sensing development platform (DMSDP) to create audio paths.

[0107] The audio service module is used to send messages selecting the streaming type to smart mobility applications based on callback functions.

[0108] In possible implementations, the application framework layer may also include one or more of the following: a display composition system, a window manager, a content provider, a resource manager, a view system, or a notification manager, etc. Figure 4 (not shown in the text), this application does not limit this aspect in the embodiments.

[0109] The purpose of the Hardware Abstraction Layer (HAL) is to abstract hardware, providing a unified interface for upper-layer applications to query hardware devices, or to provide data storage services for upper-layer applications. The HAL may include one or more of the following: a DMSDP audio processing module.

[0110] The DMSDP audio processing module is used to create (or release) audio paths and transmit audio data based on audio paths. For example, the DMSDP audio processing module can transmit audio data to the vehicle's infotainment system based on the audio path.

[0111] In possible implementations, the hardware abstraction layer may also include: a hardware editor ( Figure 4 (Not shown in the image), the hardware editor mainly provides hardware support for the display compositor, as well as supports layer compositing and display functions.

[0112] The kernel layer is the layer between hardware and software. It drives the hardware to function. The kernel layer can include one or more of the following: camera driver, display driver, or sensor driver, etc.

[0113] Among possible viewing methods, the electronic device may include a hardware layer, which may include one or more of the following: a camera, a liquid crystal display (LCD), a graphics processing unit (GPU), or a central processing unit (CPU), etc. Figure 4 (Not shown in the image).

[0114] In this application embodiment, no specific limitations are made on the software layers involved in the software architecture, the modules contained in the layers, and the functions of the modules.

[0115] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be implemented independently or in combination with each other. The same or similar concepts or processes may not be described again in some embodiments.

[0116] For example, Figure 5 This is a schematic diagram illustrating the module interaction of an audio processing method provided in an embodiment of this application. Figure 5 In the corresponding embodiments, the audio processing method involves electronic devices and vehicle systems. The electronic devices may include: an audio strategy module, an audio engine module, an audio service module, a DMSDP audio processing module, and smart mobility applications.

[0117] like Figure 5 As shown, S501-S502 illustrate the process of the electronic device loading the configuration file in response to the user's power-on operation. S503-S513 illustrate the process of the electronic device creating a playback thread and an audio path in response to the establishment of a connection between the vehicle system and the electronic device. Specifically, the audio processing method may include the following steps:

[0118] S501, in response to the user's power-on operation, the audio strategy module loads the configuration file.

[0119] Electronic devices can use configuration files to configure parameters for audio paths, and these configuration files can be pre-set in memory.

[0120] The configuration file may include one or more of the following configuration parameters: audio port name, the offloading type corresponding to the audio port name, or the routing relationship between the virtual audio device and the audio port, the data format of the audio port, the sampling rate of the audio port, or the number of channels supported by the audio port.

[0121] The audio path typically supports two channels, which can be the left channel and the right channel.

[0122] Audio channel names may include: music audio channel (or virtual output music), navigation audio channel (or virtual output navigation), and other audio channels (or virtual output other).

[0123] Music audio paths can correspond to music streaming types, navigation audio paths can correspond to navigation streaming types, and other audio paths can correspond to streaming types other than music streaming types and navigation category types. A streaming type can be understood as an audio type.

[0124] The routing relationship between virtual audio devices and audio paths can include the path information corresponding to any virtual audio device supported by the electronic device. For example, for a certain virtual audio device, such as a car infotainment system, the routing relationship between the car infotainment system and audio paths can include: car infotainment system-music audio path, car infotainment system-navigation audio path, car infotainment system-other audio paths, etc.

[0125] In this context, a virtual audio device can be understood as a device name used by an electronic device to refer to a recognized playback device. A virtual audio device can be a speaker, or an electronic device equipped with a speaker, etc. For example, in the scenario where a vehicle-mounted system establishes a connection with an electronic device as provided in this application embodiment, the vehicle-mounted system can serve as a virtual audio device within the electronic device.

[0126] It is understood that this application embodiment does not specifically limit the number of audio paths, the type of splitting, or the number of virtual audio devices in the routing relationship between audio paths in the configuration file.

[0127] For example, the configuration file may include the routing relationship between the virtual audio device and the audio path, such as deviceport_sources=virtual output music, virtual output navigation, virtual output other. Here, deviceport_sources can refer to the virtual audio device.

[0128] Configuration parameters for virtual output music can include one or more of the following: mixport name=virtual output music, format=16_bit, sampling rates=48000, channel masks=stereo.

[0129] Here, "stereo" can be understood as stereo sound, corresponding to 2 channels. The stream type can be indicated by the mixport name. For example, if mixport name = virtual output music, the stream type can be music.

[0130] Configuration parameters for virtual output navigation can include one or more of the following: mixportname=virtual output music, format=16_bit, sampling rates=48000, channel masks=stereo. When mixport name=virtual output navigation, the stream type can be navigation.

[0131] Configuration parameters for virtual output other can include one or more of the following: mixport name=virtual output music, format=16_bit, sampling rates=48000, channel masks=stereo. When mixport name=virtual output other, the stream type can be other stream types.

[0132] It is understood that the configuration parameters provided in the embodiments of this application are only examples and do not constitute a limitation on the embodiments of this application.

[0133] For example, in response to a user's power-on operation, the Linux kernel starts. The kernel can notify modules such as the audio policy module to power on via `init()`. The audio policy module then triggers `audioserver()` to load the configuration file. This configuration file can include operating system configuration information and application configuration information required when the electronic device starts up.

[0134] S502, the audio strategy module parses the configuration file and saves the configuration parameters in the configuration file.

[0135] The audio policy module can parse the configuration parameters in the configuration file and save them to the audio policy module. The contents of the configuration parameters can be found in the description in S501, and will not be repeated here.

[0136] S503, in response to the user's operation of connecting the vehicle system and electronic devices, the smart mobility application registers a callback function with the audio service module.

[0137] The callback function is used to obtain the audio stream type from the smart mobility application during audio data playback.

[0138] For example, in response to a user's operation of connecting the vehicle system and electronic devices, the smart mobility application creates a callback function and registers the callback function with the audio service module, so that the audio service module can perform the steps shown in S504 to save the callback function.

[0139] Electronic devices and vehicle infotainment systems can establish a connection via a wired method, such as through a serial interface (universal bus, USB). Alternatively, they can also establish a wireless connection, such as via Bluetooth, Wi-Fi, connecting to the same cloud account, or a trust loop. This application does not limit the specific methods used in this embodiment.

[0140] Understandably, once a connection is established between the electronic device and the vehicle's infotainment system, the vehicle's infotainment system can control audio playback via a mobile phone-vehicle interconnection application, or it can control audio playback via a trust loop.

[0141] Trust ring is a technical solution based on an identity authentication system that enables trusted interconnection of multiple devices across systems, allowing services to flow and be shared between devices. In scenarios where device connections are established based on trust ring, electronic devices logged into their accounts can use personal identification numbers (PINs) or other verification codes to achieve device authentication between the electronic device and the vehicle's infotainment system.

[0142] For example, let's illustrate trust ring authentication using a PIN code. The electronic device can send a PIN code to the vehicle's infotainment system via broadcast or other means, subsequently displaying a prompt for PIN code verification. Upon receiving the PIN code, the vehicle's infotainment system displays it and sends a response message back to the electronic device indicating receipt. In response to the user entering the PIN code into the electronic device, device authentication can be completed between the electronic device and the vehicle's infotainment system. In this scenario, the electronic device can display the trust ring interface, which may include the vehicle's infotainment system icon.

[0143] After successful device authentication, the vehicle-mounted system and the electronic device can be in the same trust ring. When the electronic device with the logged-in device account approaches the vehicle-mounted system again, the electronic device and the vehicle-mounted system can automatically establish a network connection based on the trust ring and realize communication under the trust ring.

[0144] It is understood that the embodiments of this application do not specifically limit the scenario of establishing a communication connection between the electronic device and the vehicle system.

[0145] In one possible implementation, in response to a user's connection to the vehicle's infotainment system and electronic devices, the infotainment system can send the speaker's attribute information to the smart mobility application, which can then display information such as... Figure 10 The interface shown. The speaker's attribute information may include one or more of the following: speaker location, or the number of speakers, etc.

[0146] S504, Audio Service Module saves callback functions.

[0147] S505, the smart mobility application sends a device connection event to the audio service module.

[0148] For example, a smart mobility application can send a device connection event to the audio service module through a specific interface. This specific interface could be startVirtualAudio.

[0149] For example, in response to the user's operation of connecting the vehicle's electronic device, the electronic device may execute the steps shown in S503 and S505 simultaneously, or it may execute the steps shown in S503 and S505 sequentially. This application embodiment does not specifically limit this.

[0150] S506, The audio service module sends a device connection event to the audio policy module.

[0151] The device connection event carries a device type identifier.

[0152] Device type identifiers may include DMSDP identifiers, etc. When an electronic device detects a DMSDP identifier, it can virtualize the speaker of the device corresponding to the DMSDP identifier, such as a car infotainment system or other devices, as its own local speaker, and then use this virtual speaker for audio playback. These other devices can be devices other than the local device, such as smartwatches, tablets, or smart bracelets with speakers; this embodiment does not limit the specific devices included in this application.

[0153] In some possible implementations, the device connection event may also carry a device identifier, etc. When the electronic device detects the device identifier, it can identify the specific device connected to it based on the device identifier.

[0154] Device connection events are used to indicate that a virtual audio device establishes a connection with an electronic device.

[0155] After receiving a device connection event, the audio policy module can create a playback thread and an audio path for each audio path according to the number of audio paths specified in the configuration parameters, through the audio engine module and the DMSDP audio processing module. This allows the virtual audio device to manage the streaming of audio data across different audio paths during playback.

[0156] For example, the audio strategy module can select configuration parameters by calling openoutputwithprofileanddevice() and determine which audio paths can be created under three stream types.

[0157] For example, the configuration parameters may specify audio paths including: a music audio path, a navigation audio path, and other audio paths. Therefore, the electronic device can create the music audio path and playback thread 1, the navigation audio path and playback thread 2, and other audio paths and playback thread 3. In this embodiment, the order in which the three audio paths are created is not limited. The number of audio paths created is also not limited.

[0158] The process of creating the music audio path and playback thread 1 can be found in steps S507-S514.

[0159] S507, The audio strategy module notifies the audio engine module to create playback thread 1.

[0160] Playback threads can be used to create audio players.

[0161] S508, the audio engine module creates playback thread 1 and buffer 1.

[0162] During the creation of playback thread 1, the audio engine module can determine the size of buffer1 based on the maximum data format, maximum number of channels, and maximum sampling rate, and then create buffer1. Buffer1 is used to buffer the music audio data copied from audio player 1. The size of the buffer can also be referred to as the storage space of the buffer.

[0163] Understandably, the maximum data format, maximum number of channels, and maximum sampling rate can be preset. Since buffer1 can be used to cache audio data copied from media applications, and the audio engine module cannot determine the size of the audio data copied by the media application in advance, in order to ensure that buffer1 can cache audio data of any size, the audio engine module can determine the buffer1 that can cache more data based on the maximum data format, maximum number of channels, and maximum sampling rate.

[0164] For example, taking a maximum data format of 32-bit, a maximum number of channels of 12, and a maximum sampling rate of 196000Hz as an example, the size of buffer1 is illustrated below:

[0165]

[0166] In the calculation of buffer1, the preset frame length can be 0.02 seconds, which can be understood as 20 milliseconds; the preset buffer multiplier can be 2 or 3, etc., and this embodiment does not limit it.

[0167] Understandably, to avoid stuttering caused by untimely audio data transmission, electronic devices can set the buffer multiplier to 2 or 3, so as to store more buffer data and alleviate potential stuttering.

[0168] In some possible implementations, the calculation formula for buffer1 may not include a preset buffer multiplier.

[0169] For example, `openoutputwithprofileanddevice()` can carry `profile->getname` and `iohandle`. The electronic device can use `profile->getname` to determine the audio path that the playback thread is allowed to process, and can use `iohandle` to associate the playback thread's thread identifier (such as thread number). Furthermore, the audio engine module can create this playback thread 1 using the `openoutput()` instruction.

[0170] It is understandable that electronic devices can also create playback thread 2 and playback thread 3 based on openoutput(), which will not be elaborated on further.

[0171] S509, The audio engine module notifies the DMSDP audio processing module to create a music audio path.

[0172] Creating a music audio path can be understood as creating a path for transmitting music audio data between the audio engine module and the DMSDP audio processing module.

[0173] For example, after the audio engine module creates buffer1, it notifies the DMSDP audio processing module to create a music audio path (or, in other words, sends a message to the DMSDP audio processing module to create a music audio path), causing the DMSDP audio processing module to create buffer2. Since both buffer1 and buffer2 are created within the same playback thread 1, there is a corresponding relationship between buffer1 and buffer2.

[0174] Understandably, after the music audio path is created, the media application can copy audio data to buffer1 of the audio engine module every 20 milliseconds, and the audio engine module can copy audio data to buffer2 of the DMSDP audio processing module every 20 milliseconds, thus realizing the transmission of audio data from the media application to the DMSDP audio processing module (see the description in S612-S613).

[0175] S510, DMSDP audio processing module creates buffer2.

[0176] In response to the message used to create the music audio path, the DMSDP audio processing module can calculate the size of buffer2 using the configuration parameters associated with the music audio path, and create buffer2. buffer2 is used to cache the music audio data copied by the audio engine module.

[0177] For example, taking a music audio path with a data format of 16-bit, supporting 2 channels, and a sampling rate of 48000 as an example, the size of buffer2 is illustrated below:

[0178]

[0179] The preset frame length can be the same as or different from the description above. In a possible implementation, after the DMSDP audio processing module creates buffer2, it can send a message to the audio engine module to indicate that the music audio path has been created, so that the audio engine module can determine that playback thread 1 has been created.

[0180] S511, the audio strategy module sends the configuration parameters of the music audio path to the audio engine module.

[0181] The configuration parameters for the music audio path can include one or more of the following: mixport name=virtualoutput music, format=16_bit, sampling rates=48000, channel masks=stereo, and stream type=music.

[0182] S512, the audio engine module sends the configuration parameters of the music audio path to the DMSDP audio processing module.

[0183] In a possible implementation, the electronic device can send the configuration information of the music audio path to the vehicle's infotainment system based on the steps shown in S512-S513, and the vehicle's infotainment system can then transmit audio data based on the configuration information of the music audio path. Alternatively, the vehicle's infotainment system can also pre-set the configuration information of the audio path locally; this embodiment of the application does not limit this approach.

[0184] The S513 and DMSDP audio processing modules send configuration parameters for the music audio path to the vehicle's infotainment system.

[0185] S514, the vehicle's infotainment system returns a response message to the DMSDP audio processing module.

[0186] For example, the vehicle's infotainment system can set configuration parameters after receiving them from the music audio channel, and then return a response message to the DMSDP audio processing module after the parameter settings are completed.

[0187] The process of creating the navigation audio path and playback thread 2 can be found in steps S515-S522.

[0188] S515, The audio strategy module notifies the audio engine module to create playback thread 2.

[0189] During the creation of playback thread 2, the audio engine module can determine the size of buffer4 based on the maximum data format, maximum number of channels, and maximum sampling rate, and then create buffer4. Buffer4 is used to buffer navigation audio data copied from audio player 2. The calculation method for buffer4 is similar to that for buffer1, and will not be elaborated here.

[0190] S516, the audio engine module creates playback thread 2 and buffer 4.

[0191] S517, The audio engine module notifies the DMSDP audio processing module to create a navigation audio path.

[0192] Creating a navigation audio path can be understood as creating a path for transmitting navigation audio data between the audio engine module and the DMSDP audio processing module.

[0193] For example, after the audio engine module creates buffer4, it notifies the DMSDP audio processing module to create a navigation audio path (or, in other words, sends a message to the DMSDP audio processing module to create a navigation audio path), causing the DMSDP audio processing module to create buffer5. Since both buffer4 and buffer5 are created within the same playback thread 2, there is a corresponding relationship between them.

[0194] Understandably, after the navigation audio path is created, the navigation application can copy audio data to buffer4 of the audio engine module every 20 milliseconds, and the audio engine module can copy audio data to buffer5 of the DMSDP audio processing module every 20 milliseconds, thus realizing the transmission of navigation audio data from the navigation application to the DMSDP audio processing module (see the description in S712-S713).

[0195] The S518 and DMSDP audio processing modules create buffer5.

[0196] In response to the message used to create the navigation audio path, the DMSDP audio processing module can calculate the size of buffer5 and create buffer5 using the configuration parameters associated with the navigation audio path. Buffer5 is used to cache the navigation audio data copied by the audio engine module. The calculation method for buffer5 is similar to that for buffer2, and will not be elaborated here.

[0197] In one possible implementation, after the DMSDP audio processing module creates buffer5, it can send a message to the audio engine module to indicate that the navigation audio path has been created, so that the audio engine module can determine that playback thread 2 has been created.

[0198] S519, The audio strategy module sends the configuration parameters for the navigation audio path to the audio engine module.

[0199] The configuration parameters for the navigation audio path can include one or more of the following: mixport name=virtualoutput navigation, format=16_bit, sampling rates=48000, channel masks=stereo, and stream type=navigation.

[0200] The S520 audio engine module sends configuration parameters for the navigation audio path to the DMSDP audio processing module.

[0201] The S521 and DMSDP audio processing modules send configuration parameters for the navigation audio path to the vehicle's infotainment system.

[0202] S522, the vehicle's infotainment system returns a response message to the DMSDP audio processing module.

[0203] For example, the vehicle's infotainment system can set configuration parameters after receiving the configuration parameters for the navigation audio path, and then return a response message to the DMSDP audio processing module after the parameter settings are completed.

[0204] The process of creating other audio paths and playback thread 3 can be found in steps S523-S530.

[0205] Other audio paths can be paths other than music audio paths and navigation audio paths.

[0206] S523, The audio strategy module notifies the audio engine module to create playback thread 3.

[0207] S524, the audio engine module creates playback thread 3 and buffer 7.

[0208] During the creation of playback thread 3, the audio engine module can determine the size of buffer7 based on the maximum data format, maximum number of channels, and maximum sampling rate, and then create buffer7. Buffer7 is used to buffer audio data copied by other applications (such as telephone applications). The calculation method for buffer7 is similar to that for buffer1, and will not be elaborated here.

[0209] S525, the audio engine module notifies the DMSDP audio processing module to create additional audio paths.

[0210] Creating other audio paths can be understood as creating paths between the audio engine module and the DMSDP audio processing module for transmitting other audio data.

[0211] For example, after the audio engine module creates buffer7, it notifies the DMSDP audio processing module to create other audio paths (or, in other words, sends a message to the DMSDP audio processing module to create other audio paths), causing the DMSDP audio processing module to create buffer8. Since both buffer7 and buffer8 are created within the same playback thread 3, there is a corresponding relationship between them.

[0212] Understandably, after other audio channels are created, subsequent applications (such as telephone applications) can copy audio data to buffer4 of the audio engine module every 20 milliseconds, and the audio engine module can copy audio data to buffer5 of the DMSDP audio processing module every 20 milliseconds, thus realizing the transmission of audio data from the telephone application to the DMSDP audio processing module (see the description in S812-S813).

[0213] S526, DMSDP audio processing module creates buffer8.

[0214] In response to messages used to create other audio paths, the DMSDP audio processing module can calculate the size of buffer8 and create buffer8 using configuration parameters associated with other audio paths. Buffer8 is used to buffer audio data copied from the audio engine module. The calculation method for buffer8 is similar to that for buffer2, and will not be elaborated here.

[0215] In one possible implementation, after the DMSDP audio processing module creates buffer8, it can send a message to the audio engine module to indicate that other audio paths have been created, so that the audio engine module can determine that playback thread 3 has been created.

[0216] S527, The audio strategy module sends configuration parameters for other audio paths to the audio engine module.

[0217] Other audio path configuration parameters may include one or more of the following: mixport name=virtualoutput other, format=16_bit, sampling rates=48000, channel masks=stereo, and split type=other.

[0218] S528, the audio engine module sends configuration parameters for other audio paths to the DMSDP audio processing module.

[0219] The S529 and DMSDP audio processing modules send configuration parameters for other audio channels to the vehicle's infotainment system.

[0220] S530, the vehicle's infotainment system returns a response message to the DMSDP audio processing module.

[0221] For example, the vehicle's infotainment system can set configuration parameters after receiving configuration parameters from other audio channels, and then return a response message to the DMSDP audio processing module after the parameter settings are completed.

[0222] It is understandable that the steps shown in S507-S530 can be interpreted as the initialization of the audio path. After initializing the audio path, the electronic device can... Figure 6 The corresponding implementation achieves playback of music audio data, or is based on Figure 7 The corresponding implementation achieves playback of navigation audio data, or based on Figure 8 The corresponding implementation enables the playback of ringtone data, thus speeding up the audio data playback process.

[0223] In a possible implementation, the electronic device may not initially create all audio paths supported by the electronic device (or, in other words, not execute S507-S530). When the electronic device detects that a user is playing music audio data, the required music audio path is created based on the steps shown in S507-S514; or when the electronic device detects that a user is playing navigation audio data, the required navigation audio path is created based on S515-S522; or when the electronic device detects that a user is playing ringtone audio data, other required audio paths are created based on S523-S530, to reduce memory usage. This embodiment of the application does not limit this approach.

[0224] Understandably, electronic devices can be based on Figure 5 The corresponding implementation completes the creation of the audio path and playback thread, enabling subsequent electronic devices to transmit and play audio data through the playback thread and audio path.

[0225] exist Figure 5 In corresponding embodiments, when creating music audio paths, navigation audio paths, and other audio paths, the electronic device can be based on... Figure 6 Corresponding embodiments utilize music audio pathways to play music audio data; electronic devices can also be based on... Figure 7 Corresponding embodiments utilize the navigation audio path to play navigation audio data, or electronic devices can also be based on... Figure 8 A corresponding embodiment utilizes other audio channels to play other audio data. This other audio data can be audio data other than music and navigation audio data; for example, it could be ringtone audio data or notification tone audio data.

[0226] For example, Figure 6 This is a schematic diagram illustrating a process for playing music audio data based on a music audio path, as provided in an embodiment of this application. Figure 6 In the corresponding embodiments, the example is given by taking the application that plays audio data as a media application and the audio data played by the media application as music audio data. This example does not constitute a limitation on the embodiments of this application.

[0227] S601, In response to the user's operation of playing music audio data in the media application, the media application notifies the audio engine module to create audio player 1.

[0228] Audio player 1 (or player1) is used to write music audio data to buffer1 in the audio engine module.

[0229] The user's actions for playing music audio data in a media application can be: [User actions related to...] Figure 1 The triggering operation of the play button 101 in the electronic device interface shown in Figure 'a'. In response to the user's action of playing music audio data in a media application, it can respond to the user's action... Figure 1 The triggering operation of the play button 101 in the electronic device interface shown in Figure a.

[0230] Alternatively, a user's action of playing music audio data in a media application can be: the user targets... Figure 1 The trigger operation of the play button 112 on the vehicle's infotainment interface, shown in b, refers to this. Specifically, when the vehicle's infotainment system detects that the user has access to the play button... Figure 1 When the play button 112 in the vehicle's infotainment interface (shown as b) is triggered, the vehicle's infotainment system can send a message for playing audio data to the electronic device. In response to the user's action of playing music audio data in a media application, this can be in response to receiving a message from the vehicle's infotainment system for playing music audio data.

[0231] Alternatively, the user's operation of playing music audio data in a media application can be: voice operation for playing audio, etc., which is not limited in this embodiment.

[0232] For example, in response to a user's action of playing audio in a media application, the media application can send a message to the audio engine module to create an audio player 1.

[0233] The message used to create audio player 1 may include one or more of the following: the package name of the media application (or the UID of the media application), the audio type of the music audio data, a first data format, a first number of channels, or a first sampling rate. The first data format, the first number of channels, and the first sampling rate can be audio transmission parameters supported by the media application.

[0234] The audio type of music audio data can be the audio type that electronic devices use to distinguish audio data in applications. For example, the audio type can include the following: media type, call type, ringtone type, text to speech (TTS) type, or notification type, etc.

[0235] For example, after opening a navigation app, users can activate the app's human-computer interaction functions using a wake word (such as "Hello, XX Navigation"). In response to the wake word, the navigation app can generate human-computer interaction audio data (such as "Hello, how can I help you?").

[0236] It is understandable that audio data in different applications can correspond to different audio types. For example, the human-computer interaction audio data in a navigation application can be of the TTS type, the navigation audio data in a navigation application can be of the media type, and the audio data in a music application can be of the media type.

[0237] To meet the needs of users in scenarios such as mobile phone-vehicle interconnection, the audio processing method provided in this application embodiment can divide media types into navigation streaming types and music streaming types, and set other audio types besides media types to other types. Through the re-division of audio types, the divided audio types can better meet the user's usage needs.

[0238] Media applications can notify the audio engine module to create an audio player by calling the `new AudioTrack()` instruction. The `new AudioTrack()` instruction can carry `stream_music`, `sampleRate`, `channelConfig`, and `audioFormat`. Here, `stream_music` can be understood as the audio type of the music audio data, `sampleRate` corresponds to the first sampling rate, `channelConfig` corresponds to the first number of channels, and `audioFormat` corresponds to the first data format.

[0239] It is understandable that the package name (or UID) of the media application may not be included in new AudioTrack(). In this scenario, the audio engine module can obtain the caller through attributeSource when it receives new AudioTrack(), and the caller can be the package name (or UID) of the media application (see the description in S610).

[0240] In possible implementations, in S701, the electronic device can also notify the audio engine module to create audio player 2 based on the new AudioTrack() instruction. The parameter values ​​carried in this instruction can be different from those in S601. In S801, the electronic device can also notify the audio engine module to create audio player 3 based on the new AudioTrack() instruction. The parameter values ​​carried in this instruction can be different from those in S601. This will not be elaborated further.

[0241] S602, The audio engine module sends a message to the audio strategy module to select the stream type.

[0242] The message type for the selected stream includes the media application's package name (or the media application's UID) and the audio type of the music audio data.

[0243] In one implementation, the electronic device can determine the diversion type via a smart mobility app based on the steps shown in S603-S608 (see...). Figure 6 (as described in the text).

[0244] In another implementation, the electronic device can also determine the stream type in the audio strategy module. For example, in response to S602, the audio strategy module can determine the stream type based on the first correspondence. In this scenario, the electronic device does not need to perform the steps shown in S603-S607.

[0245] S603, The audio strategy module sends a message to the audio service module to select the stream type.

[0246] S604, The audio service module notifies the call callback function to send a message selecting the streaming type to the smart mobility application.

[0247] Understandably, since the audio service module has already saved the callback function in step S504, the audio service module can send a message to the smart mobility application to select the streaming type by calling the callback function, and then obtain the streaming type corresponding to the music audio path from the smart mobility application.

[0248] S605, the smart mobility application determines the music streaming type based on the media application's package name and the audio type of the music audio data.

[0249] The package name (or UID) of the media application and the audio type of the music audio data can be carried in step S601 and passed to the smart mobility application through S602-S605.

[0250] In one implementation, the smart mobility application can pre-configure a first correspondence between the application package name, the audio type of the audio data, and the streaming type, so that the smart mobility application can obtain the streaming type related to the package name of the media application and the audio type of the music audio data from the first correspondence.

[0251] For example, the first correspondence includes the correspondence between the application package name, the audio type of the audio data, and the streaming type. The correspondence between the application UID, the audio type of the audio data, and the streaming type is similar and will not be repeated here.

[0252] The first correspondence can be shown in Table 2.

[0253] Table 2. Schematic diagram of the first correspondence.

[0254]

[0255] The amount of data in the first correspondence may not be limited to the description in Table 2. Smart mobility applications can determine the music streaming type corresponding to the package name of the media application and the media type of the music audio data based on the first correspondence shown in Table 2.

[0256] S606, The smart travel application returns the music streaming type to the audio service module.

[0257] S607, The audio service module returns the music distribution type to the audio strategy module.

[0258] In a possible implementation, the audio strategy module selects the playback device. In this embodiment, the playback device can be an in-vehicle infotainment system. For example, the electronic device can determine the playback device after detecting a connection with the in-vehicle infotainment system, or it can confirm the playback device as an in-vehicle infotainment system in response to a user's operation of playing audio in a media application.

[0259] S608, the audio strategy module selects the corresponding playback thread 1 according to the music streaming type.

[0260] Understandably, since the audio strategy module has already created playback thread 1 based on the steps shown in S507-S508, and generated the correspondence between music stream types and playback thread 1 after its creation, the audio strategy module can select the corresponding playback thread 1 according to the music stream type.

[0261] S609, The audio strategy module calls the corresponding playback thread 1 in the audio engine module to create audio player 1.

[0262] Audio Player 1 can be used to play music audio data later.

[0263] S610, the audio engine module creates audio player 1 and buffer 3.

[0264] During the process of audio player 1, the audio engine module can determine the size of buffer3 based on parameters such as the first data format, the first number of channels, and the first sampling rate, and create buffer3 in audio player 1. buffer3 can cache music audio data in media applications.

[0265] For example, taking a first data format of 16-bit, a first channel number of 2, and a first sampling rate of 36000Hz as an example, the size of buffer3 is illustrated as follows:

[0266]

[0267] It is understandable that since both buffer3 and buffer1 can be created by the audio engine module in playback thread 1, there is a corresponding relationship between buffer3 and buffer1.

[0268] In possible implementations, the calculation formula for buffer3 may not include a preset buffer multiplier, and this application embodiment does not limit this.

[0269] For example, the audio engine module can create an audio player by calling the createTrack_l() instruction. In createTrack_l(), the audio engine module can determine the application's identifier, such as the media application's package name (or the media application's UID), through the attributeSource.

[0270] In possible implementations, in S710, the audio engine module can create audio player 2 by calling createTrack_l(), which can carry the package name (or UID) of the navigation application; in S810, the audio engine module can create audio player 3 by calling createTrack_l(), which can carry the package name (or UID) of the phone application. Further details will not be provided later.

[0271] S611, The audio engine module notifies the media application audio player 1 of a message that it has been successfully created.

[0272] The message indicating that audio player 1 has been successfully created can include the identifier of audio player 1.

[0273] S612, the media application copies music audio data to buffer3 of the audio engine module every 20 milliseconds.

[0274] For example, once buffer3 is created, the media application can copy music audio data to buffer3 in audio player 1 every 20 milliseconds.

[0275] The audio engine module can start playback thread 1 by calling the startoutput() instruction. startoutput() can carry a portId, which can be understood as the identifier of the audio player. For example, in S612, the portId can be understood as the identifier of audio player 1.

[0276] In possible implementations, in S712, the audio engine module can also start playback thread 2 by calling the startoutput() instruction, and startoutput() can carry the identifier of audio player 2; in S812, the audio engine module can also start playback thread 3 by calling the startoutput() instruction, and startoutput() can carry the identifier of audio player 3.

[0277] S613, the audio engine module can copy music audio data from buffer3 to buffer1 every 20 milliseconds.

[0278] S614, the audio engine module copies the music audio data in buffer1 to buffer2 of the DMSDP audio processing module every 20 milliseconds.

[0279] Thus, with buffer1, buffer2, and buffer3 all created, the audio engine module can use S611-S612 to transmit music audio data from the media application to the DMSDP audio processing module.

[0280] The S615 and DMSDP audio processing modules send data packets containing music audio data and music stream types to the vehicle's infotainment system.

[0281] The data packet 1 may include: a header, a payload, and a trailer.

[0282] The header contains control information for data packet 1. For example, the header may include: the sender (or source address) of data packet 1, the receiver (or destination address) of data packet 1, the type of data packet 1, the length of data packet 1, and the music streaming type.

[0283] The data payload is the actual data portion carried by the data packet. It can be different types of data such as text, images, audio, and video. For example, the data payload can include music and audio data.

[0284] The end of a packet usually contains additional checksum information to detect whether data errors or loss have occurred during transmission.

[0285] For example, when the DMSDP audio processing module receives music audio data, it can set the music stream type in the header of data packet 1 and set the music audio data in the data payload, and then send data packet 1 containing music audio data and music stream type to the vehicle system.

[0286] S616, vehicle system parses data packet 1, and determines the speaker 1 for playing music audio data based on the music stream type.

[0287] For example, a second correspondence between the splitter type and the speaker can be preset in the vehicle system, so that the vehicle system can determine the speaker 1 corresponding to the music splitter type according to the second correspondence.

[0288] The second correspondence can be shown in Table 3.

[0289] Table 3. Schematic diagram of the second correspondence.

[0290]

[0291] The amount of data in the second correspondence may not be limited to the description in Table 3. The vehicle system can determine the speaker 1 corresponding to the music splitting type based on the second correspondence shown in Table 3.

[0292] As shown in Table 3, the speaker 1 described below may include one or more of the following: passenger side speaker 1, rear speaker 1, rear speaker 2, or rear speaker 3. The speaker 2 described below may include: driver's side speaker 1. The speaker 3 described below may include one or more of the following: driver's side speaker 1, passenger side speaker 1, rear speaker 1, rear speaker 2, or rear speaker 3.

[0293] S617, the vehicle's infotainment system speaker 1 plays music audio data.

[0294] It is understandable that, since a second correspondence can be set in the vehicle system, the vehicle system can control the speaker 1 that plays music audio data and instruct the speaker 1 to play music audio data, as shown in steps S614-S615.

[0295] In a possible implementation, a second correspondence can also be set in the electronic device, allowing the electronic device to specify a speaker to play audio data. For example, in response to S614, the DMSDP audio processing module can send music audio data and music stream type to the smart mobility APP. The smart mobility APP determines the speaker 1 in the vehicle system that plays music audio data based on the second correspondence, and then returns the target data packet containing the identifier of speaker 1 and the music audio data to the vehicle system. The vehicle system can then parse the target data packet and control speaker 1 to play music audio data.

[0296] The second correspondence can be based on electronic devices. Figure 8 The corresponding embodiments are pre-set, and this application does not limit this aspect in the embodiments.

[0297] In one possible implementation, if volume control is allowed in the smart mobility app, the smart mobility app can also send the volume of different speakers to the vehicle's infotainment system via a target data packet, enabling the vehicle's infotainment system to achieve precise control over the volume of different speakers.

[0298] Based on this, the embodiments of this application can realize the use of music audio path and playback thread 1 to play music audio data under music streaming type, so that the vehicle system can instruct the corresponding speaker 1 to play music audio data according to the second correspondence, realize the accurate classification of audio data, and make the playback form of music audio data meet the user's needs.

[0299] For example, Figure 7 This is a schematic diagram illustrating a process for playing navigation audio data based on a navigation audio path, as provided in an embodiment of this application. Figure 7 In the corresponding embodiments, the application playing audio data is referred to as a navigation application, and the audio data played by the navigation application is referred to as navigation audio data, as an example for illustration. This example does not constitute a limitation on the embodiments of this application. Navigation audio data may also be referred to as navigation sound, etc.

[0300] S701, In response to the user's operation of playing navigation audio data in the navigation application, the navigation application notifies the audio engine module to create audio player 2.

[0301] Audio Player 2 (or player2) is used to write navigation audio data to buffer4 in the audio engine module.

[0302] The user's action of playing navigation audio data in a navigation application can be: the user sets a navigation route and starts navigation in the electronic device interface. In response to the user's action of playing navigation audio data in a navigation application, the action can be triggered by the user setting a navigation route and starting navigation in the navigation application interface.

[0303] Alternatively, the user's action of playing navigation audio data in a navigation application can be: the user sets a navigation route and starts navigation in the vehicle's infotainment system. When the vehicle's infotainment system detects this action, it can send a message to the electronic device to play navigation audio data. The response to the user's action of playing navigation audio data in the navigation application can also be a response to the user setting a navigation route and triggering the start of navigation within the navigation application's interface.

[0304] Alternatively, the user's operation of playing navigation audio data in the navigation application can be: it can also be a voice operation to start navigation, etc., which is not limited in this embodiment.

[0305] For example, in response to a user's action of playing navigation audio data in a navigation application, the navigation application can send a message to the audio engine module to create an audio player 2.

[0306] The message used to create audio player 2 may include one or more of the following: the package name of the navigation application (or the UID of the navigation application), the audio type of the navigation audio data, a second data format, a second number of channels, or a second sampling rate. The second data format, the second number of channels, and the second sampling rate can be audio transmission parameters supported by the navigation application.

[0307] S702, the audio engine module sends a message to the audio strategy module to select the stream type.

[0308] The message carrying options for the selected feed type are: the package name of the navigation application (or the UID of the navigation application) and the audio type of the navigation audio data.

[0309] S703, The audio strategy module sends a message to the audio service module to select the stream type.

[0310] S704, The audio service module notifies the call callback function to send a message selecting the streaming type to the smart mobility application.

[0311] S705, the smart mobility application determines the navigation routing type based on the package name of the navigation application and the audio type of the navigation audio data.

[0312] Among them, the smart travel application can determine the navigation diversion type based on the description in the first correspondence relationship. The first correspondence relationship can be found in the description in S605, which will not be repeated here.

[0313] S706, The smart mobility application returns the navigation routing type to the audio service module.

[0314] S707, The audio service module returns the navigation routing type to the audio strategy module.

[0315] S708, the audio strategy module selects the corresponding playback thread 2 according to the navigation flow type.

[0316] Understandably, since the audio strategy module has already created playback thread 2 based on the steps shown in S515-S516, and generated the correspondence between navigation sub-types and playback thread 2 after its creation, the audio strategy module can select the corresponding playback thread 2 according to the navigation sub-type.

[0317] S709, The audio strategy module calls the corresponding playback thread 2 in the audio engine module to create audio player 2.

[0318] Audio Player 2 can be used to play navigation audio data later.

[0319] S710, Audio Engine module creates Audio Player 2 and buffer 6.

[0320] During the audio player 2 process, the audio engine module can determine the size of buffer6 based on parameters such as the second data format, the second number of channels, and the second sampling rate, and create buffer6 in audio player 2. Buffer6 can cache navigation audio data for navigation applications. The calculation method of buffer6 is similar to that of buffer3, and will not be elaborated here.

[0321] S711, the audio engine module notifies the navigation application that the audio player 2 has been successfully created.

[0322] The message indicating that Audio Player 2 has been successfully created can include an identifier for Audio Player 2.

[0323] S712, the navigation application copies navigation audio data to buffer6 of the audio engine module every 20 milliseconds.

[0324] For example, once buffer6 is created, the navigation application can copy navigation audio data to buffer6 in audio player 2 every 20 milliseconds.

[0325] The S713 audio engine module can copy navigation audio data from buffer6 to buffer4 every 20 milliseconds.

[0326] The S714 audio engine module can copy the navigation audio data in buffer4 to buffer5 of the DMSDP audio processing module every 20 milliseconds.

[0327] Thus, with buffer4, buffer5, and buffer6 all created, the audio engine module can use S711-S712 to transmit navigation audio data from the navigation application to the DMSDP audio processing module.

[0328] The S715 and DMSDP audio processing modules send data packets containing navigation audio data and navigation split-type data to the vehicle's infotainment system.

[0329] The data packet 2 may include a header, a payload, and a trailer.

[0330] The header contains control information for data packet 2, such as the sender (or source address) of data packet 2, the receiver (or destination address) of data packet 2, the type of data packet 2, the length of data packet 2, and the navigation routing type.

[0331] The data payload may include navigation audio data.

[0332] The end of a packet usually contains additional checksum information to detect whether data errors or loss have occurred during transmission.

[0333] S716, vehicle system parses data packet 2, and determines the speaker 2 that plays navigation audio data based on the navigation stream type.

[0334] For example, the vehicle system can determine the speaker 2 that plays navigation audio data based on the second correspondence. The second correspondence can be found in the description in S616, and will not be repeated here.

[0335] S717, vehicle infotainment system speaker 2 plays navigation audio data.

[0336] It is understandable that, since a second correspondence can be set in the vehicle infotainment system, the vehicle infotainment system can control the speaker 2 that plays navigation audio data and instruct the speaker 2 to play navigation audio data, as shown in steps S714-S715.

[0337] Based on this, the embodiments of this application can realize the use of navigation audio path and playback thread 2 to play navigation audio data under navigation split type, so that the vehicle system can instruct the corresponding speaker 2 to play navigation audio data according to the second correspondence, realize the accurate classification of audio data, and make the playback form of navigation audio data meet the user's needs.

[0338] For example, Figure 8 This is a schematic diagram illustrating a process for playing ringtone audio data based on other audio paths, provided as an embodiment of this application. Figure 8In the corresponding embodiments, the application that plays audio data is called a telephone application, and the audio data played by the telephone application is ringtone audio data (or ringtone). This example does not constitute a limitation on the embodiments of this application.

[0339] The audio data played by the phone application may also include: prompts or other audio data, etc., which is not limited in this embodiment.

[0340] S801. When the phone application plays ringtone audio data, the phone application notifies the audio engine module to create audio player 3.

[0341] Audio Player 3 (or player3) is used to write ringtone audio data to buffer7 in the audio engine module.

[0342] The message used to create audio player 3 may include one or more of the following: the package name of the phone application (or the UID of the phone application), the audio type of the ringtone audio data, the third data format, the number of third channels, or the third sampling rate. The third data format, the number of third channels, and the third sampling rate can be audio transmission parameters supported by the phone application.

[0343] S802, the audio engine module sends a message to the audio strategy module to select the stream type.

[0344] The message type for the selected routing includes the phone application's package name (or the phone application's UID) and the audio type of the ringtone audio data. The audio type of the ringtone audio data can be a ringtone type.

[0345] S803, The audio strategy module sends a message to the audio service module to select the stream type.

[0346] S804, the audio service module notifies the call callback function to send a message to the smart mobility application to select the streaming type.

[0347] S805, the smart mobility application determines other traffic routing types based on the package name of the phone application and the audio type of the ringtone audio data.

[0348] Among them, the smart mobility application can determine other diversion types based on the description in the first correspondence relationship. The first correspondence relationship can be found in the description in S605, which will not be repeated here.

[0349] S806, Smart Mobility Application returns other stream types to the audio service module.

[0350] S807, The audio service module returns other streaming types to the audio policy module.

[0351] S808, the audio strategy module selects the corresponding playback thread 3 based on other streaming types.

[0352] Understandably, since the audio strategy module has already created playback thread 3 based on the steps shown in S523-S524, and generated the correspondence between other stream types and playback thread 3 after its creation, the audio strategy module can select the corresponding playback thread 3 according to other stream types.

[0353] S809, The audio strategy module calls the corresponding playback thread 3 in the audio engine module to create audio player 3.

[0354] Audio Player 3 can be used to play ringtone audio data later.

[0355] S810, the audio engine module creates the audio player 3 and the buffer 9.

[0356] During the audio player 3 process, the audio engine module can determine the size of buffer9 based on parameters such as the third data format, the third number of channels, and the third sampling rate, and create buffer9 in audio player 3. Buffer9 can cache ringtone audio data for telephone applications. The calculation method of buffer9 is similar to that of buffer3, and will not be elaborated here.

[0357] S811, the audio engine module notifies the phone application audio player 3 of a message that creation was successful.

[0358] The message indicating that Audio Player 3 has been successfully created can include an identifier for Audio Player 3.

[0359] S812, the telephone application copies the ringtone audio data to buffer9 of the audio engine module every 20 milliseconds.

[0360] For example, once buffer9 is created, the phone application can copy the ringtone audio data to buffer9 in audio player 3 every 20 milliseconds.

[0361] S813, the audio engine module can copy the ringtone audio data in buffer9 to buffer7 every 20 milliseconds.

[0362] The S814 audio engine module can copy the ringtone audio data in buffer7 to buffer8 of the DMSDP audio processing module every 20 milliseconds.

[0363] Thus, with buffer7, buffer8, and buffer9 all created, the audio engine module can use S811-S812 to transmit ringtone audio data from the telephone application to the DMSDP audio processing module.

[0364] The S815 and DMSDP audio processing modules send data packets containing ringtone audio data and other stream types to the vehicle's infotainment system.

[0365] The data packet 3 may include: a header, a payload, and a trailer.

[0366] The header contains control information for data packet 3. For example, the header may include: the sender (or source address) of data packet 3, the receiver (or destination address) of data packet 3, the type of data packet 3, the length of data packet 3, and other routing types.

[0367] The data payload may include ringtone audio data.

[0368] The end of a packet usually contains additional checksum information to detect whether data errors or loss have occurred during transmission.

[0369] For example, when the DMSDP audio processing module receives ringtone audio data, it can play the ringtone audio data through the audio player 3, set other sub-stream types in the header of the data packet 3, and then send the data packet 3 containing the ringtone audio data and other sub-stream types to the vehicle system.

[0370] S816, vehicle system parses data packet 3, and determines the speaker 3 that plays ringtone audio data based on other stream types.

[0371] For example, the vehicle system can determine the speaker 3 that plays the ringtone audio data based on the second correspondence. The second correspondence can be found in the description in S616, and will not be repeated here.

[0372] S817, the vehicle's infotainment system speaker 3 plays ringtone audio data.

[0373] It is understandable that, since a second correspondence can be set in the vehicle system, the vehicle system can control the speaker 3 that plays the ringtone audio data and instruct the speaker 3 to play the ringtone audio data, as shown in steps S814-S815.

[0374] Based on this, the embodiments of this application can utilize other audio paths and playback thread 3 to play ringtone audio data under other streaming types. In this way, the vehicle system can instruct the corresponding speaker 3 to play ringtone audio data according to the second correspondence, thereby achieving accurate classification of audio data and enabling the playback format of ringtone audio data to meet the user's needs.

[0375] It is understood that in the audio processing method provided in this application embodiment, when the electronic device splits the audio data, the vehicle system can also control the volume of the audio data under different split types. For example, the vehicle system can provide a volume setting interface, which can display different split types and corresponding volume setting buttons.

[0376] Alternatively, the in-vehicle infotainment system can be configured to prioritize audio data under different streaming types. When receiving audio data from multiple streaming types, the system determines which streaming type to play based on priority. For example, the system could provide a priority setting interface that displays different streaming types and their corresponding priority setting buttons.

[0377] Alternatively, when electronic devices split audio data, the vehicle system can also perform one or more of the following controls: control the speakers under different split types, control the audio volume under different split types, or control the playback of audio data under higher priority split types.

[0378] In the case of electronic devices splitting audio data, the embodiments of this application do not limit the setting method of the vehicle system under different splitting types.

[0379] exist Figures 5-8 Based on the corresponding embodiments, Figure 9 This diagram illustrates a step for releasing an audio path, as provided in an embodiment of this application. Figure 9 As shown, the process of releasing the audio path may include the following steps:

[0380] S901, In response to the user disconnecting the vehicle's infotainment system from the electronic device, the smart mobility application cancels the callback function to the audio service module.

[0381] For example, in response to the user disconnecting the vehicle's infotainment system from the electronic device, the smart mobility application cancels the callback function to the audio service module, and the audio service module can then execute the steps shown in S902 to delete the callback function.

[0382] The operation of disconnecting the vehicle's infotainment system from an electronic device can correspond to the operation of connecting the vehicle's infotainment system to an electronic device. For example, in a scenario where the electronic device and the vehicle's infotainment system establish a communication connection via USB, the operation of disconnecting the connection can be the operation of disconnecting one end of the USB connection from the electronic device or the vehicle's infotainment system; when the electronic device and the vehicle's infotainment system establish a communication connection via Bluetooth, the operation of disconnecting the connection can be: the electronic device turning off Bluetooth, the vehicle's infotainment system turning off Bluetooth, or the distance between the electronic device and the vehicle's infotainment system exceeding a certain distance, which pre-sets the Bluetooth connection between the electronic device and the vehicle's infotainment system to disconnect, etc., and this application embodiment does not limit this.

[0383] S902, Audio service module deletes callback function.

[0384] S903, the smart mobility application sends a device disconnection event to the audio service module.

[0385] Device disconnection events are used to indicate that electronic devices have lost connection with the vehicle's infotainment system.

[0386] S904, The audio service module sends a device disconnection event to the audio policy module.

[0387] After receiving a device disconnection event, the audio strategy module can execute the following steps according to the three created audio paths: destroying playback thread 1 and releasing the music audio path, destroying playback thread 2 and releasing the navigation audio path, and destroying playback thread 3 and releasing the other audio paths. In this embodiment, the order in which the three audio paths are released is not limited.

[0388] The process of destroying playback thread 1 and releasing the music audio path can be found in steps S905-S908.

[0389] S905, The audio strategy module notifies the audio engine module to destroy playback thread 1.

[0390] S906, the audio engine module destroys playback thread 1 and releases buffer1 and buffer3.

[0391] During the destruction of playback thread 1, the audio engine module can release buffer1 and buffer3.

[0392] S907, the audio engine module notifies the DMSDP audio processing module to release the music audio path.

[0393] The S908 and DMSDP audio processing modules release the music audio path and release buffer2.

[0394] Since the DMSDP audio processing module has already created buffer2 in the S510, the DMSDP audio processing module can release buffer2.

[0395] The process of destroying playback thread 2 and releasing the navigation audio path can be found in steps S909-S912.

[0396] S909, The audio strategy module notifies the audio engine module to destroy playback thread 2.

[0397] S910, the audio engine module destroys playback thread 2, and releases buffer4 and buffer6.

[0398] During the destruction of playback thread 2, the audio engine module can release buffer4 and buffer6.

[0399] S911, the audio engine module notifies the DMSDP audio processing module to release the navigation audio path.

[0400] The S912 and DMSDP audio processing modules release the navigation audio path and release buffer5.

[0401] Since the DMSDP audio processing module has already created buffer5 in S518, the DMSDP audio processing module can release buffer5 during the process of determining the release of the navigation audio path.

[0402] The process of destroying playback thread 3 and releasing other audio paths can be found in steps S913-S916.

[0403] S913, The audio strategy module notifies the audio engine module to destroy playback thread 3.

[0404] S914, the audio engine module destroys playback thread 3, and releases buffer7 and buffer9.

[0405] During the destruction of playback thread 1, the audio engine module can release buffer7 and buffer9.

[0406] S915, the audio engine module notifies the DMSDP audio processing module to release other audio paths.

[0407] The S916 and DMSDP audio processing modules release other audio paths and release buffer8.

[0408] Since the DMSDP audio processing module has already created buffer8 in S526, the DMSDP audio processing module can release buffer8 during the process of determining the release of the audio path.

[0409] It is understandable that when electronic devices disconnect from the vehicle's infotainment system, they can determine and destroy the created audio paths and playback threads in order to reduce device power consumption.

[0410] In one possible implementation, after the S503 responds to the user's operation of connecting the vehicle's infotainment system and electronic devices, the smart mobility application can obtain the speaker's attribute information from the vehicle's infotainment system and display information such as... Figure 10 The interface shown.

[0411] It is understandable that the second correspondence can also be user-defined. Alternatively, the preset second correspondence can be customized by the user. Or, the electronic device can prioritize the user-defined second correspondence, in which case the preset second correspondence of the electronic device will not take effect temporarily.

[0412] Figure 10 This is a schematic diagram of an interface for setting up a speaker, provided as an embodiment of this application.

[0413] In response to the user's action of enabling smart mobility from Settings - More Connections, the electronic device displays as follows Figure 10 The interface shown in 'a' is a smart mobility settings interface, which may display one or more of the following: a button 1001 for managing the mobile phone-vehicle interconnection application, a button for connecting to the vehicle, and text information in the smart mobility settings interface.

[0414] In response to a trigger operation on button 1001, the electronic device displays as follows: Figure 10 The interface shown in b has buttons for connecting to the vehicle system, application management, automatic connection management, settings, supporting vehicle models, and audio playback settings.

[0415] In response to a trigger operation on button 1002, the electronic device displays as follows: Figure 10 The interface shown as 'c' displays information such as at least one speaker in the vehicle's infotainment system, the speaker's location, whether the speaker is on or off, and the speaker's volume. For example, the interface can display information about the driver's side speaker 1, such as: driver's side speaker 1 is on, and the driver's side speaker 1's volume is 40% of the maximum volume. Similarly, the interface can also display information about the passenger side speaker 1, rear speaker 1, rear speaker 2, and rear speaker 3, etc.

[0416] Understandably, electronic devices can enable users to control the speakers playing audio data and the volume of the speakers within smart mobility apps, thereby enhancing the user's listening experience.

[0417] The smart mobility app can also control the speakers playing music audio data. The app can select the speaker to control under the current streaming type. In this scenario, the smart mobility app does not need to pre-configure a second mapping relationship; it can control the speakers based on user input. Figure 10 The interface shown in 'c' allows for flexible settings of speaker playback.

[0418] In possible implementations, electronic devices could also offer multiple audio modes for users to choose from, such as... Figure 11 The interface shown is Figure 11 This is a schematic diagram of another interface for setting up a speaker, provided in an embodiment of this application.

[0419] In response to user requests Figure 10 When button 1002 (shown as b) is triggered, the electronic device displays as follows: Figure 11 The interface shown displays various audio playback modes provided by the smart mobility app. For example, playback modes may include one or more of the following: immersive mode, child mode, driver's mode, smart mode, or sleep mode.

[0420] In Immersive Mode, the electronic devices turn on all speakers in the vehicle's infotainment system; in Child Mode, the electronic devices turn down the volume of all rear speakers; in Driver Mode, the electronic devices turn on the driver's speaker and turn off the speakers in other locations; in Smart Mode, the electronic devices automatically turn on the speakers in occupied areas based on passenger conditions; and in Sleep Mode, the electronic devices turn off all speakers in the vehicle's infotainment system.

[0421] The smart mobility app can filter audio playback modes that match the target audio feed type. For example, when the target feed type is navigation, the electronic device can offer immersive mode and driver mode for the user to choose from; when the target feed type is music, the electronic device can offer immersive mode, child mode, driver mode, smart mode, and sleep mode for the user to choose from. Through the user's selection of audio playback modes, the electronic device can achieve more flexible control over the speakers in the vehicle's infotainment system.

[0422] In the above Figures 5-9 Based on the corresponding embodiments, in order to more clearly describe the embodiments of this application, Figure 12 This is a schematic flowchart illustrating an audio processing method provided in an embodiment of this application. Figure 12 As shown, the audio processing method may include the following steps:

[0423] S1201: The first electronic device receives a device connection event from the second electronic device, creates a first thread, and creates a second thread.

[0424] The first electronic device can be the electronic device described in the embodiments of this application (such as a mobile phone), and the second electronic device can be the vehicle system described in the embodiments of this application.

[0425] The meaning of device connection events can be found in the description in S505, and will not be repeated here.

[0426] The first thread can be playback thread 1 as described in the embodiments of this application, and the second thread can be playback thread 2 as described in the embodiments of this application.

[0427] S1202, The first electronic device receives a first operation applied to the first application.

[0428] The first application can be the media application described in the embodiments of this application.

[0429] The first operation can be the user playing music audio data in a media application, as described in S601.

[0430] S1203. After receiving the first operation, the first electronic device receives a request to create the first player.

[0431] The first player can be the audio player 1 described in the embodiments of this application.

[0432] The request to create the first player can be the request to create audio player 1 as described in S601.

[0433] The request to create the first player carries: a first media type and an identifier of a first application. For example, the first media type can be the media type described in the embodiments of this application. The identifier of the first application can be the package name of the media application or the UID of the media application.

[0434] S1204. The first electronic device responds to the request to create the first player and creates the first player.

[0435] S1205, The first electronic device binds the first player to the first thread.

[0436] The first thread corresponds to the first audio type, which can be the music streaming type described in the embodiments of this application.

[0437] S1206, The first electronic device receives a second operation applied to the second application.

[0438] The second application can be the navigation application described in the embodiments of this application.

[0439] The second operation can be the operation of playing navigation audio data in a navigation application, as described in S701.

[0440] S1207. After receiving the second operation, the first electronic device receives a request to create a second player.

[0441] The second player can be the audio player 2 described in the embodiments of this application.

[0442] The request to create a second player can be the request to create an audio player 2 as described in S701.

[0443] The request to create a second player includes: the first media type and the identifier of the second application. For example, the identifier of the second application could be the package name of the navigation application or the UID of the navigation application.

[0444] S1208, The first electronic device responds to the request to create a second player and creates a second player.

[0445] S1209, The first electronic device binds the second player to the second thread, the second thread corresponding to the second audio type.

[0446] The second audio type can be the navigation feed type described in the embodiments of this application.

[0447] Based on this, the first electronic device can achieve audio type splitting by creating different lines and different players for different audio types. Classifying audio types allows audio data of different types to be played in different ways, thus improving the flexibility of audio data playback.

[0448] It is understood that the sequential relationship between the steps described in the embodiments of this application is only an example and does not constitute a limitation on the embodiments of this application.

[0449] It should be noted that the module names involved in the embodiments of this application can all be defined as other names, as long as they can achieve the function of each module, and no specific restrictions are placed on the module names.

[0450] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0451] Figure 13This is a schematic diagram of the hardware structure of another electronic device provided in an embodiment of this application.

[0452] The electronic device includes a processor 13011304 and at least one communication interface ( Figure 13 (The example is illustrated using communication interface 1303).

[0453] The processor 1301 may be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of the program of the present application.

[0454] Communication line 1304 may include circuitry for transmitting information between the aforementioned components.

[0455] Communication interface 1303 uses any transceiver-like device for communicating with other devices or communication networks, such as Ethernet, wireless local area networks (WLAN), etc.

[0456] Possibly, the electronic device may also include a memory 1302.

[0457] The memory 1302 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory may exist independently and be connected to the processor via communication line 1304. The memory may also be integrated with the processor.

[0458] The memory 1302 stores computer execution instructions for implementing the scheme of this application, and the processor 1301 controls the execution. The processor 1301 executes the computer execution instructions stored in the memory 1302 to implement the method provided in the embodiments of this application.

[0459] It is possible that the computer execution instructions in the embodiments of this application may also be referred to as application code, and the embodiments of this application do not specifically limit this.

[0460] In a specific implementation, as one embodiment, the processor 13 may include one or more CPUs, for example... Figure 13 CPU0 and CPU1 in the CPU.

[0461] In a specific implementation, as one example, an electronic device may include multiple processors, for example... Figure 13 Processors 1301 and 1305 are mentioned. Each of these processors can be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. A processor here can refer to one or more devices, circuits, and / or processing cores used to process data (such as computer program instructions).

[0462] The broadcast processing method provided in this application can be applied to electronic devices with communication functions. Electronic devices include terminal devices, and the specific device form of the terminal device can be referred to the above-mentioned descriptions, which will not be repeated here.

[0463] This application provides a terminal device, which includes a processor and a memory; the memory stores computer execution instructions; the processor executes the computer execution instructions stored in the memory, causing the terminal device to perform the above-described method.

[0464] This application provides a chip. The chip includes a processor, which is used to call a computer program in memory to execute the technical solutions in the above embodiments. Its implementation principle and technical effects are similar to those in the related embodiments described above, and will not be repeated here.

[0465] This application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, it implements the methods described above. The methods described in the above embodiments can be implemented wholly or partially by software, hardware, firmware, or any combination thereof. If implemented in software, the functionality can be stored as one or more instructions or code on or transmitted over the computer-readable medium. The computer-readable medium can include computer storage media and communication media, and can also include any medium that can transfer a computer program from one place to another. The storage medium can be any target medium accessible by a computer.

[0466] In one possible implementation, a computer-readable medium may include RAM, ROM, compact disc read-only memory (CD-ROM) or other optical disc storage, disk storage or other magnetic storage devices, or any other medium targeted to carry or to store the required program code in the form of instructions or data structures, and accessible by a computer. Furthermore, any connection is appropriately referred to as a computer-readable medium. For example, if software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave, then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. As used herein, disks and optical discs include optical discs, laser discs, optical discs, Digital Versatile Discs (DVDs), floppy disks, and Blu-ray discs, where disks typically reproduce data magnetically, while optical discs optically reproduce data using lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0467] This application provides a computer program product, which includes a computer program that, when run, causes the computer to perform the above-described method.

[0468] This application describes embodiments of methods, apparatus (systems), and computer program products according to embodiments of this application with reference to flowchart illustrations and / or block diagrams. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processing unit of a general-purpose computer, special-purpose computer, embedded processor, or other programmable device to produce a machine, such that the instructions, which execute via the processing unit of the computer or other programmable data processing device, generate instructions for implementing the flowchart illustrations. Figure 1One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0469] The above specific embodiments further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solution of the present invention should be included within the scope of protection of the present invention.

Claims

1. An audio processing method, characterized in that, The methods include: The first electronic device receives a first operation acting on the first application; After receiving the first operation, create the first player; The first electronic device binds the first player to a first thread, the first thread corresponding to a first audio type; The first electronic device receives a second operation applied to the second application; After receiving the second operation, a second player is created; The first electronic device binds the second player to a second thread, which corresponds to a second audio type.

2. The method according to claim 1, characterized in that, Before the first electronic device receives the first operation applied to the first application, the method further includes: The first electronic device receives a device connection event from the second electronic device, creates the first thread, and creates the second thread; Prior to creating the first player, the method further includes: The first electronic device receives a request to create a first player, the request for creating the first player carrying: a first media type and an identifier of the first application; The creation of the first player includes: the first electronic device creating the first player in response to the request to create the first player; Prior to creating the second player, the method further includes: The first electronic device receives a request to create a second player, the request carrying: the first media type and the identifier of the second application; The creation of the second player includes: the first electronic device creating the second player in response to the request to create the second player.

3. The method according to claim 2, characterized in that, Before the first electronic device binds the first player to the first thread, the method further includes: the first electronic device determining that the first application corresponds to the first audio type based on the correspondence between the first media type, the identifier of the first application, and the first audio type; Before the first electronic device binds the second player to the second thread, the method further includes: the first electronic device determining that the second application corresponds to the second audio type based on the correspondence between the first media type, the identifier of the second application, and the second audio type.

4. The method according to claim 2 or 3, characterized in that, The method further includes: During the creation of the first thread, the first electronic device creates a first buffer and a second buffer. During the creation of the first player, the first electronic device creates a third buffer.

5. The method according to claim 4, characterized in that, The method further includes: After the first player is successfully created, the first electronic device receives the first audio data from the first application and caches the first audio data in the third cache area; The first electronic device caches the first audio data in the third buffer into the first buffer; The first electronic device caches the first audio data in the first buffer to the second buffer.

6. The method according to claim 5, characterized in that, The creation of the first buffer includes: determining a first storage space for the first buffer, and creating the first buffer that satisfies the first storage space, wherein the first storage space is determined based on one or more of the following: maximum data format, maximum number of channels, maximum sampling rate, preset frame length, or preset buffer multiple. The creation of the second buffer includes: determining the second storage space of the second buffer, and creating the second buffer that satisfies the second storage space, wherein the second storage space is determined based on one or more of the following: the data format of the first audio path, the sampling rate of the first audio path, the number of channels supported by the first audio path, or the preset frame length; The creation of the third buffer includes: determining the third storage space of the third buffer, and creating the third buffer that satisfies the third storage space, wherein the third storage space is determined based on one or more of the following: a first data format, a first number of channels, a first sampling rate, the preset frame length, or the preset buffer multiple.

7. The method according to claim 6, characterized in that, Before the first electronic device receives a device connection event from the second electronic device, the method further includes: In response to the operation of starting the first electronic device, the first electronic device loads the configuration file; The first electronic device obtains the first configuration parameter from the configuration file. Wherein, the first configuration parameter corresponds to the first audio type, and the first configuration parameter includes one or more of the following: the name of the first audio channel corresponding to the first audio type, the data format of the first audio channel, the correspondence between the first audio channel and the second electronic device, the sampling rate of the first audio channel, or the number of channels supported by the first audio channel.

8. The method according to any one of claims 5-7, characterized in that, The first electronic device includes an audio strategy module and an audio engine module. Upon receiving a device connection event from the second electronic device, the first electronic device creates a first thread, including: The audio policy module receives the device connection event; In response to receiving the device connection event, the audio strategy module sends a request to the audio engine module to create a first playback thread; In response to a request to create a first playback thread, the audio engine module creates the first playback thread.

9. The method according to claim 8, characterized in that, The first electronic device further includes: a DMSDP audio processing module, wherein the first electronic device creates a first buffer and creates a second buffer, including: The audio engine module creates the first buffer area; The audio engine module sends a message to the DMSDP audio processing module to create an audio path; In response to receiving the message for creating an audio path, the DMSDP audio processing module creates the second buffer.

10. The method according to claim 9, characterized in that, In response to the request to create the first player, the first electronic device creates the first player, including: The audio engine module receives the request to create the first player; In response to receiving the request to create the first player, the audio engine module sends a message to the audio strategy module for selecting the audio type; In response to receiving the message for selecting an audio type, the audio strategy module determines a first audio type and the first thread; The audio strategy module notifies the audio engine module to create the first player in the first thread; The audio engine module creates the first player.

11. The method according to claim 10, characterized in that, The first electronic device further includes: a smart mobility application, wherein the audio strategy module determines the first audio type, including: In response to receiving the message for selecting an audio type, the audio strategy module sends the message for selecting an audio type to the audio service module; In response to receiving the message for selecting an audio type, the audio service module obtains the first audio type from the smart mobility application; the smart mobility application includes: the first media type, the identifier of the first application, and the correspondence between the first audio types; The audio service module sends the first audio type to the audio policy module; The audio strategy module determines the first audio type.

12. The method according to any one of claims 9-11, characterized in that, The first electronic device creates a third buffer, including: the audio engine module creating the third buffer; The first electronic device receives first audio data from the first application and caches the first audio data in the third cache area, including: the audio engine module receives first audio data from the first application and caches the first audio data in the third cache area; The first electronic device caches the first audio data in the third buffer into the first buffer, including: the audio engine module caches the first audio data in the third buffer into the first buffer; The first electronic device caches the first audio data in the first buffer to the second buffer, including: the audio engine module sending the first audio data in the first buffer to the DMSDP audio processing module, and the DMSDP audio processing module caching the first audio data in the first buffer to the second buffer.

13. The method according to any one of claims 2-3, 5-7, and 9-11, characterized in that, The method further includes: The first electronic device sends first audio data and the first audio type to the second electronic device, wherein the first audio data corresponds to the first audio type; The first electronic device sends second audio data and the second audio type to the second electronic device, wherein the second audio data corresponds to the second audio type.

14. The method according to any one of claims 2-3, 5-7, and 9-11, characterized in that, The method further includes: When the first electronic device receives a device disconnection event from the second electronic device, it destroys the first thread and the second thread.

15. An audio processing method, characterized in that, The methods include: The second electronic device receives first audio data and a first audio type from the first electronic device, wherein the first audio data corresponds to the first audio type; The second electronic device plays the first audio data through a first speaker, the first speaker corresponding to the first audio type; The second electronic device receives second audio data and a second audio type from the first electronic device, wherein the second audio data corresponds to the second audio type; The second electronic device plays the second audio data through a second speaker, which corresponds to the second audio type, and the first speaker is different from the second speaker.

16. The method according to claim 15, characterized in that, The method further includes: When the second electronic device receives both the first audio data and the second audio data simultaneously, the second electronic device plays the second audio data, and the priority of the second audio type is higher than the priority of the first audio type.

17. An electronic device, characterized in that, The electronic device includes: one or more processors and memory; The memory is coupled to the one or more processors, the memory being used to store computer program code, the computer program code including computer instructions, the one or more processors calling the computer instructions to cause the electronic device to perform the method of any one of the first electronic devices as claimed in claims 1-14, or to cause the electronic device to perform the method of any one of the second electronic devices as claimed in claims 15-16.

18. A chip system applied to an electronic device, the chip system comprising one or more processors, the processors being configured to invoke computer instructions to cause the electronic device to perform the method of a first electronic device as claimed in any one of claims 1-14, or to cause the electronic device to perform the method of a second electronic device as claimed in any one of claims 15-16.

19. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes computer instructions that, when executed on an electronic device, cause the electronic device to perform the method of a first electronic device as claimed in any one of claims 1-14, or cause the electronic device to perform the method of a second electronic device as claimed in any one of claims 15-16.

20. A computer program product, characterized in that, The computer program product includes computer program code that, when run on an electronic device, causes the electronic device to perform the method of a first electronic device as claimed in any one of claims 1-14, or causes the electronic device to perform the method of a second electronic device as claimed in any one of claims 15-16.

Images