Audio processing method and electronic equipment

Abstract

The invention provides an audio processing method and electronic equipment, the electronic equipment comprises a digital signal processing DSP chip and an application processing AP system, the DSP chip comprises a first audio processing unit, a first coding and decoding unit and a first packet processing unit, and the AP system comprises a second audio processing unit, a third audio processing unit, a first application and an equipment node. The first application comprises a fourth audio processing unit, a second coding and decoding unit, a second packet processing unit and a transceiving unit, and in the method, the DSP chip can sequentially carry out audio processing, coding and packaging on first audio data through the first audio processing unit, the first coding and decoding unit and the first packet processing unit to obtain fourth audio data; sending the fourth audio data to the device node; the first application can acquire fourth audio data from the device node through the transceiving unit and send the fourth audio data. The method can be applied to the VoIP call, and can reduce the time delay of the VoIP call and improve the efficiency of the VoIP call.

Description

Technical Field

[0001] The present application relates to the technical field of electronic equipment, and in particular to an audio processing method and electronic equipment. Background Art

[0002] Voice over internet protocol (VoIP) calls are real-time call technology based on Internet communications. VoIP calls support network calls through applications in electronic devices, and the call charges are lower than other types of call methods, so they are widely used. However, the current VoIP call process has a high delay and low call efficiency. Summary of the invention

[0003] The present application provides an audio processing method and an electronic device for reducing VoIP call delay and improving VoIP call efficiency.

[0004] In a first aspect, an embodiment of the present application provides a control method, which is applied to an electronic device, wherein the electronic device includes a digital signal processing (DSP) chip and an application processing (AP) system, wherein the DSP chip includes a first audio processing unit, a first codec unit and a first packet processing unit, and the AP system includes a second audio processing unit, a third audio processing unit, a first application and a device node, wherein the first application includes a fourth audio processing unit, a second codec unit, a second packet processing unit and a transceiver unit, and the method includes: the DSP chip obtains first audio data; the DSP chip performs audio processing on the first audio data through the first audio processing unit to obtain second audio data, encodes the second audio data through the first codec unit to obtain third audio data, and packages the third audio data through the first packet processing unit to obtain fourth audio data; wherein the audio processing includes at least one of the following: acoustic echo cancellation, background noise suppression, and automatic gain control; the DSP chip sends the fourth audio data to the device node; the first application obtains the fourth audio data from the device node through the transceiver unit; and the first application sends the fourth audio data through the transceiver unit.

[0005] In this method, the audio processing task in the audio transmission process is completed by a DSP chip with a relatively mature and comprehensive audio processing algorithm. On the one hand, the audio processing task can be completed quickly and efficiently with the help of the existing audio processing capabilities of the DSP chip. On the other hand, the communication between the DSP chip and the application can be realized through the device node, reducing the delay of the audio processing in the application layer and the HAL layer of the electronic device, reducing the computing power consumption of the AP system (large core), and thus reducing the power consumption of the large core. At the same time, the number of buffers when the audio data is processed in the application framework layer and the HAL layer can be reduced, thereby reducing the demand for large core storage space. The above method can be applied to the VoIP call process, which can reduce the VoIP call delay and improve the efficiency of VoIP calls. In addition, when the above method is applied to the VoIP call process, the application is mainly used to forward audio data. The main audio processing algorithm in the audio transmission process can be concentrated in the DSP chip, ensuring that there is basically only one set of audio processing algorithms in the DSP chip on the entire audio transmission link, which can avoid redundancy of audio processing algorithms, improve the efficiency of audio transmission, and reduce application power consumption and improve the overall performance of the application.

[0006] In one possible design, the second audio processing unit is located in the application framework layer of the electronic device, the third audio processing unit is located in the hardware abstraction layer of the electronic device, and the fourth audio data does not pass through the second audio processing unit and the third audio processing unit.

[0007] In this method, audio data will not be processed in the application framework layer and the hardware abstraction layer, which can simplify the process of traditional VoIP applications, DSP chips, etc. that require complex interactions with the application framework layer and the hardware abstraction layer, thereby improving processing efficiency.

[0008] In one possible design, the fourth audio data does not pass through the fourth audio processing unit, the second codec unit and the second packet processing unit.

[0009] In this method, audio data will not be processed in the application, which can reduce resource consumption of the application, thereby reducing application power consumption and improving application performance.

[0010] In one possible design, the device node is located in the kernel layer of the electronic device.

[0011] In this method, bidirectional communication between the DSP chip and the AP side is achieved through the device nodes in the kernel layer, which can reduce the dependence on the application framework layer and the hardware abstraction layer and help improve processing efficiency.

[0012] In a possible design, the AP system also includes a first control information processing unit and a second control information processing unit, the first control information processing unit is located in the application framework layer of the electronic device, and the second control information processing unit is located in the hardware abstraction layer of the electronic device; the method also includes: the first application generates first control information for controlling the DSP chip to perform a first operation; the first application sends the first control information to the device node; the DSP chip obtains the first control information from the device node; the DSP chip performs the first operation according to the first control information; wherein, the first control information does not pass through the first control information processing unit and the second control information processing unit.

[0013] In this method, the transmission of control information can be realized through the device node, so that the transmission of control information can also be independent of the application framework layer and the hardware abstraction layer, thereby improving the transmission efficiency of control information.

[0014] In a possible design, the DSP chip sends the fourth audio data to the device node, including: the DSP chip sends first data including the fourth audio data and first identification information to the device node; wherein the first identification information is used to indicate the data type of the fourth audio data; the first application obtains the fourth audio data from the device node through the transceiver unit, including: the first application obtains the first data from the device node through the transceiver unit, and determines the fourth audio data based on the first data; the first application sends the first control information to the device node, including: the first application sends second data including the first control information and second identification information to the device node; wherein the second identification information is used to indicate the data type of the first control information; the DSP chip obtains the first control information from the device node, including: the DSP chip obtains the second data from the device node, and determines the first control information based on the second data.

[0015] In this method, audio data and control information can be distinguished through different identification information, so that the transmission of audio data and control information can be achieved in the electronic device through only one device node, which is highly practical.

[0016] In one possible design, a data type of the fourth audio data is an audio type or an uplink audio type, and a data type of the first control information is a control type or an uplink control type.

[0017] In one possible design, the device node corresponds to the data type of the fourth audio data; the AP system also includes a first control information processing unit and a second control information processing unit, the first control information processing unit is located in the application framework layer of the electronic device, and the second control information processing unit is located in the hardware abstraction layer of the electronic device; the method also includes: the first application generates first control information for controlling the DSP chip to perform a first operation; the first application sends the first control information to a device node corresponding to the data type of the first control information; wherein the device node corresponding to the data type of the first control information is in the kernel layer of the electronic device; the DSP chip obtains the first control information from the device node corresponding to the data type of the first control information; the DSP chip performs the first operation according to the first control information; wherein the first control information does not pass through the first control information processing unit and the second control information processing unit.

[0018] In this method, the transmission of control information can be realized through device nodes, so that the transmission of control information can also be independent of the application framework layer and the hardware abstraction layer, thereby improving the transmission efficiency of control information. At the same time, the transmission of audio data and the transmission of control information can be realized respectively through different device nodes, thereby ensuring that different types of data have independent data transmission channels, which can improve data transmission efficiency.

[0019] In a second aspect, an embodiment of the present application provides a control method, which is applied to an electronic device, wherein the electronic device includes a digital signal processing DSP chip and an application processing AP system, the DSP chip includes a first audio processing unit, a first codec unit and a first packet processing unit, the AP system includes a second audio processing unit, a third audio processing unit, a first application and a device node, the first application includes a fourth audio processing unit, a second codec unit, a second packet processing unit and a transceiver unit, the method includes: the first application receives first audio data through the transceiver unit; the first application sends the first audio data to the device node through the transceiver unit; the DSP chip obtains the first audio data from the device node; the DSP chip unpacks the first audio data through the first packet processing unit to obtain second audio, decodes the second audio through the first codec unit to obtain third audio, and performs audio processing on the third audio data through the first audio processing unit to obtain fourth audio data; wherein the audio processing includes at least one of the following: acoustic echo cancellation, background noise suppression, and automatic gain control.

[0020] In this method, the audio processing task in the audio transmission process is completed by a DSP chip with a relatively mature and comprehensive audio processing algorithm. On the one hand, the audio processing task can be completed quickly and efficiently with the help of the existing audio processing capabilities of the DSP chip. On the other hand, the communication between the DSP chip and the application can be realized through the device node, reducing the delay of the audio processing in the application layer and the HAL layer of the electronic device, reducing the computing power consumption of the AP system (large core), and thus reducing the power consumption of the large core. At the same time, the number of buffers when the audio data is processed in the application framework layer and the HAL layer can be reduced, thereby reducing the demand for large core storage space. The above method can be applied to the VoIP call process, which can reduce the VoIP call delay and improve the efficiency of VoIP calls. In addition, when the above method is applied to the VoIP call process, the application is mainly used to forward audio data. The main audio processing algorithm in the audio transmission process can be concentrated in the DSP chip, ensuring that there is basically only one set of audio processing algorithms in the DSP chip on the entire audio transmission link, which can avoid redundancy of audio processing algorithms, improve the efficiency of audio transmission, and reduce application power consumption and improve the overall performance of the application.

[0021] In one possible design, the second audio processing unit is located in the application framework layer of the electronic device, the third audio processing unit is located in the hardware abstraction layer of the electronic device, and the first audio data does not pass through the second audio processing unit and the third audio processing unit.

[0022] In one possible design, the first audio data does not pass through the fourth audio processing unit, the second codec unit and the second packet processing unit.

[0023] In a possible design, the DSP chip further includes a first anti-jitter unit, and the first application further includes a second anti-jitter unit; after the DSP chip unpacks the first audio data through the first packet processing unit to obtain the second audio, and before decoding the second audio through the first codec unit to obtain the third audio, the method further includes:

[0024] Performing anti-jitter processing on the second audio by using the first anti-jitter unit;

[0025] The first audio data does not pass through the second anti-jitter unit.

[0026] In one possible design, the device node is located in the kernel layer of the electronic device.

[0027] In a possible design, the AP system further includes a first control information processing unit and a second control information processing unit, the first control information processing unit is located in an application framework layer of the electronic device, and the second control information processing unit is located in a hardware abstraction layer of the electronic device; the method further includes:

[0028] The first application generates first control information for controlling the DSP chip to perform a first operation;

[0029] The first application sends the first control information to the device node;

[0030] The DSP chip obtains the first control information from the device node;

[0031] The DSP chip performs the first operation according to the first control information;

[0032] The first control information does not pass through the first control information processing unit and the second control information processing unit.

[0033] In a possible design, the first application sends the first audio data to the device node through the transceiver unit, including:

[0034] The first application sends first data including the first audio data and first identification information to the device node through the transceiver unit; wherein the first identification information is used to indicate the data type of the first audio data;

[0035] The DSP chip obtains the first audio data from the device node, including:

[0036] The DSP chip obtains the first data from the device node, and determines the first audio data according to the first data;

[0037] The first application sends the first control information to the device node, including:

[0038] The first application sends second data including the first control information and second identification information to the device node; wherein the second identification information is used to indicate the data type of the first control information;

[0039] The DSP chip obtains the first control information from the device node, including:

[0040] The DSP chip obtains the second data from the device node, and determines the first control information according to the second data.

[0041] In one possible design, the data type of the first audio data is an audio type or an uplink audio type, and the data type of the first control information is a control type or an uplink control type.

[0042] In a possible design, the device node corresponds to a data type of the first audio data; the AP system further includes a first control information processing unit and a second control information processing unit, the first control information processing unit is located in an application framework layer of the electronic device, and the second control information processing unit is located in a hardware abstraction layer of the electronic device; the method further includes:

[0043] The first application generates first control information for controlling the DSP chip to perform a first operation;

[0044] The first application sends the first control information to a device node corresponding to a data type of the first control information; wherein the device node corresponding to the data type of the first control information is located in a kernel layer of the electronic device;

[0045] The DSP chip obtains the first control information from a device node corresponding to a data type of the first control information;

[0046] The DSP chip performs the first operation according to the first control information;

[0047] The first control information does not pass through the first control information processing unit and the second control information processing unit.

[0048] In a third aspect, the present application provides an electronic device, comprising a memory and one or more processors; wherein the memory is used to store computer program code, and the computer program code comprises computer instructions; when the computer instructions are executed by one or more processors, the electronic device executes the method described in the above-mentioned first aspect or any possible design of the first aspect, or executes the above-mentioned second aspect or the method described in any possible design of the second aspect.

[0049] In a fourth aspect, the present application provides a computer-readable storage medium storing a computer program. When the computer program runs on an electronic device, the electronic device executes the method described in the first aspect or any possible design of the first aspect, or executes the method described in the second aspect or any possible design of the second aspect.

[0050] In a fifth aspect, the present application provides a computer program product, which includes a computer program or instructions. When the computer program or instructions are run on an electronic device, the electronic device executes the method described in the first aspect or any possible design of the first aspect, or executes the method described in the second aspect or any possible design of the second aspect.

[0051] For the beneficial effects of the second to fifth aspects mentioned above, please refer to the description of the beneficial effects of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 A schematic diagram of a VoIP call process;

[0053] Figure 2 A schematic diagram of the hardware architecture of an electronic device provided in an embodiment of the present application;

[0054] Figure 3 A schematic diagram of a software architecture of an electronic device provided in an embodiment of the present application;

[0055] Figure 4 A schematic diagram of the architecture of an audio processing system provided in an embodiment of the present application;

[0056] Figure 5 A schematic diagram of an audio sending method provided in an embodiment of the present application;

[0057] Figure 6 A schematic diagram of an audio receiving method provided in an embodiment of the present application;

[0058] Figure 7 A schematic diagram of an audio processing method provided in an embodiment of the present application;

[0059] Figure 8 A schematic diagram of an audio processing method provided in an embodiment of the present application;

[0060] Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the embodiments of the present application will be further described in detail below with reference to the accompanying drawings.

[0062] In the description of the embodiments of the present application, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features.

[0063] To facilitate understanding, exemplary descriptions of concepts related to the present application are provided for reference.

[0064] 1) Electronic devices can be devices with communication functions. In some embodiments of the present application, the electronic devices can be portable devices, such as mobile phones, tablet computers, wearable devices with wireless communication functions (such as watches, bracelets, etc.), vehicle-mounted terminal devices, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPC), netbooks, personal digital assistants (PDA), smart home devices (such as smart TVs, smart speakers, etc.), smart robots, workshop equipment, wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, or wireless terminals in smart homes, flight equipment (such as smart robots, drones, airplanes), etc.

[0065] In some embodiments of the present application, the electronic device may also be a portable terminal device that also includes other functions. Exemplary embodiments of the portable terminal device include but are not limited to: Or a portable terminal device with other operating systems. The portable terminal device may also be other portable terminal devices, such as a laptop computer with a touch-sensitive surface (e.g., a touch panel). It should also be understood that in some other embodiments of the present application, the electronic device may not be a portable terminal device, but a desktop computer with a touch-sensitive surface (e.g., a touch panel).

[0066] 2) Voice over Internet Protocol (VoIP) is a voice call technology that can achieve voice calls and multimedia conferences via the Internet Protocol (IP), that is, communication via the Internet. The basic principles of VoIP calls include: the sender encodes, compresses, and packages the audio data to obtain audio data packets, and then sends the data packets to the receiver through the network; the receiver unpacks, decompresses, and decodes the audio data packets from the sender to restore them to the original audio signal, thereby achieving the effect of transmitting audio via the Internet. VoIP calls can also be called or understood as voice transmission based on IP, IP telephony, Internet telephony, broadband telephony, broadband phone service, etc.

[0067] 3) 3A audio processing, abbreviated as 3A, is the collective name for three audio algorithms: acoustic echo cancellation (AEC), background noise suppression (ANS), and automatic gain control (AGC).

[0068] It should be understood that in the embodiments of the present application, "at least one" refers to one or more, and "plurality" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can represent: a, b, c, a and b, a and c, b and c, or a, b and c, where a, b, c can be single or multiple.

[0069] In the era of rich media information, audio and video have become an important way for people to obtain, publish and exchange information. Due to the upgrade of service experience requirements, the iteration of audio and video transmission technology, and the improvement of network infrastructure, people's requirements for audio and video service latency have become increasingly stringent, which has led to the vigorous growth of the real-time audio and video industry. At present, real-time audio and video services such as audio and video calls (such as Changlian, etc.), online meetings, interactive live broadcasts / classrooms, remote offices / medical care, etc. have gradually penetrated into all aspects of people's lives.

[0070] One of the most widely used real-time audio services is voice over internet protocol (VoIP) calls. VoIP calls support network calls through applications in electronic devices, and the call charges are lower than other types of call methods, so they are widely used. However, the delay of VoIP calls is generally high, resulting in low call efficiency. This is mainly because the VoIP call process takes a long time to process in the application framework layer (framework) and the hardware abstract layer (HAL). The following is a detailed description with reference to the accompanying drawings.

[0071] Figure 1 The following is a schematic diagram of the audio processing flow of a VoIP call. Figure 1As shown in , in the audio processing flow of a VoIP call, for the uplink audio link (i.e., the link for sending audio), the audio processing flow mainly includes: an audio acquisition device (such as a microphone, etc.) acquires audio data and sends the acquired audio data to a digital signal processing (DSP) chip. The DSP chip directly sends the received audio data to the HAL layer on the application processor (AP) side. After the HAL layer performs 3A audio processing on the audio data, the processed audio data is sent to the corresponding VoIP application through the application framework layer. Usually, in the VoIP application, there will also be corresponding audio processing, such as further 3A enhancement processing. The VoIP application can also encode the audio data after audio processing, package it with the real-time transport protocol (RTP), and call the corresponding data packet sending interface to send the packaged audio data to a modem (or modem chip). The modem can send the audio data. For the downlink audio link (i.e., the link for receiving audio), the audio processing flow mainly includes: the modem can receive audio data packets, and send the received audio data packets to the VoIP application for unpacking. The VoIP application can perform anti-jitter, decoding, and soft 3A processing on the unpacked audio data to obtain the audio data. The VoIP application obtains the audio data, which is then processed by the application framework layer and the HAL layer and then sent to the DSP chip. The DSP chip can send the audio data to the audio playback device (such as a microphone, etc.) for playback.

[0072] In the above-mentioned audio processing flow of VoIP calls, there are many audio processings (such as 3A audio processing, etc.) in the application framework layer and the HAL layer, which will cause a large delay. In addition, the audio processing flow of VoIP calls involves many processing modules, and the data interaction between processing modules and the data cache copy caused by the data processing within the processing module will also cause large delays and power consumption. In addition, in the audio processing flow of VoIP calls, the VoIP application carries a lot of audio processing, which will increase the power consumption of the VoIP application and may cause the performance of the VoIP application to decrease.

[0073] Based on the above problems, in order to reduce the latency and power consumption of VoIP calls, the embodiments of the present application provide an audio processing method and an electronic device. This solution can transfer the audio processing process of the application framework layer and HAL layer and even the VoIP application on the Android platform to the DSP chip, and use the audio processing capabilities of the DSP chip to achieve fast and efficient audio processing, thereby reducing the latency and power consumption of VoIP calls.

[0074] See below Figure 2 , the structure of the electronic device to which the method provided in the embodiments of the present application is applicable is introduced.

[0075] like Figure 2 As shown in , the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a USB interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a SIM card interface 195, etc.

[0076] The sensor module 180 may include a gyroscope sensor, an acceleration sensor, a proximity light sensor, a fingerprint sensor, a touch sensor, a temperature sensor, a pressure sensor, a distance sensor, a magnetic sensor, an ambient light sensor, an air pressure sensor, a bone conduction sensor, and the like.

[0077] Understandably, Figure 2 The electronic device 100 shown is only an example and does not constitute a limitation on the electronic device. The electronic device may have more Figure 2 More or fewer components may be shown, two or more components may be combined, or there may be a different configuration of components. Figure 2 The various components shown in the EMBODIMENTS 2000 may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application specific integrated circuits.

[0078] The processor 110 may include one or more processing units, for example: the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processor (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), a controller, a memory, a video codec, a digital signal processor (digital signal processor, DSP), a baseband processor, and / or a neural-network processing unit (neural-network processing unit, NPU), etc. Among them, different processing units can be independent devices or integrated in one or more processors. Among them, the controller can be the nerve center and command center of the electronic device 100. The controller can generate an operation control signal according to the instruction opcode and the timing signal to complete the control of fetching and executing instructions.

[0079] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0080] The execution of the audio processing method provided in the embodiment of the present application can be controlled by the processor 110 or completed by calling other components, such as calling the processing program of the embodiment of the present application stored in the internal memory 121, or calling the processing program of the embodiment of the present application stored in a third-party device through the external memory interface 120, to control the wireless communication module 160 to communicate data with other devices, improve the intelligence and convenience of the electronic device 100, and enhance the user experience. The processor 110 may include different devices, such as when the CPU and GPU are integrated, the CPU and GPU can cooperate to execute the audio processing method provided in the embodiment of the present application, such as part of the algorithm in the audio processing method is executed by the CPU, and the other part of the algorithm is executed by the GPU, so as to obtain faster processing efficiency.

[0081] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1. The display screen 194 can be used to display information input by a user or information provided to a user and various graphical user interfaces (GUIs). For example, the display screen 194 can display photos, videos, web pages, or files, etc. For another example, the display screen 194 can display the user control interface (such as a sound projection mode selection interface, a sound projection device selection interface, etc.) described in the embodiments of the present application.

[0082] In the embodiment of the present application, the display screen 194 may be an integrated flexible display screen, or a spliced ​​display screen consisting of two rigid screens and a flexible screen located between the two rigid screens.

[0083] The camera 193 (a front camera or a rear camera, or one camera can be used as both a front camera and a rear camera) is used to capture still images or videos. Generally, the camera 193 may include a photosensitive element such as a lens group and an image sensor, wherein the lens group includes a plurality of lenses (convex lenses or concave lenses) for collecting light signals reflected by the object to be photographed and transmitting the collected light signals to the image sensor. The image sensor generates an original image of the object to be photographed based on the light signal.

[0084] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area can store the code of the operating system, application program (such as the function corresponding to the present application scheme, etc.). The data storage area can store data created during the use of the electronic device 100, etc.

[0085] The internal memory 121 may also store one or more computer programs corresponding to the algorithm of the present application. The one or more computer programs are stored in the internal memory 121 and configured to be executed by one or more processors 110. The one or more computer programs include instructions, which may be used to execute the various steps in the following embodiments.

[0086] In addition, the internal memory 121 may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0087] Of course, the code of the algorithm of the embodiment of the present application can also be stored in an external memory. In this case, the processor 110 can run the code of the algorithm of the embodiment of the present application stored in the external memory through the external memory interface 120.

[0088] The sensor module 180 may include a gyro sensor, an acceleration sensor, a proximity light sensor, a fingerprint sensor, a touch sensor, and the like.

[0089] The touch sensor is also called a "touch panel". The touch sensor can be arranged on the display screen 194, and the touch sensor and the display screen 194 form a touch display screen, also called a "touch screen". The touch sensor is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor can also be arranged on the surface of the electronic device 100, which is different from the position of the display screen 194.

[0090] Exemplarily, the display screen 194 of the electronic device 100 displays a main interface, which includes icons of multiple applications (such as a camera application, etc.). For example, a user can click the icon of the camera application in the main interface through a touch sensor to trigger the processor 110 to start the camera application and turn on the camera 193. The display screen 194 displays the interface of the camera application, such as a viewfinder interface.

[0091] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0092] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0093] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110. In an embodiment of the present application, the mobile communication module 150 can also be used to interact with other devices for information.

[0094] The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a receiver 170B, etc.), or displays an image or video through a display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0095] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (WiFi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2. In the embodiment of the present application, the wireless communication module 160 is used to establish a connection with other electronic devices and exchange data. Or the wireless communication module 160 can be used to access the access point device, send control instructions to other electronic devices, or receive data sent from other electronic devices.

[0096] In addition, the electronic device 100 can implement audio functions through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone interface 170D, and the application processor. For example, music playback, recording, etc. The electronic device 100 can receive input from the key 190 and generate key signal input related to the user settings and function control of the electronic device 100. The electronic device 100 can use the motor 191 to generate a vibration prompt (such as an incoming call vibration prompt). The indicator 192 in the electronic device 100 can be an indicator light, which can be used to indicate the charging status, power changes, and can also be used to indicate messages, missed calls, notifications, etc. The SIM card interface 195 in the electronic device 100 is used to connect the SIM card. The SIM card can be inserted into the SIM card interface 195, or pulled out from the SIM card interface 195, to achieve contact and separation with the electronic device 100.

[0097] It should be understood that in practical applications, the electronic device 100 may include Figure 2The embodiments of the present application are not limited to the more or fewer components shown. The illustrated electronic device 100 is only an example, and the electronic device 100 may have more or fewer components than those shown in the figure, may combine two or more components, or may have different component configurations. The various components shown in the figure may be implemented in hardware, software, or a combination of hardware and software including one or more signal processing and / or application specific integrated circuits.

[0098] The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture. The layered architecture divides the software into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. For example, Figure 3 As shown, the software architecture can be divided into four layers, from top to bottom, namely the application layer, the application framework layer (framework, FWK), the runtime and system library, and the (Linux) kernel layer.

[0099] The application layer is the top layer of the operating system, including native applications of the operating system, such as camera, gallery, calendar, Bluetooth, music, video, information, etc., and may also include third-party applications. The application involved in the embodiment of the present application is referred to as application (APP), which is a software program that can implement one or more specific functions. Typically, multiple applications can be installed in an electronic device, such as a camera application, a mailbox application, etc. The application mentioned below may be a system application installed when the electronic device leaves the factory, or it may be a third-party application downloaded from the Internet or obtained from other electronic devices by the user while using the electronic device.

[0100] Of course, for developers, developers can write applications and install them in this layer. In a possible implementation, applications can be developed using the Java language by calling the application programming interface (API) provided by the application framework layer. Developers can interact with the bottom layer of the operating system (such as the kernel layer) through the application framework to develop their own applications.

[0101] The application framework layer is the API and programming framework of the application layer. The application framework layer may include some predefined functions. The application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, etc.

[0102] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the display screen (or screen), capture the display screen, etc.

[0103] Content providers are used to store and retrieve data and make these data accessible to applications. The data may include files (such as documents, videos, images, audio), text and other information.

[0104] The view system includes visual controls, such as controls for displaying text, images, documents, etc. The view system can be used to build applications. The interface in the display window can be composed of one or more views. For example, the display interface including the SMS notification icon can include a view for displaying text and a view for displaying images.

[0105] The phone manager is used to provide communication functions for electronic devices. The notification manager enables applications to display notification information in the status bar, which can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction.

[0106] The runtime includes the core library and the virtual machine. The runtime is responsible for the scheduling and management of the system.

[0107] The core library of the system consists of two parts: one is the function that the Java language needs to call, and the other is the core library of the system. The application layer and the application framework layer run in the virtual machine. Taking Java as an example, the virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0108] The system library can include multiple functional modules. For example: surface manager, media library, 3D graphics processing library (for example: OpenGL ES), 2D graphics engine (for example: SGL), image processing library, etc. The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications. The media library supports a variety of commonly used audio and video format playback and recording, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.564, MP3, AAC, AMR, JPG, PNG, etc. The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis and layer processing, etc. The 2D graphics engine is a drawing engine for 2D drawing.

[0109] The kernel layer provides the core system services of the operating system, such as security, memory management, process management, network protocol stack and driver model, which are all implemented based on the kernel layer. The kernel layer also serves as an abstraction layer between the hardware and software stacks. This layer has many drivers related to electronic devices, the main drivers are: display driver; keyboard driver as input device; Flash driver based on memory technology device; camera driver; audio driver; Bluetooth driver; WiFi driver, etc.

[0110] It should be understood that the functional services described above are only examples, and in actual applications, electronic devices may also be divided into more or fewer functional services according to other factors, or the functions of each service may be divided in other ways, or the functional services may not be divided but work as a whole.

[0111] The solution provided in the embodiments of the present application is described in detail below.

[0112] The solution provided in the embodiment of the present application can be applied to the scene of audio transmission and reception processing, for example, it can be applied to the VoIP call scene. Exemplarily, the VoIP call scene may include a Changlian call, a voice call of a social software, etc., which is not specifically limited in the embodiment of the present application.

[0113] Figure 4 The following is a schematic diagram of the architecture of an audio processing system provided in an embodiment of the present application. The system can be deployed in an electronic device. Figure 4 As shown in , the audio processing system at least includes a VoIP application, a DSP chip, and a device node deployed in the core layer of the electronic device.

[0114] The VoIP application may be an application installed in an electronic device for conducting VoIP calls. The device node may serve as a data transmission channel between the DSP chip and the VoIP application. Specifically, the device node may send audio data from the DSP chip to the VoIP application, or may send audio data from the VoIP application to the DSP chip.

[0115] In some embodiments of the present application, the device node may include two data queues, which may be used to transmit an upstream data stream and a downstream data stream, respectively. The upstream data stream may be a data stream from a DSP chip to a VoIP application, and the downstream data stream may be a data stream from a VoIP application to a DSP chip.

[0116] In some embodiments of the present application, the device node can also be used to transmit control information. Optionally, the control information can be control information for controlling the start and stop of the DSP chip. As an optional implementation, the audio processing system may include multiple device nodes, wherein different data nodes are used to transmit different types of data. For example, the audio processing system may include two device nodes, wherein one device node can be used to transmit an audio data stream, and the other device node can be used to transmit a control information stream (or referred to as a control data stream). As another optional implementation, the audio processing system may include a device node. The device node can transmit both an audio data stream and a control information stream. Wherein, the header of the audio data and / or control information is attached with identification information for identifying the data type to which it belongs (for example, it can include an uplink data type or a downlink data type, an audio data type or a control data type, etc.), and each component in the audio processing system can identify different types of data based on the identification information.

[0117] For example, Figure 4 As shown in the downlink control link in FIG, the VoIP application can control the DSP chip by sending control information to the DSP chip through the device node.

[0118] Optionally, the audio processing system may further include at least one of the following: a communication unit, an audio acquisition device, and an audio playback device.

[0119] In some embodiments of the present application, the audio acquisition device, DSP chip, device node, VoIP application and communication unit can form an uplink audio link (i.e., a link for sending audio, also referred to as an audio sending link), which can be used to send the audio collected by the audio acquisition device. The communication unit, VoIP application, device node, DSP chip and audio playback device can form a downlink audio link (i.e., a link for receiving audio, also referred to as an audio receiving link), which can be used to receive and play audio.

[0120] In the embodiment of the present application, the audio acquisition device can be used to collect audio in the environment, such as user audio, and send the collected audio data to the DSP chip. Exemplarily, the audio acquisition device can be a microphone, etc.

[0121] The data transmitted on the uplink audio link may be referred to as uplink data, and the data transmitted on the downlink audio link may be referred to as downlink data.

[0122] In the embodiment of the present application, the audio playback device can be used to receive audio data from the DSP chip and play the corresponding audio. Exemplarily, the audio playback device can be a speaker, earphones, etc.

[0123] In some embodiments of the present application, the communication unit may be used to send or receive audio data. Exemplarily, the communication unit may be a modem or a WiFi chip or other device that can send and receive data over a network.

[0124] In the first possible solution, for the uplink audio link, the DSP chip can be used to receive audio data from the audio acquisition device and send the received audio data to the device node. The device node can be used to send the audio data from the DSP chip to the VoIP application. The VoIP application can be used to receive audio data from the device node, perform audio processing, encoding and packaging on the received audio data, and send the processed audio data (data packet format) to the communication unit. Among them, the audio processing performed by the VoIP application on the audio data can include 3A enhancement processing, etc. The communication unit can be used to send the audio data from the VoIP application through the network. For the downlink audio link, the communication unit can be used to receive audio data through the network and send the received audio data to the VoIP application. The VOIP application can be used to receive audio data from the communication unit, and perform unpacking, anti-jitter, decoding, audio processing, etc. on the received audio data, and send the processed audio data to the device node. The device node can be used to send the audio data from the VoIP application to the DSP chip. The DSP chip can be used to send the audio data to the audio playback device for playback.

[0125] In the second possible solution, if Figure 4 As shown in, for the uplink audio link, the DSP chip can be used to receive audio data from the audio acquisition device, and perform audio processing, encoding, packaging, etc. on the received audio data, and send the processed audio data (data packet format) to the device node. The device node can be used to send the audio data from the DSP chip to the VoIP application. The VOIP application can be used to send the audio data from the device node to the communication unit. The communication unit can be used to send the audio data from the VoIP application through the network. For the downlink audio link, the communication unit can be used to receive audio data through the network, and send the received audio data to the VoIP application. The VOIP application can be used to receive audio data (data packet format) from the communication unit, and send the received audio data to the device node. Optionally, the VoIP application can receive the audio data from the communication unit through the forwarding service, and send the received audio data to the device node. The device node can be used to send the audio data from the VoIP application to the DSP chip. The DSP chip can be used to unpack, de-jitter, decode, and process the received audio data, and send the processed audio data to the audio playback device for playback.

[0126] Optional, such as Figure 4 As shown in , in the above scheme, when the VoIP application sends data to the communication unit, the data can be transparently transmitted to the kernel layer of the electronic device through the application framework layer and HAL layer of the electronic device, and the data can be delivered to the communication unit through the IP stack (IP stack), smart packet engine (smart packet engine, SPE) and IP packet filtering (IP filtering, IPF) service (used to filter upstream and downstream data and move between cores) in the kernel layer. When the communication unit sends data to the VoIP application, the data can be delivered to the VoIP application through the transmission of IPF, SPE and IP protocol stack in the kernel layer and the transparent transmission of HAL layer and application framework layer.

[0127] In some embodiments of the present application, Figure 4 The VoIP application in the audio processing system shown can be deployed in the application layer of the electronic device, the device node can be deployed in the kernel layer of the electronic device, the data of the VoIP application can be transparently transmitted to the device node through the application framework layer and HAL layer of the electronic device, and the data of the device node can be transparently transmitted to the VoIP application through the HAL layer and application framework layer of the electronic device.

[0128] It should be understood that the system architecture of the audio processing system described above is only an example. In actual applications, the audio processing system may also include more or fewer functional modules (or services), or the functions of each module may be divided in other ways, or the functional modules may not be divided, but the system may work as a whole.

[0129] Based on the above solution, the embodiment of the present application provides an audio sending method, which can be applied to the above audio processing system, such as Figure 5 As shown in , the execution process of the method may include:

[0130] S501: An audio acquisition device in an electronic device acquires audio data and encodes the acquired audio data into a PCM audio data stream.

[0131] In an optional implementation, the audio acquisition device may use an audio encoder to encode the acquired audio data into a pulse code modulation (PCM) audio data stream.

[0132] S502: The audio acquisition device sends the PCM audio data stream to the DSP chip in the electronic device.

[0133] S503: The DSP chip performs 3A processing, encoding and packaging on the PCM audio data stream to obtain an audio data packet.

[0134] The DSP chip performs 3A processing on the PCM audio data stream, which may specifically be: the DSP chip performs at least one of the 3A processing on the PCM data stream, for example, the DSP chip performs AEC processing on the PCM audio data stream.

[0135] In some embodiments of the present application, after the DSP chip performs 3A processing on the PCM audio data stream, the obtained audio data stream can be encoded into a format suitable for transmission on the network (such as the sound coding format (opus), advanced audio coding (AAC) format, etc.), and then the encoded audio data stream is packaged to obtain an audio data packet.

[0136] Optionally, the DSP chip may compress the audio data stream before encoding the audio data stream, and then encode the compressed audio data stream.

[0137] In an optional implementation, the DSP chip may package the encoded audio data stream into an RTP data packet or a real-time transport control protocol (RTCP) data packet.

[0138] S504: The DSP chip sends the audio data packet to the device node in the kernel layer of the electronic device.

[0139] S505: The VoIP application in the electronic device obtains an audio data packet from the device node.

[0140] S506: The VoIP application sends the audio data packet to the communication unit of the electronic device.

[0141] S507: The communication unit sends an audio data packet.

[0142] Optionally, the communication unit may send the audio data packet to a server or other electronic devices via a network.

[0143] Based on the above solution, the embodiment of the present application also provides an audio receiving method, which can be applied to the above audio processing system, such as Figure 6 As shown in , the execution process of the method may include:

[0144] S601: The communication unit of the electronic device receives an audio data packet.

[0145] Optionally, the communication unit may receive audio data packets from a server or other electronic devices via a network.

[0146] S602: The communication unit sends the audio data packet to the VoIP application in the electronic device.

[0147] S603: The VoIP application sends the audio data packet to the device node in the kernel layer of the electronic device.

[0148] S604: The DSP chip in the electronic device obtains the audio data packet from the device node.

[0149] S605: The DSP chip unpacks the audio data packet to obtain an audio data stream, decodes the audio data stream into a PCM audio data stream, and performs 3A processing on the PCM audio data stream.

[0150] Among them, when the audio data packet is a data packet in RTP format, the DSP chip can unpack the audio data packet based on the RTP protocol; when the audio data packet is a data packet in RTCP format, the DSP chip can unpack the audio data packet based on the RTCP protocol.

[0151] In some embodiments of the present application, after the DSP chip unpacks the audio data packet to obtain the audio data stream, before decoding the audio data stream into a PCM audio data stream, the AJB algorithm may be used to perform anti-jitter processing on the audio data stream.

[0152] In an optional implementation, the DSP chip may utilize an audio decoder to decode the audio data stream into a PCM audio data stream.

[0153] The DSP chip performs 3A processing on the PCM audio data stream, which may be: the DSP chip performs at least one of the 3A processing on the PCM data stream, for example, the DSP chip performs AEC processing on the PCM audio data stream.

[0154] Optionally, when the audio data stream is a compressed audio data stream, the DSP chip can decompress it and then perform processing such as decoding.

[0155] S606: The DSP chip sends the PCM audio data stream after 3A processing to the audio playback device in the electronic device.

[0156] S607: The audio playback device plays the received PCM audio data stream.

[0157] Above Figure 5 or Figure 6In the method shown, in addition to the transmission of audio data, it is also necessary to control the DSP chip, etc. Since the VoIP call on the existing Android platform is implemented based on the audio framework, and the VoIP uplink and downlink audio links provided by the above embodiment of the present application avoid the application layer and the HAL layer, the audio framework cannot be used, and a new control channel needs to be designed. In view of this, as described in the above method, the electronic device can create at least one device node for transmitting audio data and control information between the DSP chip and the VoIP application in the kernel layer, and at least one device node can be used as a transmission channel for uplink and downlink audio data and / or uplink and downlink control information, wherein each device node can be used to transmit at least one type of data. When a certain device node is used to transmit multiple types of data, different types of data can be distinguished by the identification information added to the data header for identifying the data type. For uplink data (including uplink audio data and / or uplink control information), the DSP chip can send the uplink data to the device node for transmitting uplink data in at least one device node, and the VoIP application can read the uplink data from the device node. For downlink data (including downlink audio data and / or downlink control information), the VoIP application may send the downlink data to a device node for transmitting downlink data in at least one device node, and the DSP chip may read the downlink data from the device node.

[0158] Exemplarily, in a possible implementation, the electronic device may create a first device node for transmitting uplink and downlink audio data and a second device node for transmitting uplink and downlink control information in the kernel layer. Figure 5 or Figure 6 The device node in the method shown can be a first device node. In addition, when the VoIP application controls the start and stop of the acquisition and playback (i.e., acquisition and playback) of the DSP chip, the corresponding control information can be sent to the second device node. The DSP chip can read the control information from the second device node and perform corresponding processing according to the instructions of the control information. In another possible implementation, the electronic device can create a device node in the kernel layer for transmitting uplink and downlink audio data and uplink and downlink control information. Figure 5 or Figure 6The device node in the method shown can be the device node. In this scenario, the uplink and downlink audio data and the uplink and downlink control information can be distinguished by the identification header, that is, the electronic device can add identification information to the header of the audio data and the control information to indicate the data type, thereby distinguishing different types of data. Among them, the identification information corresponding to the uplink audio data, the downlink audio data, the uplink control information, and the downlink control information is different. For example, the identification information corresponding to the uplink audio data (for example, the audio data sent by the DSP chip to the device node and the device node to the VoIP application) can be 0xDF50. The identification information corresponding to the downlink audio data (for example, the audio data sent by the VoIP application to the device node and the device node to the DSP chip) is 0xDF51. The identification information corresponding to the uplink control information (for example, the control information sent by the DSP chip to the device node and the device node to the VoIP application) can be 0xDF52. The identification information corresponding to the downlink control information (for example, the control information sent by the VoIP application to the device node and the device node to the DSP chip) is 0xDF53. Based on this method, each module or device in the electronic device (for example, the VoIP application, the DSP chip, the device node, etc.) can accurately identify different types of data. In another possible implementation, the electronic device may create in the kernel layer a first device node for transmitting uplink audio data, a second device node for transmitting downlink audio data, a third device node for transmitting uplink control information, and a fourth device node for transmitting downlink control information. Figure 5 The device node in the method shown may be a first device node. Figure 6 The device node in the method shown can be the second device node. In addition, when the VoIP application controls the start and stop of the DSP chip, the corresponding control information can be sent to the fourth device node, and the DSP chip can read the control information from the fourth device node and perform corresponding processing according to the instructions of the control information.

[0159] In the above method, two-way communication between the DSP chip and the AP side is realized through the device node. The VoIP application and the DSP chip can directly read and write data from the device node. Therefore, it can simplify the process of traditional VoIP applications, DSP chips, etc. that require complex interactions with the application framework layer and HAL layer, thereby improving processing efficiency.

[0160] Considering that the 3A processing, encoding and decoding, AJB and other processing algorithms in the current DSP chip are becoming more and more mature, and a large part of the delay of VoIP calls on the Android platform comes from the audio processing of the application framework layer and the HAL layer, the above method does not use the application framework layer and the HAL layer for audio processing, but uses the DSP chip to complete the audio processing tasks in the VoIP uplink audio link or the VoIP downlink audio link. On the one hand, the audio processing task can be completed quickly and efficiently with the help of the existing audio processing capabilities of the DSP chip. On the other hand, the processing delay of the application layer and the HAL layer can be reduced, and the computing power consumption of the AP side (large core) can be reduced, thereby reducing the power consumption of the large core. At the same time, the number of buffers for audio streams in the application framework layer and the HAL layer can be reduced, thereby reducing the demand for large core storage space. It has been verified by experiments that under the condition that the mean opinion score (MOS) does not decrease, under different network qualities, the average end-to-end delay of VoIP calls implemented based on the above scheme can be reduced by more than 250 milliseconds (i.e., more than 50%), which can be clearly perceived by users.

[0161] In addition, the above method removes the 3A processing, encoding and decoding and other processing solutions performed in the VoIP application in the existing solution. Therefore, the VoIP application is mainly used to forward the audio data stream, and the main audio processing algorithms in the uplink and downlink audio links are concentrated in the DSP chip, ensuring that there is basically only one set of audio processing algorithms in the DSP chip on the entire link, which can improve the efficiency of audio reception and transmission processing, while reducing the power consumption of the VoIP application and improving the overall performance of the VoIP application.

[0162] It should be noted that the specific implementation processes provided in the above embodiments are only examples of the applicable method processes of the embodiments of the present application, wherein the execution order of each step can be adjusted accordingly according to actual needs, and other steps can be added or some steps can be reduced. The specific implementation methods of each step can refer to the relevant introduction in the previous embodiments, and will not be described in detail here.

[0163] Based on the above embodiments and the same technical concept, the embodiments of the present application also provide an audio processing method, which can be applied to an electronic device, wherein the electronic device includes a DSP chip and an application processing AP system, the DSP chip includes a first audio processing unit, a first codec unit and a first packet processing unit, the AP system includes a second audio processing unit, a third audio processing unit, a first application and a device node, and the first application includes a fourth audio processing unit, a second codec unit, a second packet processing unit and a transceiver unit. The second audio processing unit is located in the application framework layer of the electronic device, the third audio processing unit is located in the hardware abstraction layer of the electronic device, and the device node is located in the kernel layer of the electronic device.

[0164] In the above scheme, the first audio processing unit, the second audio processing unit, the third audio processing unit and the fourth audio processing unit may have audio processing capabilities, for example, they may support 3A processing of audio, anti-jitter processing, etc., which are not listed one by one in the embodiments of the present application. The first codec unit and the second codec unit may have audio encoding and audio decoding functions. The first packet processing unit and the second packet processing unit may have data packing and unpacking functions.

[0165] In one example, the AP system can be Figure 4 The system on the AP side shown in FIG. 1 ; the first application can be Figure 4 For the VoIP application shown in FIG, the transceiver unit can be Figure 4 The forwarding unit in the VoIP application shown in FIG; the DSP chip can be Figure 4 The DSP chip shown in ; the second audio processing unit may include Figure 4 The third audio processing unit may include the audio tracks, audio frameworks and other functional services shown in Figure 4 3A, sound card driver and other functional services shown in.

[0166] like Figure 7 As shown in , the audio processing method may include:

[0167] S701: The DSP chip obtains first audio data.

[0168] Optionally, the first audio data may be audio data collected by an audio collection device in the electronic device described in the foregoing embodiment.

[0169] S702: The DSP chip performs audio processing on the first audio data through the first audio processing unit to obtain second audio data, encodes the second audio data through the first codec unit to obtain third audio data, and packages the third audio data through the first packet processing unit to obtain fourth audio data; wherein the audio processing includes at least one of the following: acoustic echo cancellation, background noise suppression, and automatic gain control.

[0170] Optionally, the fourth audio data may be an audio data packet processed by a DSP chip in the electronic device described in the aforementioned embodiment.

[0171] S703: The DSP chip sends the fourth audio data to the device node.

[0172] S704: The first application obtains fourth audio data from the device node through the transceiver unit.

[0173] S705: The first application sends fourth audio data through the transceiver unit.

[0174] Optionally, the electronic device may further include a communication unit, and the manner in which the first application sends the fourth audio data through the transceiver unit may be: the first application sends the fourth audio data to the communication unit of the electronic device through the transceiver unit. The communication unit may send the fourth audio data to other electronic devices. The specific process may refer to the relevant description in the aforementioned embodiment, which will not be repeated here.

[0175] In the above solution, during the transmission of the fourth audio data, the fourth audio data does not pass through the second audio processing unit and the third audio processing unit, thereby avoiding the delay caused by audio processing at the application card framework layer and the HAL layer.

[0176] In the above solution, the fourth audio data does not pass through the fourth audio processing unit, the second codec unit, and the second packet processing unit. Therefore, the delay caused by audio processing in the first application can be avoided, and the power consumption of the first application can be reduced, thereby improving the overall performance of the first application.

[0177] In the above scheme, the audio processing of audio data is mainly completed in the DSP chip, which can use the audio processing capabilities of the current DSP chip itself to complete the main audio processing tasks in the audio receiving and transmitting process of electronic equipment, thereby improving the overall processing efficiency.

[0178] In some embodiments of the present application, the device node described in the above method may correspond to the data type of the fourth audio data; optionally, the data type of the fourth audio data may be an audio type or an uplink audio type. The AP system may also include a first control information processing unit and a second control information processing unit, the first control information processing unit being located at an application framework layer of the electronic device, and the second control information processing unit being located at a hardware abstraction layer of the electronic device; the first control information processing unit and the second control information processing unit may be used to process and transmit control information.

[0179] Exemplarily, the first control information processing unit may include Figure 4 The audio framework shown in .

[0180] The AP system may also include a device node corresponding to the data type of the first control information, and the function of the device node is similar to that of the device node described in the above embodiment, except that the data type transmitted is different. The above audio processing method may also include: the first application generates the first control information for controlling the DSP chip to perform the first operation; the first application sends the first control information to the device node corresponding to the data type of the first control information; wherein the device node corresponding to the data type of the first control information is in the kernel layer of the electronic device; the DSP chip obtains the first control information from the device node corresponding to the data type of the first control information; the DSP chip performs the first operation according to the first control information; wherein the first control information does not pass through the first control information processing unit and the second control information processing unit. Optionally, the data type of the first control information may be a control type or an uplink control type.

[0181] Exemplarily, the first operation may be operations such as starting and stopping the production and broadcasting as described in the aforementioned embodiments.

[0182] In other embodiments of the present application, the AP system may also include a first control information processing unit and a second control information processing unit, the first control information processing unit is located at the application framework layer of the electronic device, and the second control information processing unit is located at the hardware abstraction layer of the electronic device; the first control information processing unit and the second control information processing unit can be used to process and transmit control information.

[0183] Exemplarily, the first control information processing unit may include Figure 4 The audio framework shown in .

[0184] The above audio processing method may further include: the first application generates first control information for controlling the DSP chip to perform the first operation;

[0185] The first application sends the first control information to the device node; the DSP chip obtains the first control information from the device node; the DSP chip performs the first operation according to the first control information; wherein the first control information does not pass through the first control information processing unit and the second control information processing unit.

[0186] Exemplarily, the first operation may be operations such as starting and stopping the production and broadcasting as described in the aforementioned embodiments.

[0187] In some embodiments of the present application, based on the above method, the DSP chip sends the fourth audio data to the device node in a manner that: the DSP chip sends first data including the fourth audio data and first identification information to the device node; wherein the first identification information is used to indicate the data type of the fourth audio data. The first application sends the first control information to the device node in a manner that: the first application sends second data including the first control information and the second identification information to the device node; wherein the second identification information is used to indicate the data type of the first control information. Optionally, the data type of the fourth audio data may be an audio type or an uplink audio type, and the data type of the first control information may be a control type or an uplink control type. Based on this method, the device node can distinguish between uplink audio data and uplink control information by identification information.

[0188] Based on the above method, the first application may obtain the fourth audio data from the device node through the transceiver unit in the following manner: the first application obtains the first data from the device node through the transceiver unit, and determines the fourth audio data according to the first data. The DSP chip may obtain the first control information from the device node in the following manner: the DSP chip obtains the second data from the device node, and determines the first control information according to the second data.

[0189] The functional services in the electronic device provided in the above embodiment can refer to the Figure 1 or Figure 4 For the relevant description in , the specific implementation method corresponding to the above method can refer to the introduction of the audio sending method provided in the aforementioned embodiment, which will not be repeated here.

[0190] Based on the above embodiments and the same technical concept, the embodiments of the present application also provide an audio processing method, which can be applied to an electronic device, wherein the electronic device includes a DSP chip and an application processing AP system, the DSP chip includes a first audio processing unit, a first codec unit and a first packet processing unit, the AP system includes a second audio processing unit, a third audio processing unit, a first application and a device node, and the first application includes a fourth audio processing unit, a second codec unit, a second packet processing unit and a transceiver unit. The second audio processing unit is located in the application framework layer of the electronic device, the third audio processing unit is located in the hardware abstraction layer of the electronic device, and the device node is located in the kernel layer of the electronic device.

[0191] In the above scheme, the first audio processing unit, the second audio processing unit, the third audio processing unit and the fourth audio processing unit may have audio processing capabilities, for example, they may support 3A processing of audio, anti-jitter processing, etc., which are not listed one by one in the embodiments of the present application. The first codec unit and the second codec unit may have audio encoding and audio decoding functions. The first packet processing unit and the second packet processing unit may have data packing and unpacking functions.

[0192] In one example, the AP system can be Figure 4 The system on the AP side shown in FIG. 1 ; the first application can be Figure 4 For the VoIP application shown in FIG, the transceiver unit can be Figure 4 The forwarding unit in the VoIP application shown in FIG; the DSP chip can be Figure 4 The DSP chip shown in ; the second audio processing unit may include Figure 4 The third audio processing unit may include the audio tracks, audio frameworks and other functional services shown in Figure 4 3A, sound card driver and other functional services shown in.

[0193] like Figure 8 As shown in , the audio processing method may include:

[0194] S801: A first application receives first audio data through a transceiver unit.

[0195] Optionally, the electronic device may further include a communication unit, which may be used to receive audio data packets from other electronic devices. Specific reference is made to the description in the aforementioned embodiment, which will not be described in detail herein. The first audio data may be an audio data packet received by the communication unit in the electronic device described in the aforementioned embodiment. The manner in which the first application receives the first audio data via the transceiver unit may be: the first application receives the first audio data from the communication unit via the transceiver unit.

[0196] S802: The first application sends the first audio data to the device node through the transceiver unit.

[0197] S803: The DSP chip obtains the first audio data from the device node.

[0198] S804: The DSP chip unpacks the first audio data through the first packet processing unit to obtain the second audio data, decodes the second audio through the first codec unit to obtain the third audio data, and performs audio processing on the third audio data through the first audio processing unit to obtain the fourth audio data; wherein the audio processing includes at least one of the following: acoustic echo cancellation, background noise suppression, and automatic gain control.

[0199] Optionally, after obtaining the fourth audio data, the DSP chip may send the fourth audio data to an audio playback device in the electronic device, and the audio playback device may play the fourth audio data.

[0200] In the above solution, during the transmission of the first audio data, the first audio data does not pass through the second audio processing unit and the third audio processing unit, thereby avoiding the delay caused by audio processing at the application card framework layer and the HAL layer.

[0201] In the above solution, the first audio data does not pass through the fourth audio processing unit, the second codec unit, and the second packet processing unit. Therefore, the delay caused by audio processing in the first application can be avoided, and the power consumption of the first application can be reduced, thereby improving the overall performance of the first application.

[0202] In the above scheme, the audio processing of audio data is mainly completed in the DSP chip, which can use the audio processing capabilities of the current DSP chip itself to complete the main audio processing tasks in the audio receiving and transmitting process of electronic equipment, thereby improving the overall processing efficiency.

[0203] In some embodiments of the present application, the device node described in the above method may correspond to the data type of the first audio data; optionally, the data type of the fourth audio data may be an audio type or an uplink audio type. The AP system may also include a first control information processing unit and a second control information processing unit, the first control information processing unit being located at the application framework layer of the electronic device, and the second control information processing unit being located at the hardware abstraction layer of the electronic device; the first control information processing unit and the second control information processing unit may be used to process and transmit control information.

[0204] Exemplarily, the first control information processing unit may include Figure 4 The audio framework shown in .

[0205] The AP system may also include a device node corresponding to the data type of the first control information, and the function of the device node is similar to that of the device node described in the above embodiment, except that the data type transmitted is different. The above audio processing method may also include: the first application generates first control information for controlling the DSP chip to perform a first operation; the first application sends the first control information to a device node corresponding to the data type of the first control information; wherein the device node corresponding to the data type of the first control information is in the kernel layer of the electronic device; the DSP chip obtains the first control information from the device node corresponding to the data type of the first control information; the DSP chip performs a first operation according to the first control information; wherein the first control information does not pass through the first control information processing unit and the second control information processing unit.

[0206] Exemplarily, the first operation may be operations such as starting and stopping the production and broadcasting as described in the aforementioned embodiments.

[0207] In other embodiments of the present application, the AP system may also include a first control information processing unit and a second control information processing unit, the first control information processing unit is located at the application framework layer of the electronic device, and the second control information processing unit is located at the hardware abstraction layer of the electronic device; the first control information processing unit and the second control information processing unit can be used to process and transmit control information.

[0208] Exemplarily, the first control information processing unit may include Figure 4 The audio framework shown in .

[0209] The above-mentioned audio processing method may also include: the first application generates first control information for controlling the DSP chip to perform a first operation; the first application sends the first control information to the device node; the DSP chip obtains the first control information from the device node; the DSP chip performs the first operation according to the first control information; wherein the first control information does not pass through the first control information processing unit and the second control information processing unit.

[0210] Exemplarily, the first operation may be operations such as starting and stopping the production and broadcasting as described in the aforementioned embodiments.

[0211] In some embodiments of the present application, based on the above method, the way in which the first application sends the first audio data to the device node through the transceiver unit may be: the first application sends the first data including the first audio data and the first identification information to the device node through the transceiver unit; wherein the first identification information is used to indicate the data type of the first audio data. The way in which the first application sends the first control information to the device node may be: the first application sends the second data including the first control information and the second identification information to the device node; wherein the second identification information is used to indicate the data type of the first control information. The data type of the first audio data may be an audio type or an uplink audio type, and the data type of the first control information may be a control type or an uplink control type. Based on this method, the device node can distinguish between downlink audio data and downlink control information by identification information.

[0212] Based on the above method, the DSP chip may obtain the first audio data from the device node in the following manner: the DSP chip obtains the first data from the device node and determines the first audio data according to the first data. The DSP chip may obtain the first control information from the device node in the following manner: the DSP chip obtains the second data from the device node and determines the first control information according to the second data.

[0213] The functional services in the electronic device provided in the above embodiment can refer to the Figure 1 or Figure 4 For the relevant description in , the specific implementation method corresponding to the above method can refer to the introduction of the audio receiving method provided in the aforementioned embodiment, which will not be repeated here.

[0214] Based on the above embodiments and the same technical concept, the present application also provides an electronic device, which is used to implement the audio processing method provided in the present application. Fig. 9As shown in the figure, the electronic device 900 may include: a memory 901, one or more processors 902, and one or more computer programs (not shown in the figure). The above-mentioned components may be coupled via one or more communication buses 903. Optionally, the electronic device 900 may also include a display screen 904.

[0215] Among them, one or more computer programs (codes) are stored in the memory 901, and the one or more computer programs include computer instructions; one or more processors 902 call the computer instructions stored in the memory 901, so that the electronic device 900 executes the audio processing method provided in the above-mentioned embodiment of the present application.

[0216] In a specific implementation, the memory 901 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more disk storage devices, flash memory devices or other non-volatile solid-state storage devices. The memory 901 may store an operating system (hereinafter referred to as system), such as an embedded operating system such as ANDROID, IOS, WINDOWS, or LINUX. The memory 901 may be used to store an implementation program of an embodiment of the present application. The memory 901 may also store a network communication program, which may be used to communicate with one or more additional devices, one or more user devices, or one or more network devices.

[0217] The one or more processors 902 may be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the present application.

[0218] The display screen 904 is used to display application interfaces and other related user interfaces, for example, it can be used to display the application interface of a VoIP application.

[0219] It should be noted that Fig. 9 This is only one implementation of the electronic device 900 provided in the embodiment of the present application. In actual applications, the electronic device 900 may also include more or fewer components. For details, please refer to Figure 2 The specific structure and description shown are not limited here.

[0220] Based on the above embodiments and the same technical concept, an embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program runs on an electronic device, the electronic device executes the method provided in the above embodiments.

[0221] Based on the above embodiments and the same technical concept, the embodiments of the present application also provide a computer program product, which includes a computer program or instructions. When the computer program or instructions are run on an electronic device, the electronic device executes the method provided in the above embodiments.

[0222] Based on the above embodiments and the same technical concept, the embodiment of the present application also provides a chip system, which includes a processor and a memory, wherein instructions are stored in the memory; when the instructions are executed by the processor, the method provided in the above embodiments can be implemented. The chip system can be composed of a chip, or it can include a chip and other discrete devices.

[0223] The method provided in the embodiment of the present application can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user device or other programmable device. Computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. Computer-readable storage media can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media integrated. Available media can be magnetic media (e.g., floppy disks, hard disks, tapes), optical media (e.g., digital video discs (DVDs), or semiconductor media (e.g., SSDs), etc.

[0224] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. An audio processing method, applied to electronic equipment, characterized in that: The electronic device comprises a digital signal processing DSP chip and an application processing AP system, the DSP chip comprises a first audio processing unit, a first codec unit and a first packet processing unit, the AP system comprises a second audio processing unit, a third audio processing unit, a first application and a device node, the first application comprises a fourth audio processing unit, a second codec unit, a second packet processing unit and a transceiver unit, and the method comprises: The DSP chip acquires first audio data; The DSP chip performs audio processing on the first audio data through the first audio processing unit to obtain second audio data, encodes the second audio data through the first codec unit to obtain third audio data, and packages the third audio data through the first packet processing unit to obtain fourth audio data; wherein the audio processing includes at least one of the following: acoustic echo cancellation, background noise suppression, and automatic gain control; The DSP chip sends the fourth audio data to the device node; The first application obtains the fourth audio data from the device node through the transceiver unit; The first application sends the fourth audio data through the transceiver unit.

2. The method according to claim 1, characterized in that The second audio processing unit is located in the application framework layer of the electronic device, the third audio processing unit is located in the hardware abstraction layer of the electronic device, and the fourth audio data does not pass through the second audio processing unit and the third audio processing unit.

3. The method according to claim 1 or 2, characterized in that The fourth audio data does not pass through the fourth audio processing unit, the second codec unit, and the second packet processing unit.

4. The method according to any one of claims 1 to 3, characterized in that: The device node is located in the kernel layer of the electronic device.

5. The method according to any one of claims 1 to 4, characterized in that: The AP system further includes a first control information processing unit and a second control information processing unit, wherein the first control information processing unit is located in an application framework layer of the electronic device, and the second control information processing unit is located in a hardware abstraction layer of the electronic device; the method further includes: The first application generates first control information for controlling the DSP chip to perform a first operation; The first application sends the first control information to the device node; The DSP chip obtains the first control information from the device node; The DSP chip performs the first operation according to the first control information; The first control information does not pass through the first control information processing unit and the second control information processing unit.

6. The method according to claim 5, characterized in that The DSP chip sends the fourth audio data to the device node, including: The DSP chip sends first data including the fourth audio data and first identification information to the device node; wherein the first identification information is used to indicate the data type of the fourth audio data; The first application obtains the fourth audio data from the device node through the transceiver unit, including: The first application obtains the first data from the device node through the transceiver unit, and determines the fourth audio data according to the first data; The first application sends the first control information to the device node, including: The first application sends second data including the first control information and second identification information to the device node; wherein the second identification information is used to indicate the data type of the first control information; The DSP chip obtains the first control information from the device node, including: The DSP chip obtains the second data from the device node, and determines the first control information according to the second data.

7. The method according to claim 6, characterized in that The data type of the fourth audio data is an audio type or an uplink audio type, and the data type of the first control information is a control type or an uplink control type.

8. The method according to any one of claims 1 to 4, characterized in that: The device node corresponds to the data type of the fourth audio data; the AP system further includes a first control information processing unit and a second control information processing unit, the first control information processing unit is located in the application framework layer of the electronic device, and the second control information processing unit is located in the hardware abstraction layer of the electronic device; the method further includes: The first application generates first control information for controlling the DSP chip to perform a first operation; The first application sends the first control information to a device node corresponding to a data type of the first control information; wherein the device node corresponding to the data type of the first control information is located in a kernel layer of the electronic device; The DSP chip obtains the first control information from a device node corresponding to a data type of the first control information; The DSP chip performs the first operation according to the first control information; The first control information does not pass through the first control information processing unit and the second control information processing unit.

9. An audio processing method, applied to an electronic device, characterized in that: The electronic device comprises a digital signal processing DSP chip and an application processing AP system, the DSP chip comprises a first audio processing unit, a first codec unit and a first packet processing unit, the AP system comprises a second audio processing unit, a third audio processing unit, a first application and a device node, the first application comprises a fourth audio processing unit, a second codec unit, a second packet processing unit and a transceiver unit, and the method comprises: The first application receives first audio data through the transceiver unit; The first application sends the first audio data to the device node through the transceiver unit; The DSP chip obtains the first audio data from the device node; The DSP chip unpacks the first audio data through the first packet processing unit to obtain the second audio, decodes the second audio through the first codec unit to obtain the third audio, and performs audio processing on the third audio data through the first audio processing unit to obtain the fourth audio data; wherein the audio processing includes at least one of the following: acoustic echo cancellation, background noise suppression, and automatic gain control.

10. The method according to claim 9, characterized in that The second audio processing unit is located in the application framework layer of the electronic device, the third audio processing unit is located in the hardware abstraction layer of the electronic device, and the first audio data does not pass through the second audio processing unit and the third audio processing unit.

11. The method according to claim 9 or 10, characterized in that The first audio data does not pass through the fourth audio processing unit, the second codec unit, and the second packet processing unit.

12. The method according to any one of claims 9 to 11, characterized in that: The DSP chip further includes a first anti-jitter unit, and the first application further includes a second anti-jitter unit; after the DSP chip unpacks the first audio data through the first packet processing unit to obtain the second audio data, and before decoding the second audio data through the first codec unit to obtain the third audio data, the method further includes: Performing anti-jitter processing on the second audio by using the first anti-jitter unit; The first audio data does not pass through the second anti-jitter unit.

13. The method according to any one of claims 9 to 12, characterized in that: The device node is located in the kernel layer of the electronic device.

14. The method according to any one of claims 9 to 13, characterized in that: The AP system further includes a first control information processing unit and a second control information processing unit, wherein the first control information processing unit is located in an application framework layer of the electronic device, and the second control information processing unit is located in a hardware abstraction layer of the electronic device; the method further includes: The first application generates first control information for controlling the DSP chip to perform a first operation; The first application sends the first control information to the device node; The DSP chip obtains the first control information from the device node; The DSP chip performs the first operation according to the first control information; The first control information does not pass through the first control information processing unit and the second control information processing unit.

15. The method according to claim 14, characterized in that The first application sends the first audio data to the device node through the transceiver unit, including: The first application sends first data including the first audio data and first identification information to the device node through the transceiver unit; wherein the first identification information is used to indicate the data type of the first audio data; The DSP chip obtains the first audio data from the device node, including: The DSP chip obtains the first data from the device node, and determines the first audio data according to the first data; The first application sends the first control information to the device node, including: The first application sends second data including the first control information and second identification information to the device node; wherein the second identification information is used to indicate the data type of the first control information; The DSP chip obtains the first control information from the device node, including: The DSP chip obtains the second data from the device node, and determines the first control information according to the second data.

16. The method according to claim 15, characterized in that The data type of the first audio data is an audio type or an uplink audio type, and the data type of the first control information is a control type or an uplink control type.

17. The method according to any one of claims 9 to 13, characterized in that: The device node corresponds to the data type of the first audio data; the AP system further includes a first control information processing unit and a second control information processing unit, the first control information processing unit is located in the application framework layer of the electronic device, and the second control information processing unit is located in the hardware abstraction layer of the electronic device; the method further includes: The first application generates first control information for controlling the DSP chip to perform a first operation; The first application sends the first control information to a device node corresponding to a data type of the first control information; wherein the device node corresponding to the data type of the first control information is located in a kernel layer of the electronic device; The DSP chip obtains the first control information from a device node corresponding to a data type of the first control information; The DSP chip performs the first operation according to the first control information; The first control information does not pass through the first control information processing unit and the second control information processing unit.

18. An electronic device, characterized in that: The electronic device includes a display screen, a memory and one or more processors; The memory is used to store computer program code, which includes computer instructions; when the computer instructions are executed by the one or more processors, the electronic device executes the method as claimed in any one of claims 1 to 8, or executes the method as claimed in any one of claims 9 to 17.

19. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program runs on an electronic device, the electronic device executes the method according to any one of claims 1 to 8, or executes the method according to any one of claims 9 to 17.

Images