Method and chip for measuring audio delay in multi-core heterogeneous system

Abstract

The invention relates to a method for measuring audio delay in a multi-core heterogeneous system and a chip, and relates to the technical field of chips, the method comprises the following steps: a first processor core obtains first audio data and attribute information of the first audio data, and transmits the first audio data and the attribute information of the first audio data to a second processor core; the second processor core processes the first audio data based on the attribute information of the first audio data to obtain a first audio signal, controls the loudspeaker to play the first audio signal, generates second audio data based on the audio signal collected by the microphone, and transmits the second audio data and the attribute information of the second audio data to the first processor core; and the first processor core processes the second audio data based on the attribute information of the second audio data to obtain a second audio signal, and calculates the delay of the audio channel based on the original audio signal corresponding to the first audio data and the second audio signal. The embodiment of the invention can measure the delay of the complete audio channel.

Description

Technical Field

[0001] The present application relates to the field of chip technology, and more specifically, to a method and chip for measuring audio delay in a multi-core heterogeneous system. Background Art

[0002] In a multi-core heterogeneous system, there are multiple independent processor cores, and each processor core has an independent operating system and audio management program. Audio data usually needs to be transmitted between multiple processor cores, which causes delays in the audio path, thereby affecting functions such as voice calls and speech recognition.

[0003] The existing method measures the delay of a certain stage in the audio path by adding a timestamp, but this method cannot measure the delay of an audio path containing multiple stages. In addition, this method requires the synchronization of timestamps between different processor cores. If the synchronization is inaccurate or there is a deviation, the calculated delay will also be affected, and the complexity of timestamp synchronization may increase the complexity and cost of the system. Summary of the invention

[0004] The present application is provided to solve the above problems existing in the prior art. According to an embodiment of the present application, a method and chip for measuring audio delay in a multi-core heterogeneous system are provided, which can measure the delay of a complete audio path including audio data transmission, audio signal playback and acquisition.

[0005] In a first aspect, an embodiment of the present application provides a method for measuring audio delay in a multi-core heterogeneous system, which is applied to a multi-core heterogeneous system on a chip, wherein the multi-core heterogeneous system includes a first processor core and a second processor core, and the method includes:

[0006] The first processor core acquires first audio data and attribute information of the first audio data, and transmits the first audio data and the attribute information of the first audio data to the second processor core;

[0007] The second processor core processes the first audio data based on the attribute information of the first audio data to obtain a first audio signal, controls the speaker to play the first audio signal, and obtains the audio signal collected by the microphone; wherein the speaker is located in an environment where the microphone is located;

[0008] The second processor core generates second audio data based on the audio signal collected by the microphone, obtains attribute information of the second audio data, and transmits the second audio data and the attribute information of the second audio data to the first processor core;

[0009] The first processor core processes the second audio data based on attribute information of the second audio data to obtain a second audio signal, and calculates a delay of an audio path based on an original audio signal corresponding to the first audio data and the second audio signal.

[0010] In a second aspect, an embodiment of the present application provides a chip, including: a multi-core heterogeneous system;

[0011] The multi-core heterogeneous system comprises: a first processor core and a second processor core;

[0012] The first processor core is configured to obtain first audio data and attribute information of the first audio data, transmit the first audio data and the attribute information of the first audio data to the second processor core; process the second audio data based on the attribute information of the second audio data to obtain a second audio signal, and calculate a delay of an audio path based on an original audio signal corresponding to the first audio data and the second audio signal;

[0013] The second processor core is used to process the first audio data based on the attribute information of the first audio data, obtain a first audio signal, control a speaker to play the first audio signal, obtain an audio signal collected by a microphone, generate second audio data based on the audio signal collected by the microphone, obtain the attribute information of the second audio data, and transmit the second audio data and the attribute information of the second audio data to the first processor core; wherein the speaker is located in an environment where the microphone is located.

[0014] Any of the embodiments in the above application has the following advantages or beneficial effects: the first audio data is transmitted to the second processor core through the first processor core, the second processor core is triggered to play and collect audio, and the obtained second audio data is fed back to the first processor core, and the first processor core calculates the delay of the audio path by comparing the audio signals of the two audio data. This process simulates the actual audio playback scenario and can measure the delay of the complete audio path including audio data transmission, audio signal playback, and audio signal collection, without adding timestamps, reducing the complexity of the system and the measurement cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In the drawings, which are not necessarily drawn to scale, the same reference numerals may describe similar parts in different views. The same reference numerals with letter suffixes or different letter suffixes may represent different instances of similar parts. The accompanying drawings generally illustrate various embodiments by way of example and not limitation, and together with the specification and claims, are used to illustrate the disclosed embodiments. When appropriate, the same reference numerals are used throughout the drawings to refer to the same or similar parts. Such embodiments are illustrative and are not intended to be exhaustive or exclusive embodiments of the present apparatus or method.

[0016] Figure 1 is a flow chart of a method for measuring audio delay in a multi-core heterogeneous system provided by an embodiment of the present application;

[0017] Figure 2 is a flow chart of a method for measuring audio delay in a multi-core heterogeneous system provided by another embodiment of the present application;

[0018] Figure 3 is a system architecture diagram of a method for measuring audio delay in a multi-core heterogeneous system provided by an embodiment of the present application;

[0019] Figure 4 It is a schematic diagram of the structure of a chip provided by an embodiment of the present application. DETAILED DESCRIPTION

[0020] In order to enable those skilled in the art to better understand the technical solution of the present application, the present application is described in detail below in conjunction with the accompanying drawings and specific embodiments. The embodiments of the present application are further described in detail below in conjunction with the accompanying drawings and specific embodiments, but are not intended to limit the present application. The terms "first", "second" and "third" used in the present application are only intended to distinguish the corresponding features, do not represent the need for such an order, and do not necessarily represent only the singular form.

[0021] like Figure 1 As shown, the embodiment of the present application embodies a method for measuring audio delay in a multi-core heterogeneous system, which is applied to a multi-core heterogeneous system on a chip, wherein the multi-core heterogeneous system includes a first processor core and a second processor core, and the method includes:

[0022] Step 101: The first processor core obtains first audio data and attribute information of the first audio data, and transmits the first audio data and the attribute information of the first audio data to the second processor core.

[0023] The first audio data can be directly obtained from the outside, or can be obtained by processing the audio signal generated by the signal generator. The attribute information of the first audio data includes: any one or more of sampling frequency, sampling bit number, number of channels, data volume, audio device information, data sending status and data receiving status.

[0024] Step 102: the second processor core processes the first audio data based on the attribute information of the first audio data to obtain a first audio signal, controls the speaker to play the first audio signal, and obtains the audio signal collected by the microphone.

[0025] Therein, the loudspeaker is located in the environment where the microphone is located.

[0026] The second processor core can decode the first audio data to obtain the first audio signal based on the attribute information of the first audio data. Based on the decoding, the second processor core can also perform a series of processing such as resampling, filtering, and noise reduction on the first audio data. The specific processing items can be adjusted based on the scene requirements. For example, in one scenario, only the first audio data is decoded. In another scenario, the first audio data is decoded and resampled.

[0027] Step 103: the second processor core generates second audio data based on the audio signal collected by the microphone, obtains attribute information of the second audio data, and transmits the second audio data and the attribute information of the second audio data to the first processor core.

[0028] The second processor core can directly encode the audio signal collected by the microphone to obtain the second audio data, and can also pre-process the audio signal collected by the microphone before encoding, and encode the pre-processed audio signal. The pre-processing may include: any one or more of frequency response adjustment, gain adjustment, mixing and sampling rate conversion.

[0029] Step 104: the first processor core processes the second audio data based on the attribute information of the second audio data to obtain a second audio signal, and calculates the delay of the audio path based on the original audio signal and the second audio signal corresponding to the first audio data.

[0030] The first processor core can decode the second audio data based on the attribute information of the second audio data to obtain a second audio signal. Based on the decoding, the first processor core can also perform a series of processing such as resampling, filtering, and noise reduction on the second audio data. The specific processing items can be adjusted based on the scene requirements.

[0031] In the embodiment of the present application, the first processor core transmits the first audio data to the second processor core, triggers the second processor core to play and collect audio, and feeds back the obtained second audio data to the first processor core, and the first processor core calculates the delay of the audio path by comparing the audio signals of the two audio data. This process simulates the actual audio playback scenario and can measure the delay of the complete audio path including audio data transmission, audio signal playback, and audio signal collection, without adding timestamps, reducing the complexity of the system and the measurement cost.

[0032] In one embodiment of the present application, the first processor core obtains the first audio data, including:

[0033] generating a baseband signal having a frequency within a preset range;

[0034] Generate a harmonic signal; wherein the frequency of the harmonic signal is an integer multiple of the frequency of the fundamental frequency signal;

[0035] Superimpose the baseband signal and the harmonic signal to obtain the original audio signal;

[0036] Based on the original audio signal, first audio data is generated.

[0037] In order to simulate more complex signals, the embodiments of the present application superimpose harmonic signals on the baseband signal. These harmonic signals can enrich the spectral components of the signal, making it closer to actual transmission requirements.

[0038] The first processor core controls the signal generator to generate a base frequency signal within a preset range of [20 Hz, 20000 Hz], and superimposes a harmonic signal of a multiple of the base frequency signal on the base frequency signal. For example, a base frequency signal of 100 Hz is generated, and a harmonic signal of 200 Hz is superimposed thereon.

[0039] A sinusoidal signal x(t) with a fundamental frequency of f0, that is, the fundamental frequency signal, is generated based on equation (1).

[0040] x(t)=A sin(2πf0t+φ) (1)

[0041] Among them, A is used to represent the amplitude, t is used to represent the time, and φ is used to represent the initial phase.

[0042] The harmonic signal y(t) is generated based on equation (2).

[0043] y(t)=A n sin(2πnf0t+φn) (2)

[0044] Where n is the index value.

[0045] The original audio signal s(t) obtained by superimposing the baseband signal and the harmonic signal is shown in formula (3).

[0046]

[0047] Wherein, N is used to represent the number of harmonics contained in the sine wave, and n is an index value.

[0048] In the embodiment of the present application, harmonic signals are superimposed on the baseband signal, and these harmonic signals can enrich the spectrum components of the signal, making it closer to the actual transmission requirements. In actual application scenarios, only the baseband signal can be used as the original audio signal.

[0049] In one embodiment of the present application, calculating the delay of the audio path based on the original audio signal and the second audio signal corresponding to the first audio data includes:

[0050] The original audio signal and the second audio signal are respectively transformed from the time domain to the frequency domain;

[0051] Calculating a phase difference between the original audio signal and the second audio signal based on the transformed original audio signal and the second audio signal;

[0052] The delay of the audio path is calculated based on the phase difference between the original audio signal and the second audio signal.

[0053] The original audio signal and the second audio signal are respectively converted into the frequency domain by fast Fourier transform, the phase of the original audio signal and the phase of the second audio signal represented in the frequency domain are extracted, and the phase difference between the original audio signal and the second audio signal is calculated based on the phase of the original audio signal and the phase of the second audio signal represented in the frequency domain.

[0054] Based on equation (4), the delay of the audio path is calculated.

[0055] Δφ[fk]=2πfkT+2πm (4)

[0056] Among them, Δφ[fk] is used to represent the phase difference corresponding to the frequency f, k is used to represent the frequency index, m is used to represent the number of cycles of a phase change, and T is used to represent the delay corresponding to the frequency f.

[0057] If only one frequency of the original audio signal is used for measurement, the delay corresponding to this frequency is the delay of the audio path. For example, when k is 1, the corresponding frequency is 1000Hz, when k is 2, the corresponding frequency is 2000Hz, and when k is 3, the corresponding frequency is 3000Hz. The value range of k is [0, N-1].

[0058] The embodiment of the present application can accurately calculate the delay of the audio path through the phase difference of two signals.

[0059] In one embodiment of the present application, calculating the delay of the audio path based on the phase difference between the original audio signal and the second audio signal includes:

[0060] Calculating the delay of the original audio signals of various frequencies based on the phase difference between the original audio signals of various frequencies and the second audio signal;

[0061] The average value of the delay of the original audio signal at various frequencies is determined as the delay of the audio path.

[0062] For example, the delays corresponding to the original audio signals of frequencies 1000Hz and 2000Hz are calculated respectively, and the average value of the delays corresponding to the two frequencies is the delay of the audio path. Considering that the frequency of the signal can exist in various situations during the actual transmission of a multi-core heterogeneous system, the embodiment of the present application considers the phase difference between the original audio signal and the second audio signal of different frequencies during the calculation of the delay of the audio path, so that the calculated delay of the audio path is more accurate and closer to the actual transmission process.

[0063] In one embodiment of the present application, the first processor core and the second processor core transmit the first audio data and the second audio data through a shared memory.

[0064] Using shared memory transmission to transmit audio data between different cores of multiple heterogeneous systems has the advantages of efficient data transmission speed, low latency, reduced resource consumption, easy implementation and management, support for large data volume transmission, and improved system scalability and flexibility.

[0065] In addition to shared memory, data can be transmitted between different processor cores using inter-core communication mechanisms, virtual sound card technology, data buffers, dedicated data channels, cache consistency protocols, and data compression and decompression.

[0066] In one embodiment of the present application, the first processor core and the second processor core transmit attribute information of the first audio data and attribute information of the second audio data through inter-core communication.

[0067] Among them, the attribute information of the first audio data includes: any one or more of the sampling frequency, the number of sampling bits, the number of channels, the data volume, the audio device information, the data sending status and the data receiving status; the attribute information of the second audio data includes: any one or more of the sampling frequency, the number of sampling bits, the number of channels, the data volume, the audio device information, the data sending status and the data receiving status.

[0068] Using inter-core communication to transmit attribute information in a multi-core heterogeneous system has many advantages, including high efficiency, low power consumption and heat dissipation, flexibility and scalability, as well as security and reliability.

[0069] In one embodiment of the present application, generating second audio data based on an audio signal collected by a microphone includes:

[0070] The audio signal collected by the microphone is input into the filter to filter out the environmental noise;

[0071] Based on the audio signal output by the filter, second audio data is generated.

[0072] Taking into account that the microphone may collect noise in the environment, in order to prevent the presence of environmental noise from causing errors in subsequent delay calculations and reducing the accuracy of the calculated delay, the embodiment of the present application filters out the environmental noise through a filter before generating the second audio data.

[0073] In actual application scenarios, the audio signal output by the filter can be encoded to obtain second audio data. Before encoding, other preprocessing operations such as noise reduction and gain control can be performed on the audio signal output by the filter. The specific preprocessing process can be adjusted based on actual business needs.

[0074] In one embodiment of the present application, generating first audio data based on an original audio signal includes:

[0075] Preprocessing the original audio signal; wherein the preprocessing includes: any one or more of frequency response adjustment, gain adjustment, mixing and sampling rate conversion;

[0076] The preprocessed original audio signal is encoded into first audio data.

[0077] In order to improve the quality of the first audio data, the original audio signal is usually preprocessed before encoding. Higher quality first audio data can improve the accuracy of subsequent delay calculation.

[0078] like Figure 2 As shown, an embodiment of the present application provides a method for measuring audio delay in a core heterogeneous system, which is applied to a multi-core heterogeneous system on a chip, wherein the multi-core heterogeneous system includes a first processor core and a second processor core, and the method includes:

[0079] Step 201: The first processor core generates a baseband signal with a frequency within a preset range, generates a harmonic signal, and superimposes the baseband signal and the harmonic signal to obtain an original audio signal.

[0080] This method can be achieved by Figure 3 The system architecture diagram shown is implemented. The timing program can send a trigger instruction to the audio signal generation program at a set time interval. The audio signal generation program generates a baseband signal and a harmonic signal based on the trigger instruction, and superimposes the two to obtain the original audio signal. Core 1 is the first processor core, and core 2 is the second processor core.

[0081] The audio playback program obtains the original audio signal and provides it to the audio data processing program.

[0082] Step 202: The first processor core generates first audio data based on the original audio signal.

[0083] The audio data processing program performs preprocessing and other operations on the original audio signal to obtain first audio data.

[0084] Step 203: The first processor core obtains attribute information of the first audio data, and transmits the attribute information of the first audio data to the second processor core through inter-core communication.

[0085] The audio framework program obtains attribute information of the first audio data.

[0086] This program belongs to the system middle layer program, which provides an audio playback interface to realize the audio playback function, and calls the audio hardware middle layer program to realize the function of operating the audio device.

[0087] In addition, the audio framework program is used to manage audio devices, unify the management and coordination of audio input and output of multiple devices, such as wired headphones, wireless Bluetooth headphones, speakers, and other external devices such as TVs and amplifiers. It can also be used to manage audio strategies, such as stopping the output of the speaker when headphones are connected; when connecting to a TV, stop other outputs and output through the TV end, etc.

[0088] The attribute information is Figure 3 The audio information in the data includes: sampling frequency, sampling bit number, number of channels, data volume, data sending status and data receiving status.

[0089] The audio framework program calls the inter-core communication program to transmit the attribute information of the first audio data to the second processor core in the form of inter-core communication.

[0090] Step 204: The first processor core transmits the first audio data to the second processor core through the shared memory.

[0091] The audio framework program calls the audio driver program through the audio hardware intermediate layer program to transmit the first audio data to the second processor core through the shared memory management program.

[0092] The audio hardware middle layer program is an abstract layer program of the audio hardware device. It only provides the interface of the audio device for the audio framework program to use. The interface is implemented by the specific audio device driver. The audio hardware middle layer program is used to uniformly manage and load the drivers of various audio devices to drive the audio devices to realize input and output functions. The audio driver is a program that controls the audio device hardware and realizes the device functions.

[0093] The shared content management program transmits the first audio data to the second processor core in a shared memory manner. In addition, the shared memory management program can also update the data pointer, memory location and synchronization information.

[0094] The audio driver is part of the kernel space and is responsible for handling the transmission and control of audio data. When the first processor core needs to send audio data, it first needs to send the audio data to the audio driver's playback buffer. This process needs to rely on the audio driver to coordinate and complete. The audio driver transfers the audio data from the playback buffer to the audio hardware device through an efficient mechanism. During this process, the audio driver monitors the status of the hardware device and the progress of data transmission to ensure the integrity and real-time nature of the data. In addition, the audio driver is also responsible for configuring and controlling audio hardware devices, such as audio codecs and digital signal processors, to ensure that audio data can be processed and output correctly. These configuration and control operations also need to rely on the audio driver to complete.

[0095] Step 205: the second processor core processes the first audio data based on the attribute information of the first audio data to obtain a first audio signal, controls the speaker to play the first audio signal, and acquires the audio signal collected by the microphone.

[0096] In the second processor core, the audio playback program obtains the first audio data through the shared memory management program and sends it to the audio data processing program. The audio data processing program can decode the first audio data based on the attribute information of the first audio data provided by the inter-core communication program, and finally obtain the first audio signal. The audio framework program calls the audio driver through the audio hardware middle layer program to play the first audio signal through the speaker and collect the audio signal through the microphone.

[0097] Step 206: The second processor core inputs the audio signal collected by the microphone into a filter to filter out environmental noise.

[0098] The audio framework program inputs the audio signal collected by the microphone into the filter to filter out environmental noise.

[0099] Step 207: the second processor core generates second audio data based on the audio signal output by the filter, obtains attribute information of the second audio data, and transmits the second audio data and the attribute information of the second audio data to the first processor core.

[0100] The audio data processing program preprocesses the audio signal output by the filter to generate second audio data. The audio recording program sends the second audio data to the first processor core through the shared memory management program. The audio framework program sends the acquired attribute information of the second audio data to the first processor core through the inter-core communication program.

[0101] Step 208: The first processor core processes the second audio data based on the attribute information of the second audio data to obtain a second audio signal.

[0102] The second audio data is transmitted to the audio framework program through the audio driver and the audio hardware middle layer program. The audio framework program provides the second audio data to the audio data processing program. The audio data processing program decodes the second audio data based on the attribute information of the second audio data provided by the inter-core communication program to obtain a second audio signal.

[0103] Step 209: The first processor core transforms the original audio signal and the second audio signal from the time domain to the frequency domain respectively, calculates the phase difference between the original audio signal and the second audio signal based on the transformed original audio signal and the second audio signal, calculates the delay of the original audio signals of various frequencies based on the phase difference between the original audio signals of multiple frequencies and the second audio signal, and determines the average value of the delay of the original audio signals of various frequencies as the delay of the audio path.

[0104] The audio recording program provides the second audio signal processed by the audio data processing program to the delay calculation program, and the delay calculation program calculates the delay of the audio path based on the original audio signal generated by the audio signal generation program and the second audio signal provided by the audio recording program.

[0105] The solution provided in the embodiment of the present application can automatically generate original audio signals, simulate the user's actual usage scenario, and measure the delay of the complete audio path starting from the audio playback program triggering the playback process, including audio data transmission, audio signal playback and acquisition.

[0106] like Figure 4 As shown, an embodiment of the present application provides a chip, including: a multi-core heterogeneous system;

[0107] The multi-core heterogeneous system includes: a first processor core and a second processor core;

[0108] The first processor core 401 is configured to obtain the first audio data and the attribute information of the first audio data, and transmit the first audio data and the attribute information of the first audio data to the second processor core; process the second audio data based on the attribute information of the second audio data to obtain a second audio signal, and calculate the delay of the audio path based on the original audio signal corresponding to the first audio data and the second audio signal;

[0109] The second processor core 402 is used to process the first audio data based on the attribute information of the first audio data, obtain the first audio signal, control the speaker to play the first audio signal, obtain the audio signal collected by the microphone, generate the second audio data based on the audio signal collected by the microphone, obtain the attribute information of the second audio data, and transmit the second audio data and the attribute information of the second audio data to the first processor core; wherein the speaker is located in the environment where the microphone is located.

[0110] In one embodiment of the present application, the first processor core 401 is used to generate a baseband signal with a frequency within a preset range; generate a harmonic signal; wherein the frequency of the harmonic signal is an integer multiple of the frequency of the baseband signal; superimpose the baseband signal and the harmonic signal to obtain an original audio signal; and generate first audio data based on the original audio signal.

[0111] In one embodiment of the present application, the first processor core 401 is used to transform the original audio signal and the second audio signal from the time domain to the frequency domain respectively; calculate the phase difference between the original audio signal and the second audio signal based on the transformed original audio signal and the second audio signal; and calculate the delay of the audio path based on the phase difference between the original audio signal and the second audio signal.

[0112] In one embodiment of the present application, the first processor core 401 is used to calculate the delay of the original audio signals of various frequencies based on the phase difference between the original audio signals of multiple frequencies and the second audio signal; and determine the average value of the delay of the original audio signals of various frequencies as the delay of the audio path.

[0113] In one embodiment of the present application, the first processor core and the second processor core transmit the first audio data and the second audio data through a shared memory.

[0114] In one embodiment of the present application, the first processor core and the second processor core transmit attribute information of the first audio data and attribute information of the second audio data through inter-core communication.

[0115] In one embodiment of the present application, the attribute information of the first audio data includes: any one or more of sampling frequency, sampling bit number, number of channels, data volume, audio device information, data sending status and data receiving status.

[0116] In one embodiment of the present application, the attribute information of the second audio data includes: any one or more of sampling frequency, sampling bit number, number of channels, data volume, audio device information, data sending status and data receiving status.

[0117] In one embodiment of the present application, the second processor core 402 is configured to input the audio signal collected by the microphone into a filter to filter out environmental noise; and generate second audio data based on the audio signal output by the filter.

[0118] In one embodiment of the present application, the first processor core 401 is used to preprocess the original audio signal; wherein the preprocessing includes: any one or more of frequency response adjustment, gain adjustment, mixing and sampling rate conversion; and encoding the preprocessed original audio signal into first audio data.

[0119] The vehicle in the embodiments of the present application may be "car", "vehicle" and "entire vehicle" or other similar terms including general motor vehicles, such as sedans, SUVs, MPVs, buses, trucks and other cargo or passenger vehicles, watercraft including various ships and boats, and aircraft, etc., including hybrid vehicles, electric vehicles, fuel vehicles, plug-in hybrid vehicles, fuel cell vehicles and other alternative fuel vehicles. Among them, hybrid vehicles refer to vehicles with two or more power sources, and electric vehicles include pure electric vehicles, extended-range electric vehicles, etc., which are not specifically limited in the present application.

[0120] It should be understood that in the embodiments of the present application, the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc.

[0121] It should also be understood that the memory mentioned in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct memory bus random access memory (Direct Rambus RAM, DR RAM).

[0122] It should be noted that when the processor is a general-purpose processor, DSP, ASIC, FPGA or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, the memory (storage module) is integrated in the processor.

[0123] It should be noted that the memory described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0124] In addition to the data bus, the bus may also include a power bus, a control bus, a status signal bus, etc. However, for the sake of clarity, various buses are marked as buses in the figure.

[0125] It should also be understood that the first, second, third, fourth and various numerical numbers involved in this document are only distinctions made for the convenience of description and are not intended to limit the scope of the present application.

[0126] It should be understood that the term "and / or" in this article is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0127] In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in a processor or an instruction in the form of software. The steps of the method disclosed in conjunction with the embodiment of the present application can be directly embodied as a hardware processor for execution, or a combination of hardware and software modules in a processor for execution. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in conjunction with its hardware. To avoid repetition, it is not described in detail here.

[0128] In various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0129] Those skilled in the art will appreciate that the various illustrative logical blocks (ILBs) and steps described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or in a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0130] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0131] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0132] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0133] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integration. The available medium can be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk), etc.

[0134] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for measuring audio delay in a multi-core heterogeneous system, characterized in that: A multi-core heterogeneous system applied to a chip, wherein the multi-core heterogeneous system includes a first processor core and a second processor core, and the method includes: The first processor core acquires first audio data and attribute information of the first audio data, and transmits the first audio data and the attribute information of the first audio data to the second processor core; The second processor core processes the first audio data based on the attribute information of the first audio data to obtain a first audio signal, controls the speaker to play the first audio signal, and obtains the audio signal collected by the microphone; wherein the speaker is located in an environment where the microphone is located; The second processor core generates second audio data based on the audio signal collected by the microphone, obtains attribute information of the second audio data, and transmits the second audio data and the attribute information of the second audio data to the first processor core; The first processor core processes the second audio data based on attribute information of the second audio data to obtain a second audio signal, and calculates a delay of an audio path based on an original audio signal corresponding to the first audio data and the second audio signal.

2. The method according to claim 1, characterized in that The first processor core acquires first audio data, including: generating a baseband signal having a frequency within a preset range; Generate a harmonic signal; wherein the frequency of the harmonic signal is an integer multiple of the frequency of the fundamental frequency signal; Superimposing the baseband signal and the harmonic signal to obtain the original audio signal; Based on the original audio signal, the first audio data is generated.

3. The method according to claim 1, characterized in that The calculating the delay of the audio path based on the original audio signal corresponding to the first audio data and the second audio signal comprises: transforming the original audio signal and the second audio signal from the time domain to the frequency domain respectively; Calculating a phase difference between the original audio signal and the second audio signal based on the transformed original audio signal and the second audio signal; The delay of the audio path is calculated based on the phase difference between the original audio signal and the second audio signal.

4. The method according to claim 3, characterized in that Calculating the delay of the audio path based on the phase difference between the original audio signal and the second audio signal comprises: Calculating the delay of the original audio signals of various frequencies based on the phase difference between the original audio signals of various frequencies and the second audio signal; The average value of the delay of the original audio signal at various frequencies is determined as the delay of the audio path.

5. The method according to claim 1, characterized in that in, The first processor core and the second processor core transmit the first audio data and the second audio data through a shared memory.

6. The method according to claim 1, characterized in that in, The first processor core and the second processor core transmit attribute information of the first audio data and attribute information of the second audio data through inter-core communication.

7. The method according to claim 1, characterized in that The attribute information of the first audio data includes: any one or more of sampling frequency, sampling bit number, number of channels, data volume, audio device information, data sending status and data receiving status; and / or, The attribute information of the second audio data includes any one or more of sampling frequency, sampling bit number, number of channels, data volume, audio device information, data sending status and data receiving status.

8. The method according to claim 1, characterized in that Generating second audio data based on the audio signal collected by the microphone, comprising: Inputting the audio signal collected by the microphone into a filter to filter out environmental noise; The second audio data is generated based on the audio signal output by the filter.

9. The method according to claim 2, characterized in that Generating the first audio data based on the original audio signal includes: Preprocessing the original audio signal; wherein the preprocessing includes: any one or more of frequency response adjustment, gain adjustment, mixing and sampling rate conversion; The preprocessed original audio signal is encoded into the first audio data.

10. A chip, characterized in that: include: Multi-core heterogeneous systems; The multi-core heterogeneous system comprises: a first processor core and a second processor core; The first processor core is configured to obtain first audio data and attribute information of the first audio data, transmit the first audio data and the attribute information of the first audio data to the second processor core; process the second audio data based on the attribute information of the second audio data to obtain a second audio signal, and calculate a delay of an audio path based on an original audio signal corresponding to the first audio data and the second audio signal; The second processor core is used to process the first audio data based on the attribute information of the first audio data, obtain a first audio signal, control a speaker to play the first audio signal, obtain an audio signal collected by a microphone, generate second audio data based on the audio signal collected by the microphone, obtain the attribute information of the second audio data, and transmit the second audio data and the attribute information of the second audio data to the first processor core; wherein the speaker is located in an environment where the microphone is located.

Images