Method and system for simultaneously transmitting audio data and non-audio data based on UAC

Abstract

The invention relates to the technical field of data transmission, and provides a UAC-based method for simultaneously transmitting audio and non-audio data, which comprises the following steps: S1, acquiring non-audio data, and adding the non-audio data into a non-audio data FIFO; s2, acquiring audio data, and adding the audio data into the audio data F IFO; s3, when the non-audio data F IFO and the audio data F IFO meet a merging condition, copying the non-audio data and the audio data from the non-audio data FIFO and the audio data F IFO to a temporary buffer area, interleaving and merging all the data in the temporary buffer area into each audio channel, and copying the data into UAC data F IFO; and S4, initializing a UAC transmission module, and when the UAC data F IFO has data, transmitting the data through the UAC. More information is transmitted by using the UAC transmission bandwidth in some resource limited scenes, the software development and debugging period is shortened by multiplexing the UAC transmission channel, and the protocol is simple and easy to package, transmit and connect with an upper computer.

Description

Technical Field

[0001] The present invention relates to the technical field of data transmission, and in particular to a method and system for simultaneously transmitting audio and non-audio data based on UAC. Background Art

[0002] The UAC protocol is used to transmit audio data in voice-related products. First of all, in the actual hardware and software environment, collecting or obtaining the original audio is the key first step. From the hardware perspective, microphones play an important role as common audio input devices. Dynamic microphones use their unique mechanical structure to efficiently convert the physical vibrations of sound into electrical signals, while condenser microphones can capture more subtle sound changes with their high sensitivity. In addition to real-time collection through microphones, reading existing audio files from storage devices is also an important way to obtain original audio. These audio files have various formats, such as WAV format, which can provide lossless audio quality, and MP3 format, which has a high compression ratio. When reading, they need to be restored to original audio data according to the corresponding decoding rules.

[0003] Next, it is necessary to package it into the buffer according to the UAC protocol format. The UAC protocol has strict regulations on the specific format of audio data transmitted on the USB bus, which covers the packaging details of audio data encoding method, number of channels, sampling rate and other information. For example, for common PCM encoded audio data, it is necessary to carefully package it according to the data packet structure specified by the UAC protocol. The buffer is an area for temporary storage of audio data. During the packaging process, the original audio data must be accurately written in sequence and in the UAC protocol format. The size of the buffer needs to take into account multiple factors. It must ensure that it can accommodate a certain amount of audio data and reduce frequent data transmission operations that interact with hardware to improve efficiency, but it must not occupy excessive memory resources and affect the overall performance of the system.

[0004] Finally, the sending module detects the buffer according to the actual situation and chooses the right time to transmit the audio data through UAC. The detection mechanism of the sending module is crucial. For example, a scientific and reasonable water level mark can be set. When the amount of audio data in the buffer accumulates to this mark, the transmission operation is automatically triggered; or a timed detection method is used to check the amount of data in the buffer at fixed time intervals. The choice of transmission timing needs to fully consider the actual situation such as the processing capacity of the device and the busyness of the USB bus. If the USB bus is busy, it may be necessary to suspend the transmission or dynamically adjust the transmission speed to ensure stable data transmission and avoid data loss. During the transmission process, the sending module accurately sends the audio data in the buffer through the USB interface in the UAC protocol format, and the receiving end unpacks and processes it according to the same protocol, thereby realizing efficient transmission of audio data.

[0005] However, this solution cannot solve some application scenarios where there is only one USB peripheral interface and audio and non-audio data need to be transmitted, and it also fails to fully utilize the data bandwidth of UAC. In application scenarios where there is only one USB peripheral interface, the shortcomings of this solution are more obvious. Since it mainly focuses on the transmission of audio data, it is powerless for situations where non-audio data needs to be transmitted at the same time. For example, in a device, it is necessary to transmit audio signals through the USB interface to realize the voice communication function, and it is also necessary to transmit non-audio data such as control instructions and sensor data. This solution cannot meet the needs of such diversified data transmission. This limits the functional expansion and integration of the device, making the applicability of the device in complex application scenarios greatly reduced. In addition, from the perspective of bandwidth utilization, the UAC protocol is mainly optimized for audio data transmission in design. However, in many cases, the bandwidth of the USB interface of the device is not fully utilized. Especially when the amount of audio data transmission is small or there is an idle period, the UAC protocol does not flexibly adjust the strategy to use these remaining bandwidths to transmit other types of data, resulting in a certain waste of resources. This not only affects the data transmission efficiency of the device, but also is not conducive to the device to achieve richer functions under limited hardware resources. Summary of the invention

[0006] In view of the above problems, the purpose of the present invention is to provide a method and system for simultaneously transmitting audio and non-audio data based on UAC, which utilizes the UAC transmission bandwidth to transmit more information in certain resource-constrained scenarios, reuses the UAC transmission channel to shorten the software development and debugging cycle, and has a simple protocol that is easy to encapsulate, transmit, and connect to a host computer.

[0007] The above-mentioned object of the present invention is achieved through the following technical solutions:

[0008] A method for transmitting audio and non-audio data simultaneously based on UAC, comprising:

[0009] S1: Acquire non-audio data and add the non-audio data to the non-audio data FIFO;

[0010] S2: Acquire audio data, and add the audio data to the audio data FIFO;

[0011] S3: when the non-audio data FIFO and the audio data FIFO meet the merging condition, copy the non-audio data and the audio data from the non-audio data FIFO and the audio data FIFO to a temporary buffer, interleave and merge all the data in the temporary buffer into each audio channel, and copy the data to the UAC data FIFO;

[0012] S4: Initialize the UAC transmission module, and when there is data in the UAC data FIFO, transmit the data through the UAC.

[0013] Further, in step S1, the non-audio data is obtained and added to the non-audio data FIFO, specifically:

[0014] Initialize the non-audio data FIFO, and perform related initialization of non-audio data acquisition, including initializing the algorithm module in the wake-up recognition project, registering the wake-up event callback function to the algorithm module, and updating the wake-up event data in the callback function when a wake-up event occurs subsequently;

[0015] Entering a loop, acquiring the non-audio data, and determining whether the non-audio data is successfully acquired by checking whether the timestamp in the wake-up information changes. If the timestamp changes, it means that the non-audio data is successfully acquired, and the execution continues; otherwise, it means that the acquisition fails, and the acquisition continues after a preset time;

[0016] Determine whether the non-audio data FIFO is full, if not, pack the non-audio data and add it to the non-audio data FIFO, otherwise discard the non-audio data and throw an exception, and then continue to collect the non-audio data.

[0017] Furthermore, the data format of the non-audio data packaging is specifically:

[0018] The data format includes a frame header, a check bit, a message type, a current sequence number, a data length, a data part and a frame tail connected in sequence;

[0019] The frame header adopts a fixed value and a length of 4 bytes;

[0020] The check bit is used to check the CRC check result of the data from the check bit to the end of the frame, and the length is 4Byte;

[0021] The message type is used to store the message type, the periodic message type is 0, the non-periodic new message start frame is 1, the intermediate frame is 2, the end frame is 3, and the length is 4Byte;

[0022] The current sequence number is used to count from the first packet, increments in sequence, and counts from 0 after overflow, and is used for packet loss detection, with a length of 4 bytes;

[0023] The data length is used to represent the length of the data part, excluding the length of the tail frame, which is 4 bytes long;

[0024] The data part is used to store the data part and has an indefinite length;

[0025] The frame tail adopts a fixed value and a length of 4 bytes.

[0026] Further, in step S2, the audio data is obtained and added to the audio data FIFO, specifically:

[0027] Initializing the audio data FIFO, and performing audio data acquisition related initialization including starting the ADC and I2S in the CODEC module for audio acquisition;

[0028] Entering a loop, acquiring the audio data, and when it is detected that the length of the read audio data is not 0, it means that the audio data is acquired successfully and the execution continues; otherwise, it means that the audio data is acquired and the preset time is waited to continue the acquisition;

[0029] Determine whether the audio data FIFO is full, if not, copy the audio data to the audio data FIFO, otherwise discard the audio data and throw an exception, and then continue to obtain the audio data.

[0030] Further, in step S3, when the non-audio data FIFO and the audio data FIFO meet the merging condition, the non-audio data and the audio data are copied from the non-audio data FIFO and the audio data FIFO to a temporary buffer, all the data in the temporary buffer are interleaved and merged into each audio channel, and the data is copied to the UAC data FIFO, specifically:

[0031] Entering a loop, determining whether the non-audio data FIFO and the audio data FIFO meet the merging condition, if not, then waiting for a period of time and then continuing the loop, if satisfied, then continuing the execution;

[0032] Copying the non-audio data and the audio data from the non-audio data FIFO and the audio data FIFO to the temporary buffer;

[0033] Interleave all the data in the temporary buffer into each audio channel according to the audio frame length, and copy the data into the UAC data FIFO;

[0034] Among them, judging whether the non-audio data FIFO and the audio data FIFO meet the merging conditions includes: the audio data satisfies that there is one frame of data in each of the audio channels, the non-audio data FIFO does not necessarily need to have one frame of data, but needs to be packaged according to the audio frame length, and at the same time, the UAC data FIFO includes enough space to hold the audio data and the non-audio data.

[0035] Further, all the data in the temporary buffer are interleaved and merged into each of the audio channels according to the audio frame length, and the data is copied to the UAC data FIFO, specifically:

[0036] The UAC transmission module includes a plurality of the audio channels and at least one periodic or non-periodic non-audio channel, and stores the data in all the channels in an interleaved manner by sequentially taking the first data that has not been taken and splicing them in sequence, wherein the data length is determined according to the audio sampling bit width;

[0037] When the length of the periodic non-audio data is less than the audio frame length, the data is packaged according to the audio frame length, and the end of the insufficient part is filled with blank data;

[0038] Non-periodic non-audio data is sent according to actual data, and is packaged according to the audio frame length each time. When the remaining length is less than the audio frame length, it is sent according to one frame length, and the end is filled with blank data; when there is no data, it can be not sent or empty data can be sent.

[0039] Further, in step S4, the UAC transmission module is initialized, and when there is data in the UAC data FIFO, the data is transmitted through the UAC, specifically:

[0040] Initialize the UAC transmission module;

[0041] Enter the loop, determine whether there is data in the UAC data FIFO, if there is data, store the data in the buffer for UAC to transmit data, otherwise wait for a preset time and continue the loop.

[0042] A system for simultaneously transmitting audio and non-audio data based on UAC for executing the method for simultaneously transmitting audio and non-audio data based on UAC as described above, comprising:

[0043] A non-audio data acquisition module, used for acquiring non-audio data and adding the non-audio data to the non-audio data FIFO;

[0044] An audio data acquisition module, used for acquiring audio data and adding the audio data to an audio data FIFO;

[0045] A data merging and interleaving module, configured to copy the non-audio data and the audio data from the non-audio data FIFO and the audio data FIFO to a temporary buffer when the non-audio data FIFO and the audio data FIFO meet a merging condition, interleave and merge all the data in the temporary buffer into each audio channel, and copy the data to the UAC data FIFO;

[0046] The UAC data transmission module is used to initialize the UAC transmission module and transmit the data through the UAC when there is data in the UAC data FIFO.

[0047] A computer device comprises a memory and one or more processors, wherein the memory stores computer codes, and when the computer codes are executed by the one or more processors, the one or more processors execute the above method.

[0048] A computer-readable storage medium stores computer codes. When the computer codes are executed, the above method is executed.

[0049] Compared with the prior art, the present invention has the following beneficial effects:

[0050] A method for transmitting audio and non-audio data simultaneously based on UAC is provided, including: S1: obtaining non-audio data, and adding the non-audio data to the non-audio data FIFO; S2: obtaining audio data, and adding the audio data to the audio data FIFO; S3: when the non-audio data FIFO and the audio data FIFO meet the merging condition, copying the non-audio data and the audio data from the non-audio data FIFO and the audio data FIFO to a temporary buffer, interweaving and merging all the data in the temporary buffer into each audio channel, and copying the data to the UAC data FIFO; S4: initializing the UAC transmission module, and transmitting the data through UAC when there is data in the UAC data FIFO. The above technical solution reuses the UAC bandwidth, transmits audio and non-audio data at the same time, mainly transmits audio, and borrows one or more audio channels of UAC to transmit non-audio data. Non-audio data supports periodic transmission and non-periodic transmission, which improves the adaptability to multiple scenarios. The merging and disassembly of non-audio data and audio data are simple, and the non-audio data protocol is simple. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 It is an overall flow chart of the method for simultaneously transmitting audio and non-audio data based on UAC of the present invention;

[0052] Figure 2 Obtaining non-audio data flow chart for the present invention;

[0053] Figure 3 This is a schematic diagram of the non-audio data packaging format of the present invention;

[0054] Figure 4 A flowchart for obtaining audio data for the present invention;

[0055] Figure 5 It is a data merging flow chart of the present invention;

[0056] Figure 6 It is a schematic diagram of the composition of UAC channel data of the present invention;

[0057] Figure 7 This is a schematic diagram of cross-storage of data in the present invention;

[0058] Figure 8 This is a data transmission flow chart of the present invention;

[0059] Fig. 9 It is an overall structural diagram of the system for simultaneously transmitting audio and non-audio data based on UAC of the present invention. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0061] Those skilled in the art will appreciate that, unless otherwise stated, the singular forms "a", "an", "said" and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present invention refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0062] In the process of offline voice product development, we often encounter scenarios where audio and non-audio data need to be transmitted. The usual practice is to use two peripheral interfaces to achieve this, one for transmitting audio and the other for transmitting non-audio data. However, in actual product development, we encounter scenarios where we need to reduce hardware interfaces and lower costs while retaining previous communication functions. The existing communication cannot meet actual requirements.

[0063] Considering the functional requirements, adaptability to multiple scenarios, transmission stability, software implementation complexity, ease of docking with the host computer, and product docking debugging cycle, USB peripherals were finally selected from general peripherals. UAC was originally a protocol dedicated to audio transmission, and it is necessary to break the routine. To transmit non-audio data through UAC, appropriate design of data packaging and synthesis should be made according to the actual scenario. And it has the following advantages: 1) Audio transmission is the main method, and one or more audio channels of UAC are used to transmit non-audio data. 2) Non-audio data supports periodic transmission and non-periodic transmission to improve multi-scenario adaptability. 3) It is simple to merge and disassemble non-audio data and audio data, and the non-audio data protocol is simple.

[0064] The following is described by specific examples:

[0065] First embodiment

[0066] like Figure 1 As shown, this embodiment provides a method for simultaneously transmitting audio and non-audio data based on UAC, including:

[0067] S1: Acquire non-audio data, and add the non-audio data to the non-audio data FIFO.

[0068] like Figure 2 As shown, in this embodiment, step S1 is specifically:

[0069] Initialize the non-audio data FIFO, and perform related initialization of non-audio data acquisition including initializing the algorithm module in the wake-up recognition project, registering the wake-up event callback function to the algorithm module, and updating the wake-up event data in the callback function when a wake-up event occurs subsequently.

[0070] The specific initialization process is as follows:

[0071] (1) Initialize non-audio data FIFO

[0072] Purpose and function:

[0073] The initialization of the non-audio data buffer FIFO (First In First Out) is intended to provide an ordered data structure for the storage and transmission of non-audio data. It ensures that data is processed in the order in which it enters, avoiding data confusion and loss. In scenarios involving simultaneous transmission of audio and non-audio data, the non-audio data buffer FIFO can help coordinate the processing rhythm of different types of data and improve the stability and efficiency of the system.

[0074] Initialization steps:

[0075] Determine the size of the FIFO: According to the expected amount of non-audio data and processing speed, set the FIFO capacity appropriately. A FIFO that is too small may cause data overflow, while a FIFO that is too large will occupy too much memory resources.

[0076] Initialization pointer: Set the read pointer and write pointer of the FIFO, usually initialized to point to the starting position of the FIFO.

[0077] Clear the FIFO: During initialization, ensure that there is no residual data in the FIFO. You can clear the FIFO by setting all storage locations to initial values ​​(such as zero or a specific invalid value).

[0078] (2) Initialization related to non-audio data acquisition

[0079] Initialization in the wake-up recognition project:

[0080] In wake-up recognition projects, the purpose of initialization is to prepare the system to receive and process wake-up events. This includes initializing the algorithm module, which is usually responsible for detecting and identifying wake-up signals.

[0081] Initializing the algorithm module may involve loading model parameters, setting the initial state of the algorithm, allocating memory resources, etc. These steps ensure that the algorithm can work properly at runtime and has the ability to accurately detect wake-up events.

[0082] Register the wake-up event callback function to the algorithm module:

[0083] A callback function is a function that is called when a specific event occurs. In the wake-up recognition project, registering a wake-up event callback function to the algorithm module means associating a specific function with the algorithm module so that it can respond in time when a wake-up event occurs.

[0084] The process of registering a callback function usually includes specifying the name or pointer of the callback function and passing it to the corresponding interface of the algorithm module. When the algorithm module detects a wake-up event, it will automatically call the registered callback function, thereby realizing real-time processing of the wake-up event.

[0085] Processing of subsequent wake-up events:

[0086] When a wake-up event occurs, the algorithm module triggers a callback function. In the callback function, a series of operations can be performed to update the wake-up event data.

[0087] First, the callback function can obtain relevant information about the wake-up event, such as the strength of the wake-up signal, timestamp, etc. Then, according to specific needs, these data can be stored in a specific data structure, or further processed and analyzed.

[0088] For example, the wake-up event data can be sent to other modules for subsequent processing, or the user interface can be updated to display the wake-up status. By processing the wake-up event data in a timely manner in the callback function, the system can be quickly responded and run efficiently.

[0089] Entering a loop, acquiring the non-audio data, and determining whether the non-audio data is successfully acquired by checking whether the timestamp in the wake-up information changes. If the timestamp changes, it means that the non-audio data is successfully acquired, and the execution continues; otherwise, it means that the acquisition fails, and the acquisition continues after a preset time;

[0090] Determine whether the non-audio data FIFO is full, if not, pack the non-audio data and add it to the non-audio data FIFO, otherwise discard the non-audio data and throw an exception, and then continue to collect the non-audio data.

[0091] Among them, Figure 3 As shown, the data format of the non-audio data packaging is specifically:

[0092] The data format includes a frame header, a check bit, a message type, a current sequence number, a data length, a data part and a frame tail connected in sequence;

[0093] The frame header adopts a fixed value and a length of 4 bytes, such as 0x5AA57E6C;

[0094] The check bit is used to check the CRC check result of the data from the check bit to the end of the frame, and the length is 4Byte;

[0095] The message type is used to store the message type, the periodic message type is 0, the non-periodic new message start frame is 1, the intermediate frame is 2, the end frame is 3, and the length is 4Byte;

[0096] The current sequence number is used to count from the first packet, increments in sequence, and counts from 0 after overflow, and is used for packet loss detection, with a length of 4 bytes;

[0097] The data length is used to represent the length of the data part, excluding the length of the tail frame, which is 4 bytes long;

[0098] The data part is used to store the data part and has an indefinite length;

[0099] The frame tail adopts a fixed value and a length of 4 bytes, such as 0x7F5E4C3B.

[0100] Since the initial requirement was to pass some operating status information to the host computer, the initial protocol was relatively simple, and it only periodically updated the status and reported it to the host computer. Later, it was found that some scenarios also required the transmission of some single transmission data, so the message type field was added; some scenarios required packet loss check, so the sequence number field was added to facilitate the judgment of whether the packet was lost.

[0101] S2: Acquire audio data, and add the audio data to the audio data FIFO.

[0102] like Figure 4 As shown, in this embodiment, step S2 is specifically:

[0103] The audio data FIFO is initialized, and the audio data acquisition related initialization of the ADC and I2S in the CODEC module including starting audio acquisition is performed.

[0104] CODEC: audio codec device, in this article it refers to the audio processing module in the Taihang hardware platform (TH1520).

[0105] ADC: refers to the module in CODEC that converts analog audio signals into digital signals.

[0106] I2S: (Inter-IC Sound) bus is a bus standard designed for audio data transmission between digital audio devices. This article refers to the I2S module that comes with the CODEC module.

[0107] Entering a loop, acquiring the audio data, and when it is detected that the length of the read audio data is not 0, it means that the audio data is acquired successfully and the execution continues; otherwise, it means that the audio data is acquired and the preset time is waited to continue the acquisition;

[0108] Determine whether the audio data FIFO is full, if not, copy the audio data to the audio data FIFO, otherwise discard the audio data and throw an exception, and then continue to obtain the audio data.

[0109] S3: When the non-audio data FIFO and the audio data FIFO meet the merging conditions, copy the non-audio data and the audio data from the non-audio data FIFO and the audio data FIFO to a temporary buffer, interleave and merge all the data in the temporary buffer into each audio channel, and copy the data to the UAC data FIFO.

[0110] like Figure 5 As shown, in this embodiment, step S3 is specifically:

[0111] Entering a loop, determining whether the non-audio data FIFO and the audio data FIFO meet the merging condition, if not, then waiting for a period of time and then continuing the loop, if satisfied, then continuing the execution;

[0112] Copying the non-audio data and the audio data from the non-audio data FIFO and the audio data FIFO to the temporary buffer;

[0113] Interleave all the data in the temporary buffer into each audio channel according to the audio frame length, and copy the data into the UAC data FIFO;

[0114] Among them, judging whether the non-audio data FIFO and the audio data FIFO meet the merging conditions includes: the audio data satisfies that there is one frame of data in each of the audio channels, the non-audio data FIFO does not necessarily need to have one frame of data, but needs to be packaged according to the audio frame length, and at the same time, the UAC data FIFO includes enough space to hold the audio data and the non-audio data.

[0115] Among them, all the data in the temporary buffer are interleaved and merged into each of the audio channels according to the audio frame length, and the data is copied to the UAC data FIFO, specifically:

[0116] like Figure 6 As shown, the UAC transmission module includes a plurality of the audio channels and at least one periodic or non-periodic non-audio channel ( Figure 6 In the example, several audio channels and two non-audio channels are included. Figure 7 As shown, the data in all channels are taken in turn in order, and the first data that has not been taken is spliced ​​in sequence and stored crosswise, wherein the data length is determined according to the audio sampling bit width, such as 16-bit sampling bit width, then the length of a single data is 16 bits;

[0117] When the length of the periodic non-audio data is less than the audio frame length, the data is packaged according to the audio frame length, and the end of the insufficient part is filled with blank data;

[0118] Non-periodic non-audio data is sent according to actual data, and is packaged according to the audio frame length each time. When the remaining length is less than the audio frame length, it is sent according to one frame length, and the end is filled with blank data; when there is no data, it can be not sent or empty data can be sent.

[0119] S4: Initialize the UAC transmission module, and when there is data in the UAC data FIFO, transmit the data through the UAC.

[0120] like Figure 8 As shown, in this embodiment, step S4 is specifically:

[0121] Initialize the UAC transmission module;

[0122] Enter the loop, determine whether there is data in the UAC data FIFO, if there is data, store the data in the buffer for UAC to transmit data, otherwise wait for a preset time and continue the loop.

[0123] Second embodiment

[0124] like Fig. 9 As shown, this embodiment provides a system for simultaneously transmitting audio and non-audio data based on UAC for executing the method for simultaneously transmitting audio and non-audio data based on UAC in the first embodiment, including:

[0125] A non-audio data acquisition module 1, used for acquiring non-audio data and adding the non-audio data to a non-audio data FIFO;

[0126] An audio data acquisition module 2, used for acquiring audio data and adding the audio data to an audio data FIFO;

[0127] A data merging and interleaving module 3 is used for, when the non-audio data FIFO and the audio data FIFO meet the merging condition, copying the non-audio data and the audio data from the non-audio data FIFO and the audio data FIFO to a temporary buffer, interleaving and merging all the data in the temporary buffer into each audio channel, and copying the data to the UAC data FIFO;

[0128] The UAC data transmission module 4 is used to initialize the UAC transmission module and transmit the data through the UAC when there is data in the UAC data FIFO.

[0129] A computer-readable storage medium stores a computer code. When the computer code is executed, the above method is executed. A person skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program. The program can be stored in a computer-readable storage medium. The storage medium can include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0130] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

[0131] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0132] It should be noted that the above embodiments can be freely combined as needed. The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered as the protection scope of the present invention.

Claims

1. A method for simultaneously transmitting audio and non-audio data based on UAC, characterized in that: include: S1: Acquire non-audio data and add the non-audio data to the non-audio data FIFO; S2: Acquire audio data, and add the audio data to the audio data FIFO; S3: when the non-audio data FIFO and the audio data FIFO meet the merging condition, copy the non-audio data and the audio data from the non-audio data FIFO and the audio data FIFO to a temporary buffer, interleave and merge all the data in the temporary buffer into each audio channel, and copy the data to the UAC data FIFO; S4: Initialize the UAC transmission module, and when there is data in the UAC data FIFO, transmit the data through the UAC.

2. The method for simultaneously transmitting audio and non-audio data based on UAC according to claim 1, characterized in that: In step S1, the non-audio data is obtained and added to the non-audio data FIFO, specifically: Initialize the non-audio data FIFO, and perform related initialization of non-audio data acquisition, including initializing the algorithm module in the wake-up recognition project, registering the wake-up event callback function to the algorithm module, and updating the wake-up event data in the callback function when a wake-up event occurs subsequently; Entering a loop, acquiring the non-audio data, and determining whether the non-audio data is successfully acquired by checking whether the timestamp in the wake-up information changes. If the timestamp changes, it means that the non-audio data is successfully acquired, and the execution continues; otherwise, it means that the acquisition fails, and the acquisition continues after a preset time; Determine whether the non-audio data FIFO is full, if not, pack the non-audio data and add it to the non-audio data FIFO, otherwise discard the non-audio data and throw an exception, and then continue to collect the non-audio data.

3. The method for simultaneously transmitting audio and non-audio data based on UAC according to claim 2, characterized in that: The data format of the non-audio data packaging is specifically: The data format includes a frame header, a check bit, a message type, a current sequence number, a data length, a data part and a frame tail connected in sequence; The frame header adopts a fixed value and a length of 4 bytes; The check bit is used to check the CRC check result of the data from the check bit to the end of the frame, and the length is 4Byte; The message type is used to store the message type, the periodic message type is 0, the non-periodic new message start frame is 1, the intermediate frame is 2, the end frame is 3, and the length is 4Byte; The current sequence number is used to count from the first packet, increments in sequence, and counts from 0 after overflow, and is used for packet loss detection, with a length of 4 bytes; The data length is used to represent the length of the data part, excluding the length of the tail frame, which is 4 bytes long; The data part is used to store the data part and has an indefinite length; The frame tail adopts a fixed value and a length of 4 bytes.

4. The method for simultaneously transmitting audio and non-audio data based on UAC according to claim 1, characterized in that: In step S2, the audio data is obtained and added to the audio data FIFO, specifically: Initializing the audio data FIFO, and performing audio data acquisition related initialization including starting the ADC and I2S in the CODEC module for audio acquisition; Entering a loop, acquiring the audio data, and when it is detected that the length of the read audio data is not 0, it means that the audio data is acquired successfully and the execution continues; otherwise, it means that the audio data is acquired and the preset time is waited to continue the acquisition; Determine whether the audio data FIFO is full, if not, copy the audio data to the audio data FIFO, otherwise discard the audio data and throw an exception, and then continue to obtain the audio data.

5. The method for simultaneously transmitting audio and non-audio data based on UAC according to claim 1, characterized in that: In step S3, when the non-audio data FIFO and the audio data FIFO meet the merging condition, the non-audio data and the audio data are copied from the non-audio data FIFO and the audio data FIFO to a temporary buffer, all the data in the temporary buffer are interleaved and merged into each audio channel, and the data is copied to the UAC data FIFO, specifically: Entering a loop, determining whether the non-audio data FIFO and the audio data FIFO meet the merging condition, if not, then waiting for a period of time and then continuing the loop, if satisfied, then continuing the execution; Copying the non-audio data and the audio data from the non-audio data FIFO and the audio data FIFO to the temporary buffer; Interleave all the data in the temporary buffer into each audio channel according to the audio frame length, and copy the data into the UAC data FIFO; Among them, judging whether the non-audio data FIFO and the audio data FIFO meet the merging conditions includes: the audio data satisfies that there is one frame of data in each of the audio channels, the non-audio data FIFO does not necessarily need to have one frame of data, but needs to be packaged according to the audio frame length, and at the same time, the UAC data FIFO includes enough space to hold the audio data and the non-audio data.

6. The method for simultaneously transmitting audio and non-audio data based on UAC according to claim 5, characterized in that: All the data in the temporary buffer are interleaved and merged into each of the audio channels according to the audio frame length, and the data are copied to the UAC data FIFO, specifically: The UAC transmission module includes a plurality of the audio channels and at least one periodic or non-periodic non-audio channel, and stores the data in all the channels in an interleaved manner by sequentially taking the first data that has not been taken and splicing them in sequence, wherein the data length is determined according to the audio sampling bit width; When the length of the periodic non-audio data is less than the audio frame length, the data is packaged according to the audio frame length, and the end of the insufficient part is filled with blank data; Non-periodic non-audio data is sent according to actual data, and is packaged according to the audio frame length each time. When the remaining length is less than the audio frame length, it is sent according to one frame length, and the end is filled with blank data; when there is no data, it can be not sent or empty data can be sent.

7. The method for simultaneously transmitting audio and non-audio data based on UAC according to claim 1, characterized in that: In step S4, the UAC transmission module is initialized. When there is data in the UAC data FIFO, the data is transmitted through the UAC. Specifically, Initialize the UAC transmission module; Enter the loop, determine whether there is data in the UAC data FIFO, if there is data, store the data in the buffer for UAC to transmit data, otherwise wait for a preset time and continue the loop.

8. A system for simultaneously transmitting audio and non-audio data based on UAC for executing the method for simultaneously transmitting audio and non-audio data based on UAC according to any one of claims 1 to 7, characterized in that: include: A non-audio data acquisition module, used for acquiring non-audio data and adding the non-audio data to the non-audio data FIFO; An audio data acquisition module, used for acquiring audio data and adding the audio data to an audio data FIFO; A data merging and interleaving module, configured to copy the non-audio data and the audio data from the non-audio data FIFO and the audio data FIFO to a temporary buffer when the non-audio data FIFO and the audio data FIFO meet a merging condition, interleave and merge all the data in the temporary buffer into each audio channel, and copy the data to the UAC data FIFO; The UAC data transmission module is used to initialize the UAC transmission module and transmit the data through the UAC when there is data in the UAC data FIFO.

9. A computer device comprising a memory and one or more processors, wherein the memory stores computer codes, and when the computer codes are executed by the one or more processors, the one or more processors execute the method according to any one of claims 1 to 7. 10 . A computer-readable storage medium storing a computer code. When the computer code is executed, the method according to claim 1 is executed.

Images