Voice acquisition and preprocessing system and method based on software and hardware collaborative acceleration

Abstract

The invention provides a voice acquisition and preprocessing system and method based on software and hardware collaborative acceleration, and relates to the technical field of voice signal processing, and the system comprises a subscriber line interface circuit module which is used for acquiring voice signals and carrying out analog-to-digital conversion on the voice signals to obtain digital voice data; the field programmable gate array module is used for preprocessing the digital voice data and packaging the preprocessed data into an Ethernet frame with a specific voice identifier; the network acceleration processing module is used for identifying an Ethernet frame of which an Ethernet type field is a specific voice identifier on a data link layer, and routing the Ethernet frame to a user mode application program; according to the system and the method, the Ethernet frame of a specific voice identifier is set, so that the voice data are processed more preferentially, kernel drive and protocol stacks are not needed, delay is reduced, and the problems that a traditional system is large in resource overhead and high in delay jitter are solved.

Description

Technical Field

[0001] This invention relates to the field of speech signal processing technology, and in particular to a speech acquisition and preprocessing system and method based on hardware and software co-acceleration. Background Technology

[0002] With the rapid development of the Internet of Things (IoT), the Industrial Internet, and real-time communication technologies, voice communication is increasingly widely used in scenarios such as industrial intercom, emergency command, voice wake-up, and smart terminals. These scenarios place extremely high demands on the processing and transmission of voice data, including low latency, low jitter, low power consumption, and high reliability. In traditional voice communication systems (such as IP phones, conferencing systems, and smart voice terminals), voice data processing typically follows a standardized software path: after being acquired by an audio codec (CODEC), voice data is first managed by drivers in the operating system kernel (such as ALSA and PulseAudio), then encapsulated and transmitted through a complex network protocol stack (such as TCP / IP), and finally delivered to the user-space application for further processing. This process involves frequent user-space / kernel-space context switching, memory copying, and scheduling delays, resulting in high system latency, high CPU load, and high power consumption, making it difficult to meet the stringent real-time requirements of scenarios (such as industrial intercom, emergency communication, and voice wake-up). While some related technologies employ DMA (Direct Memory Access) or hardware acceleration modules to reduce CPU load, they still rely on kernel drivers to manage data paths, cannot completely bypass the operating system protocol stack, lack a dynamic hardware-level scheduling mechanism for voice channel priority, and do not combine silence detection to dynamically adjust the timing of data acquisition, resulting in the continuous transmission of invalid data. The use of general Ethernet frames to carry voice data requires additional encapsulation / parsing overhead and cannot be quickly recognized by hardware. Summary of the Invention

[0003] This invention provides a voice acquisition and preprocessing system and method based on hardware and software co-acceleration, which solves the defects of existing voice technology, which relies on the operating system kernel protocol stack, resulting in long data transmission paths, high CPU load, large latency and jitter, and cannot meet the requirements of real-time scenarios such as industrial intercom.

[0004] This invention provides a voice acquisition and preprocessing system based on hardware and software co-acceleration, comprising: The user line interface circuit module is used to acquire voice signals and perform analog-to-digital conversion on the voice signals to obtain digital voice data. The field-programmable gate array module is electrically connected to the user line interface circuit module and is used to preprocess the digital voice data and encapsulate the preprocessed data into an Ethernet frame with a specific voice identifier. The network acceleration processing module is connected to the field-programmable gate array module via a network interface and is used to identify Ethernet frames whose Ethernet type field is the specific voice identifier at the data link layer and route the Ethernet frames to user-space applications. The user-mode voice processing module is used to carry the user-mode application and perform application-layer voice data processing.

[0005] According to the speech acquisition and preprocessing system based on hardware and software co-acceleration provided by the present invention, the preprocessing includes: The 8-bit nonlinear PCM code output by the user line interface circuit module is converted into a 16-bit linear PCM code.

[0006] According to the speech acquisition and preprocessing system based on hardware and software co-acceleration provided by the present invention, the preprocessing includes: Calculate the amplitude of the input signal and combine it with the zero-crossing rate of the signal to perform silence detection; When silence is detected, the silence data is excluded from the digital voice data and then encapsulated into the Ethernet frame.

[0007] According to the speech acquisition and preprocessing system based on hardware and software co-acceleration provided by the present invention, the Ethernet frame includes: Destination MAC address field, source MAC address, optional VLAN tag field, Ethernet type field, packet count field, voice channel number field, voice channel data field, and CRC check field; The Ethernet type field is used to identify frames dedicated to voice acquisition, and the packet count field is used for master / slave detection of packet loss and synchronization, as well as packet loss detection.

[0008] According to the speech acquisition and preprocessing system based on hardware and software co-acceleration provided by the present invention, the network acceleration processing module is a DPAA2 module based on the data path acceleration architecture of NXP chip. The DPAA2 module, a data path acceleration architecture based on the NXP chip, includes a regular expression matching engine, which is used to identify whether the Ethernet type field is an Ethernet frame with a specific voice identifier.

[0009] The speech acquisition and preprocessing system based on hardware and software co-acceleration provided by the present invention, after the Ethernet frame identified by the regular expression matching engine as the specific speech identifier, further includes: The Ethernet frames are redirected to a hardware frame queue, and the scheduling priority of the hardware frame queue is set to be higher than that of the ordinary data buffer queue. The hardware frame queue is bound to a dedicated user-mode input / output interface.

[0010] According to the speech acquisition and preprocessing system based on hardware and software co-acceleration provided by the present invention, the user-mode speech processing module further includes a user-mode input / output interface to obtain the Ethernet frame with a specific speech identifier from the hardware frame queue in a polling manner.

[0011] According to the speech acquisition and preprocessing system based on hardware and software co-acceleration provided by the present invention, the user-mode speech processing module is further configured to: The Ethernet frame with a specific voice identifier is parsed, the count value in the packet count field is extracted, and it is continuously compared with the count value of the previous frame to perform packet loss detection.

[0012] According to the speech acquisition and preprocessing system based on hardware and software co-acceleration provided by the present invention, the field-programmable gate array module is further used for: According to a preset time interval, the preprocessed digital voice data is sampled and cached, and after reaching a preset number of samplings, the cached multiple sampled data are encapsulated into the Ethernet frame with a specific voice identifier at once.

[0013] This invention also provides a speech acquisition and preprocessing method based on hardware and software co-acceleration, comprising: The analog voice signal is acquired and converted from analog to digital through the user line interface circuit module to generate digital voice data; The digital voice data is preprocessed by a field-programmable gate array module, and the preprocessed voice data is encapsulated into Ethernet frames with specific voice identifiers. The network acceleration processing module identifies the specific voice identifier at the data link layer, and in response to successful identification, the Ethernet frame is directly routed to the user-space application. The user-mode voice processing module receives and parses the Ethernet frames, and performs application-layer voice data processing.

[0014] This invention provides a voice acquisition and preprocessing system and method based on hardware and software collaborative acceleration. The system includes a user line interface circuit module for acquiring voice signals and performing analog-to-digital conversion on the voice signals to obtain digital voice data; a field-programmable gate array (FPGA) module electrically connected to the user line interface circuit module for preprocessing the digital voice data and encapsulating the preprocessed data into Ethernet frames with specific voice identifiers; a network acceleration processing module connected to the FPGA module via a network interface for identifying Ethernet frames with the specific voice identifier at the data link layer and routing the Ethernet frames to user-space applications; and a user-space voice processing module for carrying the user-space applications to perform application-layer voice data processing. This invention constructs a full-link hardware and software collaborative architecture of "acquisition-hardware preprocessing-dedicated frame encapsulation-hardware acceleration pass-through". By setting Ethernet frames with specific voice identifiers to be different from ordinary data transmission frames, voice data is given priority processing without going through kernel drivers and protocol stacks, achieving ultra-low latency of voice data from the physical layer to user space, fundamentally solving the technical pain points of high resource consumption and high latency jitter in traditional systems. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0016] Figure 1 This is a functional structure diagram of the voice acquisition and preprocessing system based on hardware and software collaborative acceleration provided in an embodiment of the present invention; Figure 2 This is an architecture diagram of a voice acquisition and preprocessing system based on hardware and software collaborative acceleration provided in an embodiment of the present invention; Figure 3 This is a flowchart of the voice service preprocessing process provided in an embodiment of the present invention; Figure 4 This is a message encapsulation flowchart provided in an embodiment of the present invention; Figure 5 This is a flowchart of message identification and scheduling provided in an embodiment of the present invention; Figure 6 This is a flowchart of the data processing of each module provided in the embodiments of the present invention; Figure 7 This is a flowchart of a voice acquisition and preprocessing method based on hardware and software collaborative acceleration provided in an embodiment of the present invention. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0018] Figure 1 The functional structure diagram of the speech acquisition and preprocessing system based on hardware and software co-acceleration provided in the embodiments of the present invention is as follows: Figure 1 As shown, the speech acquisition and preprocessing system based on hardware and software co-acceleration provided in this embodiment of the invention includes: The Subscriber Line Interface Circuit (SLIC) 101 is used to acquire voice signals and perform analog-to-digital conversion on the voice signals to obtain digital voice data. like Figure 2 As shown, the entire system has two standard universal external interfaces, RGMII (Reduced Gigabit Media Independent Interface) and RJ11 (Registered Jack 11), which can directly interconnect with Ethernet interfaces and ordinary telephones. The hardware system is designed as a separate hardware module, with standard power supply, and with the corresponding driver software, it can be used as a standalone module inside emergency equipment, fixed computers (servers), and portable devices.

[0019] The Field-Programmable Gate Array (FPGA) 102 is electrically connected to the user line interface circuit module and is used to preprocess the digital voice data and encapsulate the preprocessed data into an Ethernet frame with a specific voice identifier. The network acceleration processing module 103 is connected to the field-programmable gate array module through a network interface and is used to identify Ethernet frames whose Ethernet type field is the specific voice identifier at the data link layer and route the Ethernet frames to the user-space application. User-mode voice processing module 104 is used to carry the user-mode application to perform application-layer voice data processing.

[0020] Traditional voice data acquisition is first managed by drivers within the operating system kernel, then encapsulated and transmitted through a complex network protocol stack (such as TCP / IP), and finally delivered to user-space applications for further processing. This process involves frequent user-space / kernel-space context switching, memory copying, and scheduling delays, resulting in high system latency, high CPU load, and high power consumption, making it difficult to meet the stringent real-time requirements of scenarios (such as industrial intercoms, emergency communications, and voice wake-up). While some related technologies employ DMA or hardware acceleration modules to reduce the CPU burden, they still rely on kernel drivers for data path management, cannot completely bypass the operating system protocol stack, lack a dynamic hardware-level scheduling mechanism for voice channel priority, and do not incorporate silence detection to dynamically adjust the acquisition timing, leading to the continuous transmission of invalid data. Furthermore, using general Ethernet frames to carry voice data requires additional encapsulation / parsing overhead and cannot be quickly recognized by hardware.

[0021] The speech acquisition and preprocessing system based on hardware and software co-acceleration provided in this invention includes a user line interface circuit module for acquiring speech signals and performing analog-to-digital conversion on the speech signals to obtain digital speech data; a field-programmable gate array (FPGA) module electrically connected to the user line interface circuit module for preprocessing the digital speech data and encapsulating the preprocessed data into Ethernet frames with specific speech identifiers; a network acceleration processing module connected to the FPGA module via a network interface for identifying Ethernet frames with the specific speech identifier at the data link layer and routing the Ethernet frames to user-space applications; and a user-space speech processing module for carrying the user-space applications to perform application-layer speech data processing. This invention constructs a full-link hardware and software co-processing architecture of "acquisition-hardware preprocessing-dedicated frame encapsulation-hardware acceleration pass-through". By setting Ethernet frames with specific speech identifiers to be different from ordinary data transmission frames, speech data is given priority processing without going through kernel drivers and protocol stacks, achieving ultra-low latency of speech data from the physical layer to user space, fundamentally solving the technical pain points of high resource consumption and high latency jitter in traditional systems.

[0022] Based on any of the above embodiments, the preprocessing of the speech acquisition and preprocessing system based on hardware and software co-acceleration includes: The 8-bit nonlinear PCM (Pulse Code Modulation) code output by the user line interface circuit module is converted into a 16-bit linear PCM code. as well as, Calculate the amplitude of the input signal and combine it with the zero-crossing rate of the signal to perform silence detection; When silence is detected, the silence data is excluded from the digital voice data and then encapsulated into the Ethernet frame.

[0023] like Figure 3 As shown, the user line interface circuit module uses 8kHz sampling to convert analog signals to digital signals and outputs 8-bit nonlinear PCM encoding. The FPGA converts the 8-bit nonlinear encoding to 16-bit linear encoding at the same sampling frequency. The FPGA uses a combination of time-domain volume and zero-crossing rate, with volume as the primary factor and zero-crossing rate as the secondary factor. Combined with the handset off / on signal provided by SLIC, it can accurately detect and judge silence.

[0024] This invention implements full hardware preprocessing for silence detection and PCM encoding conversion using SLIC+FPGA, reducing power consumption and data redundancy. The hardware preprocessing of data reduces the CPU processing load.

[0025] Based on any of the above embodiments, the Ethernet frame includes: Destination MAC address field, source MAC address, optional VLAN tag field, Ethernet type field, packet count field, voice channel number field, voice channel data field, and CRC check field; The Ethernet type field is used to identify frames dedicated to voice acquisition, and the packet count field is used for master / slave detection of packet loss and synchronization, as well as packet loss detection.

[0026] In this embodiment of the invention, the Ethernet packet structure for voice services is shown in Table 1: Table 1. Ethernet frame structure with specific voice identifiers Field serial number Field Name Length (bytes) illustrate 1 Destination MAC address 6 Receiver MAC address (usually the CPU network card MAC address) 2 Source MAC address 6 Sender MAC (FPGA MAC) 3 VLAN tag (optional) 4 If 802.1Q VLANs are enabled, the format is: TPID(0x8100) + TCI; if no VLAN is used, this field will not exist, and subsequent fields will be moved to the beginning. 4 Ethernet type (EtherType) 2 Fixed value: 0xC00C, indicating that this frame is a dedicated frame for voice acquisition. 5 Packet count (tx_cnt) 2 A 16-bit unsigned integer, incremented by 1 for each frame sent from both ends, used for master / slave detection of packet loss and synchronization, as well as packet loss detection. 6 Number of voice channels (channel_number) 1 Value range: 1 ~ 8, indicating the number of audio channels contained in this frame, which can be expanded for multi-channel applications. 7 Payload (voice channel data) 17 × N (N is the number of channels) The converted linear PCM encoded data (16-bit) consists of 2*8 bytes of audio data (each voice channel requires 16 bytes of data based on the slice's 8kHz sampling frequency and 1ms message frame interval) + 1 byte of phone status (off-hook, on-hook, etc.). 8 CRC check (FCS) 4 The standard IEEE 802.3 32-bit CRC is automatically generated / verified by the MAC layer hardware. In this embodiment of the invention, a dedicated Ethernet frame type (0xc00c) is defined for voice acquisition data identification: a brand-new link layer voice data identification mechanism; combined with the network acceleration processing module, it significantly improves the service processing speed.

[0027] like Figure 4 As shown, the preprocessed audio data is buffered 8 times at a packet transmission interval of 1ms (sampled once every 125us). The buffered data is then packaged and sent to the CPU according to the newly defined voice service Ethernet packet structure. Simultaneously, corresponding packets sent from the CPU are received at the appropriate frequency. During data transmission and reception, the packet count field is used to determine packet continuity and packet loss, and corresponding alarm information is provided. Based on the preprocessing results (silence detection results), packet transmission and reception are dynamically controlled, and the audio data and voice status are transmitted to the CPU according to the packet frame structure.

[0028] Based on any of the above embodiments, the network acceleration processing module is a DPAA2 module based on the NXP chip data path acceleration architecture. DPAA2 (Data Path Acceleration Architecture 2) is a second-generation data path acceleration architecture designed for efficient processing of network packets. It enables protocol offloading, queue management, and traffic scheduling, thereby reducing the CPU load. NXP is the company name.

[0029] The DPAA2 module, a data path acceleration architecture based on the NXP chip, includes a regular expression matching engine, which is used to identify whether the Ethernet type field is an Ethernet frame with a specific voice identifier.

[0030] In this embodiment of the invention, hardware resources are securely exposed to user space. Through device tree configuration, the DPAA2's DPIO (Data Path I / O) hardware module is mapped to a UIO (User Space I / O) device. Device tree nodes define the device's physical address, interrupt information, and the bound dedicated CPU core. After system startup, a corresponding / dev / uioX character device is generated. User-space applications can directly access the DPIO control register area through the open and mmap system calls, thereby enabling direct user-space manipulation of hardware resources and providing a foundation for subsequent zero-copy data interaction.

[0031] It should be noted that this invention is not only applicable to NXP's DPAA2 architecture, but can also be flexibly matched with various SoC (System on Chip) solutions with regular expression matching classifications, such as Intel + DPDK, Marvel, Broadcom, etc.

[0032] Based on any of the above embodiments, after the regular expression matching engine identifies the Ethernet frame as the specific voice identifier, the method further includes: The Ethernet frames are redirected to a hardware frame queue, and the scheduling priority of the hardware frame queue is set to be higher than that of the ordinary data buffer queue. The hardware frame queue is bound to a dedicated user-mode input / output interface.

[0033] This invention utilizes the DPAA2 RegEx (Regular Expression) engine to quickly identify speech frames and combines it with a priority mechanism to achieve direct access to user-mode data: bypassing protocol stack processing and avoiding kernel mode switching, significantly reducing transmission and processing latency; it also innovatively proposes to use network hardware regular expression matching function to achieve audio data stream acquisition and acceleration.

[0034] like Figure 5As shown, the DPAA2 RegEx rule configuration (recognizing EtherType=0xC00C) achieves voice frame → high-priority FQ (Frame Queue) → dedicated DPIO → dedicated CPU core, completely isolated from other system tasks to avoid interference. Rules are defined using the MC API (C language) or DPL (Data Path Layout) configuration file. All inbound frames are checked in the pipeline; if frame[12:13] == 0xC00C, the hardware automatically redirects to the specified FQ, without CPU involvement in recognition.

[0035] Based on any of the above embodiments, the user-mode voice processing module further includes a user-mode input / output interface to obtain the Ethernet frame with a specific voice identifier from the hardware frame queue in a polling manner.

[0036] In this embodiment of the invention, the dequeue response ring (SDQRR) is continuously checked through the QBMan software port. Once data arrives at the hardware, a frame descriptor is extracted from it. This descriptor contains the physical address of the buffer storing the data and the information length. The program uses a maintained physical address-virtual address mapping hash table to instantly convert the physical address into a virtual address that can be directly accessed by the user space, thereby achieving zero-copy data access.

[0037] Based on any of the above embodiments, before performing application-layer voice data processing, the user-mode voice processing module is further configured to: The Ethernet frame with a specific voice identifier is parsed, the count value in the packet count field is extracted, and it is continuously compared with the count value of the previous frame to perform packet loss detection.

[0038] In this embodiment of the invention, after obtaining the user-space virtual address of the data frame, the program begins parsing the custom voice frame header, checks the Ethernet type field for secondary confirmation, and reads the packet count (tx_cnt) for packet loss detection and alarm. Subsequently, it parses each voice channel in the payload, extracts the linear PCM audio data and handset status information, and sends them to the backend voice processing engine (such as a speech recognition or playback queue). Finally, this module is responsible for pushing the processed buffer back into the hardware buffer pool, consuming this dequeue entry, and informing the hardware that the buffer has been reclaimed and can be reused, thus completing a full data processing lifecycle.

[0039] Based on any of the above embodiments, the field-programmable gate array module is further used for: According to a preset time interval, the preprocessed digital voice data is sampled and cached, and after reaching a preset number of samplings, the cached multiple sampled data are encapsulated into the Ethernet frame with a specific voice identifier at once.

[0040] Based on any of the above embodiments, the system includes a user line interface circuit module, an FPGA module, a DPAA2 module, and user-mode audio data processing. The FPGA module performs voice silence detection, encoding conversion, and encapsulation of a new frame structure message (TYPE=0xc00c). The DPAA2 module uses a hardware regularization engine to identify specific voice frame types (TYPE=0xc00c), bypassing the operating system kernel to directly transmit voice data to the user-mode audio data processing module. This solution effectively reduces voice acquisition and transmission latency and CPU load, achieving low-power, high-real-time voice data acquisition and transmission. The data processing flow of each module is as follows: Figure 6 As shown, the user line interface circuit module (SLIC module) completes analog sampling and A / D and D / A conversion of voice signals, and provides a standard RJ11 interface for interconnection with ordinary telephones. The FPGA module performs preprocessing of voice data, including PCM linear and non-linear encoding conversion, silence detection, and data packet encapsulation. The DPAA2 module (in conjunction with the RegEx engine) is used to identify specific types of Ethernet frames for priority scheduling, directly transmitting voice data to the user-space application. The user-space voice processing module receives voice frames from the hardware pass-through module and performs related voice content recognition, dual-transmission, and other application processing.

[0041] The system architecture provided in this invention implements a fully accelerated data processing path from "acquisition-preprocessing-transmission-user-mode reception," significantly improving performance in multiple aspects such as voice processing latency, CPU utilization, message jitter, and power consumption. Specific performance indicators can be found in the section on specific implementation effects. The entire system adopts a standard modular design (interfaces, power supply, etc.) and can be used as a plug-and-play standalone hardware module, flexibly adapting to emergency equipment, fixed computers (servers), and portable devices.

[0042] Compared to traditional solutions that use standard Linux and USB / I2S audio front-ends on the same NXP LS1028A platform and employ software scheduling via ALSA drivers and UDP protocol stacks (frame interval approximately 10ms), this patented solution, supported by a customized LSDK operating system, generates customized 0xC00C voice frames through a front-end system composed of SLIC and FPGA. It utilizes the DPAA2 regular expression matching engine and UIO / DPIO user-space pass-through mechanism to achieve hardware-level data transmission. While maintaining the 8kHz / 16-bit PCM sampling standard, it precisely controls the frame interval to 1ms and achieves independent hardware multi-channel processing, completely eliminating the high latency and high load issues caused by software protocol stacks and kernel scheduling in traditional solutions. End-to-end latency is reduced from milliseconds to microseconds (20~50 μs), a reduction of over 98%; latency jitter is optimized from ±300μs to within ±5μs; CPU utilization is reduced by over 90% (<2% with 8 channels), and memory copying and context switching are completely eliminated. Meanwhile, the system supports 8 independent hardware channels (100% improvement in expansion capability), reduces SoC power consumption by 19%, and achieves a reliable packet loss recovery mechanism by combining hardware packet counting with user-mode processing. The system startup latency is reduced by 90%, and a low-latency, low-power, and highly deterministic voice transmission effect is fully realized.

[0043] The hardware-software co-accelerated voice acquisition and preprocessing system provided in this invention defines a new Ethernet packet specifically for voice services and creatively transforms the RegEx packet filtering engine of various SoCs from "network security purposes" to "real-time voice frame recognition." Combined with its native hardware queue scheduling and user-mode I / O capabilities, it constructs a fully hardware-accelerated voice path from the physical layer to the application layer, solving the core pain points of high latency and high overhead in traditional voice systems. It possesses significant technological advancement and industrial application value. Audio data acquisition and transmission are entirely implemented in hardware, without the need for kernel drivers and protocol stacks. End-to-end latency is controlled to within 1ms, resulting in low latency and minimal jitter. Invalid data transmission is reduced through silence detection and dynamic sampling control. Encoding conversion is also completed in hardware, effectively reducing CPU load. A custom Ethernet frame format synchronization and counting mechanism enables synchronized and sequential control of data transmission and reception. It can dynamically implement single-channel and multi-channel parallel data processing and multi-priority queue scheduling to ensure service quality. With a modular design and standard universal external interfaces, the entire system can be used as a hardware module, flexibly adaptable to various application scenarios.

[0044] Figure 7 A flowchart of a speech acquisition and preprocessing method based on hardware and software co-acceleration provided in an embodiment of the present invention is shown below. Figure 7As shown, the speech acquisition and preprocessing method based on hardware and software collaborative acceleration provided in this embodiment of the invention includes: Step 701: The analog voice signal is acquired and converted from analog to digital through the user line interface circuit module to generate digital voice data; Step 702: Preprocess the digital voice data using a field-programmable gate array module, and encapsulate the preprocessed voice data into Ethernet frames with specific voice identifiers; In this embodiment of the invention, the preprocessing includes at least silence detection and PCM encoding conversion; Step 703: The network acceleration processing module identifies the specific voice identifier at the data link layer, and in response to successful identification, routes the Ethernet frame directly to user space; Step 704: Receive and parse the directly transmitted Ethernet frame through the user-mode voice processing module, and perform application-layer voice data processing.

[0045] The speech acquisition and preprocessing method based on hardware and software co-acceleration provided in this invention constructs a full-link hardware and software co-processing architecture of "acquisition-hardware preprocessing-dedicated frame encapsulation-hardware acceleration pass-through". By setting Ethernet frames with specific speech identifiers to distinguish them from ordinary data transmission frames, speech data is processed with priority and does not need to go through kernel drivers and protocol stacks. This achieves ultra-low latency of speech data from the physical layer to user space, fundamentally solving the technical pain points of high resource consumption and high latency jitter in traditional systems.

[0046] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0047] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of software products. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0048] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A voice acquisition and preprocessing system based on hardware and software co-acceleration, characterized in that, include: The user line interface circuit module is used to acquire voice signals and perform analog-to-digital conversion on the voice signals to obtain digital voice data. The field-programmable gate array module is electrically connected to the user line interface circuit module and is used to preprocess the digital voice data and encapsulate the preprocessed data into an Ethernet frame with a specific voice identifier. The network acceleration processing module is connected to the field-programmable gate array module via a network interface and is used to identify Ethernet frames whose Ethernet type field is the specific voice identifier at the data link layer and route the Ethernet frames to user-space applications. The user-mode voice processing module is used to carry the user-mode application and perform application-layer voice data processing.

2. The speech acquisition and preprocessing system based on hardware and software co-acceleration according to claim 1, characterized in that, The preprocessing includes: The 8-bit nonlinear PCM code output by the user line interface circuit module is converted into a 16-bit linear PCM code.

3. The speech acquisition and preprocessing system based on hardware and software co-acceleration according to claim 1, characterized in that, The preprocessing includes: Calculate the amplitude of the input signal and combine it with the zero-crossing rate of the signal to perform silence detection; When silence is detected, the silence data is excluded from the digital voice data and then encapsulated into the Ethernet frame.

4. The speech acquisition and preprocessing system based on hardware and software co-acceleration according to claim 1, characterized in that, The Ethernet frame includes: Destination MAC address field, source MAC address, optional VLAN tag field, Ethernet type field, packet count field, voice channel number field, voice channel data field, and CRC check field; The Ethernet type field is used to identify frames dedicated to voice acquisition, and the packet count field is used for master / slave detection of packet loss and synchronization, as well as packet loss detection.

5. The speech acquisition and preprocessing system based on hardware and software co-acceleration according to claim 1, characterized in that, The network acceleration processing module is the DPAA2 module, a data path acceleration architecture based on NXP chips. The DPAA2 module, a data path acceleration architecture based on the NXP chip, includes a regular expression matching engine, which is used to identify whether the Ethernet type field is an Ethernet frame with a specific voice identifier.

6. The speech acquisition and preprocessing system based on hardware and software co-acceleration according to claim 5, characterized in that, After the regular expression matching engine identifies an Ethernet frame as the specific voice identifier, the following is also included: The Ethernet frames are redirected to a hardware frame queue, and the scheduling priority of the hardware frame queue is set to be higher than that of the ordinary data buffer queue. The hardware frame queue is bound to a dedicated user-mode input / output interface.

7. The speech acquisition and preprocessing system based on hardware and software co-acceleration according to claim 6, characterized in that, The user-mode voice processing module also includes a user-mode input / output interface, which obtains the Ethernet frames with specific voice identifiers from the hardware frame queue in a polling manner.

8. The speech acquisition and preprocessing system based on hardware and software co-acceleration according to claim 1, characterized in that, Before performing application-layer voice data processing, the user-mode voice processing module is also used for: The Ethernet frame with a specific voice identifier is parsed, the count value in the packet count field is extracted, and it is continuously compared with the count value of the previous frame to perform packet loss detection.

9. The speech acquisition and preprocessing system based on hardware and software co-acceleration according to claim 1, characterized in that, The field-programmable gate array module is also used for: According to a preset time interval, the preprocessed digital voice data is sampled and cached, and after reaching a preset number of samplings, the cached multiple sampled data are encapsulated into the Ethernet frame with a specific voice identifier at once.

10. A speech acquisition and preprocessing method based on hardware and software co-acceleration, characterized in that, include: The analog voice signal is acquired and converted from analog to digital through the user line interface circuit module to generate digital voice data; The digital voice data is preprocessed by a field-programmable gate array module, and the preprocessed voice data is encapsulated into Ethernet frames with specific voice identifiers. The network acceleration processing module identifies the specific voice identifier at the data link layer, and in response to successful identification, the Ethernet frame is directly routed to the user-space application. The user-mode voice processing module receives and parses the Ethernet frames, and performs application-layer voice data processing.

Images