A dual-mode enhanced intercom method and system

Abstract

The invention discloses a dual-mode enhanced intercom method and system, comprising a sound pickup module (1), a digital intercom encoding and sending module (2), a network intercom encoding and sending module (3), a digital intercom receiving and decoding module (4), a network intercom receiving and decoding module (5), a digital intercom frequency domain change module (6), a network intercom frequency domain change module (7), a sound frequency domain alignment module (8), a stable noise evaluation module (9), a sound frequency domain enhancement merging module (10), a sound time domain restoration change module (11), and a loudspeaker module (12); the invention realizes the fusion of digital intercom and network intercom through a dual-mode enhanced intercom, when one of the digital intercom and the network intercom is not good, the other is automatically selected to make the intercom proceed normally, and at the same time, through the sound frequency domain enhancement merging module, when the digital intercom and the network intercom proceed normally, the sound is merged to enhance the sense of hearing.

Description

Technical Field

[0001] The present invention relates to the field of communication technology, and in particular to a dual-mode enhanced intercom method and system. Background Art

[0002] Traditional intercoms include digital intercoms and network intercoms, which generally operate independently of each other. Digital intercom and network intercom signal coverage each have their own advantages and disadvantages. Compared with network intercoms, digital intercoms have shorter voice delays due to shorter paths. Currently, a method is needed to combine digital intercoms with network intercoms to prevent intercoms from being unable to be carried out when one of the digital intercoms or network intercoms is not working well.

[0003] In the prior art, there is a network digital integrated intercom system with the publication number CN102307271A. After the analog voice signal picked up by the microphone is filtered out by the differential circuit to remove the RF signal, it is sent to the DSP inside the FM1182 chip together with the analog voice signal played by the speaker and picked up by the microphone. The chip performs calculations and comparisons to filter out interference and suppress echoes before outputting; after anti-aliasing filtering, it is sent to the A / D for analog-to-digital conversion. The resulting digital voice signal is sent to the DSP for conversion processing and then sent to the network interface, which is transmitted to the remote end by the network; the remote analog voice signal is also digitized, sent to the DSP through the network interface for D / A digital-to-analog conversion, and then smoothed and filtered and output to the DSP. After processing, it is sent to the power amplifier to drive the speaker. Although the invention provides clear communication and is low cost, once the signal is poor, it is likely to cause the inability to conduct intercom. Summary of the invention

[0004] The purpose of the present invention is to provide a dual-mode enhanced intercom method and system, which realizes the integration of digital intercom and network intercom through a dual-mode enhanced intercom. When one of them is not good, the other party is automatically selected to complete the intercom.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] A dual-mode enhanced intercom system, comprising:

[0007] Sound pickup module: The sound recognition module of the intercom is used to collect sound, obtain 32K audio PCM data, and send the PCM data to the digital intercom encoding and sending module and the network intercom encoding and sending module respectively;

[0008] Digital intercom encoding and sending module: used to process PCM data, encode the PCM data into audio compression stream, and convert the encoded audio compression stream into RF signal with built-in error correction redundant information to send to the digital intercom receiving and decoding module;

[0009] Network intercom encoding and sending module: used to process PCM data, perform opus audio encoding on the PCM data to obtain audio compression stream and network RTP audio packet, and send the network RTP audio packet to the network intercom receiving and decoding module;

[0010] Digital intercom receiving and decoding module: used to send PCM data to the digital intercom frequency domain change module, and at the same time receive the coded audio compression stream to recover it according to the redundant information of error correction, and at the same time count the error rate of the coded audio compression stream, and send the data intercom error rate to the audio domain enhancement and merging module;

[0011] Network intercom receiving and decoding module: obtains the packet loss rate based on the received network RTP audio packet, and sends the network intercom packet loss rate to the stable noise assessment module; performs opus audio decoding on the network RTP audio packet to obtain a 32K audio stream and sends it to the network intercom frequency domain change module;

[0012] Digital intercom frequency domain change module: receives PCM data and slices it according to 80 sampling points. After processing, the 128 FFT values ​​of the frequency domain slice data are sent to the audio domain alignment module and the audio domain enhancement merging module;

[0013] Network intercom frequency domain change module: receives PCM data and slices it according to 320 sampling points; takes the first 128 FFT values ​​of the frequency domain slice data after processing and sends them to the sound domain alignment module, and then sends the 512 FFT values ​​of each slice in the frequency domain to the stable noise assessment module and the sound domain enhancement merging module;

[0014] The audio frequency domain alignment module is used to calculate the alignment deviation value. The audio frequency domain alignment module sends the alignment deviation value to the audio frequency domain enhancement and merging module.

[0015] Stable noise assessment module: used to analyze the background noise of network intercom, calculate the average value of high-frequency noise and the average value of medium- and low-frequency noise, and send them to the audio frequency domain enhancement and merging module;

[0016] Audio frequency domain enhancement and merging module: used to merge the frequency domain data of digital intercom and network intercom;

[0017] Sound time domain restoration and change module: used to receive the frequency domain slice data of the sound frequency domain enhancement and merging module, restore the frequency domain sound into 32K PCM data, and send the 32K PCM data to the speaker module;

[0018] Speaker module: used to receive PCM data from the sound time domain restoration change module, perform digital to analog conversion, and play sound.

[0019] The present invention also includes a dual-mode enhanced intercom method, which uses the dual-mode enhanced intercom system as described above, and is characterized in that the method includes the following steps:

[0020] S1: The sound pickup module collects sound and obtains 32K audio PCM data, which are sent to the digital intercom encoding sending module and the network intercom encoding sending module respectively;

[0021] S2: The digital intercom coding and sending module receives the 32K PCM data from the sound pickup module. The digital intercom coding and sending module processes the PCM data using a 4K low-pass filter to obtain PCM data with a frequency less than 4K, performs audio resampling to obtain 8K PCM data, and uses a vocoder to encode the 8K PCM data into an audio compression stream. The encoded audio compression stream has built-in redundant information for error correction, which is converted into a radio frequency signal and sent to the digital intercom receiving and decoding module;

[0022] S3: The network intercom encoding and sending module receives the 32K PCM data from the sound receiving module and performs opus audio encoding to obtain an audio compression stream, performs RTP packetization, and obtains a network RTP audio packet, wherein the seq number of the network RTP audio packet increases continuously. The network intercom encoding and sending module sends the network RTP audio packet to the network intercom receiving and decoding module through the mobile network;

[0023] S4: The digital intercom receiving and decoding module receives the RF signal of the digital intercom encoding and sending module, converts it into a coded audio compression stream, and obtains 8K PCM data through the vocoder decoding; the digital intercom receiving and decoding module sends the 8K PCM data to the digital intercom frequency domain change module; the digital intercom receiving and decoding module receives the coded audio compression stream and recovers it according to the redundant information of error correction, and at the same time counts the error rate of the coded audio compression stream, and the digital intercom receiving and decoding module sends the data intercom error rate to the audio frequency domain enhancement and merging module;

[0024] S5: The network intercom receiving and decoding module receives the network RTP audio packet of the network intercom encoding and sending module; obtains the packet loss rate, and the network intercom receiving and decoding module sends the network intercom packet loss rate to the stable noise assessment module; the network intercom receiving and decoding module performs opus audio decoding on the network RTP audio packet to obtain a 32K audio stream; the network intercom receiving and decoding module sends the 32K PCM data to the network intercom frequency domain change module;

[0025] S6: Digital intercom frequency domain change module The digital intercom converts the sound time domain to the frequency domain, receives PCM data and slices it according to 80 sampling points, and after processing, sends the 128 FFT values ​​of the frequency domain slice data to the sound frequency domain alignment module and the sound frequency domain enhancement and merging module;

[0026] S7: Network intercom frequency domain change module Network intercom converts the sound time domain to frequency domain, receives PCM data and slices it according to 320 sampling points; after processing, the first 128 FFT values ​​of the frequency domain slice data are taken and sent to the sound and audio domain alignment module, and then the frequency domain 512 FFT values ​​of each slice are sent to the stable noise assessment module and the sound and audio domain enhancement merging module;

[0027] S8: The audio-frequency domain alignment module calculates the alignment deviation value, and the audio-frequency domain alignment module sends the alignment deviation value to the audio-frequency domain enhancement and merging module;

[0028] S9: The stable noise evaluation module analyzes the background noise of the network intercom, calculates the average value of the high-frequency noise and the average value of the medium- and low-frequency noise, and sends the average value of the high-frequency noise and the average value of the medium- and low-frequency noise to the sound frequency domain enhancement merging module;

[0029] S10: The audio frequency domain enhancement merging module merges the frequency domain data of the digital intercom and the network intercom;

[0030] S11: The sound time domain restoration and change module receives 512 FFT values ​​of the frequency domain slice data of the sound frequency domain enhancement and merging module, and uses 512-bit FFT inverse transformation to restore the frequency domain sound into 32K PCM data; the sound time domain restoration and change module sends the 32K PCM data to the speaker module;

[0031] S12: The speaker module receives the 32K PCM data from the sound time domain restoration and change module, performs digital to analog conversion, and plays the sound.

[0032] The steps for calculating the packet loss rate in step S5 are as follows:

[0033] The network intercom receiving and decoding module determines whether packet loss occurs based on whether the seq number of the received network RTP audio packet is continuous, counts the number of packet losses per second, and divides it by the number of packets per second to obtain the packet loss rate.

[0034] The step S6 comprises the following steps:

[0035] 6.1 The digital intercom frequency domain change module receives the 8K PCM data from the digital intercom receiving and decoding module and slices it according to 80 sampling points;

[0036] 6.2 The network intercom frequency domain change module takes the 64 sampling point data of the previous slice and splices the 64 sampling point data after the current sampling point to obtain 128 sampling point data slices, uses the 128-length FFT fast Fourier transform to obtain 128 FFT values ​​of the frequency domain number of the current slice, and takes the sampling points of the slice in an interleaved manner to make the FFT changes of the two adjacent slices smoother; the current FFT resolution is 8000 / 128, which is 62.5;

[0037] 6.3 The digital intercom frequency domain change module sends the 128 FFT values ​​of the frequency domain slice data to the audio domain alignment module and the audio domain enhancement merging module.

[0038] The step S7 comprises the following steps:

[0039] 7.1 The network intercom frequency domain change module receives the 32K PCM data of the network intercom receiving and decoding module and slices it according to 320 sampling points;

[0040] 7.2 The network intercom frequency domain change module takes the 256 sampling point data of the previous slice and splices the 256 sampling point data after the current sampling point to obtain 512 sampling point data slices, and uses the 512-length FFT fast Fourier transform to obtain 512 FFT values ​​of the frequency domain number of the current slice. The current FFT resolution is 32000 / 512, which is 62.5. All network intercom frequency domain change modules and digital intercom frequency domain change modules use the same allocation rate and different bit length FFT changes, and use the cross method to take the sampling points of the slices, so that the FFT changes of two adjacent slices are smoother;

[0041] 7.3 The network intercom frequency domain change module takes the first 128 FFT values ​​of the frequency domain slice data and sends them, and then performs FFT Fourier transform to obtain the frequency domain data and sends them to the audio domain alignment module;

[0042] 7.4 Audio and video domain alignment module The network intercom frequency domain change module sends the 512 FFT values ​​of each slice in the frequency domain to the stable noise assessment module and the audio and video domain enhancement merging module.

[0043] The step S8 comprises the following steps:

[0044] The 8.1 audio frequency domain alignment module receives 128 FFT values ​​of the frequency domain slice data of the digital intercom frequency domain change module;

[0045] 8.2 The audio frequency domain alignment module receives the first 128 FFT values ​​of the frequency domain slicing data of the network intercom frequency domain change module; because the network intercom frequency domain change module and the digital intercom frequency domain change module use the same allocation rate FFT, if the frequency domain slicing data corresponds to the FFT value of the same frequency capability;

[0046] 8.3 Compared with network intercom, digital intercom has shorter voice delay due to its shorter path. The audio and video domain alignment module is used to align the frequency domain fragmentation data of network intercom with the frequency domain fragmentation data of digital intercom.

[0047] 8.4 The audio frequency domain alignment module calculates the variance of the current network intercom frequency domain slice data and the received preset N blocks of digital intercom frequency domain slice data, that is, the 128 FFT values ​​of the network intercom frequency domain slice data and the 128 FFT values ​​of the digital intercom frequency domain slice data are subtracted from each other and the squares are taken, and the square values ​​are accumulated as the variance value;

[0048] 8.5 The audio frequency domain alignment module finds the minimum variance value among the preset N blocks of digital intercom frequency domain slice data, takes this offset position as the alignment deviation value, and calculates the variance of the subsequent network intercom frequency domain slice data with the digital intercom frequency domain slice data around the alignment deviation value, and takes the offset position of the digital intercom frequency domain slice data with the smallest variance value as the new alignment deviation value;

[0049] 8.6 If the alignment deviation value calculated by the audio frequency domain alignment module for the preset K network intercom frequency domain slices is basically unchanged, the audio frequency domain alignment module considers that a stable alignment deviation value has been found;

[0050] 8.7 The sound frequency domain alignment module sends the alignment deviation value to the sound frequency domain enhancement merging module.

[0051] The step S9 comprises the following steps:

[0052] 9.1 The stable noise evaluation module receives 512 FFT values ​​in the frequency domain of each slice;

[0053] 9.2 The stable noise evaluation module accumulates the 512 FFT values ​​in the frequency domain of the most recent 20 slices to obtain 512 FFT accumulated values;

[0054] 9.3 The stable noise evaluation module judges the 512 FFT accumulated values ​​and the preset energy threshold respectively. If the energy is higher than the quantile value, the FFT accumulated value is considered to be noise, and if it is higher than the quantile value, the FFT accumulated value is considered to be speech; the FFT number is multiplied by the FFT resolution 62.5 to obtain the corresponding frequency of the FFT;

[0055] 9.4 The stable noise evaluation module accumulates the FFT cumulative values ​​of the noise with a frequency greater than 4K and takes the average value as the mean value of the high-frequency noise;

[0056] 9.5 The stable noise evaluation module accumulates and averages the noise FFT cumulative values ​​less than or equal to the frequency 4K, and takes the average value as the mean value of the medium and low frequency noise;

[0057] 9.6 The stable noise evaluation module sends the average value of the high-frequency noise and the average value of the medium- and low-frequency noise to the sound field enhancement merging module.

[0058] The step S10 comprises the following steps:

[0059] 10.1 The error rate of the data intercom received by the audio frequency domain enhancement merging module and the data intercom receiving and decoding module.

[0060] 10.2 The network intercom packet loss rate of the network intercom receiving decoding module of the audio frequency domain enhancement merging module;

[0061] 10.3 The audio frequency domain enhancement merging module receives 128 FFT values ​​of the frequency domain slice data of the digital intercom frequency domain change module;

[0062] 10.4 The audio frequency domain enhancement merging module receives 512 FFT values ​​of the network intercom frequency domain fragmentation data of the network intercom frequency domain change module;

[0063] 10.5 The sound frequency domain enhancement merging module receives the alignment deviation value of the sound frequency domain alignment module;

[0064] 10.6 The sound frequency domain enhancement merging module receives the average value of the high frequency noise and the average value of the medium and low frequency noise of the stable noise evaluation module;

[0065] 10.7 When the data intercom error rate is less than 5% and the network intercom packet loss rate is greater than 5%, the audio frequency domain enhancement merging module selects 128 FFT values ​​of the frequency domain fragmentation data and converts them into 512 FFT values ​​and sends them to the audio time domain restoration change module;

[0066] 10.8 The method of converting 128 FFT values ​​of the frequency domain slice data of the sound domain enhancement merging module into 512 FFT values, filling 384 FFT values, and filling the FFT value with 0, finally obtaining 512 FFT values ​​of the frequency domain slice data;

[0067] 10.9 When the data intercom error rate is greater than 5% and the network intercom packet loss rate is less than 5%, the audio frequency domain enhancement merging module selects 512 FFT values ​​of the network intercom frequency domain fragmentation data and sends them to the audio time domain restoration change module;

[0068] 10.10 When the data intercom error rate is greater than 5% and the network intercom packet loss rate is greater than 5%, the data intercom error rate and the network intercom packet loss rate are judged. If the data intercom error rate is less than the network intercom packet loss rate, the sound and audio domain enhancement merging module selects 128 FFT values ​​of the frequency domain fragmentation data to be converted into 512 FFT values ​​and sends them to the sound time domain restoration change module. If the data intercom error rate is greater than the network intercom packet loss rate, the sound and audio domain enhancement merging module selects 512 FFT values ​​of the network intercom frequency domain fragmentation data to be sent to the sound time domain restoration change module.

[0069] 10.11 When the data intercom error rate is less than 5% and the network intercom packet loss rate is less than 5%, the sound merging process is carried out;

[0070] 10.12 The sound domain enhancement merging module calculates the mid-low frequency merging coefficient and the high frequency merging coefficient according to the preset mid-low frequency noise benchmark and the high frequency noise benchmark. The mid-low frequency merging coefficient is the average of the mid-low frequency noise benchmark / mid-low frequency noise, and the high frequency merging coefficient is the average of the high frequency noise benchmark / high frequency noise.

[0071] 10.13 The audio domain enhancement merging module adds the mid-low frequency merging coefficient to each FFT value of the 128 FFT values ​​of the frequency domain slice data of the digital intercom and multiplies it by the corresponding FFT value of the frequency domain slice data of the network intercom, and finally obtains the first 128 FFT values ​​of the frequency domain slice data of the merged audio. While ensuring that the listening experience of the digital intercom remains unchanged, a small amount of background noise is added according to the mid-low frequency merging coefficient;

[0072] 10.14 The audio frequency domain enhancement merging module multiplies the last 384 FFT values ​​of the frequency domain slicing data of the network intercom by the high-frequency merging coefficient, and uses them as the first 384 FFT values ​​of the frequency domain slicing data of the merged audio. On the basis of the digital intercom, a small amount of high-frequency sound is restored according to the high-frequency merging coefficient to improve the sound.

[0073] 10.15 The sound frequency domain enhancement merging module finally obtains 512 FFT values ​​of the frequency domain slice data of the merged audio and sends them to the sound time domain restoration change module.

[0074] The invention provides a dual-mode enhanced intercom method and system with the following beneficial effects:

[0075] The present invention realizes the fusion of digital intercom and network intercom based on dual-mode enhanced intercom. When one of the digital intercom and the network intercom is not good, the other one is automatically selected to make the intercom proceed normally. At the same time, through the sound and frequency domain enhancement merging module, when the digital intercom and the network intercom proceed normally, the sound is merged to enhance the hearing sense. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0077] Figure 1 The present invention is used for a dual-mode enhanced intercom method and system flow chart. DETAILED DESCRIPTION

[0078] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0079] like Figure 1As shown, a dual-mode enhanced intercom system includes: a sound pickup module 1, a digital intercom encoding and sending module 2, a network intercom encoding and sending module 3, a digital intercom receiving and decoding module 4, a network intercom receiving and decoding module 5, a digital intercom frequency domain change module 6, a network intercom frequency domain change module 7, a sound frequency domain alignment module 8, a stable noise evaluation module 9, a sound frequency domain enhancement and merging module 10, a sound time domain restoration and change module 11, and a speaker module 12.

[0080] Sound pickup module 1: The sound recognition module of the intercom is used to collect sound, obtain 32K audio PCM data, and send the PCM data to the digital intercom encoding and sending module 2 and the network intercom encoding and sending module 3 respectively;

[0081] Digital intercom coding and sending module 2: The digital intercom coding and sending module 2 receives the PCM data of the sound receiving module 132K. The digital intercom coding and sending module 2 uses a 4K low-pass filter to process the PCM data to obtain PC M data with a frequency less than 4K, performs audio resampling to obtain 8K PCM data, and uses a voice coder to encode the 8K PCM data into an audio compression stream. The encoded audio compression stream has built-in redundant information for error correction, which is converted into a radio frequency signal and sent to the digital intercom receiving and decoding module 4.

[0082] Network intercom encoding and sending module 3: The network intercom encoding and sending module 3 receives the PCM data of the sound receiving module 132K and performs opus audio encoding to obtain an audio compression stream, performs RTP packetization, and obtains a network RTP audio packet, wherein the seq number of the network RTP audio packet increases continuously. The network intercom encoding and sending module 3 sends the network RTP audio packet to the network intercom receiving and decoding module 5 via the mobile network.

[0083] Digital intercom receiving and decoding module 4: The digital intercom receiving and decoding module 4 receives the RF signal of the digital intercom encoding and transmitting module 2, converts it into a coded audio compression stream, and obtains 8K PCM data through the vocoder decoding. The digital intercom receiving and decoding module 4 sends the 8K PCM data to the digital intercom frequency domain change module 6. The digital intercom receiving and decoding module 4 receives the coded audio compression stream and restores it according to the redundant information of error correction, and at the same time counts the error rate of the coded audio compression stream. The digital intercom receiving and decoding module 4 sends the data intercom error rate to the audio domain enhancement and merging module 10.

[0084] Network intercom receiving and decoding module 5: Network intercom receiving and decoding module 5 receives the network RTP audio packet from the network intercom encoding and sending module 3. Network intercom receiving and decoding module 5 determines whether packet loss occurs according to whether the seq sequence number of the received network RTP audio packet is continuous, counts the number of packet losses per second, and divides it by the number of packets per second to obtain the packet loss rate. Network intercom receiving and decoding module 5 sends the network intercom packet loss rate to the stable noise assessment module 9. Network intercom receiving and decoding module 5 performs opus audio decoding on the network RTP audio packet to obtain a 32K audio stream. Network intercom receiving and decoding module 5 sends 32K PCM data to the network intercom frequency domain change module 7.

[0085] Digital intercom frequency domain change module 6: The digital intercom frequency domain change module 6 receives the 8K PCM data of the digital intercom receiving and decoding module 4 and slices it according to 80 sampling points. The network intercom frequency domain change module 7 takes the 64 sampling point data of the previous slice and splices the 64 sampling point data after the current sampling point to obtain 128 sampling point data slices, uses the 128-length FFT fast Fourier transform to obtain 128 FFT values ​​of the frequency domain number of the current slice, and takes the sampling points of the slice in an interleaved manner to make the FFT change of the two adjacent slices smoother. The current FFT resolution is 8000 / 128, which is 62.5. The digital intercom frequency domain change module 6 sends the 128 FFT values ​​of the frequency domain slice data to the audio domain alignment module 8 and the audio domain enhancement merging module 10.

[0086] Network intercom frequency domain change module 7: Network intercom frequency domain change module 7 receives 32K PCM data from network intercom receiving and decoding module 5 and slices it according to 320 sampling points. Network intercom frequency domain change module 7 takes 256 sampling point data of the previous slice and splices the 256 sampling point data after the current sampling point to obtain 512 sampling point data slices, and uses 512 length FFT fast Fourier transform to obtain 512 FFT values ​​of the frequency domain number of the current slice. The current FFT resolution is 32000 / 512, which is 62.5. All network intercom frequency domain change modules 7 and digital intercom frequency domain change modules 6 use the same allocation rate and different bit length FFT changes. The sampling points of the slices are taken in an interleaved manner to make the FFT changes of the two adjacent slices smoother. Network intercom frequency domain change module 7 takes the first 128 FFT values ​​of the frequency domain slice data and sends them, and sends the frequency domain data obtained by performing FFT Fourier transform to the sound and audio domain alignment module 8. The audio frequency domain alignment module 8 and the network intercom frequency domain change module 7 send the 512 FFT values ​​of each slice in the frequency domain to the stable noise assessment module 9 and the audio frequency domain enhancement merging module 10.

[0087] Audio and video domain alignment module 8: The audio and video domain alignment module 8 receives 128 FFT values ​​of the frequency domain slicing data of the digital intercom frequency domain change module 6, and the audio and video domain alignment module 8 receives the first 128 FFT values ​​of the frequency domain slicing data of the network intercom frequency domain change module 7. Because the network intercom frequency domain change module 7 and the digital intercom frequency domain change module 6 use the same allocation rate FFT, if the frequency domain slicing data corresponds to the FFT value of the same frequency capacity value. Compared with the network intercom, the digital intercom has a shorter voice delay because the path is shorter. The audio and video domain alignment module 8 aligns the network intercom frequency domain slicing data with the digital intercom frequency domain slicing data. The audio frequency domain alignment module 8 calculates the variance of the current network intercom frequency domain slice data and the received N (e.g., 100) blocks of digital intercom frequency domain slice data, that is, the 128 FFT values ​​of the network intercom frequency domain slice data and the 128 FFT values ​​of the digital intercom frequency domain slice data are subtracted and squared, and the square values ​​are accumulated as the variance value. The audio frequency domain alignment module 8 finds the minimum variance value among the N (e.g., 100) blocks of digital intercom frequency domain slice data, and uses this offset position as the alignment deviation value. For the subsequent network intercom frequency domain slice data, the variance is calculated with the digital intercom frequency domain slice data around the alignment deviation value, and the offset position of the digital intercom frequency domain slice data with the smaller variance value is used as the new alignment deviation value. If the alignment deviation value calculated by the audio frequency domain alignment module 8 in K consecutive (e.g., 20) network intercom frequency domain slices is basically unchanged, the audio frequency domain alignment module 8 considers that a stable alignment deviation value has been found, and the audio frequency domain alignment module 8 sends the alignment deviation value to the audio frequency domain enhancement merging module 10.

[0088] Stable noise assessment module 9: Stable noise assessment module 9 receives 512 FFT values ​​in the frequency domain of each slice. Stable noise assessment module 9 accumulates the 512 FFT values ​​in the frequency domain of the most recent 20 slices to obtain 512 FFT cumulative values. Stable noise assessment module 9 judges the 512 FFT cumulative values ​​and the preset energy threshold respectively. If the energy is higher, the FFT cumulative value is considered to be noise, and if it is higher than the quantile value, the FFT cumulative value is considered to be speech. The FFT number is multiplied by the FFT resolution 62.5 to obtain the corresponding frequency of the FFT. Stable noise assessment module 9 accumulates and averages the noise FFT cumulative values ​​greater than the frequency 4K, and uses the average value as the mean of high-frequency noise. Stable noise assessment module 9 accumulates and averages the noise FFT cumulative values ​​less than or equal to the frequency 4K, and uses the average value as the mean of medium and low frequency noise. Stable noise assessment module 9 sends the mean of high-frequency noise and the mean of medium and low frequency noise to the sound field enhancement merging module 10.

[0089] The audio-frequency domain enhancement and merging module 10: The audio-frequency domain enhancement and merging module 10 receives the data intercom error rate of the digital intercom receiving and decoding module 4, and the audio-frequency domain enhancement and merging module 10 receives the network intercom packet loss rate of the network intercom receiving and decoding module 5. The audio-frequency domain enhancement and merging module 10 receives 128 FFT values ​​of the frequency domain slicing data of the digital intercom frequency domain change module 6. The audio-frequency domain enhancement and merging module 10 receives 512 FFT values ​​of the network intercom frequency domain slicing data of the network intercom frequency domain change module 7. The audio-frequency domain enhancement and merging module 10 receives the alignment deviation value of the audio-frequency domain alignment module 8. The audio-frequency domain enhancement and merging module 10 receives the mean of the high-frequency noise and the mean of the medium- and low-frequency noise of the stable noise evaluation module 9. When the data intercom error rate is less than 5% and the network intercom packet loss rate is greater than 5%, the audio-frequency domain enhancement and merging module 10 selects 128 FFT values ​​of the frequency domain slicing data to convert them into 512 FFT values ​​and sends them to the sound time domain restoration change module 11. The method of converting 128 FFT values ​​of the frequency domain slice data of the audio frequency domain enhancement merging module 10 into 512 FFT values, filling 384 FFT values, filling FFT values ​​with 0, and finally obtaining 512 FFT values ​​of the frequency domain slice data. When the data intercom error rate is greater than 5% and the network intercom packet loss rate is less than 5%, the audio frequency domain enhancement merging module 10 selects 512 FFT values ​​of the network intercom frequency domain slice data and sends them to the sound time domain restoration change module 11. When the data intercom error rate is greater than 5% and the network intercom packet loss rate is greater than 5%, the data intercom error rate and the network intercom packet loss rate are judged. If the data intercom error rate is less than the network intercom packet loss rate, the sound frequency domain enhancement merging module 10 selects 128 FFT values ​​of the frequency domain slicing data to be converted into 512 FFT values ​​and sends them to the sound time domain restoration change module 11. If the data intercom error rate is greater than the network intercom packet loss rate, the sound frequency domain enhancement merging module 10 selects 512 FFT values ​​of the network intercom frequency domain slicing data and sends them to the sound time domain restoration change module 11. When the data intercom error rate is less than 5% and the network intercom packet loss rate is less than 5%, the sound merging process is performed. The sound frequency domain enhancement merging module 10 calculates the mid-low frequency merging coefficient and the high frequency merging coefficient according to the preset mid-low frequency noise benchmark and the high frequency noise benchmark. The mid-low frequency merging coefficient is the average of the mid-low frequency noise benchmark / mid-low frequency noise, and the high frequency merging coefficient is the average of the high frequency noise benchmark / high frequency noise. The audio domain enhancement merging module 10 adds the low- and medium-frequency merging coefficient to each FFT value of the 128 FFT values ​​of the frequency domain slice data of the digital intercom and multiplies it by the corresponding FFT value of the frequency domain slice data of the network intercom, and finally obtains the first 128 FFT values ​​of the frequency domain slice data of the merged audio, while ensuring that the listening experience of the digital intercom remains unchanged, and adds a small amount of background noise. The audio domain enhancement merging module 10 multiplies the last 384 FFT values ​​of the frequency domain slice data of the network intercom by the high-frequency merging coefficient as the first 384 FFT values ​​of the frequency domain slice data of the merged audio, and restores a small amount of high-frequency sound on the basis of the digital intercom to improve the sound.The sound frequency domain enhancement merging module 10 finally obtains 512 FFT values ​​of the frequency domain slice data of the merged audio and sends them to the sound time domain restoration change module 11.

[0090] Sound time domain restoration and change module 11: The sound time domain restoration and change module 11 receives 512 FFT values ​​of the frequency domain slice data of the sound frequency domain enhancement and merging module 10, and uses 512-bit FFT inverse transformation to restore the frequency domain sound into 32K PCM data. The sound time domain restoration and change module 11 sends the 32K PCM data to the speaker module 12.

[0091] Speaker module 12: The speaker module 12 receives the 32K PCM data from the sound time domain restoration and change module 11, performs digital to analog conversion, and plays the sound.

[0092] The present invention also includes a dual-mode enhanced intercom method, which uses the above dual-mode enhanced intercom system and includes the following steps:

[0093] S1: The sound pickup module 1 is used to collect sound, obtain 32K audio PCM data, and send the PCM data to the digital intercom encoding and sending module 2 and the network intercom encoding and sending module 3 respectively;

[0094] S2: The digital intercom coding and sending module 2 receives the PCM data of the sound receiving module 132K, processes the PCM data with a 4K low-pass filter, obtains PCM data with a frequency less than 4K, performs audio resampling to obtain 8K PCM data, and uses a vocoder to encode the 8K PCM data into an audio compression stream. The encoded audio compression stream has built-in redundant information for error correction, which is converted into a radio frequency signal and sent to the digital intercom receiving and decoding module 4;

[0095] S3: The network intercom encoding and sending module 3 receives the PCM data of the sound receiving module 132K and performs opus audio encoding to obtain an audio compression stream, performs RTP packetization, and obtains a network RTP audio packet, wherein the seq sequence number of the network RTP audio packet increases continuously. The network intercom encoding and sending module 3 sends the network RTP audio packet to the network intercom receiving and decoding module 5 through the mobile network;

[0096] S4: The digital intercom receiving and decoding module 4 receives the radio frequency signal of the digital intercom encoding and sending module 2, converts it into a coded audio compression stream, and obtains 8K PCM data through the vocoder decoding; the digital intercom receiving and decoding module 4 sends the 8K PCM data to the digital intercom frequency domain change module 6; the digital intercom receiving and decoding module 4 receives the coded audio compression stream and recovers it according to the redundant information of error correction, and at the same time counts the error rate of the coded audio compression stream, and the digital intercom receiving and decoding module 4 sends the data intercom error rate to the voice and audio domain enhancement and merging module 10;

[0097] S5: the network intercom receiving and decoding module 5 receives the network RTP audio packet of the network intercom encoding and sending module 3; the network intercom receiving and decoding module 5 determines whether packet loss occurs according to whether the seq sequence number of the received network RTP audio packet is continuous, counts the number of packet losses per second, and divides it by the number of packets per second to obtain the packet loss rate, and the network intercom receiving and decoding module 5 sends the network intercom packet loss rate to the stable noise evaluation module 9; the network intercom receiving and decoding module 5 performs opus audio decoding on the network RTP audio packet to obtain a 32K audio stream; the network intercom receiving and decoding module 5 sends the 32K PCM data to the network intercom frequency domain change module 7;

[0098] S6: The digital intercom frequency domain change module 6 converts the sound time domain to the frequency domain, receives the PCM data and slices it according to 80 sampling points, and after processing, sends the 128 FFT values ​​of the frequency domain slice data to the sound frequency domain alignment module 8 and the sound frequency domain enhancement and merging module 10;

[0099] 6.1 The digital intercom frequency domain change module 6 receives the 8K PCM data from the digital intercom receiving and decoding module 4 and slices it according to 80 sampling points;

[0100] 6.2 Network intercom frequency domain change module 7 takes 64 sampling point data of the previous slice and splices the 64 sampling point data after the current sampling point to obtain 128 sampling point data slices, uses 128 length FFT fast Fourier transform to obtain 128 FFT values ​​of the frequency domain number of the current slice, and takes the sampling points of the slice in an interleaved manner to make the FFT change of the two adjacent slices smoother; the current FFT resolution is 8000 / 128, which is 62.5;

[0101] 6.3 The digital intercom frequency domain change module 6 sends the 128 FFT values ​​of the frequency domain slice data to the audio domain alignment module 8 and the audio domain enhancement merging module 10.

[0102] S7: Network intercom frequency domain change module 7 Network intercom converts the sound time domain to frequency domain, receives PCM data and slices it according to 320 sampling points; after processing, the first 128 FFT values ​​of the frequency domain slice data are taken and sent to the sound and audio domain alignment module 8, and then the frequency domain 512 FFT values ​​of each slice are sent to the stable noise assessment module 9 and the sound and audio domain enhancement merging module 10;

[0103] 7.1 The network intercom frequency domain change module 7 receives the 32K PCM data of the network intercom receiving and decoding module 5 and slices it according to 320 sampling points;

[0104] 7.2 The network intercom frequency domain change module 7 takes the 256 sampling point data of the previous slice and splices the 256 sampling point data after the current sampling point to obtain 512 sampling point data slices, and uses the 512-length FFT fast Fourier transform to obtain 512 FFT values ​​of the frequency domain number of the current slice. The current FFT resolution is 32000 / 512, which is 62.5. All network intercom frequency domain change modules 7 and digital intercom frequency domain change modules 6 use the same allocation rate and different bit lengths of FFT changes, and use a cross method to take the sampling points of the slices, so that the FFT changes of two adjacent slices are smoother;

[0105] 7.3 The network intercom frequency domain change module 7 takes the first 128 FFT values ​​of the frequency domain slice data and sends them, and sends the frequency domain data obtained by performing FFT Fourier transform to the audio domain alignment module 8;

[0106] 7.4 The audio frequency domain alignment module 8 and the network intercom frequency domain change module 7 send the 512 FFT values ​​of each slice in the frequency domain to the stable noise assessment module 9 and the audio frequency domain enhancement merging module 10.

[0107] S8: the audio frequency domain alignment module 8 calculates the alignment deviation value, and the audio frequency domain alignment module 8 sends the alignment deviation value to the audio frequency domain enhancement and merging module 10;

[0108] 8.1 The audio frequency domain alignment module 8 receives 128 FFT values ​​of the frequency domain slice data from the digital intercom frequency domain change module 6;

[0109] 8.2 The audio frequency domain alignment module 8 receives the first 128 FFT values ​​of the frequency domain slicing data of the network intercom frequency domain change module 7; because the network intercom frequency domain change module 7 and the digital intercom frequency domain change module 6 use the same allocation rate FFT, if the frequency domain slicing data corresponds to the FFT value of the same frequency capability;

[0110] 8.3 Compared with network intercom, digital intercom has shorter voice delay due to shorter path. The audio frequency domain alignment module 8 is used to align the frequency domain fragmentation data of network intercom with the frequency domain fragmentation data of digital intercom.

[0111] 8.4 The audio frequency domain alignment module 8 calculates the variance of the current network intercom frequency domain slice data and the received N (e.g., 100) blocks of digital intercom frequency domain slice data, that is, the 128 FFT values ​​of the network intercom frequency domain slice data and the 128 FFT values ​​of the digital intercom frequency domain slice data are subtracted from each other and the squares are taken, and the square values ​​are accumulated as the variance value;

[0112] 8.5 The audio frequency domain alignment module 8 finds the minimum variance value in N (e.g., 100) blocks of digital intercom frequency domain slice data, takes this offset position as the alignment deviation value, and performs variance calculation on the subsequent network intercom frequency domain slice data with the digital intercom frequency domain slice data around the alignment deviation value, and takes the offset position of the digital intercom frequency domain slice data with the smallest variance value as the new alignment deviation value;

[0113] 8.6 If the alignment deviation value calculated by the audio frequency domain alignment module 8 in K consecutive (e.g., 20) network intercom frequency domain slices is basically unchanged, the audio frequency domain alignment module 8 considers that a stable alignment deviation value has been found;

[0114] 8.7 The sound frequency domain alignment module 8 sends the alignment deviation value to the sound frequency domain enhancement and merging module 10.

[0115] S9: The stable noise evaluation module 9 analyzes the background noise of the network intercom, calculates the average value of the high-frequency noise and the average value of the medium- and low-frequency noise, and sends the average value of the high-frequency noise and the average value of the medium- and low-frequency noise to the sound frequency domain enhancement merging module 10;

[0116] 9.1 Stable noise assessment module 9 receives 512 FFT values ​​in the frequency domain of each slice;

[0117] 9.2 Stable noise evaluation module 9 accumulates the 512 FFT values ​​in the frequency domain of the most recent 20 slices respectively to obtain 512 FFT accumulated values;

[0118] 9.3 Stable noise evaluation module 9 judges the 512 FFT accumulated values ​​and the preset energy threshold respectively. If the energy is higher than the quantile value, the FFT accumulated value is considered to be noise; if it is higher than the quantile value, the FFT accumulated value is considered to be speech; the FFT number is multiplied by the FFT resolution 62.5 to obtain the corresponding frequency of the FFT;

[0119] 9.4 Stable noise evaluation module 9 accumulates and averages the noise FFT cumulative values ​​greater than the frequency 4K, and takes the average value as the mean value of the high-frequency noise;

[0120] 9.5 Stable noise evaluation module 9 accumulates and averages the noise FFT cumulative values ​​less than or equal to the frequency 4K, and takes the average value as the mean value of the medium and low frequency noise;

[0121] 9.6 The stable noise assessment module 9 sends the average value of the high-frequency noise and the average value of the medium- and low-frequency noise to the sound field enhancement merging module 10.

[0122] S10: The audio frequency domain enhancement merging module 10 merges the frequency domain data of the digital intercom and the network intercom;

[0123] 10.1 The data intercom error rate of the digital intercom receiving and decoding module 4 received by the audio frequency domain enhancement merging module 10;

[0124] 10.2 The network intercom packet loss rate received by the audio frequency domain enhancement merging module 10 and the network intercom receiving decoding module 5;

[0125] 10.3 The audio frequency domain enhancement and merging module 10 receives 128 FF T values ​​of the frequency domain slice data from the digital intercom frequency domain change module 6;

[0126] 10.4 The audio frequency domain enhancement merging module 10 receives 512 FFT values ​​of the network intercom frequency domain fragmentation data of the network intercom frequency domain change module 7;

[0127] 10.5 The sound frequency domain enhancement and merging module 10 receives the alignment deviation value of the sound frequency domain alignment module 8;

[0128] 10.6 The sound domain enhancement merging module 10 receives the average value of the high frequency noise and the average value of the medium and low frequency noise from the stable noise evaluation module 9;

[0129] 10.7 When the data intercom error rate is less than 5% and the network intercom packet loss rate is greater than 5%, the sound frequency domain enhancement merging module 10 selects 128 FFT values ​​of the frequency domain slice data and converts them into 512 FFT values ​​and sends them to the sound time domain restoration change module 11;

[0130] 10.8 The method of converting 128 FFT values ​​of the frequency domain slice data of the sound domain enhancement merging module 10 into 512 FFT values, filling 384 FFT values, and filling the FFT values ​​with 0, and finally obtaining 512 FFT values ​​of the frequency domain slice data;

[0131] 10.9 When the data intercom error rate is greater than 5% and the network intercom packet loss rate is less than 5%, the audio frequency domain enhancement merging module 10 selects 512 FFT values ​​of the network intercom frequency domain fragmentation data and sends them to the audio time domain restoration change module 11;

[0132] 10.10 When the data intercom error rate is greater than 5% and the network intercom packet loss rate is greater than 5%, the data intercom error rate and the network intercom packet loss rate are judged. If the data intercom error rate is less than the network intercom packet loss rate, the sound and frequency domain enhancement merging module 10 selects 128 FFT values ​​of the frequency domain slicing data to be converted into 512 FFT values ​​and sends them to the sound time domain restoration change module 11. If the data intercom error rate is greater than the network intercom packet loss rate, the sound and frequency domain enhancement merging module 10 selects 512 FFT values ​​of the network intercom frequency domain slicing data to be sent to the sound time domain restoration change module 11.

[0133] 10.11 When the data intercom error rate is less than 5% and the network intercom packet loss rate is less than 5%, the sound merging process is carried out.

[0134] 10.12 The sound domain enhancement merging module 10 calculates a mid-low frequency merging coefficient and a high frequency merging coefficient according to a preset mid-low frequency noise benchmark and a high frequency noise benchmark, wherein the mid-low frequency merging coefficient is the average of the mid-low frequency noise benchmark / mid-low frequency noise, and the high frequency merging coefficient is the average of the high frequency noise benchmark / high frequency noise;

[0135] 10.13 The audio domain enhancement merging module 10 adds each FFT value of the 128 FFT values ​​of the frequency domain slice data of the digital intercom to the mid-low frequency merging coefficient and multiplies it by the corresponding FFT value of the frequency domain slice data of the network intercom, and finally obtains the first 128 FFT values ​​of the frequency domain slice data of the merged audio. Under the condition of ensuring that the listening experience of the digital intercom remains unchanged, a small amount of background noise is added according to the mid-low frequency merging coefficient;

[0136] 10.14 The audio frequency domain enhancement merging module 10 multiplies the last 384 FFT values ​​of the frequency domain slicing data of the network intercom by the high-frequency merging coefficient respectively, as the first 384 FFT values ​​of the frequency domain slicing data of the merged audio, and restores a small amount of high-frequency sound according to the high-frequency merging coefficient on the basis of the digital intercom to improve the sound;

[0137] 10.15 The sound frequency domain enhancement merging module 10 finally obtains 512 FFT values ​​of the frequency domain slice data of the merged audio and sends them to the sound time domain restoration change module 11.

[0138] S11: the sound time domain restoration and change module 11 receives 512 FFT values ​​of the frequency domain slice data of the sound frequency domain enhancement and merging module 10, and uses 512-bit FFT inverse transformation to restore the frequency domain sound into 32K PCM data; the sound time domain restoration and change module 11 sends the 32K PCM data to the speaker module 12;

[0139] S12: The speaker module 12 receives the 32K PCM data from the sound time domain restoration and transformation module 11, performs digital to analog conversion, and plays the sound.

[0140] The specific embodiments described above further illustrate the technical problems, technical solutions and beneficial effects solved by the present invention. It should be understood that the above are only specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A dual-mode enhanced intercom system, characterized in that: include: Sound pickup module (1): a sound recognition module for intercom, used for collecting sound, obtaining 32K audio PCM data, and sending the PCM data to the digital intercom encoding and sending module (2) and the network intercom encoding and sending module (3) respectively; Digital intercom encoding and sending module (2): used for processing PCM data, encoding the PCM data into an audio compression stream, encoding the audio compression stream with built-in error correction redundant information, converting it into a radio frequency signal and sending it to the digital intercom receiving and decoding module (4); The network intercom encoding and sending module (3) is used to process PCM data, perform opus audio encoding on the PCM data to obtain an audio compression stream and a network RTP audio packet, and send the network RTP audio packet to the network intercom receiving and decoding module (5); A digital intercom receiving and decoding module (4) is used to send PCM data to a digital intercom frequency domain change module (6), and at the same time receive a coded audio compression stream to recover it according to redundant information of error correction, and at the same time count the error rate of the coded audio compression stream, and send the data intercom error rate to a voice frequency domain enhancement merging module (10); The network intercom receiving and decoding module (5) obtains the packet loss rate according to the received network RTP audio packet, and sends the network intercom packet loss rate to the stable noise evaluation module (9); performs opus audio decoding on the network RTP audio packet to obtain a 32K audio stream and sends it to the network intercom frequency domain change module (7); Digital intercom frequency domain change module (6): receives PCM data and slices it according to 80 sampling points, and after processing, sends 128 FFT values ​​of the frequency domain slice data to the voice domain alignment module (8) and the voice domain enhancement merging module (10); Network intercom frequency domain change module (7): receives PCM data and divides it into slices according to 320 sampling points; After processing, the first 128 FFT values ​​of the frequency domain slice data are sent to the sound domain alignment module (8), and then the 512 FFT values ​​of the frequency domain of each slice are sent to the stable noise evaluation module (9) and the sound domain enhancement merging module (10); A sound frequency domain alignment module (8): used for calculating an alignment deviation value, and the sound frequency domain alignment module (8) sends the alignment deviation value to the sound frequency domain enhancement and merging module (10); Stable noise evaluation module (9): used to analyze the background noise of the network intercom, calculate the average value of high-frequency noise and the average value of medium- and low-frequency noise and send them to the sound field enhancement and merging module (10); The audio and video domain enhancement and merging module (10) receives the data intercom error rate of the digital intercom receiving and decoding module (4), the network intercom packet loss rate of the network intercom receiving and decoding module (5), 128 FFT values ​​of the frequency domain fragmentation data of the digital intercom frequency domain change module (6), 512 FFT values ​​of the network intercom frequency domain fragmentation data of the network intercom frequency domain change module (7), the alignment deviation value of the audio and video domain alignment module (8), and the average value of the high frequency noise and the average value of the medium and low frequency noise of the stable noise evaluation module (9), and selects or merges the frequency domain data of the digital intercom and the network intercom according to the above received data; The sound time domain restoration and change module (11) is used to receive the frequency domain slice data from the sound frequency domain enhancement and merging module (10), restore the frequency domain sound into 32K PCM data, and send the 32K PCM data to the speaker module (12); The loudspeaker module (12) is used to receive the PCM data from the sound time domain restoration and change module (11), perform digital to analog conversion, and play the sound.

2. A dual-mode enhanced intercom method, using a dual-mode enhanced intercom system as claimed in claim 1, characterized in that: The method comprises the following steps: S1: The sound pickup module (1) collects sound and obtains 32K audio PCM data. The sound pickup module (1) sends the data to the digital intercom coding sending module (2) and the network intercom coding sending module (3) respectively; S2: The digital intercom coding and sending module (2) receives the 32K PCM data from the sound receiving module (1), processes the PCM data with a 4K low-pass filter to obtain PCM data with a frequency less than 4K, performs audio resampling to obtain 8K PCM data, and uses a vocoder to encode the 8K PCM data into an audio compression stream. The encoded audio compression stream has built-in redundant information for error correction, which is converted into a radio frequency signal and sent to the digital intercom receiving and decoding module (4); S3: The network intercom encoding and sending module (3) receives the 32K PCM data of the sound receiving module (1), performs opus audio encoding to obtain an audio compression stream, performs RTP packetization, obtains a network RTP audio packet, wherein the seq number of the network RTP audio packet increases continuously, and the network intercom encoding and sending module (3) sends the network RTP audio packet to the network intercom receiving and decoding module (5) via the mobile network; S4: The digital intercom receiving and decoding module (4) receives the radio frequency signal of the digital intercom encoding and sending module (2), converts it into a coded audio compression stream, and obtains 8K PCM data through decoding by the vocoder; the digital intercom receiving and decoding module (4) sends the 8K PCM data to the digital intercom frequency domain change module (6); the digital intercom receiving and decoding module (4) receives the coded audio compression stream and restores it according to the redundant information of error correction, and at the same time counts the error rate of the coded audio compression stream, and the digital intercom receiving and decoding module (4) sends the data intercom error rate to the voice frequency domain enhancement and merging module (10); S5: the network intercom receiving and decoding module (5) receives the network RTP audio packet from the network intercom encoding and sending module (3); obtains the packet loss rate, and the network intercom receiving and decoding module (5) sends the network intercom packet loss rate to the stable noise evaluation module (9); the network intercom receiving and decoding module (5) performs opus audio decoding on the network RTP audio packet to obtain a 32K audio stream; the network intercom receiving and decoding module (5) sends the 32K PCM data to the network intercom frequency domain change module (7); S6: Digital intercom frequency domain change module (6) The digital intercom converts the sound time domain to the frequency domain, receives PCM data and slices it according to 80 sampling points, and after processing, sends 128 FFT values ​​of the frequency domain slice data to the sound frequency domain alignment module (8) and the sound frequency domain enhancement and merging module (10); S7: The network intercom frequency domain change module (7) transforms the sound time domain into the frequency domain, receives the PCM data and slices it according to 320 sampling points; after processing, the first 128 FFT values ​​of the frequency domain slice data are taken and sent to the sound and frequency domain alignment module (8), and then the 512 FFT values ​​of the frequency domain of each slice are sent to the stable noise evaluation module (9) and the sound and frequency domain enhancement merging module (10); S8: the sound frequency domain alignment module (8) calculates the alignment deviation value, and the sound frequency domain alignment module (8) sends the alignment deviation value to the sound frequency domain enhancement and merging module (10); S9: The stable noise evaluation module (9) analyzes the background noise of the network intercom, calculates the average value of the high-frequency noise and the average value of the medium- and low-frequency noise, and sends the average value of the high-frequency noise and the average value of the medium- and low-frequency noise to the sound frequency domain enhancement merging module (10); S10: The audio frequency domain enhancement and merging module (10) receives the data sent by each module, and then merges the frequency domain data of the digital intercom and the network intercom according to the different situations of the data intercom error rate and the network intercom packet loss rate. Specifically: 10.1 The data intercom error rate of the digital intercom receiving and decoding module (4) received by the audio frequency domain enhancement and merging module (10); 10.2 The audio frequency domain enhancement merging module (10) receives the network intercom packet loss rate from the network intercom receiving decoding module (5); 10.3 The audio frequency domain enhancement merging module (10) receives 128 FFT values ​​of the frequency domain slice data from the digital intercom frequency domain change module (6); 10.4 The audio frequency domain enhancement merging module (10) receives 512 FFT values ​​of the network intercom frequency domain fragmentation data from the network intercom frequency domain change module (7); 10.5 The sound frequency domain enhancement merging module (10) receives the alignment deviation value of the sound frequency domain alignment module (8); 10.6 The sound frequency domain enhancement merging module (10) receives the average value of the high frequency noise and the average value of the medium and low frequency noise from the stable noise evaluation module (9); 10.7 When the data intercom error rate is less than 5% and the network intercom packet loss rate is greater than 5%, the sound frequency domain enhancement merging module (10) selects 128 FFT values ​​of the frequency domain slice data and converts them into 512 FFT values ​​and sends them to the sound time domain restoration change module (11); 10.8 The method of converting 128 FFT values ​​of the frequency domain slice data of the sound domain enhancement merging module (10) into 512 FFT values, padding 384 FFT values, and padding the FFT values ​​to 0, and finally obtaining 512 FFT values ​​of the frequency domain slice data; 10.9 When the data intercom error rate is greater than 5% and the network intercom packet loss rate is less than 5%, the sound frequency domain enhancement merging module (10) selects 512 FFT values ​​of the network intercom frequency domain fragmentation data and sends them to the sound time domain restoration change module (11); 10.10 When the data intercom error rate is greater than 5% and the network intercom packet loss rate is greater than 5%, the data intercom error rate and the network intercom packet loss rate are judged. If the data intercom error rate is less than the network intercom packet loss rate, the sound frequency domain enhancement merging module (10) selects 128 FFT values ​​of the frequency domain fragmentation data to be converted into 512 FFT values ​​and sends them to the sound time domain restoration change module (11). If the data intercom error rate is greater than the network intercom packet loss rate, the sound frequency domain enhancement merging module (10) selects 512 FFT values ​​of the network intercom frequency domain fragmentation data to be sent to the sound time domain restoration change module (11); 10.11 When the data intercom error rate is less than 5% and the network intercom packet loss rate is less than 5%, the sound merging process is carried out; 10.12 The sound frequency domain enhancement merging module (10) calculates a mid-low frequency merging coefficient and a high frequency merging coefficient according to a preset mid-low frequency noise benchmark and a preset high frequency noise benchmark, wherein the mid-low frequency merging coefficient is the average of the mid-low frequency noise benchmark / mid-low frequency noise, and the high frequency merging coefficient is the average of the high frequency noise benchmark / high frequency noise; 10.13 The audio frequency domain enhancement merging module (10) adds the mid-low frequency merging coefficient to each FFT value of the 128 FFT values ​​of the frequency domain slice data of the digital intercom and multiplies it by the corresponding FFT value of the frequency domain slice data of the network intercom, and finally obtains the first 128 FFT values ​​of the frequency domain slice data of the merged audio. While ensuring that the listening experience of the digital intercom remains unchanged, a small amount of background noise is added according to the mid-low frequency merging coefficient; 10.14 The audio frequency domain enhancement merging module (10) multiplies the last 384 FFT values ​​of the frequency domain slice data of the network intercom by the high frequency merging coefficient respectively, and uses the first 384 FFT values ​​of the frequency domain slice data of the merged audio as the first 384 FFT values, and restores a small amount of high frequency sound according to the high frequency merging coefficient on the basis of the digital intercom to improve the sound; 10.15 The sound frequency domain enhancement merging module (10) finally obtains 512 FFT values ​​of the frequency domain slice data of the merged audio and sends them to the sound time domain restoration change module (11); S11: The sound time domain restoration and change module (11) receives 512 FFT values ​​of the frequency domain slice data of the sound frequency domain enhancement and merging module (10), and uses 512-bit FFT inverse transformation to restore the frequency domain sound into 32K PCM data; the sound time domain restoration and change module (11) sends the 32K PCM data to the speaker module (12); S12: The speaker module (12) receives the 32K PCM data from the sound time domain restoration and change module (11), performs digital to analog conversion, and plays the sound.

3. The dual-mode enhanced intercom method according to claim 2, characterized in that: The steps for calculating the packet loss rate in step S5 are as follows: The network intercom receiving and decoding module (5) determines whether packet loss occurs according to whether the seq sequence number of the received network RTP audio packet is continuous, counts the number of packet losses per second, and divides it by the number of packets per second to obtain the packet loss rate.

4. A dual-mode enhanced intercom method according to claim 2 or 3, characterized in that: The step S6 comprises the following steps: 6.1 The digital intercom frequency domain change module (6) receives the 8K PCM data from the digital intercom receiving and decoding module (4) and divides it into slices according to 80 sampling points; 6.2 Network intercom frequency domain change module (7) takes 64 sampling point data of the previous slice and splices the 64 sampling point data after the current sampling point to obtain 128 sampling point data slices, uses 128 length FFT fast Fourier transform to obtain 128 FFT values ​​of the frequency domain number of the current slice, and takes the sampling points of the slice in an interleaved manner to make the FFT change of two adjacent slices smoother; the current FFT resolution is 8000 / 128, which is 62.5; 6.3 The digital intercom frequency domain change module (6) sends the 128 FFT values ​​of the frequency domain slice data to the audio domain alignment module (8) and the audio domain enhancement merging module (10).

5. The dual-mode enhanced intercom method according to claim 4, characterized in that: The step S7 comprises the following steps: 7.1 The network intercom frequency domain change module (7) receives the 32K PCM data from the network intercom receiving and decoding module (5) and divides it into slices according to 320 sampling points; 7.2 The network intercom frequency domain change module (7) takes the 256 sampling point data of the previous slice and splices the 256 sampling point data after the current sampling point to obtain 512 sampling point data slices, and uses the 512-length FFT fast Fourier transform to obtain 512 FFT values ​​of the frequency domain number of the current slice. The current FFT resolution is 32000 / 512, which is 62.

5. All network intercom frequency domain change modules (7) and digital intercom frequency domain change modules (6) use the same allocation rate and different bit lengths of FFT changes, and use a cross-mode to take the sampling points of the slices, so that the FFT changes of two adjacent slices are smoother; 7.3 The network intercom frequency domain change module (7) obtains the first 128 FFT values ​​of the frequency domain slice data and sends them, and then performs FFT Fourier transform to obtain the frequency domain data and sends them to the audio frequency domain alignment module (8); 7.4 The audio frequency domain alignment module (8) and the network intercom frequency domain change module (7) send the 512 FFT values ​​of each slice in the frequency domain to the stable noise evaluation module (9) and the audio frequency domain enhancement merging module (10).

6. The dual-mode enhanced intercom method according to claim 5, characterized in that: The step S8 comprises the following steps: 8.1 The audio frequency domain alignment module (8) receives 128 FFT values ​​of the frequency domain slice data from the digital intercom frequency domain change module (6); 8.2 The audio frequency domain alignment module (8) receives the first 128 FFT values ​​of the frequency domain slicing data of the network intercom frequency domain change module (7); because the network intercom frequency domain change module (7) and the digital intercom frequency domain change module (6) use the same allocation rate FFT, if the corresponding FFT values ​​of the frequency domain slicing data are the same frequency capacity values; 8.3 Compared with network intercom, digital intercom has shorter voice delay because of its shorter path. The audio frequency domain alignment module (8) is used to align the frequency domain fragmentation data of network intercom with the frequency domain fragmentation data of digital intercom. 8.4 The audio frequency domain alignment module (8) calculates the variance of the current network intercom frequency domain slice data and the received preset N blocks of digital intercom frequency domain slice data, that is, the 128 FFT values ​​of the network intercom frequency domain slice data and the 128 FFT values ​​of the digital intercom frequency domain slice data are subtracted from each other and the squares are taken, and the square values ​​are accumulated as the variance value; 8.5 The audio frequency domain alignment module (8) finds the minimum variance value among the preset N blocks of digital intercom frequency domain slice data, takes the offset position as the alignment deviation value, and performs variance calculation on the subsequent network intercom frequency domain slice data with the digital intercom frequency domain slice data around the alignment deviation value, and takes the offset position of the digital intercom frequency domain slice data with the smaller variance value as the new alignment deviation value; 8.6 The audio frequency domain alignment module (8) If the alignment deviation value calculated by the audio frequency domain alignment module (8) for the K consecutive preset network intercom frequency domain slices is substantially unchanged, the audio frequency domain alignment module (8) considers that a stable alignment deviation value has been found; 8.7 The sound frequency domain alignment module (8) sends the alignment deviation value to the sound frequency domain enhancement merging module (10).

7. The dual-mode enhanced intercom method according to claim 6, characterized in that: The step S9 comprises the following steps: 9.1 The stable noise evaluation module (9) receives 512 FFT values ​​in the frequency domain of each slice; 9.2 Stable noise evaluation module (9) accumulates the 512 FFT values ​​of the frequency domain of the most recent 20 slices respectively to obtain 512 FFT accumulated values; 9.3 The stable noise evaluation module (9) judges the 512 FFT accumulated values ​​and the preset energy threshold respectively. If the energy is higher than the quantile value, the FFT accumulated value is considered to be noise; if the energy is higher than the quantile value, the FFT accumulated value is considered to be speech; the FFT number is multiplied by the FFT resolution 62.5 to obtain the corresponding frequency of the FFT; 9.4 Stable noise evaluation module (9) accumulates and averages the noise FFT cumulative values ​​greater than the frequency 4K, and takes the average value as the mean value of the high-frequency noise; 9.5 Stable noise evaluation module (9) accumulates and averages the noise FFT cumulative values ​​less than or equal to the frequency 4K, and takes the average value as the mean value of the medium and low frequency noise; 9.6 The stable noise evaluation module (9) sends the average value of the high-frequency noise and the average value of the medium- and low-frequency noise to the sound field enhancement merging module (10).

Images