Interactive dialogue method and device for drone voice broadcasting, flying drone, electronic device and storage medium

Abstract

The present invention relates to an interactive dialogue method for drone voice broadcasting, comprising: decoding and preprocessing according to control information to be played, obtaining playable audio, transmitting the playable audio to a sound amplification device, and obtaining the current environmental sound signal; performing channel separation, echo cancellation, and human voice detection on the current environmental sound signal in sequence, identifying a reference noise channel and a human voice channel, and constructing a noise filter based on the reference noise channel, filtering the human voice channel through the noise filter, and obtaining a denoised sound signal; performing amplitude modulation coding and secondary coding on the denoised sound signal, and returning the coded sound signal. The interactive dialogue method for drone voice broadcasting described in the present invention enables the drone voice broadcasting system to achieve high-quality two-way conversation in a high-noise environment through precise channel separation and noise filtering, overcoming the problems of poor noise reduction effect, low audio quality, and low transmission efficiency in the prior art.

Description

Technical Field

[0001] The present invention relates to unmanned aerial vehicle voice broadcasting and processing technology, and in particular to an interactive dialogue method and device for unmanned aerial vehicle voice broadcasting, a flying unmanned aerial vehicle, an electronic device and a computer storage medium. Background Art

[0002] Existing drone voice broadcast systems usually use a one-way output method without considering interactive functions. However, in emergency rescue applications, this one-way output method will result in the victims being unable to communicate effectively or instantly with rescuers, thus affecting the rescuers' understanding of the on-site situation and hindering the advancement of rescue work.

[0003] Based on this, general two-way communication technologies are used in walkie-talkies and mobile phone communications, which rely on high-bandwidth, low-latency mobile communication networks to achieve stable two-way voice transmission. However, when these technologies are directly transplanted to drones, they usually face complex noise interference, especially the high-frequency noise of drone blades, wind noise and broadcast echo generated when drones move quickly, which makes the human voice collected by the drone microphone easily covered by noise.

[0004] In order to solve the noise interference, the existing technology usually adopts microphone array and sound wave positioning and other related technologies. The multi-microphone array is used to collect sound signals from different directions, so as to enhance the microphone pickup in the direction of human voice through beamforming technology and suppress the noise in other directions. At the same time, the noise of microphones in other directions is extracted through machine learning or other methods, and anti-noise is generated through the principle of anti-phase interference of sound waves, and the sound in the direction of human voice is denoised through anti-noise. Among them, the multi-microphone array can identify human voices in different directions through time difference and sound pressure amplitude. The time difference method relies on the time difference of sound reaching different microphones to determine the source direction of the sound, while the sound pressure amplitude law determines the direction of the sound by measuring the sound pressure intensity of the sound at different microphones. However, the existing technology has limited noise reduction effect under the influence of high-frequency noise interference and complex environment generated by drones during flight, and remote or weak human voice signals are easily drowned by noise, resulting in delays in the best rescue time. Summary of the invention

[0005] Based on this, the object of the present invention is to provide an interactive dialogue method for drone voice broadcasting.

[0006] An interactive dialogue method for drone voice broadcasting, comprising the following steps:

[0007] S1: Decoding and preprocessing the control information to be played to obtain playable audio, transmitting the playable audio to a sound amplification device, and obtaining a current ambient sound signal; wherein the control information to be played is control information output by the mobile terminal;

[0008] S2: Perform channel separation, echo cancellation and human voice detection on the current environmental sound signal in sequence, identify the reference noise channel and the human voice channel, and construct a noise filter based on the reference noise channel. Use the noise filter to filter out the noise of the human voice channel to obtain a denoised sound signal;

[0009] S3: Perform amplitude modulation coding and secondary coding on the denoised sound signal, and transmit the coded sound signal back.

[0010] Compared with the prior art, the interactive dialogue method of drone voice broadcasting described in the present invention distinguishes reference noise and human voice channels through channel separation, echo cancellation and human voice detection, and constructs a noise filter based on the reference noise channel, thereby accurately filtering out noise and ensuring voice clarity in high-frequency and complex noise environments. At the same time, the present invention encodes the sound signal through amplitude modulation to avoid sound signal distortion, and secondary encoding can further compress the data volume of the sound signal, thereby improving transmission efficiency and real-time performance, thereby ensuring efficient transmission under limited bandwidth.

[0011] Furthermore, the step S2 comprises the following steps:

[0012] S21: performing channel separation on the current ambient sound signal to obtain a number of microphone signals;

[0013] S22: performing echo cancellation and voice detection on the plurality of microphone signals to obtain a reference noise channel and a voice channel;

[0014] It includes the following sub-steps:

[0015] S221: using the playable audio as a reference signal, and performing echo cancellation on a plurality of microphone signals according to the reference signal to obtain a plurality of microphone signals to be detected;

[0016] S222: performing frame division, windowing, and fast Fourier transformation on a plurality of microphone signals to be detected to obtain frequency spectrum information of the microphone signals;

[0017] S223: extracting features from the spectrum information of the microphone signal through a Mel filter bank to obtain microphone feature parameters;

[0018] S224: performing voice recognition on microphone feature parameters through a trained binary classification model to obtain a number of voice features;

[0019] S225: Calculate the maximum sound pressure amplitude among the plurality of human voice features, and use the corresponding microphone signal as a human voice channel, and the remaining microphone signals as reference noise channels;

[0020] S23: Perform time-frequency analysis on the reference noise channel to obtain noise characteristic parameters;

[0021] It includes the following sub-steps:

[0022] S231: performing frame division, windowing and Fourier transformation on the reference noise channel to obtain spectrum information of the noise;

[0023] S232: performing energy calculation on the frequency spectrum information of the noise to obtain an energy spectrum of the noise;

[0024] S233: Obtain the maximum amplitude value in the energy spectrum of the plurality of noises as a frequency point, obtain the current plurality of noise frequency points, and use them as noise characteristic parameters;

[0025] S24: Perform filtering construction according to noise characteristic parameters to obtain a noise filter;

[0026] S25: Filter the human voice channel for noise through a noise filter to obtain a denoised sound signal, and use the denoised sound signal as the current sound signal to execute step S3.

[0027] The present invention extracts Mel-frequency cepstral coefficients through a Mel filter bank to characterize the main frequency characteristics of human voice, thereby realizing accurate recognition of human voice characteristics through a binary classification model.

[0028] Furthermore, the filtering construction comprises the following steps:

[0029] A second-order notch filter is constructed for each frequency point in the noise characteristic parameter to obtain a plurality of second-order notch filters;

[0030] And several second-order notch filters are cascaded to obtain a noise filter.

[0031] Since the second-order notch filter has a high attenuation characteristic at a specific frequency point, a notch filter is constructed by selecting several frequency points with the largest amplitude in a frequency spectrum of one cycle to accurately attenuate the main components of the noise. At the same time, by cascading the second-order notch filter, multiple noise frequency points can be effectively filtered out.

[0032] Furthermore, the control information further includes text-generated control information, and the control information to be played further includes text-generated control information built into a drone, and the steps for acquiring the text-generated control information are as follows:

[0033] SA1: By extracting the scene image of the current environment;

[0034] SA2: Extract features from the scene image of the environment through image processing algorithms to obtain scene features;

[0035] SA3: Use the pre-built detection model to perform scene recognition on scene features and obtain the current scene keywords;

[0036] SA4: Index a preset scene text library according to the current scene keyword to obtain scene text data;

[0037] SA5: Convert scene text data through a text-to-speech model to obtain broadcast audio;

[0038] SA6: Encode the broadcast audio, obtain the text-generated control information, and use the text-generated control information as the current control information.

[0039] Among them, the control information generated by text is used in drone voice broadcasts to provide more timely rescue guidance, disaster warnings and other guidance information. In addition, if the drone is in a weak signal situation, it can autonomously analyze the scene and conduct simple guidance broadcasts to provide more help for rescue in harsh environments. At the same time, the text-to-speech model can also support multiple languages ​​(or dialects), and can broadcast in multiple languages ​​according to needs, thereby providing equally efficient rescue guidance and disaster warnings in different language environments.

[0040] An interactive dialogue device for drone voice broadcasting includes a broadcast audio processing unit, an audio noise reduction processing unit and a sound pickup audio processing unit;

[0041] The broadcast audio processing unit is used to decode and preprocess the control information to be played, obtain playable audio, transmit the playable audio to a sound amplification device, and obtain the current ambient sound signal; wherein the control information to be played is the control information output by the mobile terminal;

[0042] The audio noise reduction processing unit is used to perform channel separation, echo cancellation and human voice detection on the current environmental sound signal in sequence, identify a reference noise channel and a human voice channel, and construct a noise filter based on the reference noise channel, and filter the noise of the human voice channel through the noise filter to obtain a denoised sound signal;

[0043] The sound pickup audio processing unit is used to perform amplitude modulation coding and secondary coding on the denoised sound signal, and to transmit the coded sound signal back.

[0044] Further, the audio noise reduction processing unit includes a channel separation module, a human voice detection module, a noise extraction module, a noise filter construction module and a noise filtering module;

[0045] The channel separation module is used to perform channel separation on the current ambient sound signal to obtain a number of microphone signals;

[0046] The human voice detection module is used to perform echo cancellation and human voice detection on a plurality of microphone signals to obtain a reference noise channel and a human voice channel; specifically, it includes the following sub-steps:

[0047] The playable audio is used as a reference signal, and echo cancellation is performed on a plurality of microphone signals according to the reference signal to obtain a plurality of microphone signals to be detected;

[0048] Performing frame division, windowing and fast Fourier transformation on a number of microphone signals to be detected to obtain spectrum information of the microphone signals;

[0049] The spectral information of the microphone signal is extracted by using a Mel filter bank to obtain microphone characteristic parameters;

[0050] Using a trained binary classification model to perform voice recognition on microphone feature parameters, several voice features are obtained;

[0051] Calculate the maximum sound pressure amplitude among several human voice features, and use the corresponding microphone signal as the human voice channel, and the remaining microphone signals as reference noise channels;

[0052] The noise extraction module is used to perform time-frequency analysis on the reference noise channel to obtain noise characteristic parameters;

[0053] It includes the following sub-steps:

[0054] The reference noise channel is framed, windowed and Fourier transformed to obtain the spectrum information of the noise;

[0055] Perform energy calculation on the frequency spectrum information of the noise to obtain the energy spectrum of the noise;

[0056] The maximum value of the amplitude in the energy spectrum of several noises is obtained as the frequency point, and the current several noise frequency points are obtained as the noise characteristic parameters;

[0057] The noise filter construction module is used to perform filtering construction according to noise characteristic parameters to obtain a noise filter;

[0058] The filter construction includes the following steps:

[0059] A second-order notch filter is constructed for each frequency point in the noise characteristic parameter to obtain a plurality of second-order notch filters;

[0060] And several second-order notch filters are cascaded to obtain a noise filter;

[0061] The noise filtering module is used to filter the noise of the human voice channel through a noise filter to obtain a denoised sound signal, and use the denoised sound signal as the current sound signal to call the sound pickup audio processing unit.

[0062] Furthermore, the control information further includes text-generated control information, and the control information to be played further includes text-generated control information built into a drone, and the steps for acquiring the text-generated control information are as follows:

[0063] SA1: By extracting the scene image of the current environment;

[0064] SA2: Extract features from the scene image of the environment through image processing algorithms to obtain scene features;

[0065] SA3: Use the pre-built detection model to perform scene recognition on scene features and obtain the current scene keywords;

[0066] SA4: Index a preset scene text library according to the current scene keyword to obtain scene text data;

[0067] SA5: Convert scene text data through a text-to-speech model to obtain broadcast audio;

[0068] SA6: Encode the broadcast audio, obtain the text-generated control information, and use the text-generated control information as the current control information.

[0069] A flying drone, comprising a sound amplification device, three microphones and the interactive dialogue device for drone voice broadcasting as described above;

[0070] The amplifying device is used to play the playable audio output by the interactive dialogue device of the drone voice broadcast;

[0071] The three microphones are respectively arranged on the upper end and left and right sides of the flying drone and are arranged in a triangle, and are used to obtain environmental sound signals and input the environmental sound signals into the interactive dialogue device of the drone voice broadcast described above.

[0072] For better understanding and implementation, the present invention is described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Figure 1 A simple schematic diagram of the equipment model for flying drones;

[0074] Figure 2 A schematic diagram of an interactive dialogue device for drone voice broadcasting according to the present invention;

[0075] Figure 3This is a flow chart of the interactive dialogue method for drone voice broadcasting according to the present invention;

[0076] Figure 4 Schematic diagram of the amplitude-frequency curve of the notch filter. DETAILED DESCRIPTION

[0077] In order to solve the problem of limited noise reduction effect of the prior art when conducting two-way calls in high-frequency noise interference and complex environments, the present invention performs channel separation on the acquired sound signal to obtain a number of microphone signals. A reference noise channel and a human voice channel are identified from the multiple microphone signals, and a noise filter is constructed based on the reference noise channel. The human voice channel is filtered out by the noise filter to achieve sound pickup denoising in a high-frequency and complex noise environment, solving the problem of poor noise reduction effect of the prior art when conducting two-way calls in high-frequency noise interference and complex environments.

[0078] Based on the above design, the present invention proposes an interactive dialogue method for drone voice broadcasting, and based on this method, proposes an interactive dialogue device for drone voice broadcasting.

[0079] See also Figure 1 , Figure 1 A simplified schematic diagram of a flying drone equipment model.

[0080] The interactive dialogue device for drone voice broadcasting is installed on a flying drone, and the flying drone includes a sound amplification device and three microphones. The sound amplification device is used to play the playable audio output by the interactive dialogue device for drone voice broadcasting. The playable audio is obtained by the user shouting to the drone remote control or mobile phone APP, and the drone remote control and mobile phone APP are used to receive or output signals from the interactive dialogue device for drone voice broadcasting. The three microphones are respectively arranged at the upper end and left and right sides of the flying drone, and the three microphones are arranged in a triangle on the same plane, and are used to collect the environmental sound signal of the current flying drone, and input the environmental sound signal into the interactive dialogue device for drone voice broadcasting described in the present invention.

[0081] See also Figure 2 and Figure 3 , Figure 2 This is a schematic diagram of the interactive dialogue device for drone voice broadcasting according to the present invention. Figure 3 The figure is a flow chart of the interactive dialogue method for drone voice broadcasting according to the present invention.

[0082] The interactive dialogue device for drone voice broadcasting includes a broadcast audio processing unit 1, an audio noise reduction processing unit 2 and a sound pickup audio processing unit 3.

[0083] The broadcast audio processing unit 1 is used to execute step S1: decode and preprocess according to the control information, obtain playable audio, transmit the playable audio to the sound amplification device, and determine whether it is currently in monitoring mode: if yes, call the pickup audio processing unit 3; if not, call the audio noise reduction processing unit 2.

[0084] Specifically, the broadcast audio processing unit 1 includes a broadcast audio decoding module 11 , a broadcast audio initialization module 12 , an audio control module 13 and a monitoring mode determination module 14 .

[0085] The broadcast audio decoding module 11 is used to execute step S11: obtain control information to be played, and decode the control information to obtain audio information.

[0086] In one embodiment, the control information to be played is the control information output by the mobile terminal. The broadcast audio of the user shouting in real time through the remote control or mobile phone APP of the drone is encoded into an audio signal by the remote control or mobile phone APP of the drone, and the audio signal is the control information to be played.

[0087] The control information is transmitted via the drone's own link (the drone's own transmission channel) or the mobile network link (the mobile phone's transmission channel).

[0088] The decoding is to perform reverse processing on the control information to restore the original broadcast audio signal, thereby obtaining the audio information.

[0089] Through the communication link of the drone's built-in remote control or the communication link of the mobile phone APP, audio information can be data transmitted through different links, thereby improving the robustness of the drone's voice broadcast.

[0090] In addition, the control information to be played can be looped or delayed according to the user's settings, further improving the humanized operation of drone voice broadcasting. The present invention does not specifically limit its playback conditions.

[0091] In another embodiment, the control information to be played also includes control information generated by a built-in text of the drone, and the steps for obtaining the control information generated by the text are as follows:

[0092] SA1. Obtain a scene image of the current environment of the drone, and extract features from the scene image through an image processing algorithm to obtain scene features.

[0093] The image processing algorithm may be a convolutional neural network (CNN), including a convolutional layer, a pooling layer, and a fully connected layer, all of which are used to extract features from the scene image of the current environment of the drone. Since the deep learning algorithm for image feature extraction can be changed according to demand, the present invention does not specifically limit the selection of its image processing algorithm.

[0094] SA2: Use the pre-built detection model to perform scene recognition on scene features and obtain the current scene keywords.

[0095] The pre-built detection model can be selected as a target detection algorithm of the YOLO (You Only Look Once) series, and its structure includes feature fusion and a detection head. The feature fusion is used to fuse scene features to obtain fused feature parameters, and the detection head is used to identify the category to which the fused feature parameters belong to obtain the current scene keywords. The current scene keywords are the classifications in the training data set used in the training process of the pre-built detection model. Since the target detection algorithm can be selected according to demand, and the training data set and its classification are user-defined content, the present invention does not make specific limitations on this.

[0096] SA3. Index the current scene keywords in a preset scene text library to obtain scene text data.

[0097] The preset scene text library is a mapping library customized by the user according to needs, that is, one scene keyword corresponds to one or more sentences.

[0098] For example, if there are currently scene keywords: "high mountain", "rain", "steep slope" and "landslide".

[0099] The scene text library may contain the following contents:

[0100] "High Mountains": You are currently in a high mountain area and walking is difficult. Please try to avoid acting alone and keep the group close.

[0101] “Rain”: There is heavy rain in the current area and the road surface is slippery. Please be careful to avoid slipping and try to choose a stable path.

[0102] “Steep slope”: There is a steep slope in the current area. Please pay attention to your feet, try to avoid climbing, and choose a gentler path.

[0103] "Landslide": There is a risk of landslide in the current area. Please leave the landslide area quickly and seek shelter in a safe high place.

[0104] The combined scene text data is:

[0105] "Attention: We are currently in a high mountain area, and it is difficult to walk. Please try to avoid acting alone and keep the group close. There is a lot of rain in the current area, and the road is slippery. Please pay attention to anti-slip and try to choose a stable path."

[0106] "Attention: There is a steep slope in the current area. Please pay attention to your feet, try to avoid climbing, and choose a gentler path. There is a risk of landslide in the current area. Please leave the landslide area quickly and find a safe high place to avoid danger."

[0107] Since the user can define the combination rules of the scene texts according to the requirements, the present invention will not specifically limit its application scenarios here.

[0108] SA4. Convert scene text data into broadcast audio through a text-to-speech model.

[0109] The text-to-speech model generally includes a text analysis module, an acoustic model module, and a speech synthesis module. The text analysis module is used to perform word segmentation, part-of-speech tagging, and grammatical analysis on the input text (scene text data) to extract the speech feature information of the text. The speech feature information may include rhythm, stress, and intonation.

[0110] The acoustic model module is used to map the speech feature information to specific acoustic features to obtain acoustic features. The acoustic features may include frequency, duration, amplitude, etc.

[0111] The speech synthesis module is used to synthesize and convert the acoustic features into waveform data to obtain a speech signal, namely, broadcast audio.

[0112] Since users can select different text-to-speech models according to their needs, such as DeepVoice or Tacotron TTS models, the present invention is not specifically limited here.

[0113] SA5. Encode the broadcast audio to obtain control information for text generation.

[0114] Among them, the control information generated by text is used in drone voice broadcasts to provide more timely rescue guidance, disaster warnings and other guidance information. In addition, if the drone is in a weak signal situation, it can autonomously analyze the scene and conduct simple guidance broadcasts to provide more help for rescue in harsh environments. At the same time, the TTS model can also support multiple languages ​​(or dialects) and can broadcast in multiple languages ​​according to needs, thereby providing equally efficient rescue guidance and disaster warnings in different language environments.

[0115] The broadcast audio initialization module 12 is used to execute step S12: frequency cutting, gain and compression of the audio information to obtain playable audio.

[0116] Specifically, the method includes the following sub-steps:

[0117] S121: frequency-slicing the audio data through a filter to obtain human voice audio information.

[0118] The filter is used to filter unnecessary frequency bands or noise frequency bands in the audio data to retain the required audio features, such as removing low-frequency background noise and high-frequency electromagnetic interference by frequency cutting to retain the intermediate-frequency speech part.

[0119] S122: Gain the human voice audio information to obtain enhanced human voice audio information.

[0120] The gain is performed by an amplifier, specifically, by an electronic amplification circuit to amplify the voltage, current or power of the audio signal.

[0121] S123: Compressing the enhanced human voice audio information to obtain playable audio.

[0122] The compression includes a compression part and a limiting part. The compression part compresses the enhanced human voice audio information to a dynamic range, specifically adjusting the signal strength through a set threshold to ensure the stability of the output volume and avoid large fluctuations in the volume.

[0123] The limiting part is used to limit the maximum amplitude of the compressed enhanced human voice audio data, thereby preventing the audio signal from exceeding the set maximum value, thereby avoiding audio distortion and overload.

[0124] The present invention ensures the quality and stability of broadcast audio through frequency cutting, gain and compression, thereby ensuring that the broadcast audio can be effectively transmitted to on-site personnel.

[0125] The audio control module 13 is used to execute step S13: control the sound amplification device to play the playable audio, and control a plurality of microphones to pick up the sound to obtain the picked-up audio signal.

[0126] The monitoring mode determination module 14 is used to execute step S14: determine whether the current state is in monitoring mode, if yes, output the picked-up audio signal as the current sound signal to the picked-up audio processing unit 3; if no, execute the next step.

[0127] Specifically, the monitoring mode is activated by the user to monitor whether the current broadcast is correctly broadcast.

[0128] Since the drone voice broadcast process is generally not within the user's audible range, it is impossible to ensure that the current sound amplification equipment is working properly, especially in harsh environments such as blizzards, storms, hail, rain and snow, sandstorms and electromagnetic interference environments. These environmental factors may cause the sound amplification equipment to be damaged or work abnormally, affecting the effective transmission of the broadcast. Accordingly, by turning on the monitoring mode, users can monitor the quality and accuracy of the broadcast audio in real time, discover and deal with possible abnormal situations in a timely manner, and ensure that voice communication or broadcasting in emergency rescue and other tasks is unimpeded.

[0129] The audio noise reduction processing unit 2 is used to execute step S2: perform channel separation, echo elimination and human voice detection on the current environmental sound signal in sequence, identify the reference noise channel and the human voice channel, and construct a noise filter based on the reference noise channel, and filter the noise of the human voice channel through the noise filter to obtain a denoised sound signal.

[0130] Specifically, the audio noise reduction processing unit 2 includes a channel separation module 21 , a human voice detection module 22 , a noise extraction module 23 , a noise filter construction module 24 and a noise filtering module 25 .

[0131] The channel separation module 21 is used to execute step S21: performing channel separation on the current ambient sound signal to obtain a plurality of microphone signals.

[0132] Specifically, the channel separation is to separate the sound signals received by different microphones to obtain a plurality of microphone signals.

[0133] By separating the sound signals from different microphones, more sophisticated processing can be performed, such as determining the source of the sound direction or noise reduction filtering.

[0134] The human voice detection module 22 is used to execute step S22: perform echo cancellation and human voice detection on a plurality of microphone signals to obtain a reference noise channel and a human voice channel.

[0135] Specifically, the method includes the following sub-steps:

[0136] S221: Using the monitored sound signal or the playable audio as a reference signal, and performing echo cancellation on a plurality of microphone signals according to the reference signal to obtain a plurality of microphone signals to be detected.

[0137] The echo cancellation may select different filtering algorithms according to user requirements, such as an adaptive filtering algorithm of a least mean squares (LMS) filter or a delay and subtraction method, which is not specifically limited in the present invention.

[0138] S222: performing frame division, windowing and fast Fourier transformation on a plurality of microphone signals to be detected to obtain frequency spectrum information of the microphone signals.

[0139] Wherein, the framing is to divide the microphone signal to be detected according to a certain frame length, obtain the framed microphone signal, and obtain multiple small segments of processable microphone signals through framing. The windowing is to multiply the framed microphone signal of each frame by a window function (such as a Hening window or a Hamming window) to obtain a windowed microphone signal to reduce spectrum leakage and improve the accuracy of spectrum estimation. Then, the windowed microphone signal is subjected to a Fast Fourier Transform (FFT) to obtain the spectrum information of the microphone signal.

[0140] S223: Extracting features of the frequency spectrum information of the microphone signal through a Mel filter bank to obtain microphone feature parameters.

[0141] The Mel filter bank calculates the energy of the spectral information of the microphone signal through a number of bandpass filters with different center frequencies; then, the energy is logarithmically transformed and discrete cosine transformed (DCT) to obtain Mel frequency cepstral coefficients (MFCC) to represent the characteristic parameters of the microphone.

[0142] The Mel-frequency cepstral coefficient is used to characterize the main frequency characteristics of human voice and is widely used in speech recognition and human voice detection.

[0143] S224: Performing voice recognition on microphone feature parameters through a trained binary classification model to obtain a number of voice features.

[0144] The binary classification model is used to classify and identify the characteristic parameters of the microphone, and to distinguish the features belonging to the human voice from all the microphone characteristic parameters, so as to obtain a number of human voice features.

[0145] Among them, the binary classification model can be selected as a support vector machine, a decision tree or a neural network, and the Mel-frequency cepstral coefficients with labels (human voice and non-human voice) are used as training data to train the binary classification model, so that the binary classification model learns the characteristics of human voice and non-human voice, and finally obtains a trained binary classification model. Since the binary classification model can be replaced according to user needs, the present invention does not make specific limitations here.

[0146] S225: Calculate the maximum sound pressure amplitude among the plurality of human voice features, and use the corresponding microphone signal as a human voice channel, and the remaining microphone signals as reference noise channels.

[0147] If there is no human voice feature, any microphone signal is used as the human voice channel, and the remaining microphone signals are used as reference noise channels to ensure that noise processing can be effectively performed even in the absence of human voice to prevent users from being disturbed by noise.

[0148] The noise extraction module 23 is used to execute step S23: perform time-frequency analysis on the reference noise channel to obtain noise characteristic parameters.

[0149] Specifically, the method includes the following sub-steps:

[0150] S231: Frame, window and Fourier transform the reference noise channel to obtain frequency spectrum information of the noise.

[0151] S232: Perform energy calculation on the frequency spectrum information of the noise to obtain an energy spectrum of the noise.

[0152] S233: Obtain the maximum amplitude value in the energy spectrum of the plurality of noises as a frequency point, obtain the current plurality of noise frequency points, and use them as noise characteristic parameters.

[0153] In this embodiment, the maximum value of the amplitude in the energy spectrum of the noise is selected as the frequency point three, and the calculation amount of constructing the filter with three frequency points is moderate, so that the filter calculation can achieve a balance between real-time performance and filtering effect.

[0154] The noise filter construction module 24 is used to execute step S24: perform filtering construction according to noise characteristic parameters to obtain a noise filter.

[0155] Specifically, a second-order notch filter is constructed for each frequency point in the noise characteristic parameter to obtain a plurality of second-order notch filters, and the plurality of second-order notch filters are cascaded to obtain a noise filter.

[0156] Since the second-order notch filter has the characteristic of high attenuation at specific frequency points, by selecting several frequency points with the largest amplitude in a period of spectrum to construct a notch filter, the main components of the noise can be accurately attenuated. At the same time, by cascading the second-order notch filter, multiple noise frequency points can be effectively filtered out. For more information about frequency points, please refer to Figure 4 , Figure 4 Schematic diagram of the amplitude-frequency curve of the notch filter.

[0157] The noise filtering module 25 is used to execute step S25: filtering the human voice channel for noise through a noise filter to obtain a denoised sound signal, and outputting the denoised sound signal as the current sound signal to the sound pickup audio processing unit 3.

[0158] The sound pickup audio processing unit 3 is used to execute step S3: perform amplitude modulation encoding and secondary encoding on the current sound signal, and return the encoded sound signal.

[0159] The picked-up sound audio processing unit 3 includes a picked-up sound audio compression module 31 , a picked-up sound audio encoding module 32 and a picked-up sound audio playing module 33 .

[0160] The picked-up audio compression module 31 is used to execute step S31: encoding the current sound signal through amplitude modulation to obtain a preliminary compressed sound signal.

[0161] Specifically, the amplitude modulation is used to convert the current sound signal into a digital form while maintaining the amplitude information of the sound signal, thereby avoiding distortion.

[0162] The picked-up audio encoding module 32 is used to execute step S32: performing secondary encoding on the initially compressed sound signal to obtain an encoded sound signal.

[0163] The secondary encoding may be selected as mp3 encoding, opus encoding, AAC encoding or Vorbis encoding, etc., which is used to encode the primary compressed sound signal, so that the encoded sound signal has a smaller data volume, thereby achieving better transmission efficiency.

[0164] Among them, the initially compressed sound signal is secondary encoded to fully compress the sound signal to improve transmission efficiency and adapt it to different communication links.

[0165] The sound pickup audio playing module 33 is used to execute step S33: returning the encoded sound signal according to the original communication link.

[0166] Specifically, the backhaul is to transmit the encoded sound signal to the drone remote controller or mobile phone APP according to the drone's own link or mobile network link.

[0167] The drone remote controller or mobile phone APP restores and decodes the encoded sound signal to obtain playable audio and plays it to achieve high real-time audio playback.

[0168] Compared with the prior art, the present invention uses a three-microphone design and channel separation to enable the drone voice broadcast to fully pick up the sound signals of the environment, and extracts the human voice features in different sound channels through the Mel filter group feature extraction combined with the binary classification algorithm to accurately distinguish the human voice channel from the noise channel. The noise channel is constructed and a noise filter is applied to effectively filter out complex noise from the human voice channel to ensure that high-frequency noise can be effectively filtered out.

[0169] In addition, the present invention also performs scene recognition on the scene images of the drone environment by combining an artificial neural network to generate corresponding scene text data in combination with a preset scene text library, and can flexibly use a text-to-speech model (Text-to-Speech) to convert the scene text data into broadcast audio to provide timely rescue guidance and disaster warning information.

[0170] Based on the same inventive concept, the present application also provides an electronic device, which may be a terminal device such as a server, a desktop computing device or a mobile computing device (e.g., a laptop computing device, a handheld computing device, a tablet computer, a netbook, etc.). The device includes one or more processors and a memory, wherein the processor is used to execute a program to implement the interactive dialogue method of the drone voice broadcast of the embodiment of the present invention; and the memory is used to store a computer program executable by the processor.

[0171] Based on the same inventive concept, the present application also provides a computer-readable storage medium, corresponding to the embodiment of the interactive dialogue method of the drone voice broadcast mentioned above, and the computer-readable storage medium stores a computer program thereon, which, when executed by the processor, implements the steps of the interactive dialogue method of the drone voice broadcast recorded in any of the above embodiments.

[0172] The present application may take the form of a computer program product implemented on one or more storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing program code. Computer-usable storage media include permanent and non-permanent, removable and non-removable media, and information storage can be achieved by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include but are not limited to: phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

[0173] The above-mentioned embodiments only express several implementation methods of the present invention, and the description is relatively specific and detailed, but it cannot be understood as limiting the scope of the invention patent. It should be pointed out that for ordinary technicians in this field, several modifications and improvements can be made without departing from the concept of the present invention, and the present invention is also intended to include these modifications and modifications.

Claims

1. An interactive dialogue method for drone voice broadcasting, characterized in that: The following steps are involved: S1: Decoding and preprocessing the control information to be played to obtain playable audio, transmitting the playable audio to a loudspeaker device, and obtaining a current ambient sound signal; wherein the control information to be played is control information output by a mobile terminal; and the loudspeaker device is used to play the playable audio; S2: performing channel separation, echo cancellation and human voice detection on the current environmental sound signal in sequence, identifying a reference noise channel and a human voice channel, and constructing a noise filter based on the reference noise channel, filtering the human voice channel with the noise filter, and obtaining a denoised sound signal; wherein, step S2 includes the following steps: S21: performing channel separation on the current ambient sound signal to obtain a number of microphone signals; S22: performing echo cancellation and voice detection on the plurality of microphone signals to obtain a reference noise channel and a voice channel; S23: Perform time-frequency analysis on the reference noise channel to obtain noise characteristic parameters; S24: Performing filter construction according to the noise characteristic parameters to obtain a noise filter; wherein the filter construction comprises the following steps: A second-order notch filter is constructed for each frequency point in the noise characteristic parameter to obtain a plurality of second-order notch filters; And several second-order notch filters are cascaded to obtain a noise filter; S25: Filter the noise of the human voice channel through a noise filter to obtain a denoised sound signal, and use the denoised sound signal as the current sound signal to execute step S3; S3: Perform amplitude modulation encoding and secondary encoding on the denoised sound signal, and return the encoded sound signal.

2. The interactive dialogue method of drone voice broadcasting according to claim 1, characterized in that: The step S22 specifically includes the following sub-steps: S221: using the playable audio as a reference signal, and performing echo cancellation on a plurality of microphone signals according to the reference signal to obtain a plurality of microphone signals to be detected; S222: performing frame division, windowing, and fast Fourier transformation on a plurality of microphone signals to be detected to obtain frequency spectrum information of the microphone signals; S223: extracting features from the spectrum information of the microphone signal through a Mel filter bank to obtain microphone feature parameters; S224: performing voice recognition on microphone feature parameters through a trained binary classification model to obtain a number of voice features; S225: Calculate the maximum sound pressure amplitude among the plurality of human voice features, and use the corresponding microphone signal as a human voice channel, and the remaining microphone signals as reference noise channels; The step S23 specifically includes the following sub-steps: S231: performing frame division, windowing and Fourier transformation on the reference noise channel to obtain spectrum information of the noise; S232: performing energy calculation on the frequency spectrum information of the noise to obtain an energy spectrum of the noise; S233: Obtain the maximum amplitude value in the energy spectrum of the plurality of noises as a frequency point, obtain the current plurality of noise frequency points, and use them as noise characteristic parameters.

3. The interactive dialogue method of drone voice broadcasting according to claim 2, characterized in that: The control information to be played also includes control information generated by a text built into the drone, and the steps for obtaining the control information generated by the text are as follows: SA1: By extracting the scene image of the current environment; SA2: Extract features from the scene image of the environment through image processing algorithms to obtain scene features; SA3: Use the pre-built detection model to perform scene recognition on scene features and obtain the current scene keywords; SA4: Index a preset scene text library according to the current scene keyword to obtain scene text data; SA5: Convert scene text data through a text-to-speech model to obtain broadcast audio; SA6: Encode the broadcast audio, obtain the text-generated control information, and use the text-generated control information as the current control information.

4. An interactive dialogue device for drone voice broadcasting, characterized in that: It includes a broadcast audio processing unit, an audio noise reduction processing unit and a pickup audio processing unit; The broadcast audio processing unit is used to decode and preprocess the control information to be played, obtain the playable audio, transmit the playable audio to a sound amplification device, and obtain the current ambient sound signal; wherein the control information to be played is the control information output by the mobile terminal; the sound amplification device is used to play the playable audio; The audio noise reduction processing unit is used to perform channel separation, echo elimination and human voice detection on the current environmental sound signal in sequence, identify the reference noise channel and the human voice channel, and construct a noise filter based on the reference noise channel, and filter the noise of the human voice channel through the noise filter to obtain a denoised sound signal; wherein the audio noise reduction processing unit includes a channel separation module, a human voice detection module, a noise extraction module, a noise filter construction module and a noise filtering module; The channel separation module is used to perform channel separation on the current ambient sound signal to obtain a number of microphone signals; The human voice detection module is used to perform echo cancellation and human voice detection on a plurality of microphone signals to obtain a reference noise channel and a human voice channel; The noise extraction module is used to perform time-frequency analysis on the reference noise channel to obtain noise characteristic parameters; The noise filter construction module is used to perform filtering construction according to noise characteristic parameters to obtain a noise filter; The filter construction includes the following steps: A second-order notch filter is constructed for each frequency point in the noise characteristic parameter to obtain a plurality of second-order notch filters; And several second-order notch filters are cascaded to obtain a noise filter; The noise filtering module is used to filter the noise of the human voice channel through a noise filter to obtain a denoised sound signal, and use the denoised sound signal as the current sound signal to call the sound pickup audio processing unit; The sound pickup audio processing unit is used to perform amplitude modulation coding and secondary coding on the denoised sound signal, and to transmit the coded sound signal back.

5. According to the interactive dialogue device of drone voice broadcasting according to claim 4, the human voice detection module specifically comprises executing the following sub-steps: The playable audio is used as a reference signal, and echo cancellation is performed on a plurality of microphone signals according to the reference signal to obtain a plurality of microphone signals to be detected; Performing frame division, windowing and fast Fourier transformation on a number of microphone signals to be detected to obtain spectrum information of the microphone signals; The spectral information of the microphone signal is extracted by using a Mel filter bank to obtain microphone characteristic parameters; Using a trained binary classification model to perform voice recognition on microphone feature parameters, several voice features are obtained; Calculate the maximum sound pressure amplitude among several human voice features, and use the corresponding microphone signal as the human voice channel, and the remaining microphone signals as reference noise channels; The noise extraction module specifically includes executing the following sub-steps: The reference noise channel is framed, windowed and Fourier transformed to obtain the spectrum information of the noise; Perform energy calculation on the frequency spectrum information of the noise to obtain the energy spectrum of the noise; The maximum value of the amplitude in the energy spectrum of several noises is obtained as the frequency point, and the current several noise frequency points are obtained and used as the noise characteristic parameters.

6. The interactive dialogue device for drone voice broadcasting according to claim 5, characterized in that: The control information to be played also includes control information generated by a text built into the drone, and the steps for obtaining the control information generated by the text are as follows: SA1: By extracting the scene image of the current environment; SA2: Extract features from the scene image of the environment through image processing algorithms to obtain scene features; SA3: Use the pre-built detection model to perform scene recognition on scene features and obtain the current scene keywords; SA4: Index a preset scene text library according to the current scene keyword to obtain scene text data; SA5: Convert scene text data through a text-to-speech model to obtain broadcast audio; SA6: Encode the broadcast audio, obtain the text-generated control information, and use the text-generated control information as the current control information.

7. A flying drone, comprising a sound amplification device and three microphones, characterized in that: It also includes the interactive dialogue device for drone voice broadcasting as described in any one of claims 4-6; The amplifying device is used to play the playable audio output by the interactive dialogue device of the drone voice broadcast; The three microphones are respectively arranged at the upper end and left and right sides of the flying drone and are arranged in a triangle, and are used to obtain environmental sound signals and input the environmental sound signals into the interactive dialogue device of the drone voice broadcast.

8. An electronic device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the interactive dialogue method for drone voice broadcasting as described in any one of claims 1 to 3 is implemented.

9. A computer-readable storage medium storing computer-executable instructions, characterized in that: The computer executable instructions are used in the interactive dialogue method for drone voice broadcasting as described in any one of claims 1 to 3.

Images