Semantic audio interaction system of unmanned equipment

By using the combination technology of voice signal acquisition, preprocessing and recognition modules in the semantic audio interaction system of unmanned devices, the environmental noise is eliminated and the voice signal recognition model is constructed, which solves the problem of inconvenience in traditional interaction methods and realizes efficient people Computer interaction and device control improve the accuracy of speech recognition and the intelligence of interaction.

CN119068874APending Publication Date: 2024-12-03INST OF WAR STUDIES ACAD OF MILITARY SCI OF THE CHINESE PEOPLES LIBERATION ARMY
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410962046.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-18
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

The prior art is difficult to effectively preprocess the user's voice signals to eliminate environmental impacts, thereby improving the accuracy of speech recognition. Especially in semantic audio interaction systems of unmanned devices, there are inconveniences in traditional interaction methods and cannot achieve efficient Human-computer interaction.

Method used

A semantic audio interaction system with unmanned equipment was designed, including voice signal acquisition module, voice signal preprocessing module, voice signal recognition module and control execution module. Through technical means such as Fourier transform and inverse transform, filtering processing, etc., it eliminates the The ambient noise is divided into short-time frames, pre-emphasis and bandpass filtering are performed, and a speech signal recognition model is constructed to realize speech text recognition and equipment control.

Benefits of technology

It realizes accurate recognition of user voice and equipment control, improves the naturalization and intelligence of human-computer interaction, and enhances the efficiency of work and life.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119068874A_ABST
    Figure CN119068874A_ABST
Patent Text Reader

Abstract

The invention discloses a semantic audio interaction system of unmanned equipment, and relates to the field of signal processing, the semantic audio interaction system comprises an interaction center, the interaction center is in communication connection with a voice signal acquisition module, a voice signal preprocessing module, a voice signal identification module and a control execution module; the method comprises the following steps: acquiring a voice signal of a user, preprocessing the acquired voice signal of the user to obtain a voice signal of the user after the environmental influence is eliminated, identifying the obtained voice signal of the user after the environmental influence is eliminated to obtain a user voice text, and controlling the unmanned equipment according to the obtained user voice text. The unmanned device is controlled through the user voice, man-machine interaction is made to be more natural and intelligent, the user voice can be recognized more accurately, and therefore work and life efficiency is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to the field of signal processing, and in particular to a semantic audio interaction system for unmanned equipment. Background Art

[0002] With the improvement of people's living standards and changes in lifestyles, the demand for convenient and efficient human-computer interaction is increasing. In particular, in some special scenarios, such as driving, home, industrial production, etc., the traditional touch screen or keyboard interaction methods are inconvenient, so the demand for semantic audio interaction systems is becoming more and more urgent; intelligence has become one of the main trends in the development of today's society. The intelligence of unmanned equipment requires that the equipment can actively understand user needs and make corresponding feedback and control. As a part of intelligence, the semantic audio interaction system can realize the recognition, understanding and execution of voice commands, thereby realizing the intelligent control and interaction of equipment; How to eliminate environmental influences by preprocessing user voice signals and obtain user voice text, thereby improving the accuracy of voice recognition, is a problem we need to solve. To this end, a semantic audio interaction system for unmanned equipment is now provided. Summary of the invention

[0003] In order to solve the above technical problems, the purpose of the present invention is to provide a semantic audio interaction system for unmanned equipment.

[0004] The object of the present invention can be achieved by the following technical solutions: A semantic audio interaction system for unmanned equipment, comprising an interaction center, wherein the interaction center is communicatively connected with a voice signal acquisition module, a voice signal preprocessing module, a voice signal recognition module, and a control execution module; The voice signal acquisition module is used to collect the user's voice signal; The voice signal preprocessing module is used to preprocess the collected user's voice signal to obtain the user's voice signal after eliminating the environmental influence; The voice signal recognition module is used to recognize the user's voice signal obtained after eliminating environmental influences to obtain the user's voice text; The control execution module is used to control the unmanned equipment according to the obtained user voice text.

[0005] Furthermore, the process of the voice signal acquisition module acquiring the user's voice signal includes: The voice signal acquisition module is provided with a voice signal acquisition terminal, which is installed at the corresponding position of the unmanned equipment. When the user's voice signal is detected, the user's voice signal is collected through the installed voice signal acquisition terminal. When the detected user's voice signal disappears, wait for t seconds and end the collection.

[0006] Furthermore, the process of the voice signal preprocessing module preprocessing the collected user's voice signal includes: Acquire a voice signal that does not contain the user's voice from the collected user's voice signal, which is recorded as environmental noise data, and acquire a voice signal that contains the user's voice from the collected user's voice signal, which is recorded as a user pure voice signal; The obtained environmental noise data and the user pure voice signal are converted into the frequency domain by Fourier transform, and a spectrogram of the environmental noise data and a spectrogram of the user pure voice signal are obtained, the spectrogram of the obtained environmental noise data is recorded as an environmental noise spectrogram, and the spectrogram of the obtained user pure voice signal is recorded as a user pure voice spectrogram; Obtain the mean value of the ambient noise spectrum according to the ambient noise spectrum graph, subtract the user's pure speech spectrum from the obtained mean value of the ambient noise spectrum, obtain the user's pure speech spectrum graph after the ambient noise is eliminated, and record the obtained pure speech spectrum graph after the ambient noise is eliminated as the user's pure speech spectrum graph after the ambient noise is eliminated; The pure speech spectrum of the user after eliminating the environmental influence is converted into the time domain through inverse Fourier transform to obtain the user's speech signal after eliminating the environmental influence.

[0007] Furthermore, the process of the voice signal recognition module recognizing the user's voice signal obtained after eliminating the environmental influence includes: The continuous user's voice signal after eliminating the environmental influence is divided into a plurality of short time frames, the user's voice signal after eliminating the environmental influence corresponding to the obtained short time frames is pre-emphasized, and a first-order filter is set; Inputting the user's voice signal after eliminating the environmental influence corresponding to the short-time frame into the set first-order filter to obtain the user's voice signal after pre-emphasis processing and eliminating the environmental influence; Performing Fourier transform on the user's voice signal obtained after pre-emphasis processing and elimination of environmental influence, converting the time domain signal into the frequency domain, and obtaining the length when performing Fourier transform; Setting a plurality of bandpass filters, inputting the user's voice signal converted to the frequency domain after pre-emphasis processing and after eliminating environmental influences into the set bandpass filters, obtaining the output results of the bandpass filters, and recording the obtained output results of the bandpass filters as user voice recognition signals; Construct a speech signal recognition model, input the obtained user speech recognition signal into the speech signal recognition model, and obtain the user speech text corresponding to the user speech recognition signal.

[0008] Furthermore, the process of segmenting the continuous user's voice signal after eliminating the environmental influence into a plurality of short time frames includes: The duration of each frame is recorded as the frame length, and the time interval between adjacent frames is recorded as the frame shift. The frame length and the frame shift are set according to actual conditions, and a sliding window is set. The length of the sliding window is consistent with the set frame length. The set sliding window is slid on the user's voice signal after the environmental influence is continuously eliminated. The distance of each sliding is the same as the set frame shift. A short-time frame is obtained each time it slides. When the sliding window slides to the end of the user's voice signal after the environmental influence is continuously eliminated, the sliding is stopped to obtain several short-time frames.

[0009] Furthermore, the process of setting a plurality of bandpass filters includes: Setting a plurality of bandpass filters according to the frequency spectrum range of the user's voice signal after pre-emphasis processing and elimination of environmental influence, and setting a bandpass filter function and an inverse function of the bandpass filter; Input the lowest frequency within the frequency range of the bandpass filter into the set bandpass filter function to obtain the output result of the bandpass filter function, and record the obtained output result of the bandpass filter function as the lowest frequency result; input the highest frequency within the frequency range of the bandpass filter into the set bandpass filter function to obtain the output result of the bandpass filter function, and record the obtained output result of the bandpass filter function as the highest frequency result; input the sampling frequency of the bandpass filter into the set bandpass filter function to obtain the output result of the bandpass filter function, and record the obtained output result of the bandpass filter function as the sampling frequency result; The center frequency of the bandpass filter is set according to the obtained minimum frequency result, maximum frequency result and sampling frequency result.

[0010] Furthermore, the process of building a speech signal recognition model includes: A speech signal recognition model is constructed based on historical data. The historical user speech recognition signal is used as the input value of the speech signal recognition model. The historical user speech text corresponding to the historical user speech recognition signal is used as the output value of the speech signal recognition model. The speech signal recognition model is trained using historical data. The speech signal recognition model is constructed.

[0011] Furthermore, the process of the control execution module controlling the unmanned device according to the obtained user voice text includes: The obtained user voice text is converted into binary representation, and the corresponding control execution instruction is issued according to the converted binary. The unmanned device performs the corresponding operation according to the issued control execution instruction. If the unmanned device cannot perform the corresponding operation according to the issued control execution instruction, a red alarm is issued and a voice reminder is given.

[0012] Compared with the prior art, the beneficial effects of the present invention are: collecting user voice signals, preprocessing the collected user voice signals to obtain the user voice signals after eliminating environmental influences, recognizing the obtained user voice signals after eliminating environmental influences, dividing the continuous user voice signals after eliminating environmental influences into several short-time frames, pre-emphasizing the user voice signals after eliminating environmental influences corresponding to the obtained short-time frames, converting the obtained user voice signals after pre-emphasis and after eliminating environmental influences to the frequency domain, filtering, constructing a voice signal recognition model, obtaining user voice text according to the constructed voice signal recognition model, and controlling unmanned equipment according to the obtained user voice text, thereby realizing the control of unmanned equipment through user voice, making human-computer interaction more natural and intelligent, and being able to more accurately recognize user voices, thereby improving work and life efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0014] Figure 1 It is a schematic diagram of the present invention. DETAILED DESCRIPTION

[0015] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0016] like Figure 1 As shown, a semantic audio interaction system for unmanned equipment includes an interaction center, wherein the interaction center is communicatively connected to a voice signal acquisition module, a voice signal preprocessing module, a voice signal recognition module, and a control execution module; The voice signal acquisition module is used to collect the user's voice signal; It should be further explained that, in a specific implementation process, the process of the voice signal acquisition module acquiring the user's voice signal includes: The voice signal acquisition module is provided with a voice signal acquisition terminal, which is installed at the corresponding position of the unmanned equipment. When the user's voice signal is detected, the user's voice signal is collected through the installed voice signal acquisition terminal. When the detected user's voice signal disappears, wait for t seconds and end the collection.

[0017] The voice signal preprocessing module is used to preprocess the collected user's voice signal to obtain the user's voice signal after eliminating the environmental influence; It should be further explained that, in a specific implementation process, the process of the voice signal preprocessing module preprocessing the collected user's voice signal includes: Acquire a voice signal that does not contain the user's voice from the collected voice signal of the user, record the acquired voice signal that does not contain the user's voice as environmental noise data, acquire a voice signal that contains the user's voice from the collected voice signal of the user, and record the acquired voice signal that contains the user's voice as a user pure voice signal; The obtained environmental noise data and the user pure voice signal are converted into the frequency domain by Fourier transform, and a spectrogram of the environmental noise data and a spectrogram of the user pure voice signal are obtained, the spectrogram of the obtained environmental noise data is recorded as an environmental noise spectrogram, and the spectrogram of the obtained user pure voice signal is recorded as a user pure voice spectrogram; Obtain the mean value of the ambient noise spectrum according to the ambient noise spectrum graph, subtract the user's pure speech spectrum from the obtained mean value of the ambient noise spectrum, obtain the user's pure speech spectrum graph after the ambient noise is eliminated, and record the obtained pure speech spectrum graph after the ambient noise is eliminated as the user's pure speech spectrum graph after the ambient noise is eliminated; The pure speech spectrum of the user after eliminating the environmental influence is converted into the time domain through inverse Fourier transform to obtain the user's speech signal after eliminating the environmental influence.

[0018] The voice signal recognition module is used to recognize the user's voice signal obtained after eliminating environmental influences to obtain the user's voice text; It should be further explained that, in a specific implementation process, the process in which the voice signal recognition module recognizes the user's voice signal obtained after eliminating the environmental influence includes: The continuous voice signal of the user after eliminating the environmental influence is divided into several short time frames, the duration of each frame is recorded as the frame length, the time interval between adjacent frames is recorded as the frame shift, the frame length and the frame shift are set according to the actual situation, and a sliding window is set, the length of the sliding window is consistent with the set frame length, the set sliding window is slid on the continuous voice signal of the user after eliminating the environmental influence, the size of each sliding distance is the same as the size of the set frame shift, and a short time frame is obtained each time the sliding is performed. When the sliding window slides to the end of the continuous voice signal of the user after eliminating the environmental influence, the sliding is stopped to obtain several short time frames, each short time frame is labeled and recorded as i, i=1, 2, 3, ..., n, n is a positive integer, and the voice signal of the user after eliminating the environmental influence corresponding to the short time frame is recorded as x i (t); Performing pre-emphasis processing on the user's voice signal after eliminating environmental influences corresponding to the obtained short-time frame, and setting a first-order filter; The differential equation of the first-order filter is: y(t)=x(t)+β×x(t-1), where x(t) is the input of the first-order filter, y(t) is the output of the first-order filter, and β is the pre-emphasis; The user's voice signal after eliminating the environmental influence corresponding to the short-time frame is input into the set first-order filter to obtain the user's voice signal after pre-emphasis processing and eliminating the environmental influence, and the obtained user's voice signal after pre-emphasis processing and eliminating the environmental influence is recorded as y i (t); It should be further explained that, in the specific implementation process, the speech signal of the user after the environmental influence is eliminated corresponding to the obtained short-time frame is pre-emphasized, thereby reducing the low-frequency component of the speech signal of the user after the environmental influence is eliminated in the frame, and increasing the high-frequency component of the speech signal of the user after the environmental influence is eliminated, so that its energy is more prominent, which is conducive to improving the clarity and recognizability of the speech signal of the user after the environmental influence is eliminated; Performing Fourier transform on the user's speech signal obtained after pre-emphasis processing and elimination of environmental influence, converting the time domain signal into the frequency domain, and recording the length of the Fourier transform as N; According to the frequency spectrum range of the user's voice signal after pre-emphasis processing and elimination of environmental influence, several band-pass filters are set, and the set band-pass filters are labeled as m, where m=1, 2, 3, ..., j, where j is a positive integer, and the set band-pass filters are recorded as Hm(k), 1≤m≤M, where M is the number of band-pass filters, and a band-pass filter function and an inverse function of the band-pass filter are set, and the set band-pass filter function is recorded as F(m), and the set band-pass filter inverse function is recorded as F -1(m), setting the center frequency of the bandpass filter according to the set bandpass filter function and the bandpass filter inverse function; Input the lowest frequency within the frequency range of the bandpass filter into the set bandpass filter function to obtain the output result of the bandpass filter function, and record the obtained output result of the bandpass filter function as the lowest frequency result; input the highest frequency within the frequency range of the bandpass filter into the set bandpass filter function to obtain the output result of the bandpass filter function, and record the obtained output result of the bandpass filter function as the highest frequency result; input the sampling frequency of the bandpass filter into the set bandpass filter function to obtain the output result of the bandpass filter function, and record the obtained output result of the bandpass filter function as the sampling frequency result; Subtract the lowest frequency result from the highest frequency result, add the number of bandpass filters to one, divide the subtraction result by the addition result, multiply the division result by m, add the multiplication result to the lowest frequency result, input the addition result to the set inverse function of the bandpass filter, obtain the output result of the inverse function of the bandpass filter, divide the length when Fourier transform is performed by the sampling frequency of the bandpass filter, multiply the division result by the output result of the inverse function of the bandpass filter, the multiplication result is the center frequency of the set bandpass filter, and the center frequency of the set bandpass filter is recorded as f(m); Among them, F^(-1) (m) = a_1×(〖10〗^(m / b_1 )-1), F(m) = b_1×log_10〖(1+m / a_1 )〗, a1 and b1 are coefficients set according to actual conditions; Where, f(m)=(N / f_s)×F^(-1) (F(f_w)+m×(F(f_h)-F(f_w)) / (M+1)) fw is the lowest frequency within the frequency range of the bandpass filter, fh is the highest frequency within the frequency range of the bandpass filter, and fs is the sampling frequency of the bandpass filter; When k<f(m-1), Hm(k)=0, when f(m-1)≤k≤f(m), H_m (k)=(kf(m-1)) / (f(m)-f(m-1)), when f(m)<k≤f(m+1), H_m (k)=(f(m+1)-k) / (f(m+1)-f(m)), when k>f(m+1), Hm(k)=0; Inputting the user's voice signal converted to the frequency domain after pre-emphasis processing and after eliminating environmental influence into the set bandpass filter, obtaining the output result of the bandpass filter, and recording the obtained output result of the bandpass filter as the user voice recognition signal; A speech signal recognition model is constructed based on historical data, and the historical user speech recognition signal is used as the input value of the speech signal recognition model. The historical user speech text corresponding to the historical user speech recognition signal is used as the output value of the speech signal recognition model. The speech signal recognition model is trained using historical data to achieve continuous optimization of the speech signal recognition model. The obtained user speech recognition signal is input into the speech signal recognition model to obtain the user speech text corresponding to the user speech recognition signal.

[0019] The control execution module is used to control the unmanned device according to the obtained user voice text; It should be further explained that, in the specific implementation process, the process of the control execution module controlling the unmanned device according to the obtained user voice text includes: The obtained user voice text is converted into binary representation, and the corresponding control execution instruction is issued according to the converted binary. The unmanned device performs the corresponding operation according to the issued control execution instruction. If the unmanned device cannot perform the corresponding operation according to the issued control execution instruction, a red alarm is issued and a voice reminder is given.

[0020] The above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. A semantic audio interaction system for unmanned equipment, comprising an interaction center, characterized in that: The interactive center is communicatively connected to a voice signal acquisition module, a voice signal preprocessing module, a voice signal recognition module and a control execution module; The voice signal acquisition module is used to collect the user's voice signal; The voice signal preprocessing module is used to preprocess the collected user's voice signal to obtain the user's voice signal after eliminating the environmental influence; The voice signal recognition module is used to recognize the user's voice signal obtained after eliminating environmental influences to obtain the user's voice text; The control execution module is used to control the unmanned equipment according to the obtained user voice text.

2. According to the semantic audio interaction system of unmanned equipment of claim 1, it is characterized in that: The process of the voice signal acquisition module acquiring the user's voice signal includes: The voice signal acquisition module is provided with a voice signal acquisition terminal, which is installed at the corresponding position of the unmanned equipment. When the user's voice signal is detected, the user's voice signal is collected through the installed voice signal acquisition terminal. When the detected user's voice signal disappears, wait for t seconds and end the collection.

3. The semantic audio interaction system for unmanned equipment according to claim 2, characterized in that: The process of preprocessing the collected user's voice signal by the voice signal preprocessing module includes: Acquire a voice signal that does not contain the user's voice from the collected user's voice signal, which is recorded as environmental noise data, and acquire a voice signal that contains the user's voice from the collected user's voice signal, which is recorded as a user pure voice signal; The obtained environmental noise data and the user pure voice signal are converted into the frequency domain by Fourier transform, and a spectrogram of the environmental noise data and a spectrogram of the user pure voice signal are obtained, the spectrogram of the obtained environmental noise data is recorded as an environmental noise spectrogram, and the spectrogram of the obtained user pure voice signal is recorded as a user pure voice spectrogram; Obtain the mean value of the ambient noise spectrum according to the ambient noise spectrum graph, subtract the user's pure speech spectrum from the obtained mean value of the ambient noise spectrum, obtain the user's pure speech spectrum graph after the ambient noise is eliminated, and record the obtained pure speech spectrum graph after the ambient noise is eliminated as the user's pure speech spectrum graph after the ambient noise is eliminated; The pure speech spectrum of the user after eliminating the environmental influence is converted into the time domain through inverse Fourier transform to obtain the user's speech signal after eliminating the environmental influence.

4. The semantic audio interaction system for unmanned equipment according to claim 3, characterized in that: The process of the voice signal recognition module recognizing the user's voice signal obtained after eliminating the environmental influence includes: The continuous user's voice signal after eliminating the environmental influence is divided into a plurality of short time frames, the user's voice signal after eliminating the environmental influence corresponding to the obtained short time frames is pre-emphasized, and a first-order filter is set; Inputting the user's voice signal after eliminating the environmental influence corresponding to the short-time frame into the set first-order filter to obtain the user's voice signal after pre-emphasis processing and eliminating the environmental influence; Performing Fourier transform on the user's voice signal obtained after pre-emphasis processing and elimination of environmental influence, converting the time domain signal into the frequency domain, and obtaining the length when performing Fourier transform; Setting a plurality of bandpass filters, inputting the user's voice signal converted to the frequency domain after pre-emphasis processing and after eliminating environmental influences into the set bandpass filters, obtaining the output results of the bandpass filters, and recording the obtained output results of the bandpass filters as user voice recognition signals; Construct a speech signal recognition model, input the obtained user speech recognition signal into the speech signal recognition model, and obtain the user speech text corresponding to the user speech recognition signal.

5. The semantic audio interaction system for unmanned equipment according to claim 4, characterized in that: The process of segmenting the continuous user's voice signal after eliminating the environmental influence into several short time frames includes: The duration of each frame is recorded as the frame length, and the time interval between adjacent frames is recorded as the frame shift. The frame length and the frame shift are set according to actual conditions, and a sliding window is set. The length of the sliding window is consistent with the set frame length. The set sliding window is slid on the user's voice signal after the environmental influence is continuously eliminated. The distance of each sliding is the same as the set frame shift. A short-time frame is obtained each time it slides. When the sliding window slides to the end of the user's voice signal after the environmental influence is continuously eliminated, the sliding is stopped to obtain several short-time frames.

6. The semantic audio interaction system for unmanned equipment according to claim 4, characterized in that: The process of setting up several bandpass filters involves: According to the frequency spectrum range of the user's voice signal after pre-emphasis processing and elimination of environmental influence, several bandpass filters are set, bandpass filter functions and bandpass filter inverse functions are set, and the center frequency of the bandpass filter is set according to the set bandpass filter functions and bandpass filter inverse functions.

7. The semantic audio interaction system for unmanned equipment according to claim 4, characterized in that: The process of building a speech signal recognition model includes: A speech signal recognition model is constructed based on historical data. The historical user speech recognition signal is used as the input value of the speech signal recognition model. The historical user speech text corresponding to the historical user speech recognition signal is used as the output value of the speech signal recognition model. The speech signal recognition model is trained using historical data. The speech signal recognition model is constructed.

8. The semantic audio interaction system for unmanned equipment according to claim 4, characterized in that: The process of the control execution module controlling the unmanned device according to the obtained user voice text includes: The obtained user voice text is converted into binary representation, and the corresponding control execution instruction is issued according to the converted binary. The unmanned device performs the corresponding operation according to the issued control execution instruction. If the unmanned device cannot perform the corresponding operation according to the issued control execution instruction, a red alarm is issued and a voice reminder is given.