Human-machine face-to-face interaction method and device based on voice and electronic equipment

Abstract

The invention provides a human-machine face-to-face interaction method and device based on voice and electronic equipment. The method comprises the steps that real person recognition is carried out on a target which is away from a voice robot by a preset distance; under the condition that the target is determined to be a real person, performing noise reduction and enhancement processing on the collected voice to obtain a preprocessed audio; intercepting the preprocessed audio under the quantitative sampling frequency to obtain a sampled audio, calculating the audio intensity of the sampled audio, and performing release identification on the sampled audio based on an audio intensity threshold; and carrying out radio processing on the sampled audio meeting the release identification condition to identify the audio content. According to the invention, a man-machine voice interaction machine can be more intelligent, voice interaction can be started only when a real person is recognized, and the embarrassing situation of unmanned interaction or non-real person interaction is avoided; the face-to-face interaction (within 2 meters) voice recognition is free from noise interference, face-to-face man-machine voice interaction has higher recognition accuracy, and better interaction experience is achieved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent human-computer interaction technology, and in particular to a voice-based human-computer face-to-face interaction method, device, electronic device and storage medium. Background Art

[0002] Machines such as digital humans or AI smart robots that use face-to-face voice interaction can already be seen everywhere in many exhibition halls or high-end supermarkets. In this scenario, the public can easily interact with the machine through voice conversations, and then get help from the smart machine, which can greatly reduce the daily manpower expenses of the corresponding venues.

[0003] When voice robots are working, the most critical part is voice recognition. The machine needs to know what people are saying first, and the recorded voice is indeed from the person in front of the machine, so that it can further understand and respond; otherwise, if the first step is wrong, all the subsequent processes will be in vain. Although the current voice-to-text apps on mobile phones have a high recognition accuracy, this is because the sound source is recorded close to the microphone, naturally avoiding a lot of interference from environmental noise.

[0004] However, in noisy places, such as supermarkets, stations, markets, etc., embarrassing situations may occur. The content of the speech of people interacting at a close distance (within about 2 meters) is often recognized as not the original intention of the speaker due to the interference of background noise or distant voices; or the speech of people farther away (for example, not at all in front of the device) immediately triggers the voice interaction of the machine, resulting in the embarrassing situation of "no one" interaction. These will seriously affect the human-machine voice interaction experience of such machines.

[0005] Therefore, it is necessary to improve the interaction mode of existing voice robots. Summary of the invention

[0006] In view of this, it is necessary to provide a voice-based human-computer face-to-face interaction method, device, electronic device and storage medium that can solve the technical problem in the prior art that the interaction between voice robots is easily invalid due to environmental influences.

[0007] In order to solve the above technical problems, the present invention provides a human-computer face-to-face interaction method based on speech, comprising: Perform real person recognition on targets at a preset distance from the voice robot; When the target is confirmed to be a real person, the collected voice is subjected to noise reduction and enhancement processing to obtain pre-processed audio; Intercepting the preprocessed audio at the quantitative sampling frequency to obtain sampled audio, calculating the audio intensity of the sampled audio, and performing release identification on the sampled audio based on the audio intensity threshold; The sampled audio that meets the release identification conditions is collected and processed to identify the audio content.

[0008] In a possible implementation, the performing real person recognition on a target at a preset distance from the voice robot includes: Based on the face recognition model, real person recognition is performed on the target at a preset distance from the voice robot; and / or; Based on binocular live anti-counterfeiting detection technology, real-person recognition is performed on targets at a preset distance from the voice robot.

[0009] In a possible implementation, real person recognition of a target at a preset distance from the voice robot is performed based on a face recognition model, including: Based on the Adaboost algorithm as a benchmark model of the face recognition model, multiple weak classifiers are combined into a strong classifier, and the face recognition model is obtained by weighting and repeatedly training the data set; Real person recognition based on face recognition model.

[0010] In a possible implementation, real person recognition of a target at a preset distance from the voice robot is performed based on binocular liveness anti-counterfeiting detection technology, including: Adopt visible light plus near infrared optoelectronic integrated face liveness detection technology as binocular liveness anti-counterfeiting detection technology; The spectral information reflected by facial skin under different lighting conditions is analyzed and classified, and the heterogeneous facial images are associated and judged to distinguish the real facial skin.

[0011] In a possible implementation, performing noise reduction processing on the collected speech includes: Perform noise reduction processing on the collected speech based on the audio noise reduction model; The audio noise reduction model is a new architecture model composed of a convolutional cyclic codec obtained by adding a cyclic layer on the basis of a convolutional codec architecture and introducing a feedforward sequence memory network.

[0012] In a possible implementation, performing enhancement processing on the collected speech includes: performing enhancement processing on the recognized human voice; In a possible implementation, the calculation formula of the audio intensity of the sampled audio is: ; in, Indicates the audio intensity of the sampled audio. is the RMS value of the audio sampling point amplitude, The maximum amplitude of the audio sampling point; In a possible implementation, the step of performing release identification on the sampled audio based on the audio intensity threshold includes: Set release threshold and release quantity, which are obtained based on actual equipment testing and use; Based on the release volume, multiple groups of sampled data that follow are released and sent to speech recognition. Each time a group of data is released, this value is subtracted by 1 until it reaches 0, and then no more data is released. Among them, the sampling point groups above the release threshold will not consume the release volume, and the release volume will be restored to the default value. The sampling point groups below the release threshold will consume the release volume one by one until it is consumed. Then the release of audio sampling data will be stopped and the human voice threshold screening process will be entered again.

[0013] The present invention also provides a human-computer face-to-face interaction device based on speech, comprising: A real person recognition module, used to perform real person recognition on a target at a preset distance from the voice robot; The audio processing module is used to reduce noise and enhance the collected voice to obtain pre-processed audio when the target is confirmed to be a real person; A release identification module, used to intercept the pre-processed audio at the quantitative sampling frequency to obtain sampled audio, calculate the audio intensity of the sampled audio, and perform release identification on the sampled audio based on the audio intensity threshold; The audio receiving module is used to receive and process the sampled audio that meets the release identification conditions to identify the audio content.

[0014] The present invention also provides an electronic device, comprising a memory and a processor, wherein: The memory is used to store programs; The processor is coupled to the memory and is used to execute the program stored in the memory to implement the steps in the voice-based human-computer face-to-face interaction method in any of the above implementations.

[0015] The present invention also provides a computer-readable storage medium for storing computer-readable programs or instructions. When the program or instructions are executed by a processor, the steps in the voice-based human-computer face-to-face interaction method in any of the above-mentioned implementation modes can be implemented.

[0016] The beneficial effects of adopting the above embodiment are: The present invention can make machines for human-machine voice interaction more intelligent. Voice interaction is only started when a "real person" is recognized, avoiding the embarrassing situation of no-human interaction or "non-real person" interaction; voice recognition of face-to-face interaction (within 2 meters) is free from noise interference, so that face-to-face human-machine voice interaction has a higher recognition accuracy rate and achieves a better interactive experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0018] Figure 1 A schematic flow chart of an embodiment of the human-computer face-to-face interaction method based on speech provided by the present invention; Figure 2 A structural schematic diagram of an embodiment of an audio noise reduction model provided by the present invention; Figure 3 A schematic diagram of the structure of an embodiment of a human-computer face-to-face interaction device based on voice provided in an embodiment of the present invention; Figure 4 A schematic diagram of the structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0019] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0020] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0021] The present invention provides a human-computer face-to-face interaction method, device, electronic device and storage medium based on voice, which are respectively described below.

[0022] like Figure 1 FIG. 1 is a flow chart of an embodiment of a human-computer face-to-face interaction method based on voice provided by an embodiment of the present invention, and the method includes: Step S101, performing real person recognition on a target at a preset distance from the voice robot; Step S102: When it is confirmed that the target is a real person, noise reduction and enhancement processing are performed on the collected voice to obtain pre-processed audio; Step S103: intercepting the preprocessed audio at the quantitative sampling frequency to obtain a sampled audio, calculating the audio intensity of the sampled audio, and performing release identification on the sampled audio based on the audio intensity threshold; Step S104: collect and process the sampled audio that meets the release identification conditions to identify the audio content.

[0023] It should be noted that the embodiments of the present invention are mainly applied to scenarios of face-to-face interaction between humans and machines, and the main purpose is to ensure that a "real person" is in front of the machine before entering the voice interaction mode. At the same time, the voice of the face-to-face interaction personnel can be more accurately recognized to ensure the smoothness of human-machine interaction.

[0024] Compared with the prior art, the present invention can make the human-machine voice interaction machine more intelligent. The voice interaction is only started when a "real person" is recognized, avoiding the embarrassing situation of no-human interaction or "non-real person" interaction; the voice recognition of face-to-face interaction (within 2 meters) is free from noise interference, so that the face-to-face human-machine voice interaction has a higher recognition accuracy and achieves a better interactive experience.

[0025] In some embodiments of the present invention, the performing real person recognition on a target at a preset distance from the voice robot includes: Based on the face recognition model, real person recognition is performed on the target at a preset distance from the voice robot; and / or; Based on binocular live anti-counterfeiting detection technology, real-person recognition is performed on targets at a preset distance from the voice robot.

[0026] In some embodiments of the present invention, real person recognition of a target at a preset distance from a voice robot is performed based on a face recognition model, including: Based on the Adaboost algorithm as a benchmark model of the face recognition model, multiple weak classifiers are combined into a strong classifier, and the face recognition model is obtained by weighting and repeatedly training the data set; Real person recognition based on face recognition model.

[0027] It should be noted that the Adaboost algorithm is a face detection algorithm that solves the binary classification problem. It can combine multiple weak classifiers (such as decision trees) into a strong classifier. Its basic idea is to improve the accuracy of the classifier by weighting and repeatedly training the data set. Specifically, the mathematical expression of the Adaboost algorithm is: ; Among them, m represents the number of samples, G(x) represents the base classifier, and α represents the weight of each sample.

[0028] The specific algorithm flow of the Adaboost algorithm includes: 1. Initialize / update the weight distribution of the current training data: 1) The first training requires initializing the sample weights: Initialize the weights of the training samples to equal values. If there are N samples, each training sample is initially assigned the same weight: 1 / N. m =(1 / N, 1 / N, 1 / N ... 1 / N) 2) Weight update method for non-first training samples: D m =(W m,1 ,...,W m,i,..., W m,N ), where i represents any sample number and 1≤i≤N; in, ; and ; Because the result of the base classifier is either -1 or 1, it can be simplified to the following formula: ; 2. Train the current base classifier Gm(x): 1) Calculate the classification error rate e of the current Gm(x) on the training data set m : ; Divide the candidate thresholds in the training set x, find the threshold that minimizes the error rate em, and construct Gm(x) for this round.

[0029] 2) Use the training data set with weight distribution Dm to learn and obtain Gm(x).

[0030] 3. Calculate the weight α of the current base classifier m : 1) Calculate the classification error rate em of the current Gm(x) on the training data set: ; 2) According to the classification error rate em, calculate the weight coefficient of the base classifier Gm(x): ; 3) α m Gm(x) is updated to the additive model: ; 4. Determine whether the loop exit condition is met: 1) Whether the number of classifiers reaches M.

[0031] 2) Whether the total classifier error rate is lower than the set accuracy sign[f(x)].

[0032] After the above process, the binary classification function can be realized.

[0033] In some embodiments of the present invention, real person recognition of a target at a preset distance from a voice robot is performed based on binocular live anti-counterfeiting detection technology, including: Adopt visible light plus near infrared optoelectronic integrated face liveness detection technology as binocular liveness anti-counterfeiting detection technology; The spectral information reflected by facial skin under different lighting conditions is analyzed and classified, and the heterogeneous facial images are associated and judged to distinguish the real facial skin.

[0034] It should be noted that the principle of the "visible light + near-infrared" optoelectronic integrated face liveness detection technology is to analyze and classify the spectral information reflected by the face skin under different lighting conditions, make correlation judgments on heterogeneous face images, and effectively distinguish the difference between real face skin and all other attacking materials. Visible light technology can realize rapid face recognition, and near-infrared imaging technology has the characteristics of being insensitive to light, unable to form images on electronic screens, and able to penetrate sunglasses for imaging. In actual application scenarios, it can prevent malicious people from forging and stealing other people's biometrics for identity authentication, and can more effectively prevent attacks by various means such as photos, videos, and 3D masks, thereby improving the security of users' remote identity verification.

[0035] In some embodiments of the present invention, noise reduction processing is performed on the collected speech, including: Perform noise reduction processing on the collected speech based on the audio noise reduction model; The audio noise reduction model is a new architecture model composed of a convolutional cyclic codec obtained by adding a cyclic layer on the basis of a convolutional codec architecture and introducing a feedforward sequence memory network.

[0036] Specifically, the audio denoising model is based on the convolutional encoder-decoder (Convolutional Encoder-Decoder) architecture. By further adding recurrent layers, a new convolutional recurrent encoder-decoder (Convolutional Recurrent Encoder-Decoder) architecture is obtained. This can significantly improve the field of view limitations of the convolution kernel and enhance the feature expression of the frequency dimension of the denoising model, especially in the expression of long-distance frequency correlation. It can eliminate noise while performing more targeted recognition and protection of speech.

[0037] In addition, the introduction of the Feedforward Sequential Memory Network (FSMN) reduces the complexity of the recurrent network, and combines complex domain network operations to implement a full complex deep network model algorithm, which not only models long-sequence speech more effectively, but also enhances the amplitude and phase of speech simultaneously. The related model performed well in the IEEE / INTERSpeech DNS Challenge. The open model uses two Unet cascades and SE layers to achieve a more stable effect.

[0038] The neural network structure of this model is as follows Figure 2 As shown in the figure. The input and output of the model are both single-channel speech time domain waveform signals with a sampling rate of 16kHz. The input signal can be directly recorded by a single-channel microphone, and the output is a speech audio signal after noise suppression [1]. The model input signal is converted into complex spectral features as input through STFT transform, and Complex FSMN is used to perform correlation processing in the frequency domain and long sequence processing on the time series features to predict the intermediate output target Complexideal ratio mask. The predicted mask is then multiplied with the input spectrum to obtain the enhanced spectrum. Finally, the enhanced speech waveform signal is obtained through the inverse STFT transform.

[0039] In some embodiments of the present invention, the collected speech is enhanced, including: enhancing the recognized human voice. It should be noted that the received sound (personnel voice + environmental noise) is first subjected to noise reduction processing, the environmental noise is filtered, and the human voice part is enhanced.

[0040] Furthermore, the amplitude data of a certain number of sampling points (the specific number depends on the sampling rate of the audio) is intercepted and averaged, and then the DBFS (decibel full scale: a unit used to measure the strength of an audio signal) of this part of the audio is calculated. This value is the audio intensity of this part of the audio; In some embodiments of the present invention, the calculation formula of the audio intensity of the sampled audio is: ; in, Indicates the audio intensity of the sampled audio. is the RMS value of the audio sampling point amplitude, The maximum amplitude of the audio sampling point.

[0041] In some embodiments of the present invention, the step of performing release identification on the sampled audio based on the audio intensity threshold comprises: Set release threshold and release quantity, which are obtained based on actual equipment testing and use; Based on the release volume, multiple groups of sampled data that follow are released and sent to speech recognition. Each time a group of data is released, this value is subtracted by 1 until it reaches 0, and then no more data is released. Among them, the sampling point groups above the release threshold will not consume the release volume, and the release volume will be restored to the default value. The sampling point groups below the release threshold will consume the release volume one by one until it is consumed. Then the release of audio sampling data will be stopped and the human voice threshold screening process will be entered again.

[0042] The purpose of processing the received audio repeatedly in this way is to ensure the correctness of the recognized content and achieve smooth near-field interaction free from interference from ambient noise and far-field sound sources.

[0043] It should be noted that the innovation of the present invention is that: combining live face recognition technology, it ensures that a "real person" faces the machine before human-machine voice interaction, avoiding the situation of unmanned interaction or "non-real person" interaction. At the same time, it has the ability to filter out environmental noise and far-field (3 meters away) human voices, and only retain the human voices of face-to-face interaction (within 2 meters), ensuring that the voice of the current face-to-face interaction with the machine is fully retained, improving the accuracy of speech recognition (ASR), and improving the human-machine face-to-face voice interaction experience.

[0044] In order to better implement the human-computer face-to-face interaction method based on voice in the embodiment of the present invention, based on the human-computer face-to-face interaction method based on voice, correspondingly, Figure 3 As shown, the embodiment of the present invention further provides a human-computer face-to-face interaction device 300 based on speech, comprising: A real person recognition module 301 is used to perform real person recognition on a target at a preset distance from the voice robot; The audio processing module 302 is used to perform noise reduction and enhancement processing on the collected voice to obtain pre-processed audio when the target is confirmed to be a real person; The release identification module 303 is used to intercept the pre-processed audio at the quantitative sampling frequency to obtain the sampled audio, calculate the audio intensity of the sampled audio, and perform release identification on the sampled audio based on the audio intensity threshold; The sound receiving module 304 is used to receive and process the sampled audio that meets the release identification condition to identify the audio content.

[0045] It should be noted here that the voice-based human-computer face-to-face interaction device 300 provided in the above embodiment can implement the technical solutions described in the above method embodiments. The specific implementation principles of the above modules or units can refer to the corresponding contents in the above method embodiments, which will not be repeated here.

[0046] like Figure 4As shown, based on the above-mentioned voice-based human-computer face-to-face interaction method, the present invention also provides an electronic device 400. The electronic device 400 includes a processor 401, a memory 402 and a display 403. Figure 4 Only some components of the electronic device 400 are shown, but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0047] In some embodiments, the memory 402 may be an internal storage unit of the electronic device 400, such as a hard disk or memory of the electronic device 400. In other embodiments, the memory 402 may also be an external storage device of the electronic device 400, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc., equipped on the electronic device 400.

[0048] Furthermore, the memory 402 may include both an internal storage unit of the electronic device 400 and an external storage device. The memory 402 is used to store application software installed in the electronic device 400 and various data. In some embodiments, the processor 401 may be a central processing unit (CPU), a microprocessor or other data processing chip, used to run the program code or process data stored in the memory 402, such as the voice-based human-computer face-to-face interaction method of the present invention.

[0049] In some embodiments, the display 403 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, etc. The display 403 is used to display information on the electronic device 400 and to display a visual user interface. The components 401-403 of the electronic device 400 communicate with each other via a system bus.

[0050] In one embodiment, when the processor 401 executes the voice-based human-computer face-to-face interaction program 404 in the memory 402, the following steps may be implemented: Perform real person recognition on targets at a preset distance from the voice robot; When the target is confirmed to be a real person, the collected voice is subjected to noise reduction and enhancement processing to obtain pre-processed audio; Intercepting the preprocessed audio at the quantitative sampling frequency to obtain sampled audio, calculating the audio intensity of the sampled audio, and performing release identification on the sampled audio based on the audio intensity threshold; The sampled audio that meets the release identification conditions is collected and processed to identify the audio content.

[0051] It should be understood that: when the processor 402 executes the voice-based human-computer face-to-face interaction program 404 in the memory 401, in addition to the above functions, other functions can also be implemented. For details, please refer to the description of the corresponding method embodiment above.

[0052] Furthermore, the embodiment of the present invention does not specifically limit the type of the electronic device 400 mentioned, and the electronic device 400 may be a portable electronic device such as a mobile phone, a tablet computer, a personal digital assistant (PDA), a wearable device, a laptop computer, etc. Exemplary embodiments of portable electronic devices include but are not limited to portable electronic devices equipped with iOS, Android, Microsoft or other operating systems. The above-mentioned portable electronic devices may also be other portable electronic devices, such as a laptop computer with a touch-sensitive surface (e.g., a touch panel). It should also be understood that in some other embodiments of the present invention, the electronic device 400 may not be a portable electronic device, but a desktop computer with a touch-sensitive surface (e.g., a touch panel).

[0053] Accordingly, an embodiment of the present application also provides a computer-readable storage medium, which is used to store computer-readable programs or instructions. When the program or instructions are executed by a processor, the method steps or functions provided by the above-mentioned method embodiments can be implemented.

[0054] Those skilled in the art will appreciate that all or part of the processes of the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, wherein the computer-readable storage medium is a disk, an optical disk, a read-only storage memory, or a random access memory, etc.

[0055] The above is a detailed introduction to the voice-based human-computer face-to-face interaction method, device, electronic device and storage medium provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A human-computer face-to-face interaction method based on speech, characterized in that: include: Perform real person recognition on targets at a preset distance from the voice robot; When the target is confirmed to be a real person, the collected voice is subjected to noise reduction and enhancement processing to obtain pre-processed audio; Intercepting the preprocessed audio at the quantitative sampling frequency to obtain sampled audio, calculating the audio intensity of the sampled audio, and performing release identification on the sampled audio based on the audio intensity threshold; The sampled audio that meets the release identification conditions is collected and processed to identify the audio content.

2. The human-computer face-to-face interaction method based on voice according to claim 1, characterized in that: The real person recognition of the target at a preset distance from the voice robot includes: Based on the face recognition model, real person recognition is performed on the target at a preset distance from the voice robot; and / or; Based on binocular live anti-counterfeiting detection technology, real-person recognition is performed on targets at a preset distance from the voice robot.

3. The human-computer face-to-face interaction method based on voice according to claim 2, characterized in that: Based on the face recognition model, real person recognition is performed on the target at a preset distance from the voice robot, including: Based on the Adaboost algorithm as a benchmark model of the face recognition model, multiple weak classifiers are combined into a strong classifier, and the face recognition model is obtained by weighting and repeatedly training the data set; Real person recognition based on face recognition model.

4. The human-computer face-to-face interaction method based on speech according to claim 2, characterized in that: Based on binocular live anti-counterfeiting detection technology, real-person recognition is performed on targets at a preset distance from the voice robot, including: Adopt visible light plus near infrared optoelectronic integrated face liveness detection technology as binocular liveness anti-counterfeiting detection technology; The spectral information reflected by facial skin under different lighting conditions is analyzed and classified, and the heterogeneous facial images are associated and judged to distinguish the real facial skin.

5. The human-computer face-to-face interaction method based on speech according to claim 1, characterized in that: Perform noise reduction on the collected speech, including: Perform noise reduction processing on the collected speech based on the audio noise reduction model; The audio noise reduction model is a new architecture model composed of a convolutional cyclic codec obtained by adding a cyclic layer on the basis of a convolutional codec architecture and introducing a feedforward sequence memory network.

6. The human-computer face-to-face interaction method based on speech according to claim 1, characterized in that: The collected speech is enhanced, including: the recognized human voice is enhanced.

7. The human-computer face-to-face interaction method based on speech according to claim 1, characterized in that: The calculation formula of the audio intensity of the sampled audio is: ； in, Indicates the audio intensity of the sampled audio. is the RMS value of the audio sampling point amplitude, The maximum amplitude of the audio sampling point.

8. The human-computer face-to-face interaction method based on speech according to claim 1, characterized in that: The step of releasing and identifying the sampled audio based on the audio intensity threshold includes: Set release threshold and release quantity, which are obtained based on actual equipment testing and use; Based on the release volume, multiple groups of sampled data that follow are released and sent to speech recognition. Each time a group of data is released, this value is subtracted by 1 until it reaches 0, and then no more data is released. Among them, the sampling point groups above the release threshold will not consume the release volume, and the release volume will be restored to the default value. The sampling point groups below the release threshold will consume the release volume one by one until it is consumed. Then the release of audio sampling data will be stopped and the human voice threshold screening process will be entered again.

9. A voice-based human-computer face-to-face interaction device, characterized in that: include: A real person recognition module, used to perform real person recognition on a target at a preset distance from the voice robot; The audio processing module is used to reduce noise and enhance the collected voice to obtain pre-processed audio when the target is confirmed to be a real person; A release identification module, used to intercept the pre-processed audio at the quantitative sampling frequency to obtain sampled audio, calculate the audio intensity of the sampled audio, and perform release identification on the sampled audio based on the audio intensity threshold; The audio receiving module is used to receive and process the sampled audio that meets the release identification conditions to identify the audio content.

10. An electronic device, characterized in that: comprising a memory and a processor, wherein: The memory is used to store programs; The processor is coupled to the memory and is used to execute the program stored in the memory to implement the steps in the voice-based human-computer face-to-face interaction method described in any one of claims 1 to 8.

Images