Audio talkback processing method and device and electronic equipment
By analyzing the environmental noise characteristics and semantic recognition, and automatically adjusting the configuration information of the intercom device, the problem of low audio intercom quality in noisy environments is solved, and high-quality dialogue and intelligent interactive experience is achieved.
Patent Information
- Application Number
- CN202411093535.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-09
- Publication Date
- 2024-12-03
AI Technical Summary
During the audio intercom, noisy environment interference leads to a decrease in communication efficiency and experience. The prior art cannot effectively and automatically adjust the configuration information of the intercom device, resulting in uncertain adjustment effect.
By collecting environmental audio samples, analyzing noise feature information, performing denoising processing and semantic recognition, determining the target situation mode, automatically adjusting the configuration information of the intercom device, and adapting to different environments and needs.
It improves the quality of intercom communication, realizes an intelligent interactive experience, ensures the clarity and intelligibility of voice under different environmental conditions, reduces the rate of misrecognition, and increases the convenience of user interaction.
Smart Images

Figure CN119068872A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present invention relate to computer technology, and in particular to an audio intercom processing method, device and electronic equipment. Background Art
[0002] With the rapid development of computer technology, intercom equipment can be used to achieve two-way audio intercom. During the audio intercom process, it is easy to be disturbed by various interferences, such as interference from noisy environments, which greatly reduces the communication efficiency and experience. Usually, intercom users can manually adjust the configuration information of the intercom equipment based on their own needs to improve the quality of intercom communication. However, this manual adjustment method is cumbersome and cannot guarantee the adjustment effect, which reduces the intercom interaction experience. Summary of the invention
[0003] The embodiments of the present invention provide an audio intercom processing method, device and electronic device to realize automatic intelligent adjustment of configuration information of an intercom device, thereby improving the intercom communication quality and realizing an intelligent interactive experience.
[0004] In a first aspect, an embodiment of the present invention provides an audio intercom processing method, comprising:
[0005] Collect audio samples from the current environment of the intercom user, and analyze the audio samples to determine the current environmental noise characteristic information;
[0006] Based on the current environmental noise characteristic information, denoising the first audio input by the intercom user to obtain a denoised second audio;
[0007] Performing semantic recognition on the second audio to determine current demand information of the intercom user;
[0008] Based on the current environmental noise characteristic information and the current demand information, determining the target situation mode of the intercom user;
[0009] Based on the target context mode, the configuration information of the intercom device used by the intercom user is adjusted.
[0010] In a second aspect, an embodiment of the present invention further provides an audio intercom processing device, comprising:
[0011] The audio sample analysis module is used to collect audio samples in the current environment of the intercom user and analyze the audio samples to determine the characteristic information of the current environmental noise;
[0012] An audio denoising module, configured to denoise the first audio input by the intercom user based on the current environmental noise feature information to obtain a denoised second audio;
[0013] An audio semantic recognition module, used for performing semantic recognition on the second audio to determine current demand information of the intercom user;
[0014] A target situation mode determination module, used to determine the target situation mode of the intercom user based on the current environmental noise characteristic information and the current demand information;
[0015] The configuration information adjustment module is used to adjust the configuration information of the intercom device used by the intercom user based on the target context mode.
[0016] In a third aspect, an embodiment of the present invention further provides an electronic device, the electronic device comprising:
[0017] one or more processors;
[0018] A memory for storing one or more programs;
[0019] When the one or more programs are executed by the one or more processors, the one or more processors implement the audio intercom processing method provided by any embodiment of the present invention.
[0020] One embodiment of the above invention has the following advantages or beneficial effects:
[0021] By collecting audio samples in the current environment of the intercom user and analyzing the audio samples, the current environmental noise characteristic information is determined; based on the current environmental noise characteristic information, the first audio input by the intercom user is denoised to obtain the denoised second audio; semantic recognition is performed on the denoised second audio to accurately determine the current demand information of the intercom user; based on the current environmental noise characteristic information and the current demand information, the target situation mode of the intercom user can be automatically determined; based on the target situation mode, the configuration information of the intercom device used by the intercom user is automatically and intelligently adjusted, so that the adjusted configuration information of the intercom device is adapted to the target situation mode, so that the intercom user can have a higher quality conversation in the target situation mode, thereby improving the intercom communication quality and realizing an intelligent interactive experience. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0023] Figure 1 is a flow chart of an audio intercom processing method provided by an embodiment of the present invention;
[0024] Figure 2 is a flow chart of another audio intercom processing method provided by an embodiment of the present invention;
[0025] Figure 3 It is a structural schematic diagram of an audio intercom processing device provided by an embodiment of the present invention;
[0026] Figure 4 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0027] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.
[0028] Figure 1 This is a flowchart of an audio intercom processing method provided by an embodiment of the present invention. This embodiment is applicable to the case where the configuration information of the intercom device is intelligently adjusted during the audio intercom process. The method can be executed by an audio intercom processing device, which can be implemented by software and / or hardware and integrated into an electronic device. Figure 1 As shown, the method specifically comprises the following steps:
[0029] S110: Collect audio samples from the current environment of the intercom user, and analyze the audio samples to determine characteristic information of the current environmental noise.
[0030] Among them, the intercom user may refer to the user of the current voice call. The audio sample refers to the audio signal collected from the environment. The current environmental noise feature information may be used to characterize the background noise characteristics of the environment in which the intercom user is currently located. Exemplarily, the current environmental noise feature information may include but is not limited to: the current environmental noise frequency distribution, the current environmental noise type and the current environmental noise intensity. Among them, the current environmental noise frequency distribution refers to the frequency band in which the current environmental noise signal is located. The current environmental noise type may refer to white noise or mechanical noise, etc. Different environmental noise types correspond to different environmental noise frequency distributions. The current environmental noise intensity refers to the volume of the current environmental noise.
[0031] Specifically, the audio samples in the current environment of the intercom user can be collected in real time or periodically, and the real-time spectrum analysis of the audio samples can be performed to identify the characteristic information of the current environmental noise.
[0032] S120: Based on the current environmental noise feature information, denoise the first audio input by the intercom user to obtain a denoised second audio.
[0033] The first audio refers to an audio signal input by the intercom user through the intercom device, and the second audio refers to an audio signal obtained after removing the environmental noise in the first audio.
[0034] Specifically, based on the identified current environmental noise feature information, the audio noise reduction parameters are intelligently and dynamically adjusted, so as to denoise the first audio currently input by the intercom user, thereby suppressing the environmental noise and obtaining a clearer second audio. Exemplarily, the denoised second audio is transmitted, thereby ensuring the clarity and intelligibility of the voice under various environmental conditions, and achieving environmental adaptation.
[0035] Exemplarily, step S120 may include: adjusting the filter parameters in the adaptive filter group based on the current environmental noise characteristic information; filtering and denoising the first audio input by the intercom user based on the adjusted filter parameters to obtain the denoised second audio.
[0036] Specifically, the filtering parameters in the adaptive filter group can be dynamically adjusted based on the current ambient noise frequency distribution and the current ambient noise intensity, and based on the adjusted filtering parameters, the noise within a specific frequency range and intensity in the first audio input by the intercom user can be suppressed, thereby achieving filtering denoising, optimizing the voice signal, and obtaining a clearer second audio.
[0037] S130: Perform semantic recognition on the second audio to determine current demand information of the intercom user.
[0038] Among them, the current demand information refers to what the intercom user currently needs to do, such as making an emergency call. The current demand information can represent the intention information of the intercom user. Specifically, by utilizing natural speech processing technology, the second audio can be semantically recognized to understand the purpose or request of the intercom user, so that the current demand information of the intercom user can be extracted. For example, after obtaining the current demand information of the intercom user, a response that is more in line with the user's intention can be automatically provided to the intercom user based on the current demand information, such as searching for information, executing a command, providing guidance, or answering questions. After completing the task, appropriate feedback or response can also be generated and conveyed to the intercom user in voice or text form.
[0039] Exemplarily, step S130 may include: performing speech-to-text conversion on the second audio to obtain text information of the second audio; performing text semantic recognition on the text information of the second audio based on a natural language processing model to determine current demand information of the intercom user.
[0040] Specifically, a deep neural network model can be used to convert the second audio from speech to text to obtain the text information of the second audio, thereby improving the recognition accuracy. The text information of the second audio is input into the natural language processing model for semantic recognition and analysis of the text to obtain the current demand information of the intercom user. By performing semantic recognition on the denoised second audio, environmental noise interference can be avoided, and the current demand information can be extracted more accurately, thereby improving the accuracy of the configuration information adjustment of the intercom device.
[0041] S140: Determine the target situation mode of the intercom user based on the current environmental noise characteristic information and the current demand information.
[0042] Among them, the target situation mode refers to the situation mode that the intercom user is currently in. Based on different environmental noise characteristic information and different demand information, it can be divided into multiple situation modes, and each situation mode corresponds to different environmental noise characteristic information and / or different demand information. For example, if the current demand information is empty information and the current environmental noise intensity is greater than the preset noise intensity, the target situation mode is determined to be a high noise mode. Each situation mode can be named based on different environmental noise characteristic information and demand information, which is not limited here.
[0043] Advantageously, the environmental noise characteristic information and demand information in each pre-set situation mode can be matched with the current environmental noise characteristic information and the current demand information respectively, and the successfully matched situation mode is determined as the target situation mode in which the intercom user is currently located.
[0044] Exemplarily, each situation mode may include at least one situation sub-mode. Each situation sub-mode may be determined based on the current environment noise intensity or the current demand information in the current environment noise characteristic information. For example, if the current environment noise intensity is less than the first noise intensity, it is determined that the target situation mode in which the intercom user is located includes a low noise sub-mode; if the current environment noise intensity is greater than the second noise intensity, it is determined that the target situation mode in which the intercom user is located includes a high noise sub-mode; if there is emergency information in the current demand information, it is determined that the target situation mode in which the intercom user is located includes an emergency sub-mode. Among them, the first noise intensity is less than the second noise intensity. If the current environment noise intensity is between the first noise intensity and the second noise intensity, it indicates that there is no need to adjust the dialogue device based on the current noise, and the target situation mode determined at this time does not include a low noise sub-mode or a high noise sub-mode. Emergency information refers to instructions that the intercom user needs to handle urgently, such as alarm instructions. By combining all sub-modes, multiple different situation modes can be obtained.
[0045] S150: Based on the target scenario mode, adjust the configuration information of the intercom device used by the intercom user.
[0046] The configuration information of the intercom device refers to the system configuration information in the intercom device that can adjust the intercom interaction strategy. For example, the configuration information of the intercom device may include but is not limited to: the configuration information of the microphone in the intercom device, the configuration information of the speaker, the output control information of the reminder instruction, and the opening control information of the function button.
[0047] Specifically, based on the target context mode, the configuration information of the intercom device used by the intercom user is automatically adjusted to adapt to the target context mode, thereby automatically adjusting the interaction strategy of the intercom device, improving the quality of voice communication, reducing the error recognition rate, and increasing the convenience and intelligence level of user interaction.
[0048] Exemplarily, in the case where the target scenario mode includes at least one scenario sub-mode, in response to the low noise sub-mode in the target scenario mode, the microphone gain of the intercom device used by the intercom user is reduced to prevent excessive amplification of noise. In response to the high noise sub-mode in the target scenario mode, the microphone gain of the intercom device used by the intercom user is increased and the speaker output is fine-tuned, and the intercom user is reminded to repeat the instruction or suggested to switch to the text input mode. By increasing the microphone gain based on the intensity of the ambient noise, it can be ensured that the voice can still be clearly captured in a high noise environment. By fine-tuning the speaker output, the clarity of voice transmission can be guaranteed and signal distortion can be avoided. By dynamically adjusting the microphone gain and speaker output, it can be ensured that the audio signal is not distorted, and voice enhancement is achieved, thereby ensuring the clarity and intelligibility of the voice under various environmental conditions. By automatically reminding the user that voice recognition may be affected due to increased ambient noise, it is recommended that the user repeat the instruction or switch to the text input mode, so as to avoid communication barriers in a noisy environment, and guide the user to adopt a more effective communication method through intelligent prompts and suggestions. In response to the emergency sub-mode in the target scenario mode, the intercom device used by the intercom user is controlled to prioritize the alarm instruction and automatically turn on the location tracking function. In an emergency, the intercom device will immediately prioritize the alarm command to ensure the rapid transmission of emergency information, and can automatically turn on the location tracking function to help rescuers quickly locate. It can also provide a one-click call for help function to simplify the operating procedures in emergency situations.
[0049] For example, when the intercom environment changes, such as from a quiet environment to a noisy environment, the intercom device will automatically adjust parameters such as microphone sensitivity to ensure that voice clarity is not affected.
[0050] The technical solution of this embodiment determines the current environmental noise characteristic information by collecting audio samples in the environment in which the intercom user is currently located and analyzing the audio samples; based on the current environmental noise characteristic information, denoising is performed on the first audio input by the intercom user to obtain the denoised second audio; semantic recognition is performed on the denoised second audio to accurately determine the current demand information of the intercom user; based on the current environmental noise characteristic information and the current demand information, the target situation mode in which the intercom user is located can be automatically determined; based on the target situation mode, the configuration information of the intercom device used by the intercom user is automatically and intelligently adjusted, so that the adjusted configuration information of the intercom device is adapted to the target situation mode, so that the intercom user can have a higher quality conversation in the target situation mode, thereby improving the intercom communication quality and realizing an intelligent interactive experience.
[0051] Based on the above technical solution, the method may also include: obtaining the historical conversation audio content of the intercom user; predicting the potential needs of the intercom user based on the historical conversation audio content and the content of interest to the intercom user, and providing feedback response to the intercom user based on the predicted potential needs.
[0052] Specifically, the historical conversation audio content and historical interaction behavior of the intercom user can be obtained from the context management module, and the content of interest to the intercom user can be determined based on the historical interaction behavior of the intercom user. Modeling is performed based on the historical conversation audio content and the content of interest to the intercom user, and the reconstructed model is used to predict the current potential needs of the intercom user, and based on the predicted potential needs, feedback responses are automatically given to the intercom user, thereby ensuring that the model understands the context in a continuous conversation and provides a coherent and reasonable response, thereby providing the user with a coherent and personally tailored interactive experience.
[0053] Figure 2 This is a flow chart of another audio intercom processing method provided by an embodiment of the present invention. Based on the above embodiments, this embodiment optimizes the step of "determining the target situation mode of the intercom user based on the current environmental noise characteristic information and the current demand information". The explanations of the terms that are the same or corresponding to the above embodiments are not repeated here.
[0054] See also Figure 2 Another audio intercom processing method provided in this embodiment specifically includes the following steps:
[0055] S210: Collect audio samples from the current environment of the intercom user, and analyze the audio samples to determine characteristic information of the current environmental noise.
[0056] S220: Based on the current environmental noise feature information, denoise the first audio input by the intercom user to obtain a denoised second audio.
[0057] S230: Perform semantic recognition on the second audio to determine current demand information of the intercom user.
[0058] S240: Collect sensor data through sensors installed on the intercom device used by the intercom user.
[0059] The sensor is a sensor for sensing changes in the surrounding environment. The sensor may be of one or more types. For example, the sensor may include but is not limited to a light sensor and a motion sensor. The light sensor may be used to collect the light intensity in the environment where the intercom user is located. The motion sensor may be used to collect the moving speed of the intercom user. The light sensor may be used to determine whether the intercom user is indoors or outdoors. The motion sensor may be used to determine whether the intercom user is stationary or moving, etc.
[0060] Specifically, by collecting sensor data using sensors installed on intercom devices, more dimensional environmental information can be obtained, thereby enhancing the ability to perceive environmental changes and further improving the accuracy of situation recognition.
[0061] S250: Determine the target situation mode of the intercom user based on the sensor data, the current environmental noise characteristic information and the current demand information.
[0062] Specifically, in addition to considering the current environmental noise characteristic information and current demand information, the collected sensor data can also be taken into account, so that the surrounding environmental information can be more comprehensively perceived, and the target situation mode of the intercom user can be determined more accurately.
[0063] Exemplarily, step S250 may include: determining the current ambient brightness of the intercom user based on light sensor data collected by the light sensor; determining the current moving speed of the intercom user based on motion sensor data collected by the motion sensor; and determining the target situation mode of the intercom user based on the current ambient brightness, current moving speed, current ambient noise intensity in the current ambient noise characteristic information, and current demand information.
[0064] Specifically, the current light intensity in the light sensor data collected by the light sensor can be determined as the current ambient brightness of the intercom user. The specific current moving speed of the intercom user is obtained from the motion sensor data collected by the motion sensor. The ambient brightness, moving speed, ambient noise characteristic information and demand information in each pre-set scenario mode can be matched with the current ambient brightness, current moving speed, current ambient noise characteristic information and current demand information, respectively, and the successfully matched scenario mode is determined as the target scenario mode that the intercom user is currently in.
[0065] Exemplarily, each situation mode may include at least one situation sub-mode. Each situation sub-mode may be determined based on the current environment brightness, the current moving speed, the current environment noise intensity, or the current demand information. For example, based on the current environment brightness, the current moving speed, the current environment noise intensity in the current environment noise feature information, and the current demand information, determining the target situation mode that the intercom user is in may include:
[0066] If the current environment brightness is less than the preset dark brightness, it is determined that the target situation mode of the intercom user includes a dark sub-mode; if the current environment brightness is greater than the preset bright brightness, it is determined that the target situation mode of the intercom user includes a bright sub-mode; if the current moving speed is greater than the preset speed, it is determined that the target situation mode of the intercom user includes a moving sub-mode; if the current environment noise intensity in the current environment noise characteristic information is less than the first noise intensity, it is determined that the target situation mode of the intercom user includes a low noise sub-mode; if the current environment noise intensity is greater than the second noise intensity, it is determined that the target situation mode of the intercom user includes a high noise sub-mode; if there is emergency information in the current demand information, it is determined that the target situation mode of the intercom user includes an emergency sub-mode.
[0067] Specifically, if the current environment brightness is less than the preset dark brightness, it indicates that it may be at night or indoors, and the target situation mode of the intercom user is determined to include the dark sub-mode; if the current environment brightness is greater than the preset bright brightness, it indicates that it may be during the day or outdoors, and the target situation mode of the intercom user is determined to include the bright sub-mode. If the current moving speed is greater than the preset speed, it indicates that the user is currently in a moving state, and the target situation mode of the intercom user is determined to include the moving sub-mode. By combining all sub-modes, multiple different situation modes can be obtained.
[0068] S260: Based on the target scenario mode, adjust the configuration information of the intercom device used by the intercom user.
[0069] Specifically, based on the identified more comprehensive target situation mode, the configuration information of the intercom device used by the intercom user is automatically adjusted to adapt to the target situation mode, so that the interaction strategy of the intercom device can be adjusted more comprehensively, the quality of voice communication can be improved, the error recognition rate can be reduced, and the convenience and intelligence level of user interaction can be increased.
[0070] Exemplarily, step S260 may include: in response to the dark sub-mode or the bright sub-mode in the target scenario mode, adjusting the screen brightness of the intercom device used by the intercom user; in response to the mobile sub-mode in the target scenario mode, reducing the microphone sensitivity of the intercom device used by the intercom user; in response to the low noise sub-mode in the target scenario mode, reducing the microphone gain of the intercom device used by the intercom user; in response to the high noise sub-mode in the target scenario mode, increasing the microphone gain of the intercom device used by the intercom user and fine-tuning the speaker output, as well as reminding the intercom user to repeat the instructions or suggesting switching to text input mode; in response to the emergency sub-mode in the target scenario mode, controlling the intercom device used by the intercom user to prioritize alarm instructions and automatically turn on the location tracking function.
[0071] Specifically, in the case where the target scenario mode includes at least one scenario sub-mode, in response to the dark sub-mode or the bright sub-mode in the target scenario mode, the screen brightness of the intercom device used by the intercom user is automatically adjusted to a suitable brightness to save power. In response to the mobile sub-mode in the target scenario mode, the microphone sensitivity of the intercom device used by the intercom user is reduced to reduce wind noise interference. In response to the low noise sub-mode in the target scenario mode, the microphone gain of the intercom device used by the intercom user is reduced to prevent excessive noise amplification. In the high noise sub-mode, by increasing the microphone gain, it can be ensured that the voice can still be clearly captured in a high noise environment. By fine-tuning the speaker output, the clarity of voice transmission can be guaranteed and signal distortion can be avoided. By dynamically adjusting the microphone gain and the speaker output, it can be ensured that the audio signal is not distorted, and voice enhancement is achieved, thereby ensuring the clarity and intelligibility of the voice under various environmental conditions. By automatically reminding the user that voice recognition may be affected due to increased environmental noise, it is recommended that the user repeat the instruction or switch to the text input mode, so as to avoid communication barriers in a noisy environment, and guide the user to adopt a more effective communication method through intelligent prompts and suggestions, thereby avoiding misreading of instructions or communication failure caused by noise interference. In an emergency, the intercom device will immediately prioritize the alarm command to ensure the rapid transmission and timely response of emergency information, and can automatically turn on the location tracking function to help rescuers quickly locate. It can also provide a one-click call for help function to simplify the operating procedures in emergency situations.
[0072] The technical solution of this embodiment integrates sensors on the intercom device used by the intercom user, and utilizes the collected sensor data to more comprehensively perceive environmental changes, thereby more accurately determining the target situation mode of the intercom user, and automatically adjusting the configuration information of the intercom device to the optimal state based on different situation modes, thereby further improving the intelligence of the interaction and the user experience.
[0073] The following is an embodiment of an audio intercom processing device provided in an embodiment of the present invention. The device and the audio intercom processing methods of the above-mentioned embodiments belong to the same inventive concept. For details not described in detail in the embodiment of the audio intercom processing device, reference can be made to the embodiment of the above-mentioned audio intercom processing method.
[0074] Figure 3 This is a schematic diagram of the structure of an audio intercom processing device provided by an embodiment of the present invention. This embodiment is applicable to the case where the configuration information of the intercom device is intelligently adjusted during the audio intercom process. Figure 3 As shown, the device specifically includes: an audio sample analysis module 310 , an audio denoising module 320 , an audio semantic recognition module 330 , a target context mode determination module 340 and a configuration information adjustment module 350 .
[0075] Among them, the audio sample analysis module 310 is used to collect audio samples in the current environment of the intercom user, and analyze the audio samples to determine the current environmental noise characteristic information; the audio denoising module 320 is used to denoise the first audio input by the intercom user based on the current environmental noise characteristic information to obtain the denoised second audio; the audio semantic recognition module 330 is used to perform semantic recognition on the second audio to determine the current demand information of the intercom user; the target situation mode determination module 340 is used to determine the target situation mode of the intercom user based on the current environmental noise characteristic information and the current demand information; the configuration information adjustment module 350 is used to adjust the configuration information of the intercom device used by the intercom user based on the target situation mode.
[0076] The technical solution of this embodiment determines the current environmental noise characteristic information by collecting audio samples in the environment in which the intercom user is currently located and analyzing the audio samples; based on the current environmental noise characteristic information, denoising is performed on the first audio input by the intercom user to obtain the denoised second audio; semantic recognition is performed on the denoised second audio to accurately determine the current demand information of the intercom user; based on the current environmental noise characteristic information and the current demand information, the target situation mode in which the intercom user is located can be automatically determined; based on the target situation mode, the configuration information of the intercom device used by the intercom user is automatically and intelligently adjusted, so that the adjusted configuration information of the intercom device is adapted to the target situation mode, so that the intercom user can have a higher quality conversation in the target situation mode, thereby improving the intercom communication quality and realizing an intelligent interactive experience.
[0077] Optionally, the current environmental noise characteristic information includes: current environmental noise frequency distribution, current environmental noise type and current environmental noise intensity;
[0078] The audio denoising module 320 is specifically used to: adjust the filter parameters in the adaptive filter group based on the current environmental noise characteristic information; and filter and denoise the first audio input by the intercom user based on the adjusted filter parameters to obtain the denoised second audio.
[0079] Optionally, the audio semantic recognition module 330 is specifically used to: convert the second audio into speech to obtain text information of the second audio; based on a natural language processing model, perform text semantic recognition on the text information of the second audio to determine the current demand information of the intercom user.
[0080] Optionally, the target context mode determination module 340 includes:
[0081] A sensor data acquisition unit, used to collect sensor data through a sensor installed on an intercom device used by the intercom user;
[0082] The target situation mode determination unit is used to determine the target situation mode of the intercom user based on the sensor data, the current environmental noise characteristic information and the current demand information.
[0083] Optionally, the sensor includes a light sensor and a motion sensor;
[0084] The target situation model determination unit includes:
[0085] A current environment brightness determination subunit, used to determine the current environment brightness of the intercom user based on the light sensor data collected by the light sensor;
[0086] A current moving speed determining subunit, configured to determine the current moving speed of the intercom user based on the motion sensor data collected by the motion sensor;
[0087] The target situation mode determination subunit is used to determine the target situation mode of the intercom user based on the current environment brightness, the current moving speed, the current environment noise intensity in the current environment noise characteristic information and the current demand information.
[0088] Optionally, the target context mode determination subunit is specifically configured to:
[0089] If the current environment brightness is less than the preset dark brightness, it is determined that the target situation mode in which the intercom user is located includes a dark sub-mode; if the current environment brightness is greater than the preset bright brightness, it is determined that the target situation mode in which the intercom user is located includes a bright sub-mode; if the current moving speed is greater than the preset speed, it is determined that the target situation mode in which the intercom user is located includes a moving sub-mode; if the current environment noise intensity in the current environment noise characteristic information is less than the first noise intensity, it is determined that the target situation mode in which the intercom user is located includes a low noise sub-mode; if the current environment noise intensity is greater than the second noise intensity, it is determined that the target situation mode in which the intercom user is located includes a high noise sub-mode; if there is emergency information in the current demand information, it is determined that the target situation mode in which the intercom user is located includes an emergency sub-mode.
[0090] Optionally, the configuration information adjustment module 350 is specifically configured to:
[0091] In response to the dark sub-mode or the bright sub-mode in the target scenario mode, the screen brightness of the intercom device used by the intercom user is adjusted; in response to the moving sub-mode in the target scenario mode, the microphone sensitivity of the intercom device used by the intercom user is reduced; in response to the low noise sub-mode in the target scenario mode, the microphone gain of the intercom device used by the intercom user is reduced; in response to the high noise sub-mode in the target scenario mode, the microphone gain of the intercom device used by the intercom user is increased and the speaker output is fine-tuned, and the intercom user is reminded to repeat the instructions or suggested to switch to text input mode; in response to the emergency sub-mode in the target scenario mode, the intercom device used by the intercom user is controlled to give priority to alarm instructions and automatically turn on the location tracking function.
[0092] Optionally, the device further comprises:
[0093] The potential demand prediction and response module is used to obtain the historical conversation audio content of the intercom user; based on the historical conversation audio content and the content of interest of the intercom user, predict the potential needs of the intercom user, and provide feedback response to the intercom user based on the predicted potential needs.
[0094] The audio intercom processing device provided in the embodiment of the present invention can execute the audio intercom processing method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of executing the audio intercom processing method.
[0095] It is worth noting that in the embodiment of the above-mentioned audio intercom processing device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be realized; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.
[0096] Figure 4 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. Figure 4 A block diagram of an exemplary electronic device 12 suitable for implementing embodiments of the present invention is shown. The electronic device 12 may be referred to as an intercom device. Figure 4 The electronic device 12 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.
[0097] like Figure 4 As shown, the electronic device 12 is in the form of a general purpose computing device. The components of the electronic device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 that connects various system components (including the system memory 28 and the processing unit 16).
[0098] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. By way of example, these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.
[0099] The electronic device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device 12, including volatile and non-volatile media, removable and non-removable media.
[0100] The system memory 28 may include computer system readable media in the form of volatile memory, such as random access memory 30 (RAM) and / or cache memory 32. The electronic device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Figure 4 not shown, usually called a "hard drive"). Although Figure 4Not shown in the figure, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The system memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present invention.
[0101] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. Program modules 42 generally perform the functions and / or methods of the embodiments described herein.
[0102] The electronic device 12 may also communicate with one or more external devices 14 (e.g., keyboards, pointing devices, displays 24, etc.), may communicate with one or more devices that enable a user to interact with the electronic device 12, and / or may communicate with any device that enables the electronic device 12 to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed via an input / output (I / O) interface 22. Furthermore, the electronic device 12 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 20. As shown, the network adapter 20 communicates with other modules of the electronic device 12 via a bus 18. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.
[0103] The processing unit 16 executes various functional applications and data processing by running the program stored in the system memory 28, for example, implementing the steps of an audio intercom processing method provided by any embodiment of the present invention, the method comprising:
[0104] Collect audio samples from the current environment of the intercom user, and analyze the audio samples to determine the current environmental noise characteristic information;
[0105] Based on the current environmental noise characteristic information, denoising the first audio input by the intercom user to obtain a denoised second audio;
[0106] Performing semantic recognition on the second audio to determine current demand information of the intercom user;
[0107] Based on the current environmental noise characteristic information and the current demand information, determining the target situation mode of the intercom user;
[0108] Based on the target context mode, the configuration information of the intercom device used by the intercom user is adjusted.
[0109] Of course, those skilled in the art can understand that the processor can also implement the technical solution of the audio intercom processing method provided by any embodiment of the present invention.
[0110] It should be understood by those skilled in the art that the modules or steps of the present invention described above can be implemented by a general-purpose computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, optionally, they can be implemented by a program code executable by a computer device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.
[0111] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims
1. An audio intercom processing method, characterized in that: include: Collect audio samples from the current environment of the intercom user, and analyze the audio samples to determine the current environmental noise characteristic information; Based on the current environmental noise characteristic information, denoising the first audio input by the intercom user to obtain a denoised second audio; Performing semantic recognition on the second audio to determine current demand information of the intercom user; Based on the current environmental noise characteristic information and the current demand information, determining the target situation mode of the intercom user; Based on the target context mode, the configuration information of the intercom device used by the intercom user is adjusted.
2. The method according to claim 1, characterized in that The current environmental noise characteristic information includes: the current environmental noise frequency distribution, the current environmental noise type and the current environmental noise intensity; The step of performing denoising on the first audio input by the intercom user based on the current environmental noise feature information to obtain the denoised second audio includes: Based on the current environmental noise characteristic information, adjusting the filtering parameters in the adaptive filter group; Based on the adjusted filtering parameters, the first audio input by the intercom user is filtered and denoised to obtain a denoised second audio.
3. The method according to claim 1, characterized in that Performing semantic recognition on the second audio to determine current demand information of the intercom user includes: Convert the second audio into text to obtain text information of the second audio; Based on a natural language processing model, text semantic recognition is performed on the text information of the second audio to determine current demand information of the intercom user.
4. The method according to claim 1, characterized in that Determining the target situation mode of the intercom user based on the current environmental noise characteristic information and the current demand information includes: Collecting sensor data through sensors installed on the intercom device used by the intercom user; Based on the sensor data, the current environmental noise characteristic information and the current demand information, a target situation mode in which the intercom user is located is determined.
5. The method according to claim 4, characterized in that The sensors include light sensors and motion sensors; The determining, based on the sensor data, the current environmental noise characteristic information and the current demand information, the target situation mode in which the intercom user is located comprises: Determine the current brightness of the environment in which the intercom user is located based on the light sensor data collected by the light sensor; Determining the current moving speed of the intercom user based on the motion sensor data collected by the motion sensor; The target situation mode in which the intercom user is located is determined based on the current environment brightness, the current moving speed, the current environment noise intensity in the current environment noise characteristic information, and the current demand information.
6. The method according to claim 5, characterized in that The determining the target situation mode of the intercom user based on the current environment brightness, the current moving speed, the current environment noise intensity in the current environment noise feature information, and the current demand information includes: If the current environment brightness is less than the preset dark brightness, it is determined that the target situation mode of the intercom user includes a dark sub-mode; if the current environment brightness is greater than the preset bright brightness, it is determined that the target situation mode of the intercom user includes a bright sub-mode; If the current moving speed is greater than a preset speed, determining that the target situation mode in which the intercom user is located includes a moving sub-mode; If the current environmental noise intensity in the current environmental noise characteristic information is less than the first noise intensity, it is determined that the target situation mode in which the intercom user is located includes a low noise sub-mode; if the current environmental noise intensity is greater than the second noise intensity, it is determined that the target situation mode in which the intercom user is located includes a high noise sub-mode; If emergency information exists in the current demand information, it is determined that the target situation mode in which the intercom user is located includes an emergency sub-mode.
7. The method according to claim 6, characterized in that The adjusting, based on the target context mode, configuration information of the intercom device used by the intercom user includes: In response to the dark sub-mode or the bright sub-mode in the target context mode, adjusting the screen brightness of the intercom device used by the intercom user; In response to the mobile sub-mode in the target situation mode, reducing the microphone sensitivity of the intercom device used by the intercom user; In response to the low noise sub-mode in the target situation mode, reducing the microphone gain of the intercom device used by the intercom user; In response to the high noise sub-mode in the target situation mode, increasing the microphone gain and fine-tuning the speaker output of the intercom device used by the intercom user, and reminding the intercom user to repeat the instruction or suggesting switching to a text input mode; In response to the emergency sub-mode in the target scenario mode, the intercom device used by the intercom user is controlled to give priority to processing the alarm instruction and automatically start the location tracking function.
8. The method according to any one of claims 1 to 7, characterized in that: The method further comprises: Obtaining the audio content of the historical conversation of the intercom user; Based on the historical conversation audio content and the content of interest to the intercom user, potential needs of the intercom user are predicted, and based on the predicted potential needs, feedback response is given to the intercom user.
9. An audio intercom processing device, characterized in that: include: The audio sample analysis module is used to collect audio samples in the current environment of the intercom user and analyze the audio samples to determine the characteristic information of the current environmental noise; An audio denoising module, configured to denoise the first audio input by the intercom user based on the current environmental noise feature information to obtain a denoised second audio; An audio semantic recognition module, used for performing semantic recognition on the second audio to determine current demand information of the intercom user; A target situation mode determination module, used to determine the target situation mode of the intercom user based on the current environmental noise characteristic information and the current demand information; The configuration information adjustment module is used to adjust the configuration information of the intercom device used by the intercom user based on the target context mode.
10. An electronic device, characterized in that: The electronic device comprises: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the audio intercom processing method as described in any one of claims 1-7.
