Audio information processing method, device and electronic equipment

Abstract

This application proposes an audio information processing method, apparatus, and electronic device. When the electronic device detects that the audio information it acquires contains a first audio that meets specific conditions, in order to eliminate the noise interference of the first audio, the electronic device can extract the first audio contained in the audio information. Based on the relative positional relationship between the sound source of the first audio, the audio playback device, and the audio receiving object, it obtains an initial inverted first audio for the first audio and sends it to the output of the audio playback device. This allows the target inverted first audio transmitted to the audio receiving object to cancel each other out with the first audio, thereby achieving active noise reduction processing of the first audio acquired by the electronic device. This avoids the first audio from causing noise interference to the user of the electronic device and improves the user's experience of using the electronic device.

Description

Technical Field

[0001] The present application mainly relates to the field of audio signal processing, and more specifically to an audio information processing method, device and electronic equipment. Background Art

[0002] At present, during the operation of electronic devices such as laptops, steady-state noise may be collected, such as the electrical noise of the capacitor generated during the operation of low-cost ordinary capacitors usually installed in laptops; the noise generated by the laptop's built-in fan, etc. If the laptop is running in a very quiet environment, or requires a quieter environment, this type of steady-state noise will cause noise interference to the user and reduce the user's experience of the laptop. Summary of the invention

[0003] In view of this, in order to solve the above technical problems, this application proposes the following technical solutions:

[0004] On the one hand, the present application proposes an audio information processing method, the method comprising:

[0005] Get audio information;

[0006] When detecting that the audio information includes a first audio satisfying a specific condition, extracting the first audio included in the audio information;

[0007] Based on the relative position relationship between the sound source of the first audio, the audio playback device, and the audio receiving object, obtaining an initial inverted first audio for the extracted first audio;

[0008] The initial inverted first audio is sent to the audio playback device for output, so that the target inverted first audio transmitted to the audio receiving object and the first audio cancel each other out.

[0009] In some embodiments, when detecting that the audio information includes a first audio that satisfies a specific condition, extracting the first audio included in the audio information includes:

[0010] The first processor detects whether the audio information includes a first audio that meets a specific condition;

[0011] If yes, the first processor sends noise reduction prompt information to the second processor;

[0012] The second processor extracts the first audio included in the audio information;

[0013] In the same period of time, a first power consumption generated by the operation of the first processor is less than a second power consumption generated by the operation of the second processor.

[0014] In some embodiments, detecting that the audio information includes the first audio satisfying a specific condition includes:

[0015] detecting that the audio information contains steady-state noise, and determining the steady-state noise as the first audio; or,

[0016] Acquire steady-state noise and non-steady-state noise contained in the audio information;

[0017] It is determined that the steady-state noise and the non-steady-state noise meet a specific condition, or it is determined that the steady-state noise and the non-steady-state noise meet a specific condition and an abnormal frequency signal exists in the steady-state noise, and the steady-state noise is determined as the first audio.

[0018] In some embodiments, the steady-state noise includes electrical noise generated by the electronic device itself; and determining that the steady-state noise and the non-steady-state noise meet a specific condition includes:

[0019] Acquire a signal strength difference between the steady-state noise and the non-steady-state noise;

[0020] It is determined that the signal strength difference is greater than a steady-state noise reduction threshold, where the steady-state noise reduction threshold is determined based on a signal strength of ambient noise of the electronic device.

[0021] In some embodiments, obtaining the initial inverted first audio for the extracted first audio based on the relative position relationship among the sound source of the first audio, the audio playback device, and the audio receiving object includes:

[0022] Acquire a first sound source position of the first audio, a sound outlet position of an audio playback device, and a sound receiving position of an audio receiving object;

[0023] According to the first sound source position, the sound outlet position and the sound receiving position, the first audio is subjected to inversion processing to obtain an initial inverted first audio;

[0024] The step of sending the initial inverted first audio to an audio playback device for output comprises:

[0025] The initial inverted first audio is mixed with the audio to be played, and the obtained mixed audio is sent to the audio playback device for output.

[0026] In some embodiments, detecting that the audio information includes the first audio satisfying a specific condition includes:

[0027] Detecting that the audio information includes user audio, performing sound source positioning on the user audio to obtain a position of a speaker of the user audio;

[0028] Determining that the positional relationship between the speaker's position and the electronic device meets a specific condition, and determining the user audio as the first audio;

[0029] The method further comprises:

[0030] Based on the first audio, authenticating the speaker of the first audio;

[0031] If the identity authentication is passed, the voice recognition engine of the electronic device is awakened, and the voice recognition engine executes the voice command contained in the first audio.

[0032] On the other hand, the present application also proposes an audio information processing device, the device comprising:

[0033] An audio information acquisition module, used to acquire audio information;

[0034] A first audio extraction module, configured to extract the first audio contained in the audio information when detecting that the audio information contains the first audio that satisfies a specific condition;

[0035] an inversion processing module, configured to obtain an initial inverted first audio for the extracted first audio based on a relative position relationship between a sound source of the first audio, an audio playback device, and an audio receiving object;

[0036] The audio transmission module is used to send the initial inverted first audio to the audio playback device for output, so that the target inverted first audio transmitted to the audio receiving object and the first audio cancel each other out.

[0037] On the other hand, the present application also proposes an electronic device, the electronic device comprising a body, and electronic components, an audio acquisition device, an audio playback device, and an audio processing device deployed in the body, wherein:

[0038] The audio collection device is used to collect audio information when the electronic device is running;

[0039] The audio processing device is used to call and execute a first program to implement the various steps of the audio information processing method as described above, and the first program is a program for implementing the audio information processing method as described above.

[0040] In some embodiments, the audio processing device comprises:

[0041] A first processor is used to obtain audio information, detect that the audio information contains a first audio that meets a specific condition, and generate noise reduction prompt information;

[0042] a second processor, configured to receive the noise reduction prompt information sent by the first processor, extract the first audio contained in the audio information, obtain an initial inverted first audio for the extracted first audio based on the relative position relationship between the sound source of the first audio, the audio playback device, and the audio receiving object, and send the initial inverted first audio to the audio playback device for output, so that the target inverted first audio transmitted to the audio receiving object and the first audio cancel each other out;

[0043] In the same period of time, a first power consumption generated by the operation of the first processor is less than a second power consumption generated by the operation of the second processor.

[0044] In some embodiments, the first processor is deployed in the audio acquisition device;

[0045] The audio collection device includes a plurality of audio collectors deployed in an array;

[0046] The electronic component includes a capacitor element, and during the operation of the electronic device, the electronic component can generate steady-state noise.

[0047] It can be seen that the present application proposes an audio information processing method, device and electronic device. When it is detected that the audio information obtained by the electronic device contains a first audio that meets specific conditions, in order to eliminate the noise interference of the first audio, the electronic device can extract the first audio contained in the audio information, and thus obtain the initial inverted first audio for the first audio based on the relative position relationship between the sound source of the first audio, the audio playback device and the audio receiving object, and send it to the audio playback device for output, so that the target inverted first audio transmitted to the audio receiving object and the first audio can cancel each other, thereby realizing active noise reduction processing of the first audio collected by the electronic device, avoiding the first audio from causing noise interference to the user using the electronic device, and improving the user experience of the electronic device. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0049] Figure 1 A schematic diagram of an optional example of an electronic device applicable to the audio information processing method and device proposed in the present application;

[0050] Figure 2A schematic diagram of the hardware structure of another optional example of an electronic device applicable to the audio information processing method and device proposed in the present application;

[0051] Figure 3 A schematic structural diagram of another optional example of an electronic device applicable to the audio information processing method and device proposed in the present application;

[0052] Figure 4 A schematic diagram of the hardware structure of another optional example of an electronic device applicable to the audio information processing method and device proposed in the present application;

[0053] Figure 5 A flowchart of an optional example of the audio information processing method proposed in this application;

[0054] Figure 6 A signaling flow diagram of another optional example of the audio information processing method proposed in this application;

[0055] Figure 7 A flowchart diagram of another optional example of the audio information processing method proposed in this application;

[0056] Figure 8 A flowchart diagram of another optional example of the audio information processing method proposed in this application;

[0057] Fig. 9 A flowchart diagram of another optional example of the audio information processing method proposed in this application;

[0058] Fig.10 A schematic diagram of the structure of an optional example of the audio information processing device proposed in this application;

[0059] Fig.11 This is a structural diagram of another optional example of the audio information processing device proposed in the present application. DETAILED DESCRIPTION

[0060] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0061] It should be noted that, for the convenience of description, only the parts related to the relevant invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other.

[0062] It should be understood that the "system", "device", "unit" and / or "module" used in this application is a method for distinguishing different components, elements, parts, portions or assemblies at different levels. However, if other words can achieve the same purpose, the word can be replaced by other expressions.

[0063] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "a kind" and / or "the" do not refer to the singular, but also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of clearly identified steps and elements, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements. The elements defined by the sentence "includes a..." do not exclude the existence of other identical elements in the process, method, commodity or device that includes the elements.

[0064] Among them, in the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" refers to two or more than two. The following terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features.

[0065] In addition, flow charts are used in the present application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, the various steps may be processed in reverse order or simultaneously. At the same time, other operations may also be added to these processes, or a certain step or several steps of operations may be removed from these processes.

[0066] Reference Figure 1 , is a structural diagram of an optional example of an electronic device suitable for the audio information processing method and device proposed in the present application. The electronic device may include but is not limited to a smart phone, a tablet computer, a wearable device, a personal computer (PC), a netbook, a personal digital assistant (PDA), a smart watch, a smart speaker, a robot, a desktop computer, etc. Figure 1 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0067] like Figure 1 As shown, the electronic device proposed in this embodiment may include a body 100, and an electronic component 200, an audio collection device 300, an audio playback device 400 and an audio processing device 500 deployed in the body 100, wherein:

[0068] The electronic component 200 can be used to build a hardware device that supports the normal functions of the electronic device. It can be understood that the electronic components that constitute the hardware device are diverse and can be determined according to the specific functions to be implemented. In the embodiment of the present application, the electronic component 200 mainly refers to an electronic component that generates a first audio that meets specific conditions, such as an electronic component that can generate steady-state noise (such as an audio signal whose signal properties remain basically unchanged over time), which can specifically include capacitor elements, etc.

[0069] The audio collection device 300 can be used to collect audio information when the electronic device is running.

[0070] In the embodiment of the present application, the audio information collected by the audio collection device 300 may include various audios generated in the environment in which the electronic device is located, audio generated by the electronic device itself, etc. The present application does not limit the audio category contained in the audio information, which may be determined according to the circumstances.

[0071] In some embodiments, in order to facilitate the positioning of the sound source, the audio collection device 300 may include a plurality of audio collectors deployed in an array, such as an array microphone, etc. The present application does not limit the specific composition structure of the audio collection device 300.

[0072] The audio playback device 400 can be used to output the audio generated or acquired by the electronic device itself, depending on the situation. This application does not elaborate on the structure and application scenarios of the audio playback device 400. Optionally, the audio playback device 400 may include an audio player deployed in the body of the electronic device, and its specific deployment location can be determined according to the type of electronic device and structural deployment requirements, and this application does not limit this.

[0073] In some other embodiments, the audio playback device 400 may also include an independent audio player, which may be connected to the sound outlet of the electronic device via a data transmission line to play the audio output by the electronic device. In this case, the user may adjust the position of the independent audio player and its audio transmission direction according to needs, which is not described in detail in this application.

[0074] The audio processing device 500 can be used to execute the audio information processing method and device proposed in the embodiments of the present application. Specifically, it can be used to call and execute a first program to implement the various steps of the audio information processing method described in the corresponding embodiments below. The first program can be a program for implementing the corresponding audio information processing method.

[0075] In some embodiments, the above-mentioned first program can be stored in a memory in the audio processing device 500, and the processor included in the audio processing device 500 calls and executes the first program stored in the memory according to needs to implement the steps of the audio information processing method described in the corresponding embodiments below. The specific implementation process is not described in detail in this application.

[0076] In the embodiments of the present application, the working environment of the electronic device is often not very quiet, and there is usually noise interference. In order to reduce at least part of the noise interference to the audio receiving object (such as a user, an audio acquisition device, etc., the present application does not limit the category of the audio receiving object and it may depend on the situation) of the audio output of the electronic device, the present application hopes to adopt an active noise reduction method, such as an inverse noise reduction method, to eliminate the first audio received by the audio receiving object that meets specific conditions. The specific implementation process of the active noise reduction is not described in detail in the present application, and can refer to but is not limited to the description of the corresponding part of the following embodiments.

[0077] Among them, the present application can record the noise that needs active noise reduction processing as the first audio. When the electronic device is in different environments and based on different noise reduction requirements during the use of the electronic device, the category and characteristics of the first audio that needs active noise reduction processing in the present application may be different, which can be recorded as the first audio that meets specific conditions. As for the specific content of the specific condition, it depends on the situation, and the present application will not describe them in detail one by one.

[0078] To summarize, in an embodiment of the present application, for the audio information collected by the electronic device, it can be detected whether there is a first audio that meets specific conditions. If the first audio exists, it is necessary to perform active noise reduction processing on it to obtain an initial inverted first audio for the first audio, so as to ensure that the target inverted first audio transmitted to the audio receiving object where the electronic device plays the audio can offset the first audio transmitted to the audio receiving object, thereby eliminating the noise interference of the first audio on the audio receiving object, and ensuring that the audio receiving object can clearly and accurately receive the audio content played by the electronic device.

[0079] In some embodiments of the present application, the audio information processing process implemented by the audio processing device 500 can be implemented based on artificial intelligence (AI) technology, and can be specifically implemented by a control chip that supports the implementation of artificial intelligence technology, such as Figure 2 The AI ​​chip in the audio acquisition device 300 (such as Figure 2After the microphone Mic in the audio information is collected, it can be sent to the AI ​​chip through the PDM audio interface (it can also be other data audio interfaces, and this application does not impose any restrictions on this) for active noise reduction processing. For the initial phase first audio obtained by the processing, it can be sent to the audio codec (Audio Codec) through the integrated circuit built-in audio bus (I2S (Inter-IC Sound) bus) for audio format conversion processing to obtain analog audio supported by the audio player 400 (Speaker) output. The specific implementation process of the audio encoding and decoding in this process is not described in detail in this application.

[0080] In which, during the use of the electronic device, the music player 400 may also play other audio generated or received by the electronic device itself, recorded as audio to be played, which can be sent by the HD (High Definition) audio controller in the corresponding chipset to the audio codec with high definition to be played. The audio codec mixes the received audio to be played with the above-mentioned initial inverted first audio, and sends the obtained mixed signal to the audio player output.

[0081] It should be noted that since there is often a certain distance between the audio player and the audio receiving object, if the audio player directly outputs audio with an opposite phase to the first audio, when it is transmitted to the audio receiving object, the phase may not be completely opposite to the first audio transmitted here, that is, the phase difference is greater than or less than 180°, which will cause the two audios to be unable to completely cancel each other out, and may even be interfered with by the audio being played.

[0082] Therefore, in order to ensure reliable elimination of the noise of the first audio at the audio receiving object, the present application needs to perform an inversion operation on the collected first audio according to the distances from the above-mentioned sound source to the audio collection device and the audio collection object, as well as the distance between the audio receiving object and the audio playback device, so as to obtain the initial inverted first audio, that is, the audio whose phase difference with the first audio may be greater than or less than 180°. Of course, it may also be the audio with the opposite phase to the first audio, which depends on the above-mentioned distance values ​​obtained in the specific scenario. Thereby ensuring that the phase between the first audio generated by the above-mentioned sound source and the corresponding audio signal transmitted to the audio receiving object is opposite, such as the phase difference is 108°, so that the two channels of audio here can cancel each other, so that the audio receiving object will not be disturbed by the noise of the first audio during the process of receiving the required audio. Among them, the present application does not elaborate on the specific process of obtaining the above-mentioned initial phase first audio.

[0083] In such Figure 2In the electronic device shown, in order to achieve the above-mentioned noise reduction purpose, usually when the electronic device is in operation, the audio collection device 300 collects audio information in real time and sends it to the audio processing device 500 for audio information processing to determine whether to perform active noise reduction processing on the audio information in the current environment. Figure 2 The program executed by the AI ​​chip shown, or the CPU or other processor of the electronic device calling the program to execute, analyzes and processes the audio information collected in real time, which often causes the system power consumption of the electronic device to be very high.

[0084] In order to solve the above-mentioned high power consumption problem, the present application further proposes that a low-power processor, denoted as the first processor, performs a preliminary analysis on the collected audio information, and preliminarily determines that it contains the first audio that requires active noise reduction (that is, the first audio that meets specific conditions), and then notifies the above-mentioned AI chip or CPU and other second processors to perform complex analysis on the audio information and complete the inverse noise reduction processing of the first audio in the above-mentioned manner; conversely, if the first processor preliminarily determines that the collected audio information does not contain the first audio that meets the specific conditions, there is no need for the second processor to perform responsible analysis of the audio information.

[0085] It can be seen that since the power consumption generated by the operation of the first processor is lower than that of the second processor within the same period of time, the first processor replaces the second processor to perform preliminary analysis on the audio information collected in real time, and determines whether to trigger the second processor to execute the program for implementing the audio information processing method proposed in this application based on the preliminary analysis results. The second processor is not required to execute the program in real time, which greatly reduces the power consumption generated by the second processor executing the program. At the same time, it also meets the noise reduction requirements of the application scenario and ensures the best recording and playback effects.

[0086] Based on the above technical concept, Figure 3 As shown, the audio processing device 500 may include the first processor 510 and the second processor 520. As analyzed above, the first processor 510 obtains audio information and detects whether the audio information contains the first audio that meets specific conditions. If not, the audio information collected by the audio collection device may continue to be detected; if it is included, noise reduction prompt information may be generated and sent to the second processor 520, and the second processor 520 performs inverse noise reduction processing on the first audio in the above manner. The specific implementation process of this application embodiment will not be described in detail here.

[0087] It should be noted that the present application does not limit the specific types of the first processor 510 and the second processor 520 and their deployment locations, which may be determined according to the actual situation.

[0088] In some other embodiments, the above-described technical concept of determining whether to trigger the second processor 520 to run based on the initial analysis result of the audio information obtained by the first processor 510 to achieve low-power audio detection analysis can also be applied to other application scenarios of electronic devices, such as waking up the operating system and speech recognition engine (i.e., voice assistant) of the electronic device after certain conditions are met, so as to solve the high power consumption problem caused by the operating system and speech recognition engine of the electronic device always being in operation. Of course, this technical concept is not limited to the several application scenarios listed in this application, and this application will not be described in detail one by one.

[0089] Exemplarily, during the process of a user using an electronic device, in order to reduce the power consumption of the electronic device, after the electronic device is not in use, the electronic device is usually controlled to enter a low power consumption state, such as a shutdown state or a standby state. When the user needs to use the electronic device, the electronic device operating system is started to operate normally. During this process, the present application may use a voice interaction method without trigger words (such as no wake-up words) to wake up the electronic device. In this case, combined with the above-mentioned technical concepts, the first processor and the audio acquisition device of the electronic device may be in a normal operating state, and the above-mentioned second processor will be in a corresponding low power consumption state, waiting to be awakened; of course, it may also be in a normal operating state, but the collected audio information will not be directly analyzed in real time, so as to reduce the power consumption generated by the second processor analyzing the audio information in real time.

[0090] Specifically, when the electronic device is in a low power consumption state, the audio collector 300 can still collect audio information normally, and the first processor 510 performs an initial analysis on it to determine that there is a first audio that meets specific conditions, such as the presence of user voice, and combined with the sound source positioning, it is determined that the relative position relationship between the speaker of the user voice and the electronic device meets specific conditions, such as determining that the user is facing the display screen of the electronic device. It can be determined that the user wants to use the electronic device without the user saying the preset wake-up word of the electronic device. Based on the analysis result, the second processor can be notified to perform complex analysis on the collected audio information, and identity authentication is performed based on the user voice contained in the audio information to determine that the user is a legitimate user of the electronic device, thereby directly waking up the operating system of the electronic device and the voice recognition engine installed in the electronic device, and then the voice recognition engine executes the collected user voice instructions to meet application requirements, but is not limited to the audio information processing method described in this embodiment.

[0091] It can be understood that when the electronic device is in normal working condition, it can also include the first processor and the second processor to execute the corresponding method steps described above. In this case, after completing the user authentication, there is no need to wake up the operating system and the voice recognition engine, and the voice recognition engine can directly execute the user voice instructions contained in the cached audio information. The specific implementation process is not described in detail in this application.

[0092] In a possible implementation, the first processor 510 may be deployed in the audio acquisition device 300, that is, the audio acquisition device 300 may be an intelligent audio acquisition device with a built-in first processor. The present application does not limit the specific hardware deployment relationship between the audio collector and the first processor, and the first processor 510 is not limited to being deployed in the audio acquisition device 300. This hardware deployment relationship is described in the present application only as an example.

[0093] Reference Figure 4 The structural diagram of another optional example of the electronic device shown is that the first processor 510 can be a neural network processor (Neural Network Processing Unit, NPU), which is a general term for a new type of processor based on neural network algorithm and acceleration. In this way, after the audio collector 310 in the audio collection device 300 collects audio information and sends it to the first processor 510, a simple signal analysis can be performed on it using a neural network algorithm to determine whether there is a first audio that meets specific conditions. The specific analysis implementation process is not described in detail in this application.

[0094] Among them, Figure 4 As shown, the audio collector 310 in the above-mentioned audio collection device 300 and the second processor 520 can realize the transmission of audio information through a PDM audio interface (but not limited to this audio interface, the type of audio interface to be used can be determined according to the audio format requirements, and this application does not list them one by one). The first processor 510 and the second processor 520 interact with each other through their respective SPI (Serial Peripheral interface) and / or GPIO (General-Purpose IO ports) and other data interfaces, such as transmitting corresponding instructions and working status information generated based on the preliminary analysis results of the audio information. This application does not limit the specific data communication method between the first processor 510 and the second processor 520, which can be determined according to the actual situation.

[0095] In addition, in some application scenarios of electronic devices, after the second processor 520 receives the audio information sent by the audio collector, the audio information can be cached and sent to other chips in the electronic device for processing when necessary. Therefore, the second processor 520 can communicate with other chips in the electronic device, and the specific connection method is not limited. Figure 3 The USB connection method is used as an example to illustrate the communication connection between the second processor 520 and the chipset of the electronic device. The specific communication connection method and the type of the connected chip may depend on the situation, and this application will not describe them in detail.

[0096] Exemplarily, in a voice interaction application scenario without trigger words (such as no wake-up words required) as described above, the second processor 520 can send the received audio information to the chipset, so that when the second processor 520 determines that the speaker of the user voice contained in the audio information is a legitimate user of the electronic device, that is, the user voice is authenticated, the operating system of the electronic device is awakened, and the voice assistant is triggered to enter the working state, that is, the chipset enters the normal operating state, and the chip that implements the voice assistant function can recognize the voice command contained in the audio information and send it to the corresponding chip to execute the voice command to meet the application requirements. The control process for other application scenarios is similar, and this application will not be described in detail one by one.

[0097] Reference Figure 5 , is a flowchart of an optional example of the audio information processing method proposed in this application. The method can be applied to electronic devices. This application does not limit the product category and component structure of the electronic device. The structure described in the above electronic device embodiment can be referred to but not limited to. Figure 5 As shown, the audio information processing method proposed in this embodiment may include:

[0098] Step S11, obtaining audio information;

[0099] Combined with the above description, the audio information can be collected by the audio collector in the audio collection device 300 of the electronic device, and the specific audio collection implementation process is not described in detail. Specifically, the audio information collected by the audio collection device can be actively or passively acquired by the audio processing device in the electronic device, and this application does not limit the specific implementation process of the audio processing device acquiring the audio information.

[0100] It can be understood that in different application scenarios, the audio content and category contained in the audio information may be different, such as the ambient noise of the environment in which the electronic device is located (such as the noise generated by other devices working in the environment, the noise generated by the natural environment itself, etc.), the electrical noise generated by the electronic device itself, the user voice output by the user to the electronic device, etc. This application does not limit the specific content contained in the above audio information.

[0101] Step S12, when detecting that the audio information includes a first audio that meets a specific condition, extracting the first audio included in the audio information;

[0102] In actual applications, for different application scenarios of electronic devices and their application requirements, the content and category of the first audio that meets specific conditions may be different. This application does not impose any restrictions on this and it may depend on the circumstances.

[0103] In one possible implementation, the first audio that meets the specific conditions mentioned above may refer to steady-state audio, such as the noise generated by the operation of a system radiator (such as a fan) of an electronic device, such as the noise generated by the operation of household appliances such as vacuum cleaners / range hoods, etc. This can be determined based on the current environment of the electronic device, and this application does not limit the steady-state audio category and its generation method. In this case, the audio processing device can extract the steady-state audio when detecting that the acquired audio information contains steady-state audio, and continue to process the steady-state audio in the following manner to achieve anti-phase cancellation of the steady-state audio at the receiving object of the transmitted audio.

[0104] In another possible implementation, in some scenarios, the signal strength of the non-steady-state noise in the environment where the electronic device is located is relatively large, resulting in the audio receiving object (such as a user) in the environment being basically unable to receive the steady-state noise in the environment. Take the example of the capacitive electrical noise generated by the capacitive element in the electronic device as the steady-state noise, and the actual environmental noise as the non-steady-state noise. If the current environment is relatively noisy, it can basically cover up the capacitive electrical noise generated by the electronic device. In this case, the electronic device can handle the capacitive electrical noise generated by the electronic device without using the inverting noise reduction method proposed in this application. Specifically, no processing can be performed, or other noise reduction filtering methods can be used for processing. This application does not limit this, and it can be determined according to the circumstances. On the contrary, if the environment in which the electronic device is located is relatively quiet, the capacitive electrical noise as described above will be highlighted. In this case, it is usually necessary to perform inverting noise reduction on the capacitive electrical noise to reduce interference to users using the electronic device.

[0105] Therefore, for the application scenarios described above, the electronic device can obtain the steady-state noise and non-steady-state noise in the collected audio information, and then determine whether the two types of noise meet specific conditions, such as whether the signal strength difference between the steady-state noise and the non-steady-state noise described in the above scenario is greater than the corresponding steady-state noise reduction threshold, and then determine whether the steady-state noise needs to be subjected to inverse noise reduction processing.

[0106] Furthermore, in another possible implementation method, in certain application scenarios, it may be necessary to perform active noise reduction processing on abnormal steady-state noise with sudden frequency changes. Then, for such application scenarios, when it is determined in the above manner that the steady-state noise and the non-steady-state noise meet the corresponding specific conditions, the present application can further determine whether there is an abnormal frequency signal in the steady-state noise, and in turn determine whether to perform inverse noise reduction processing on the currently detected steady-state noise.

[0107] It should be noted that the first audio that meets the specific conditions is not limited to the content described above, and can be determined according to the application scenario of the electronic device. This application will not describe them in detail here.

[0108] Step S13, obtaining an initial inverted first audio for the extracted first audio based on the relative position relationship between the sound source of the first audio, the audio playback device, and the audio receiving object;

[0109] In combination with the description of the corresponding parts of the above embodiments, the distances between the sound source of the first audio and the audio playback device and the audio receiving object are often different, and there is often a certain distance between the audio playback device and the audio receiving object. In this way, when the audio signal directly output by the audio player is transmitted to the audio receiving object, the signal properties will often change to a certain extent, such as a decrease in signal strength due to signal attenuation, and the phase of the audio signal directly output by the audio player may be different from the phase of the audio signal transmitted to the audio receiving object.

[0110] Therefore, in order to ensure that the noise signal to be eliminated transmitted to the audio receiving object and the inverted noise signal used to eliminate the noise signal are in opposite phases and can cancel each other out, the present application needs to consider the relative position relationship between the sound source of the first audio, the audio playback device and the audio receiving object during the inverting process of the extracted first audio, so as to ensure that the audio playback device outputs the calculated initial inverted first audio, which can achieve audio inverted cancellation when transmitted to the audio receiving object.

[0111] It can be understood that if the distance between the audio receiving object and the audio playback device is very small, less than a specific distance threshold, it can be considered that the audio signal received by the audio receiving object is the same as the audio signal directly played by the audio playback device. In this case, the initial inverted first audio obtained by this application can be an audio with a phase opposite to that of the first audio. It can be seen that in the face of different application scenarios, the phase difference between the initial inverted first audio obtained above and the extracted first audio may be different. This application needs to specifically calculate the corresponding initial inverted first audio based on the relative position relationship between the sound source of the first audio, the audio playback device, and the audio receiving object in a specific application scenario. The specific inverted calculation process is not described in detail in this application.

[0112] Step S14, sending the initial inverted first audio to the audio playback device for output, so that the target inverted first audio transmitted to the audio receiving object and the first audio cancel each other out.

[0113] Continuing the above description, since the present application takes into account the distance difference between the sound source of the first audio and the audio playback device and the audio receiving object, respectively, during the process of performing the inversion calculation on the extracted first audio, for the initial inverted first audio output by the audio playback device, if the audio transmitted to the audio receiving object is recorded as the target inverted first audio, the present application can ensure that the target inverted first audio and the first audio transmitted to the audio receiving object cancel each other out, thereby preventing the audio receiving object from being disturbed by the noise of the first audio.

[0114] Exemplarily, assuming that the first distance between the sound source of the first audio and the audio receiving object is d1, and the second distance between the audio playback device and the audio receiving object is d2, under the same environment, the transmission speed of the audio signal is relatively fixed. Therefore, the time taken for the audio signal to transmit the first distance d1 is t1, and the time taken for the audio signal to transmit the second distance d2 is t2.

[0115] In actual applications, if the above d1 is less than d2, the audio receiving object is closer to the above sound source than the audio playback device, then the time t1 taken for the first audio generated by the above sound source to be transmitted to the audio receiving object is less than the time t2 taken for the initial inverted first audio output by the audio playback device to be transmitted to the audio receiving object, and during the time period (t2-t1), the audio receiving object will still be disturbed by the noise of the first audio. However, starting from the next moment of this time period, the target inverted audio will be transmitted from the audio receiving object, which can cancel each other with the transmitted first audio.

[0116] On the contrary, if the above d1 is greater than d2, the audio receiving object is closer to the audio playing device relative to the above sound source, then the above t1 is greater than t2. If the audio playing device directly plays the initial inverted first audio after obtaining it in the above manner, then, for the first audio generated by the sound source and the initial inverted first audio output by the audio playing device at the same time, subject to the above positional relationship, the initial inverted first audio will reach the audio receiving object earlier than the first audio, specifically, it will arrive at the duration (t1-t2) first. The initial inverted first audio that arrives within this duration will also be an unnecessary noise signal for the audio receiving object, and the audio receiving object does not need to receive the initial inverted first audio.

[0117] Therefore, when d1 is greater than d2, the initial inverted first audio sent to the audio playback device can be delayed to ensure that when the played initial inverted first audio is transmitted to the audio receiving object, the first audio generated by the above sound source is also transmitted to the audio receiving object, so that the two audios transmitted to the audio receiving object can cancel each other out and reduce noise interference. Among them, the specific duration of the delay of the initial inverted first audio can be determined based on the distance difference between the first distance d1 and the second distance d2, the audio transmission speed and other parameters. The specific implementation process of the delay processing is not described in detail in this application.

[0118] In summary, in an embodiment of the present application, when it is detected that the audio information obtained by the electronic device contains a first audio that meets specific conditions, in order to eliminate the noise interference of the first audio, the electronic device can extract the first audio contained in the audio information, and thus obtain the initial inverted first audio for the first audio based on the relative position relationship between the sound source of the first audio, the audio playback device and the audio receiving object, and send it to the audio playback device for output, so that the target inverted first audio transmitted to the audio receiving object and the first audio can cancel each other, thereby realizing active noise reduction processing of the first audio collected by the electronic device, avoiding the first audio from causing noise interference to the user using the electronic device, and improving the user experience of the electronic device.

[0119] Reference Figure 6 , which is a signaling flow diagram of another optional example of the audio information processing method proposed in the present application. This embodiment can be an optional detailed implementation method of the audio information processing scheme described in the above embodiment. Combined with the above description of the composition structure of the electronic device, this embodiment can reduce the power consumption of the electronic device while realizing active noise reduction, mainly reducing the power consumption consumed in the audio detection process. In this case, the embodiment of the present application can be applicable to but not limited to the above Figure 3 and Figure 4 The electronic equipment shown, such as Figure 6As shown, the audio information processing method proposed in this embodiment may include:

[0120] Step S21, the first processor obtains audio information;

[0121] Combined with the description of the corresponding part of the above embodiment, for the first processor and the second processor in the electronic device, in the same time, the first power consumption generated by the operation of the first processor is less than the second power consumption generated by the operation of the second processor. In some scenarios, the value of the first power consumption will be much smaller than the value of the second power consumption, such as the difference between the second power consumption and the first power consumption is greater than the power consumption threshold (which is a relatively large value, which can be determined based on the power consumption consumed by the operation of the electronic device, and this application does not limit its value), etc. This application does not limit the specific values ​​of the above-mentioned first power consumption and second power consumption, which can be determined according to the circumstances.

[0122] In some embodiments, the audio acquisition device of the electronic device performs audio acquisition and sends the collected audio information to the first processor actively or passively (i.e., the first processor actively reads the audio information collected by the audio acquisition device, etc.). In some scenarios, the audio acquisition device can also actively or passively send the audio information to the second processor so that the second processor can perform complex analysis and processing on the corresponding audio information; of course, the audio acquisition device or the first processor can also send the audio information to the second processor if the audio information meets certain conditions. This application does not limit the execution stage of sending the audio information to the second processor.

[0123] Among them, the audio information collected by the electronic device may include at least one type of audio. Since the working environment of the electronic device is not an absolutely quiet environment, it usually contains multiple types of audio. However, this application does not limit the audio category contained in the audio information, which can be determined according to the situation.

[0124] Step S22, the first processor detects whether the audio information contains the first audio that meets the specific condition, if not, continue to detect the acquired audio information; if yes, proceed to step S23;

[0125] Further to the above analysis, in order to avoid the high power consumption problem caused by the real-time analysis of the audio information collected by the electronic device directly by the second processor, the embodiment of the present application chooses to implement real-time detection of audio information by a first processor with low power consumption characteristics to determine whether the acquired audio information contains the first audio that meets specific conditions. The specific detection implementation process can refer to but is not limited to the description of the corresponding part of the above embodiment, and this embodiment will not be repeated here.

[0126] After the above detection and analysis, the first processor determines that the acquired audio information contains the first audio that meets the specific conditions, and then notifies the second processor to work and perform detailed detection on the audio information; conversely, if it is determined that the acquired audio information does not contain the first audio that meets the specific conditions, it is not necessary to notify the second processor to detect the audio information. In other words, for such audio information that does not contain the first audio that meets the specific conditions, the second processor does not need to perform detection and analysis, thereby reducing the power consumption caused by the second processor detecting and analyzing such audio information.

[0127] Step S23, the first processor sends noise reduction prompt information to the second processor;

[0128] Since the noise reduction prompt information is mainly used to notify the second processor to perform complex detection and analysis on the corresponding audio information, in order to ensure that the second processor accurately determines the corresponding audio information containing the above-mentioned first audio, the noise reduction prompt information may include the time information corresponding to the acquired audio information, and the detection result that the audio information contains the first audio, etc., so that the second processor can determine the audio information corresponding to the time information from the multiple audio information received, and complete subsequent detection and analysis on it.

[0129] Of course, in some other embodiments, if the first processor completes the above detection of the acquired audio information, and then determines whether to send it to the second processor for subsequent processing based on the detection result, then the noise reduction prompt information sent by the first processor to the second processor does not need to include the above time information, and the second processor can also accurately know which audio information needs to be complexly analyzed. It can be seen that in different application scenarios, the content of the noise reduction prompt information sent by the first processor may be different, and this application does not limit the content of the noise reduction prompt information, which can be determined according to the situation.

[0130] In one possible implementation, the noise reduction prompt information may be in the form of high / low level signals, events or instructions. When the second processor receives the noise reduction prompt information, it may process the corresponding audio information according to a preset audio analysis method. The present application does not impose any restrictions on the generation method and output form of the noise reduction prompt information, which may be determined according to the circumstances.

[0131] Step S24, the second processor obtains the above audio information, receives the noise reduction prompt information, and extracts the first audio included in the audio information;

[0132] As analyzed above, the embodiment of the present application does not limit the execution steps of the second processor to obtain audio information, which can be determined according to the circumstances.

[0133] In some embodiments, the first processor may include an NPU, and the second processor may include an AI chip, etc., but is not limited to these two processors. This embodiment is only used as an example to illustrate that when the NPU performs a preliminary analysis on the audio information and determines that it contains the first audio, the second processor will perform a complex analysis on the audio information based on AI technology, such as using neural networks, sound source localization algorithms, etc., to extract the first audio that meets specific conditions from the acquired audio information. This application does not limit the specific extraction implementation method of the first audio, which can be determined according to the circumstances.

[0134] Step S25, the second processor obtains an initial inverted first audio for the extracted first audio based on the relative position relationship between the sound source of the first audio, the audio playback device, and the audio receiving object;

[0135] Step S26: the second processor sends the initial inverted first audio to the audio playback device for output, so that the target inverted first audio transmitted to the audio receiving object and the first audio cancel each other out.

[0136] Regarding the specific implementation process of step S25 and step S26, reference may be made to the description of the corresponding parts of the above embodiment, which will not be elaborated in this embodiment.

[0137] In summary, in the embodiment of the present application, a low-power first processor will be used to perform real-time analysis on the collected audio information. Only when it is determined that the collected audio information contains the first audio that meets specific conditions, will the high-power second processor be notified to extract the first audio from the corresponding audio information, and construct the corresponding initial inverted first audio, which is output by the audio playback device to ensure that the target inverted first audio transmitted to the audio receiving object and the first audio can offset each other, thereby achieving active noise reduction of the first audio and avoiding noise interference of the first audio on the audio receiving object.

[0138] When the first processor determines that the collected audio information does not include the above-mentioned first audio, the second processor does not need to perform complex analysis on the audio information collected this time. Compared with the second processor directly completing the entire audio information processing process, the power consumption generated by the second processor performing audio processing is greatly reduced. It can be seen that the embodiment of the present application realizes low-power real-time audio detection and analysis, while meeting the optimal requirements for recording and playback effects, the impact of system power consumption is minimized.

[0139] It should be noted that in the actual application of the present application, according to the application requirements in different scenarios, the audio information processing method proposed in the present application can be executed by the above-mentioned second processor in the electronic device, so as to achieve the purpose of active noise reduction of the first audio transmitted to the audio receiving object in the above-mentioned manner; if the application scenario also needs to reduce power consumption, the first processor and the second processor can cooperate to implement the audio information processing method proposed in the present application in accordance with the method described in the above embodiment to reduce system power consumption. The specific implementation process will not be repeated in this application.

[0140] Reference Figure 7 , which is a flowchart of another optional example of the audio information processing method proposed in this application. This embodiment can be another optional detailed implementation method of the audio information processing scheme described in the above embodiment. It mainly uses the above-mentioned first audio as an example of steady-state noise for explanation, but does not limit the specific application scenario of eliminating steady-state noise, as well as the specific audio content and category of the steady-state noise, which can be determined according to the situation. Based on this, Figure 7 As shown, the method may include:

[0141] Step S31, obtaining audio information;

[0142] Regarding the specific implementation process of step S31, reference may be made to the description of the corresponding part of the above embodiment, which will not be elaborated in this embodiment.

[0143] Step S32, detecting that the audio information contains steady-state noise, and extracting the steady-state noise from the audio information;

[0144] In an embodiment of the present application, in different application scenarios, the first processor or the second processor can perform signal feature analysis on the acquired audio information to determine the noise categories of different audios contained in the audio information. If the audio information contains steady-state noise, the steady-state noise can be directly determined as the first audio. Combined with the description of the corresponding part of the above embodiment, the second processor can extract the steady-state noise contained in the audio information, that is, the first audio, and the specific extraction method is not limited.

[0145] Among them, the signal feature analysis process of audio of different noise categories can be determined according to the audio signal characteristics of this type of noise. For example, if the attribute values ​​such as signal amplitude and frequency of a certain audio signal contained therein change with time and are basically stable within a certain range, this audio signal can be considered to be steady-state noise; if the attribute value of a certain audio signal changes irregularly with time, this audio signal can be considered to be non-steady-state noise, etc. Therefore, the electronic device can analyze the changes in the attribute values ​​of each audio signal in the acquired audio information to realize noise category identification, but it is not limited to these two noise categories and their identification methods. In some application scenarios, the noise source position can also be combined to determine the noise category, etc. For example, for capacitive electrical noise, the sound source position is determined to be located in the area where the electronic device is located through sound source positioning, and the audio generated by the sound source can be considered to be capacitive electrical noise, etc. This application will not describe them in detail here.

[0146] It should be noted that since steady-state noise also includes multiple categories, such as electronic noise generated by the electronic device itself, noise generated by specific categories of household appliances working around the electronic device, etc., in different application scenarios, all noise reduction processing may be required, or some categories of steady-state noise may be required. Therefore, when it is determined that the audio information contains steady-state noise, all or part of the steady-state noise contained in the audio information can be extracted based on the noise reduction requirements of the application scenario and the category of each steady-state noise, that is, the steady-state noise that needs noise reduction processing. The specific implementation process is not described in detail in this application.

[0147] Step S33, obtaining the first sound source position of the steady-state noise, the sound outlet position of the audio playback device, and the sound receiving position of the audio receiving object;

[0148] Further to the above description, when the first audio is steady-state noise, the embodiment of the present application can determine that the acquired audio information contains steady-state noise, and determine that the steady-state noise needs to be subjected to inverse noise reduction processing. Since the present application needs to ensure the inverse cancellation of the steady-state noise at the sound receiving position of the audio receiving object, considering the phase deviation caused by the long-distance transmission of the audio signal, the present application can obtain the relative position relationship between the sound source of the steady-state noise (i.e., the first audio), the audio playback device, and the audio receiving object.

[0149] Specifically, the present application can perform sound source positioning on each channel of audio collected by the audio collection device to obtain the sound source position of steady-state noise, such as using the attribute values ​​of each channel of audio collected by the array microphone and the positional relationship between the microphones included in the array microphone to achieve sound source positioning of steady-state noise. The present application does not elaborate on the specific implementation process of sound source positioning.

[0150] In addition, the present application also needs to obtain the position of the sound outlet of the audio playback device and the sound receiving position of the audio receiving object (such as the position of the user's ear, the sound receiving hole position of other audio receivers, etc.). In order to ensure the positioning consistency of the above-mentioned position information obtained, the above-mentioned sound source position, sound outlet position and sound receiving position can be determined based on the same position reference point. The position reference point can be a certain position in the electronic device, etc. The present application does not limit the specific position of the position reference point.

[0151] Among them, in different application scenarios, the sound receiving position for the above-mentioned audio receiving object can be determined according to the category of the audio receiving object. For example, if the audio receiving object is a user, the sound receiving position is the user's ear, then the sound receiving position can be determined by counting the distance between the ear of an ordinary user and the above-mentioned position reference point during the use of the electronic device. The sound receiving position can be used as a parameter for the audio information processing application as needed, and can also be personalized by the user to make it more matched with the user's own sound receiving position, thereby improving the audio information processing effect. Similarly, the other position information mentioned above can also be adaptively adjusted according to changes in the electronic device model. The present application does not elaborate on the adjustment implementation method of each position information.

[0152] Step S34, performing inversion processing on the steady-state noise according to the first sound source position, the sound outlet position and the sound receiving position to obtain initial inverted steady-state noise;

[0153] In combination with the description of the corresponding parts of the above embodiments, in some embodiments, based on the position information such as the first sound source position, the sound outlet position and the sound receiving position, it is determined that the distance between the sound outlet position and the sound receiving position (i.e., the second distance) is within a certain range. If the second distance is less than a certain distance threshold, it can be considered that the audio played by the sound outlet has the same attribute information and content as the audio received by the sound receiving hole of the audio receiving object, and there is no phase shift problem. In this case, the extracted steady-state noise can be directly inverted to obtain an initial inverted steady-state noise with a phase difference of 180° from the steady-state noise, i.e., the target inverted steady-state noise transmitted to the sound receiving position of the audio receiving object, thereby achieving inverted cancellation of the corresponding steady-state noise transmitted to the sound receiving position of the audio receiving object and avoiding interference from the steady-state noise.

[0154] In some other embodiments, if the second distance between the sound outlet position and the sound receiving position exceeds the above range, a certain phase deviation may occur when the audio signal is transmitted over a longer distance. Therefore, there may be a phase deviation between the audio directly output from the sound outlet of the audio playback device and the audio at the sound receiving position transmitted to the audio receiving object. The present application needs to ensure that the phase of the audio signal directly played from the audio transmitted to the sound receiving position is opposite to the phase of the extracted steady-state noise signal, so that the superposition of the two audio channels can cancel each other out, thereby achieving the purpose of reliable noise reduction of steady-state noise.

[0155] Therefore, in order to achieve the above purpose, in the process of constructing the inverted steady-state noise, the size of the second distance between the above-mentioned sound outlet position and the sound receiving position is taken into account, so that there is a certain phase difference between the constructed initial inverted steady-state noise (i.e., the initial inverted first audio) and the extracted steady-state noise, but the phase difference may not be 180°, which can be specifically determined based on the phase deviation from the audio played by the audio playback device to the above-mentioned sound receiving position. The present application does not limit the size of the phase difference here, which can be determined according to the circumstances. After this processing, the initial inverted steady-state noise played is phase-shifted, and the target inverted steady-state noise transmitted to the above-mentioned sound receiving position can be inversely offset with the above-mentioned steady-state noise.

[0156] In some other embodiments, combined with the above-mentioned embodiment, if the second distance is smaller than the first distance, delay processing may be required in the process of constructing the initial inverse steady-state noise to avoid the initial inverse steady-state noise played from causing noise interference to the audio receiving object. The specific implementation process of the delay processing can refer to the description of the corresponding part of the above-mentioned embodiment, and this embodiment will not be repeated here.

[0157] It can be understood that, when the extracted steady-state noise includes multiple steady-state noises, the constructed initial inverted steady-state noise may include initial inverted steady-state noise for each steady-state noise, so as to ensure all noise reduction processes for the extracted multiple steady-state noises.

[0158] Step S35, mixing the initial inverse steady-state noise with the audio to be played to obtain a mixed audio;

[0159] Step S36, sending the mixed audio to the audio playback device for output, so that the target anti-phase steady-state noise in the mixed audio transmitted to the sound receiving position of the audio receiving object and the above-mentioned steady-state noise cancel each other out.

[0160] In an embodiment of the present application, the content of the audio to be played can be determined according to a specific application scenario. For example, in application scenarios such as a user controlling an electronic device to play music, using headphones to listen to music, or in audio and video conferencing, online learning, etc., the audio to be played can be the music to be played, the audio of speeches by other participants, the audio of the teacher in class, etc. In order to eliminate noise interference in the corresponding application scenarios and improve the recording and playback effects, the above-mentioned method can be used to simultaneously perform inverse noise reduction processing on the existing steady-state noise while the user is listening to the audio to be played. The noise reduction processing process will not affect the user's listening to the audio to be played in the corresponding application scenario.

[0161] It can be understood that in the application scenario where the user uses headphones to listen to audio, after the audio codec obtains the initial inverted first audio, if there is audio to be played at this time, the two audio channels will be mixed and then sent to the headphones for playback. Therefore, in this embodiment, the above-mentioned audio playback device may include headphones connected to the sound outlet of the electronic device itself, and the sound receiving position of the above-mentioned audio receiving object may be the position of the headphone sound outlet. Of course, if the user does not use headphones to listen to audio, but the audio is played by the audio player of the electronic device itself, the sound receiving position of the above-mentioned audio receiving object may be the position of the user's ear. In other application scenarios, since the audio receiving object is not limited to the user, its sound receiving position does not have the position of the user's ear, it may depend on the situation, and this application will not describe them in detail one by one.

[0162] In addition, if there is no audio to be played in the electronic device, the initial inverse steady-state noise can be directly sent to the audio playback device output, which can ensure that the target inverse steady-state noise and the steady-state noise transmitted to the audio receiving object at the receiving position cancel each other out.

[0163] In summary, in the embodiment of the present application, the electronic device determines that the acquired audio information contains steady-state noise. In order to eliminate the noise interference of the steady-state noise on the audio receiving object, the first sound source position of the steady-state noise, the sound outlet position of the audio playback device, and the sound receiving position of the audio receiving object can be obtained, so as to invert the extracted steady-state noise, mix the obtained initial inverted steady-state noise with the audio to be played and send it to the audio playback device for output, and the target inverted steady-state noise in the mixed audio transmitted to the sound receiving position of the audio receiving object cancels out the above-mentioned steady-state noise, thereby avoiding the noise interference of the steady-state noise on the audio receiving object, so that the audio content to be played can be reliably received.

[0164] It can be understood that in the above-mentioned audio information processing process, in order to reduce the system power consumption of the electronic device as much as possible, a low-power first processor can be used to perform real-time audio analysis. When it is determined that there is steady-state noise in the audio information, a high-power second processor can perform complex analysis on the audio information to avoid the power consumption caused by the second processor performing complex analysis on the audio information that does not contain steady-state noise. The specific implementation process can refer to the description of the corresponding part of the above-mentioned embodiment, and the embodiment of the present application will not be described in detail here.

[0165] In the actual application of the present application, in different application scenarios, the first audio that meets the specific conditions is not limited to the presence of steady-state noise described in the above embodiment; it can also include that the steady-state noise and non-steady-state noise in the acquired audio information meet specific conditions. In this case, only determining that the acquired audio information contains steady-state noise cannot directly determine that the steady-state noise needs to be denoised. It is also necessary to further determine whether the steady-state noise and non-steady-state noise in the audio information meet specific conditions, such as whether the signal strength difference between the steady-state noise and the non-steady-state noise is greater than the steady-state noise reduction threshold, so as to determine that the steady-state noise can interfere with the audio receiving object, wherein the steady-state noise reduction threshold is determined based on the signal strength of the ambient noise of the electronic device. The specific application implementation process can refer to the description of the corresponding part of the above embodiment, and this embodiment will not be described in detail here.

[0166] In another possible implementation, when it is determined that the steady-state noise and non-steady-state noise in the acquired audio information meet specific conditions, it is also possible to further determine whether the detected steady-state noise is abnormal, so as to determine whether the steady-state noise needs to be denoised. It can be seen that in different application scenarios, the content of the first audio that meets the specific conditions may be different. This application can be determined based on the specific noise denoising requirements of the application scenario. This application does not list them one by one. Here, only this possible implementation is taken as an example to describe the audio information processing process applicable to it.

[0167] Reference Figure 8 , which is a flowchart of another optional example of the audio information processing method proposed in this application. Different from the first audio content that meets the specific conditions defined in the above embodiment, this embodiment of the application describes the first audio of other content. The subsequent processing process of extracting the first audio is similar, and the description of the corresponding part of the above embodiment can be referred to. This embodiment will not be described in detail. Figure 8 As shown, the method may include:

[0168] Step S41, obtaining audio information;

[0169] Step S42, obtaining steady-state noise and non-steady-state noise contained in the audio information;

[0170] Step S43, obtaining a signal strength difference between the steady-state noise and the non-steady-state noise;

[0171] Step S44, determining that the signal strength difference is greater than the steady-state noise reduction threshold, and that the steady-state noise has an abnormal frequency signal, and determining the steady-state noise as the first audio;

[0172] In an embodiment of the present application, steady-state noise and non-steady-state noise contained in the audio information can be extracted. If the steady-state noise is masked by the non-steady-state noise, it can be considered that the steady-state noise will not interfere with the audio receiving object, and it is not necessary to perform inverse noise reduction processing on it in the manner of the present application. Therefore, the present application can first detect whether the signal strength difference between the steady-state noise and the non-steady-state noise is greater than the steady-state noise reduction threshold, that is, whether the signal strength of the steady-state noise is large enough to interfere with the audio receiving object, so as to determine whether the steady-state noise and the non-steady-state noise meet specific conditions.

[0173] According to the above method, it is determined that the obtained signal strength difference is greater than the steady-state noise reduction threshold, that is, the steady-state noise and the non-steady-state noise meet specific conditions. It can be considered that the steady-state noise can interfere with the audio receiving object in terms of signal strength. In some embodiments, no other detection is required, and the steady-state noise is directly used as the first audio for subsequent processing; and in the embodiment of the present application, it can be considered that normal steady-state noise does not need to be filtered using the inverted noise reduction processing method of the present application, and no processing or other filtering methods can be used for noise reduction. For abnormal steady-state noise, the audio information processing method proposed in the present application is used to perform inverted noise reduction processing on it, thereby ensuring reliable inverted cancellation of the abnormal steady-state noise at the sound receiving position of the audio receiving object.

[0174] Therefore, the embodiment of the present application can further analyze whether the extracted steady-state noise has an abnormal frequency signal when it is determined that the obtained signal strength difference is greater than the steady-state noise reduction threshold. Specifically, the spectrum diagram of the steady-state noise can be analyzed. If there is an abnormal frequency signal with a sudden amplitude change, it can be determined that the steady-state noise is abnormal and determined as the first audio, and the subsequent steps can be continued.

[0175] It can be understood that in the above-mentioned different application scenarios, according to the corresponding detection method, it is determined that the acquired audio information does not contain the first audio that meets the specific conditions, and the subsequent acquired audio information can continue to be detected. The specific implementation process can be combined with the application scenario to determine the definition content of the first audio that meets the specific conditions, which is not described in detail in this application.

[0176] Step S45, obtaining an initial inverted first audio for the extracted first audio based on the relative position relationship between the sound source of the first audio, the audio playback device, and the audio receiving object;

[0177] Step S46, sending the initial inverted first audio to the audio playback device for output, so that the target inverted first audio transmitted to the audio receiving object and the first audio cancel each other out.

[0178] Regarding the specific implementation process of step S45 and step S46, please refer to the description of the corresponding parts of the above embodiment, and this application will not go into details here.

[0179] In summary, in the embodiment of the present application, after the electronic device obtains the audio information, it determines that the signal strength difference between the steady-state noise and the non-steady-state noise contained therein is greater than the steady-state noise reduction threshold, that is, the signal strength of the steady-state noise is sufficient to interfere with the audio receiving object, and determines that there is an abnormal frequency signal in the steady-state noise, and the steady-state noise needs to be inverted and eliminated, so it is determined as the first audio, and based on the relative position relationship between its sound source position, the audio playback device and the audio receiving object, the corresponding initial inverted first audio is obtained, so that it is output by the audio playback device, and the target inverted first audio transmitted to the audio receiving object can offset each other with the first audio, thereby ensuring that the audio receiving object will not be interfered by this type of steady-state noise.

[0180] Moreover, in combination with the description of the other embodiments above, in order to reduce system power consumption, the audio information processing process in this embodiment can be implemented by the cooperation of the first processor and the second processor in the electronic device. The specific implementation process will not be described in detail in this embodiment of the present application.

[0181] In some other embodiments proposed in the present application, the first audio that meets the specific conditions is not limited to steady-state audio, but can also be a preset user audio, so that voice control of the electronic device can be implemented based on it. The user does not need to remember and say the preset wake-up word, and can also wake up the system, voice recognition engine, etc. of the electronic device, thereby meeting the user's control needs for the electronic device.

[0182] Based on the above analysis, refer to Fig. 9 , which is a flowchart of another optional example of the audio information processing method proposed in the present application, which is different from the definition of the first audio in the above detailed embodiment, specifically as follows Fig. 9 As shown, the method may include:

[0183] Step S51, obtaining audio information;

[0184] Step S52, detecting that the audio information contains user audio, performing sound source positioning on the user audio, and obtaining the position of the speaker of the user audio;

[0185] Step S53, determining that the positional relationship between the speaker's position and the electronic device meets a specific condition, and determining the user audio as the first audio;

[0186] In combination with the description of the corresponding parts of the electronic device implementation example above, in the application scenario of voice interaction without wake-up words, the electronic device can determine the relative position relationship between it and the electronic device through the user audio to determine whether the user has the intention to use the electronic device. If it is determined that the user has the intention to use the electronic device, the user audio can be subjected to complex analysis.

[0187] Based on this, after the audio collection device collects audio information in real time, the sound source of the user audio contained in the audio information is located to determine whether the position relationship between the speaker's position and the electronic device meets specific conditions, such as whether the user is speaking facing the direction of the electronic device system, etc. This application does not limit the specific conditions in this scenario and can be determined according to the situation.

[0188] Step S54, extracting the first audio included in the audio information;

[0189] Step S55, based on the first audio, authenticating the speaker of the first audio;

[0190] Step S56: If the identity authentication is successful, the voice recognition engine of the electronic device is awakened, and the voice recognition engine executes the voice instruction contained in the first audio.

[0191] In practical applications, in order to reduce system power consumption, especially when the electronic device is in a low power consumption state, the first processor (such as an NPU integrated in an array audio collector, etc.) can execute the above steps S51 to S53 and notify the second processor to execute subsequent steps to solve the problem of high power consumption of the system caused by the high power consumption second processor executing steps S51 to S53 in real time.

[0192] Among them, after the electronic device extracts the first audio (i.e., user audio) contained in the audio information, it can use voiceprint recognition technology to realize the identity authentication of the speaker of the first audio. The specific identity authentication implementation process is not described in detail. For example, the extracted first audio can be directly input into a pre-built identity recognition model to determine whether the speaker of the first audio is the preset legitimate user of the electronic device. If so, the operating system and speech recognition engine of the electronic device will be awakened, and the cached user audio will be sent to the operating system, and the speech recognition engine will execute the corresponding voice command. It can be understood that in order to ensure the reliability of identity authentication, the first audio obtained within a continuous period of time (such as 2 seconds or longer) can be processed.

[0193] After the above detection and analysis, if the positional relationship between the speaker's position and the electronic device does not meet specific conditions, the subsequently acquired audio information can continue to be analyzed; if the identity authentication fails, the subsequently acquired audio information can also continue to be authenticated. After a preset number of consecutive authentication failures, the electronic device is locked to improve the safety of the use of the electronic device, but it is not limited to the security measures described in this embodiment.

[0194] In addition, in actual applications, the collected audio information may only include or mainly include the first audio. In this case, the electronic device does not need to extract the first audio from the collected audio information, and can use the audio information as the first audio to complete subsequent processing; if the user's environment is relatively noisy, the directly collected audio information can be combined with noise reduction technology to obtain a clearer and cleaner first audio, so as to improve the reliability of voice control of the electronic device.

[0195] In summary, in the embodiments of the present application, there is no need to pre-configure the wake-up words of the electronic device operating system and the voice recognition engine. The user can directly speak to the electronic device according to the application requirements. The electronic device determines that the collected audio information contains user audio, locates the sound source, and determines that the positional relationship between the speaker's position and the electronic device meets specific conditions. The user audio is used as the first audio, and the speaker is authenticated based on this. If the authentication is successful, the system and voice recognition engine of the electronic device are directly woken up, the user audio is sent to the voice recognition engine, and the corresponding voice commands are determined and executed to meet the voice control requirements of the electronic device.

[0196] Based on the technical concept of the audio information processing method described in the above embodiments, the composition structure of the virtual device that implements the audio information processing method will be described below, but it is not limited to the composition structure described in the device embodiment below.

[0197] Reference Fig.10 , is a schematic diagram of an optional example of the structure of the audio information processing device proposed in this application, which can be applied to the above-mentioned electronic equipment, such as Fig.10 As shown, the device may include:

[0198] An audio information acquisition module 11, used to acquire audio information;

[0199] A first audio extraction module 12 is used to extract the first audio contained in the audio information when detecting that the audio information contains the first audio that meets a specific condition;

[0200] An inversion processing module 13, configured to obtain an initial inverted first audio for the extracted first audio based on a relative position relationship between a sound source of the first audio, an audio playback device, and an audio receiving object;

[0201] The audio transmission module 14 is used to send the initial inverted first audio to the audio playback device for output, so that the target inverted first audio transmitted to the audio receiving object and the first audio cancel each other out.

[0202] In a possible implementation, the inversion processing module 13 may include:

[0203] A position acquisition unit, used to acquire a first sound source position of the first audio, a sound outlet position of the audio playback device, and a sound receiving position of an audio receiving object;

[0204] an inversion processing unit, configured to perform inversion processing on the first audio according to the first sound source position, the sound outlet position, and the sound receiving position, so as to obtain an initial inverted first audio;

[0205] Accordingly, the audio transmission module 14 may include:

[0206] The audio mixing output unit is used to mix the initial inverted first audio with the audio to be played, and send the obtained mixed audio to the audio playback device for output.

[0207] In some embodiments, the first audio extraction module 12 may include:

[0208] A first detection unit located in a first processor of the electronic device, configured to detect whether the audio information includes a first audio that meets a specific condition;

[0209] A first noise reduction prompt unit located in the first processor, configured to send noise reduction prompt information to the second processor if the detection result of the first detection unit is yes;

[0210] Based on this, the above-mentioned inversion processing module 13 and audio transmission module 14 can be located in the second processor of the electronic device to ensure that the electronic device uses the first processor and the second processor to minimize system power consumption during audio information processing.

[0211] In some further embodiments, in order to determine that the audio information includes the first audio that meets a specific condition, the first audio extraction module 12 may include:

[0212] A first determining unit is configured to detect that the audio information contains steady-state noise, and determine the steady-state noise as the first audio; or

[0213] A first acquisition unit, configured to acquire steady-state noise and non-steady-state noise contained in the audio information;

[0214] A second determination unit is used to determine that the steady-state noise and the non-steady-state noise meet specific conditions, and determine the steady-state noise as the first audio; or a third determination unit is used to determine that the steady-state noise and the non-steady-state noise meet specific conditions and there is an abnormal frequency signal in the steady-state noise, and determine the steady-state noise as the first audio.

[0215] In the practical application of the present application, if the above-mentioned steady-state noise includes electrical noise generated by the electronic device itself, the above-mentioned second determining unit and the third determining unit may both include:

[0216] A signal strength difference acquisition unit, used to acquire a signal strength difference between the steady-state noise and the non-steady-state noise;

[0217] The fourth determining unit is configured to determine that the signal strength difference is greater than a steady-state noise reduction threshold, where the steady-state noise reduction threshold is determined based on the signal strength of the ambient noise of the electronic device.

[0218] In some other embodiments, Fig.11 As shown, the first audio extraction module 12 may further include:

[0219] A sound source localization unit 121 is used to detect that the audio information contains user audio, perform sound source localization on the user audio, and obtain the position of the speaker of the user audio;

[0220] A position relationship determination unit 122, configured to determine that the position relationship between the speaker's position and the electronic device satisfies a specific condition, and determine the user audio as the first audio;

[0221] Accordingly, the above device may further include:

[0222] An identity authentication module 15, configured to authenticate the speaker of the first audio based on the first audio;

[0223] The voice control module 16 is used to wake up the voice recognition engine of the electronic device when the identity authentication result of the identity authentication module 15 is that the identity authentication is passed, so that the voice recognition engine executes the voice instruction included in the first audio.

[0224] It should be noted that the various modules, units, etc. in the above-mentioned device embodiments can be stored in the memory as program modules, and the processor executes the above-mentioned program modules stored in the memory to implement the corresponding functions. For the functions implemented by each program module and its combination, as well as the technical effects achieved, please refer to the description of the corresponding parts of the above-mentioned method embodiments, which will not be repeated in this embodiment.

[0225] Finally, it should be noted that the various embodiments in this specification are described in a progressive or parallel manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the various embodiments can be referred to each other. For the devices and electronic devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0226] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for processing audio information, the method comprising: Acquiring audio information while the electronic device is in operation; In the case where it is detected that the audio information contains a first audio that meets a specific condition, the first audio contained in the audio information is extracted; wherein, detecting that the audio information contains the first audio that meets the specific condition comprises: obtaining capacitive electrical noise generated by a capacitive element in an electronic device and non-steady-state actual environmental noise contained in the audio information; determining whether the capacitive electrical noise generated by the capacitive element in the electronic device meets a specific condition based on the capacitive electrical noise generated by the capacitive element in the electronic device and the non-steady-state actual environmental noise; if it is determined that the capacitive electrical noise generated by the capacitive element in the electronic device meets the specific condition, determining the capacitive electrical noise generated by the capacitive element in the electronic device as the first audio; the satisfying of the specific condition comprises that the signal strength difference between the capacitive electrical noise generated by the capacitive element in the electronic device and the non-steady-state actual environmental noise is greater than a noise reduction threshold; Based on the relative position relationship between the sound source of the first audio, the audio playback device, and the audio receiving object, obtaining an initial inverted first audio for the extracted first audio; The initial inverted first audio is sent to the audio playback device for output, so that the target inverted first audio transmitted to the audio receiving object and the first audio cancel each other out.

2. According to the method of claim 1, when the detecting that the audio information contains a first audio that satisfies a specific condition, extracting the first audio contained in the audio information comprises: The first processor detects whether the audio information includes a first audio that meets a specific condition; If yes, the first processor sends noise reduction prompt information to the second processor; The second processor extracts the first audio included in the audio information; In the same period of time, a first power consumption generated by the operation of the first processor is less than a second power consumption generated by the operation of the second processor. The method according to claim 1 , wherein the noise reduction threshold is determined based on a signal strength of ambient noise of the electronic device.

4. The method according to any one of claims 1 to 3, wherein obtaining the initial inverted first audio for the extracted first audio based on the relative position relationship among the sound source of the first audio, the audio playback device, and the audio receiving object comprises: Acquire a first sound source position of the first audio, a sound outlet position of an audio playback device, and a sound receiving position of an audio receiving object; According to the first sound source position, the sound outlet position and the sound receiving position, the first audio is subjected to inversion processing to obtain an initial inverted first audio; The step of sending the initial inverted first audio to an audio playback device for output comprises: The initial inverted first audio is mixed with the audio to be played, and the obtained mixed audio is sent to the audio playback device for output.

5. The method according to any one of claims 1 to 3, wherein the detecting that the audio information contains the first audio satisfying a specific condition further comprises: Detecting that the audio information includes user audio, performing sound source positioning on the user audio to obtain a position of a speaker of the user audio; Determining that the positional relationship between the speaker's position and the electronic device meets a specific condition, and determining the user audio as the first audio; The method further comprises: Based on the first audio, authenticating the speaker of the first audio; If the identity authentication is passed, the voice recognition engine of the electronic device is awakened, and the voice recognition engine executes the voice command contained in the first audio.

6. An audio information processing device, the device comprising: An audio information acquisition module, used to acquire audio information when the electronic device is in operation; A first audio extraction module is used to extract the first audio contained in the audio information when detecting that the audio information contains a first audio that satisfies a specific condition; wherein detecting that the audio information contains the first audio that satisfies a specific condition comprises: obtaining capacitive electrical noise generated by a capacitive element in an electronic device and non-steady-state actual environmental noise contained in the audio information; determining whether the capacitive electrical noise generated by the capacitive element in the electronic device satisfies a specific condition based on the capacitive electrical noise generated by the capacitive element in the electronic device and the non-steady-state actual environmental noise; if it is determined that the capacitive electrical noise generated by the capacitive element in the electronic device satisfies the specific condition, determining the capacitive electrical noise generated by the capacitive element in the electronic device as the first audio; satisfying the specific condition comprises that the signal strength difference between the capacitive electrical noise generated by the capacitive element in the electronic device and the non-steady-state actual environmental noise is greater than a noise reduction threshold; an inversion processing module, configured to obtain an initial inverted first audio for the extracted first audio based on a relative position relationship between a sound source of the first audio, an audio playback device, and an audio receiving object; The audio transmission module is used to send the initial inverted first audio to the audio playback device for output, so that the target inverted first audio transmitted to the audio receiving object and the first audio cancel each other out.

7. An electronic device, comprising a body, and electronic components, an audio acquisition device, an audio playback device, and an audio processing device disposed in the body, wherein: The audio collection device is used to collect audio information when the electronic device is running; The audio processing device is used to call and execute a first program to implement the steps of the audio information processing method as described in any one of claims 1 to 5, and the first program is a program for implementing the audio information processing method as described in any one of claims 1 to 5.

8. The electronic device according to claim 7, wherein the audio processing device comprises: A first processor is used to obtain audio information, detect that the audio information contains a first audio that meets a specific condition, and generate noise reduction prompt information; a second processor, configured to receive the noise reduction prompt information sent by the first processor, extract the first audio contained in the audio information, obtain an initial inverted first audio for the extracted first audio based on the relative position relationship between the sound source of the first audio, the audio playback device, and the audio receiving object, and send the initial inverted first audio to the audio playback device for output, so that the target inverted first audio transmitted to the audio receiving object and the first audio cancel each other out; In the same period of time, a first power consumption generated by the operation of the first processor is less than a second power consumption generated by the operation of the second processor.

9. The electronic device according to claim 7 or 8, wherein the first processor is disposed in the audio acquisition device; The audio collection device includes a plurality of audio collectors deployed in an array; The electronic component includes a capacitor element, and during the operation of the electronic device, the electronic component can generate steady-state noise.

Images