Noise reduction equipment and audio processing method and device thereof

Abstract

The invention provides an audio processing method of noise reduction equipment. The audio processing method comprises the following steps: acquiring a plurality of source audios; in response to a setting operation of a user, displaying a setting interface containing a plurality of source audios, so that the user sets the types of the plurality of source audios in different scene modes and generates a setting instruction; in response to the setting instruction, setting attributes of a plurality of source audios in different scene modes so as to obtain a plurality of sample audios; feeding the plurality of sample audios into a preset training model for learning to obtain a noise elimination model; picking up the sound of the noise reduction equipment in each direction in the current surrounding environment to obtain a plurality of to-be-processed audios, wherein each to-be-processed audio comprises a plurality of types of audios; utilizing a noise elimination model to identify attributes of various types of audios of the to-be-processed audios; and performing audio processing according to the attributes of various types of audios in the plurality of to-be-processed audios so as to perform noise reduction on each plurality of to-be-processed audios. In addition, the invention also provides noise reduction equipment and an audio processing device thereof.

Description

Technical Field

[0001] The present application relates to the field of voice playback technology, and in particular to a noise reduction device and an audio processing method and apparatus thereof. Background Art

[0002] The current active noise reduction technology mainly reduces noise by generating a reverse noise reduction signal, and all collected sounds are processed for noise reduction. However, this noise reduction method only performs noise reduction on sounds collected from any direction without differentiation. For example, when a sound with warning characteristics is collected, it will be regarded as noise for noise reduction due to the frequency difference with other sounds. The lack of directionality can easily cause the isolation of the real world and the "noise reduction world", resulting in the loss of audio. At the same time, this noise reduction method cannot adapt to different scenarios and perform noise reduction in different ways. For example, in outdoor scenes, the sounds of birds or insects in the field are often not expected to be filtered out, but they are regarded as noise for noise reduction due to their frequency difference with other sounds. Summary of the invention

[0003] In view of this, it is necessary to provide a noise reduction device and an audio processing method and apparatus thereof.

[0004] In a first aspect, an embodiment of the present application provides an audio processing method for a noise reduction device, the audio processing method for the noise reduction device comprising: obtaining a plurality of source audios, each of the source audios comprising a type of audio; in response to a user's setting operation, displaying a setting interface containing the plurality of source audios, so that the user can set the types of the plurality of source audios in different scene modes and generate setting instructions; in response to the setting instructions, setting the attributes of the plurality of source audios in different scene modes to obtain a plurality of sample audios, the sample audios comprising one or more scene tags and one or more attribute flags, each scene mode corresponding to a scene tag, the attribute flags comprising noise and non-noise; feeding the plurality of sample audios into a preset training model to learn and obtain a noise elimination model; picking up sounds from various directions of the current surrounding environment of the noise reduction device to obtain a plurality of audios to be processed, each of the audios to be processed comprising multiple types of audios; using the noise elimination model to identify the attributes of various types of audios of the audios to be processed; performing audio processing according to the attributes of various types of audios in the plurality of audios to be processed, so as to perform noise reduction on each of the plurality of audios to be processed.

[0005] In a second aspect, an embodiment of the present application provides an audio processing device of a noise reduction device, wherein the noise reduction device includes a sound sensor array, wherein the sound sensor array includes multiple sound sensors for acquiring audio in different directions; the audio processing device includes: an audio acquisition module, a setting module, a model training module, an audio pickup module, and an audio noise reduction module, wherein the audio acquisition module is used to acquire a number of source audios, wherein each source audio includes a type of audio; the setting module is used to respond to a user's setting operation, display a setting interface containing the number of source audios, so that the user can set the type of the number of source audios in different scene modes and generate setting instructions; the setting module is also used to respond to the setting instructions, set the attributes of the number of source audios in different scene modes, thereby obtaining a number of samples. Audio, the sample audio includes one or more scene tags and one or more attribute tags, each scene mode corresponds to a scene tag, and the attribute tags include noise and non-noise; the model training module is used to feed the several sample audios into a preset training model to learn and obtain a noise elimination model; the audio pickup module is used to pick up the sounds of various directions of the current surrounding environment of the noise reduction device to obtain several audios to be processed, and each audio to be processed includes multiple types of audios; the several audios to be processed are input into the noise elimination model for identification to obtain the attributes of various types of audios to be processed; the audio noise reduction module is used to perform audio processing according to the attributes of various types of audios in the several audios to be processed to reduce the noise of each of the several audios to be processed.

[0006] In a third aspect, an embodiment of the present application provides a noise reduction device, which includes a sound sensor array and a main control device, wherein the sound sensor array includes multiple sound sensors for acquiring audio in different directions; the main control device includes a memory and a processor, the memory is used to store a computer program; the processor is used to execute the computer program to implement the audio processing method of the above-mentioned noise reduction device.

[0007] The above-mentioned noise reduction device and its audio processing method and device obtain a number of source audios and respond to the user's setting operation to allow the user to set the types of the several source audios in different scene modes and generate setting instructions. When responding to the setting instructions, the properties of the several source audios in different scene modes are set to obtain a number of sample audios, and the several sample audios are fed into a preset training model to learn to obtain a noise elimination model. After picking up sounds from various directions of the current surrounding environment of the noise reduction device to obtain a number of audios to be processed, the noise elimination model is used to identify the properties of various types of audios to be processed, so as to perform noise reduction on each of the several audios to be processed, so as to adapt to different scenes and perform noise reduction in different noise reduction methods, thereby realizing intelligent noise reduction. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying any creative work.

[0009] Figure 1 This is a first flow chart of the audio processing method of the noise reduction device provided in an embodiment of the present application.

[0010] Figure 2 This is a first schematic diagram of a setting interface provided in an embodiment of the present application.

[0011] Figure 3 A second schematic diagram of the setting interface provided in an embodiment of the present application.

[0012] Figure 4 This is a second flow chart of the audio processing method of the noise reduction device provided in an embodiment of the present application.

[0013] Figure 5 A schematic diagram of the structure of an audio processing device of a noise reduction device provided in an embodiment of the present application.

[0014] Figure 6 A schematic diagram of the structure of the noise reduction device provided in an embodiment of the present application.

[0015] Figure 7 A schematic diagram of the internal structure of the main control device provided in an embodiment of the present application.

[0016] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0018] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar planning objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate. In other words, the described embodiments are implemented according to an order other than that illustrated or described herein. In addition, the terms "including" and "having" and any of their variations may also include other content. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to only those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0019] It should be noted that the descriptions involving "first", "second", etc. in this application are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in this field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0020] Please see Figure 1 , which is the first flow chart of the audio processing method of the noise reduction device provided in the embodiment of the present application. The present application provides an audio processing method for a noise reduction device. The noise reduction device includes a sound sensor array. The sound sensor array includes a plurality of sound sensors for acquiring audio in different directions. The audio processing method for the noise reduction device provided in the present application can set the type of each audio in different scene modes according to the user's setting operation and train the set audio, and then perform audio processing on the audio to be processed picked up by the current surrounding environment of the noise reduction device to realize intelligent noise reduction. The audio processing method for the noise reduction device includes steps S101-S106.

[0021] Step S101, obtaining some source audio.

[0022] In step S101, a plurality of source audios are obtained by the noise reduction device collecting and extracting the audios existing in the current surrounding environment, or by the user inputting the historical audios collected from other devices or by the noise reduction device. Each source audio includes a type of audio. The plurality of source audios may be, but are not limited to, different types of audios such as human voices, vehicle driving sounds, alarm sounds, bird calls, wind sounds, etc. The noise reduction device uses a sound sensor array to obtain various audios existing in the surrounding environment in real time.

[0023] Step S102, in response to the user's setting operation, a setting interface including a plurality of source audios is displayed, so that the user can set the types of the plurality of source audios in different scene modes and generate setting instructions.

[0024] In step S102, the user's setting operations include but are not limited to setting the scene mode of the source audio, setting the audio properties of the source audio in the scene mode, and other operations. The setting interface allows the user to input corresponding setting operations to complete the settings of each source audio. Different scene modes include but are not limited to pointing transparency mode, warning transparency mode, original sound transparency mode, full noise reduction mode, sleep mode and other modes. It can be understood that a scene mode corresponds to a usage scenario of the noise reduction device, and different scene modes can be determined by reading the usage scenarios corresponding to each historical audio stored locally or in the cloud of the noise reduction device. The setting interface will be explained in detail below.

[0025] Step S103, in response to the setting instruction, setting the properties of a plurality of source audios in different scene modes to obtain a plurality of sample audios.

[0026] In step S103, the sample audio includes one or more scene tags and one or more attribute tags. Each scene mode corresponds to a scene tag. The attribute tags include noise and non-noise, which are respectively used to mark the audio attributes of the source audio in the corresponding scene. In other words, for any source audio, it can be set to multiple scene modes and the audio attributes under each set scene mode. Therefore, for the same source audio, it can be noise or non-noise in different scene modes. The following will illustrate how to respond to user operations to set different source audios to obtain several sample audios by schematically illustrating a setting interface.

[0027] Please see Figure 2 and Figure 3 , Figure 2 This is a first schematic diagram of a setting interface provided in an embodiment of the present application. Figure 3 A second schematic diagram of the setting interface provided in an embodiment of the present application.

[0028] like Figure 2As shown, the setting interface 222 can realize the setting of each source audio by generating a visual component that can be operated by the user. The user's operation mode of the visual component includes but is not limited to operations such as clicking and dragging on the setting interface. After obtaining the source audio corresponding to different types of audio such as source audio 1, source audio 2, source audio 3, source audio 4, source audio 5, etc., each source audio is configured to be displayed on the setting interface 222. The display mode of each source audio can be to assign a corresponding first visual component to each source audio for display through the setting interface 222. Each first visual component is configured to respond to the corresponding operation input by the user, so that the user can intuitively set each source audio. After receiving the setting operation of the user, the setting operation is parsed to obtain "setting source audio 1 to the noise of scene mode 1", "setting source audio 1 to the non-noise of scene mode 2, and setting source audio 2 to the noise of scene mode 2", "setting source audio 3 and source audio 4 to the noise of scene mode 3", "setting source audio 2 and source audio 3 to the non-noise of scene mode 4, and setting source audio 5 to the noise of scene mode 4" and other user expected setting operations for each source audio. Afterwards, according to the expected setting operation, the control setting interface 222 is configured to allocate a corresponding second visual component to each scene mode for display. Each second visual component is configured to establish an association relationship with other components. Whenever a first visual component establishes an association relationship with a second visual component, it indicates that the corresponding original audio has been set to the corresponding scene mode, and a corresponding setting instruction is generated to indicate that the corresponding source audio needs to further set its audio properties in the corresponding scene mode.

[0029] like Figure 3 As shown, since a source audio may be noise or non-noise in different scene modes, according to the expected setting operation, the setting interface 222 allocates corresponding third visual components to the scene tags such as scene tag 1, scene tag 2, scene tag 3, and scene tag 4 corresponding to the scene modes of each source audio to distinguish different scene modes, and allocates corresponding fourth visual components to the attribute flag 1 and attribute flag 2 corresponding to the scene modes of each source audio to distinguish the audio attributes of each source audio in each scene mode. Among them, attribute flag 1 and attribute flag 2 represent noise and non-noise respectively, so as to adapt to the use scenarios where any source audio may be noise or non-noise in different scene modes, and further improve the intelligence of noise reduction.

[0030] In this embodiment, the sample audio can be obtained according to the scene mode. Specifically, the sample audio includes a number of source audios in different scene modes, and the corresponding audio attributes of each source audio. For example, after the source audios are set through the above setting interface 222, the sample audio corresponding to scene mode 4 can be expressed as "scene mode 4, scene tag 4: (source audio 2, attribute tag 2), (source audio 3, attribute tag 2), (source audio 5, attribute tag 5)". Based on this, the source audios of each scene mode and their corresponding audio attributes can be intuitively illustrated.

[0031] In some feasible embodiments, the sample audio can also be obtained based on the source audio. Specifically, the sample audio includes several scene modes of a source audio and the audio attributes of the source audio in each scene mode. For example, after the above-mentioned setting interface 222 sets each source audio, the sample audio corresponding to source audio 1 can also be expressed as "source audio 1: (scene tag 1, attribute tag 1), (scene tag 2, attribute tag 2)". Based on this, the scene mode of each source audio and its corresponding audio attributes can be intuitively illustrated.

[0032] In the above embodiment, the display method, display style, operation method of the source audio, scene mode, setting instructions, scene mark, attribute flag, etc. in the setting interface 222, and each visual component generated by the setting interface 222 are only an example of the setting interface 222, not a limitation of the setting interface 222, and will not be elaborated here.

[0033] Step S104, feeding a number of sample audios into a preset training model to learn and obtain a noise elimination model.

[0034] In step S104, the preset training model is obtained by feeding each source audio and its audio attributes in different scene modes into the initial training model for training. The noise elimination model is used to describe the correlation between each audio and its corresponding noise reduction strategy in the scene mode. In this embodiment, the initial training model is an initial neural network model. The preset training model is a neural network model obtained after deep learning of the initial neural network model. The initial neural network model can be an existing neural network model, which will not be described in detail here.

[0035] Furthermore, the noise elimination model includes an audio recognition model and a noise reduction model. The audio recognition model is used to identify scenes of a plurality of audios to be processed according to scene tags. The noise reduction model is used to match corresponding noise reduction strategies to a plurality of audios to be processed according to the scenes of the plurality of audios to be processed, and to perform noise reduction processing on the corresponding audios to be processed according to the corresponding noise reduction strategies.

[0036] It is understandable that different audio scenes correspond to different noise reduction strategies. Specifically, scenes include but are not limited to warning transparency scenes, directional transparency scenes, human voice transparency scenes, original sound transparency scenes, sleeping scenes and other scenes. Optionally, scenes include warning transparency scenes, directional transparency scenes, human voice transparency scenes and original sound transparency scenes. Among them, the noise reduction strategy corresponding to the warning transparency scene is to transparently transmit the audio of the warning sound in the audio to be processed. The noise reduction strategy corresponding to the directional transparency scene is to transparently transmit the audio of the human voice in a specific direction in the audio to be processed. The noise reduction strategy corresponding to the human voice transparency scene is to transparently transmit the audio of the human voice. The noise reduction strategy corresponding to the original sound transparency scene is to transparently transmit the audio of the sounds of nature. The noise reduction strategies of each scene will be elaborated in detail below.

[0037] Step S105, picking up sounds from various directions of the current surrounding environment of the noise reduction device to obtain a number of audios to be processed.

[0038] In step S105, each audio to be processed includes multiple types of audio. Each position can be determined based on the pickup range of the sound sensor array in the noise reduction device, or can be determined based on the pickup range of a pickup component such as a miniature microphone integrated with the sound sensor array in the noise reduction device. For example, when the noise reduction device is used in a meeting, the pickup range of the sound sensor array can be adjusted according to the different positions of each participant in the meeting, so that the sound sensor can collect multiple types of audio in each position in the meeting and obtain a number of audio to be processed.

[0039] Step S106: using the noise elimination model to identify the properties of various types of audio to be processed.

[0040] In step S106, after the sound sensor array picks up a number of audios to be processed, the noise elimination model parses each audio to be processed to obtain the audio frequencies of various audios in each audio to be processed. The noise elimination model obtains scene tags of various types of audio by comparing the audio frequencies of various audios with the audio frequencies of each source audio. The noise elimination model obtains the scene mode adapted to the current surrounding environment through the scene tags of various audios to obtain the audio properties of various audios in the adapted scene mode. Therefore, when the source audios and their corresponding scene tags corresponding to sufficient scene modes have been set through the user's setting operation, the scene modes adapted to various types of audio in a number of audios to be processed can be determined through the scene tags of various types of audio, and then the audio properties of various types of audio can be determined.

[0041] Step S107: performing audio processing according to the properties of various types of audio in the plurality of audios to be processed, so as to perform noise reduction on each of the plurality of audios to be processed.

[0042] In step S107, the audio processing includes sound elimination processing and sound enhancement processing. The sound elimination processing includes the following steps: determining the orientation of a plurality of audios to be processed; outputting the audios in the opposite orientation to offset one or more types of audios with the attribute mark of noise. The sound enhancement processing includes the following steps: amplifying one or more types of audios with the attribute mark of non-noise. The specific steps of each audio processing will be described in detail below.

[0043] In this embodiment, when performing sound elimination processing, after picking up a number of audios to be processed, the orientation of each audio to be processed is obtained according to the sound sensor array. Specifically, the audio picked up by each sound sensor is positioned based on the sound source localization technology, for example, beamforming, super-resolution spectrum estimation or based on the time difference of arrival (TDOA), etc., to form the orientation corresponding to each sound sensor, and then obtain the orientation of each audio. The orientation of each audio can be used to confirm whether each audio to be processed is related, such as whether different audios to be processed are picked up in the same orientation. In this embodiment, in order to realize the collection of multiple types of audio by the noise reduction device, each orientation is preset with multiple sensors for collecting different types of audio, so that each orientation can realize the pickup of different types of audio through a sufficient number of sound sensors, thereby improving the comprehensiveness of the audio pickup of the noise reduction device. Each audio to be processed is analyzed to obtain the audio frequencies of various types of audio in each audio to be processed. When the attribute mark of the audio is noise, the audio in the opposite orientation is output through the orientation number of each audio to be processed and the audio frequency of various types of audio to offset. The audio frequency of the reverse audio corresponds to the audio frequency of the audio whose audio attribute mark is noise.

[0044] In this embodiment, when performing sound enhancement processing, while amplifying the audio, it is also necessary to ensure that each audio with the attribute mark of non-noise is within the normal acceptance range of the user. For example, in a noisy environment, the recognized human voice audio is subjected to sound enhancement processing such as wavelet transform method to ensure that both parties can communicate normally.

[0045] In the above embodiment, the noise reduction strategy corresponding to the plurality of audios to be processed is to perform sound elimination processing or sound enhancement processing on various types of audios in each audio to be processed according to the scene mode, so as to achieve transparent transmission of audios in different scene modes. The following will specifically describe how to achieve transparent transmission of audios in different scene modes.

[0046] In this embodiment, transparently transmitting the audio of the warning sound in the audio to be processed specifically includes performing sound enhancement processing on the audio of the warning sound. Specifically, it is determined whether there is an audio of the warning sound in each audio to be processed. When there is an audio of the warning sound in the audio to be processed, the audio of the warning sound in the corresponding audio to be processed is enhanced, and other audio is sound eliminated or not processed.

[0047] In the present embodiment, transparent transmission of the audio of human voice in a specific direction in the audio to be processed is specifically to perform sound elimination processing on the audio that is not human voice in the specific direction. Specifically, it is determined whether each audio direction is a specific direction. For example, the front, left front side and right front side relative to the noise reduction device are set as specific directions. At this time, it is necessary to determine whether each audio direction is one of the front, left front side and right front side. When it is determined that the audio to be processed is in a specific direction, it is determined whether there is human voice audio in the corresponding audio to be processed. When there is human voice audio in the corresponding audio to be processed, the non-human voice audio in the corresponding audio to be processed and other audio to be processed that is not in the specific direction are all subjected to sound elimination processing.

[0048] In this embodiment, the transparent transmission of the audio of the human voice is specifically to perform sound enhancement processing on the audio of the human voice. Specifically, it is determined whether there is human voice audio in each audio to be processed. When there is human voice audio in the audio to be processed, the sound enhancement processing is performed on the human voice audio in the corresponding audio to be processed, and the sound elimination processing is performed on other audio or it is not processed.

[0049] In the present embodiment, the transparent transmission of the audio of the nature sound specifically includes performing sound elimination processing on the audio that is not the nature sound, or performing sound elimination processing on the audio that is not the nature sound and the audio that is part of the nature sound. Specifically, it is first determined whether there is nature sound audio in each audio to be processed. When there is nature sound audio in the audio to be processed, all the audio that is not nature sound audio in the audio to be processed is subjected to sound elimination processing. If the attributes of various audio in the nature sound audio are adjusted through the user's setting operation, the attributes set for each audio in the nature sound audio after the response setting operation are obtained to determine whether the various audio contained in the nature audio in the audio to be processed is noise or non-noise, and then the audio in the nature sound audio whose attribute is marked as noise is subjected to sound elimination processing.

[0050] Please see Figure 4 , which is a second flow chart of the audio processing method of the noise reduction device provided in the embodiment of the present application. The audio processing method of the noise reduction device also includes steps S201-S202.

[0051] Step S201: receiving feedback information input by a user.

[0052] In step S201, due to the different life scenes of different users, the same audio may be accepted differently by different users in the same scene mode, and may be confirmed as noise or non-noise. Therefore, when receiving the user's setting operation, a corresponding feedback mechanism can also be established to receive the user's feedback information on the output audio after audio processing, so that the audio that has been processed in the same scene mode can be reprocessed when necessary, thereby improving the intelligence of the noise reduction device.

[0053] Step S202: According to the feedback information input by the user, the noise elimination model is modified or noise reduction is performed on the audio to be processed.

[0054] In step S202, the user can pre-record or select audio with a property mark of non-noise predetermined by the system, such as recording the voice information of relatives, selecting the property mark of source audio such as alarm sound, bird sound, wind sound as non-noise in each scene mode, and obtain the corresponding sample audio. When the feedback information input by the user is received through the feedback mechanism to obtain feedback information such as "the bird sound is too noisy in a certain scene mode", the original "the property mark of the bird sound audio is non-noise" in the corresponding scene mode can be adjusted to "the property mark of the bird sound audio is noise" through the feedback information, and the noise elimination model is corrected based on the adjustment to obtain the corrected noise elimination model, or the corresponding audio such as the bird sound audio in the audio to be processed or the bird sound audio that has been processed can be directly de-noised based on the feedback information.

[0055] Please see Figure 5 , which is a structural diagram of the audio processing device of the noise reduction device provided in the embodiment of the present application. The present application also provides an audio processing device 2 of a noise reduction device. Among them, the noise reduction device 1000 includes a sound sensor array 1. The sound sensor array 1 includes a plurality of sound sensors 11 for acquiring audio in different directions. The audio processing device 2 of the noise reduction device includes an audio acquisition module 21, a setting module 22, a model training module 23, an audio pickup module 24, and an audio noise reduction module 25. In this embodiment, the audio acquisition module 21, the setting module 22, the model training module 23, the audio pickup module 24, and the audio noise reduction module 25 can be a program module, or hardware with corresponding functions, such as: a chip, a module, etc., or any combination of software and hardware. Each module is a separate program module or an independent hardware, or multiple modules can be integrated in a program module or a hardware, which is not limited here.

[0056] The audio acquisition module 21 is used to acquire a plurality of source audios, each of which includes a type of audio.

[0057] The setting module 22 is used to respond to the user's setting operation, display a setting interface 222 containing a plurality of source audios, so that the user can set the types of the plurality of source audios in different scene modes and generate setting instructions. The setting module 22 is also used to respond to the setting instructions, set the attributes of the plurality of source audios in different scene modes to obtain a plurality of sample audios, the sample audios include one or more scene tags, and one or more attribute flags, each scene mode corresponds to a scene tag, and the attribute flags include noise and non-noise.

[0058] The model training module 23 is used to feed a number of sample audios into a preset training model to learn and obtain a noise elimination model.

[0059] The audio pickup module 24 is used to pick up the sounds of various directions of the current surrounding environment of the noise reduction device to obtain a number of audios to be processed, each of which includes multiple types of audios. The multiple audios to be processed are input into the noise elimination model for identification to obtain the properties of various types of audios to be processed.

[0060] The audio noise reduction module 25 is used to perform audio processing according to the properties of various types of audio in the plurality of audios to be processed, so as to perform noise reduction on each of the plurality of audios to be processed.

[0061] In this embodiment, the audio processing device 2 of the noise reduction device further includes a feedback module 26. The feedback module 26 includes an information feedback unit 261 and a correction unit 262. The information feedback unit 261 is used to receive feedback information input by a user. The correction unit 262 is used to correct the noise elimination model or perform noise reduction on the audio to be processed according to the feedback information input by the user.

[0062] Please see Figure 6 , which is a structural diagram of the noise reduction device provided in an embodiment of the present application. The present application also provides a noise reduction device 1000. The noise reduction device 1000 includes a sound sensor array 1 and a main control device 100. The sound sensor array 1 includes a plurality of sound sensors 11 for acquiring audio in different directions. The main control device 100 is used to implement an audio processing method of the noise reduction device. The audio processing method of the noise reduction device has been described in detail above and will not be repeated here.

[0063] Please see Figure 7 , which is a schematic diagram of the internal structure of the main control device provided in the embodiment of the present application. The main control device 100 includes a memory 901 and a processor 902. The processor 902 is used to run the computer program instructions in the memory 901 to implement the audio processing method of the noise reduction device.

[0064] The memory 901 includes at least one type of readable storage medium, which includes a flash memory, a hard disk, a multimedia card, a card-type memory (for example, an SD or DX memory, etc.), a magnetic memory, a disk, an optical disk, etc. In some embodiments, the memory 901 may be an internal storage unit of a computer device, such as a hard disk of a computer device. In other embodiments, the memory 901 may also be a storage device of an external computer device, such as a plug-in hard disk configured in a computer device, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Further, the memory 901 may also include both an internal storage unit of a computer device and an external storage device. The memory 901 may not only be used to store application software and various types of data installed in a computer device, such as the code of an audio processing method of a noise reduction device, etc., but may also be used to temporarily store data that has been output or is to be output.

[0065] Furthermore, the noise reduction device 1000 further includes a bus 903. The bus 903 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0066] Furthermore, the noise reduction device 1000 may also include a display component 904. The display component 904 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an organic light-emitting diode (OLED) touch device, etc. The display component 904 may also be appropriately referred to as a display device or a display unit, which is used to display information processed in the noise reduction device 1000 and to display a visual user interface.

[0067] Furthermore, the noise reduction device 1000 may also include a communication component 905. The communication component 905 may optionally include a wired communication component and / or a wireless communication component (such as a Wi-Fi communication component, a Bluetooth communication component, etc.), which is generally used to establish a communication connection between the noise reduction device 1000 and other computer devices.

[0068] Figure 7 Only some components and the audio processing method of the noise reduction device 1000 are shown. It can be understood by those skilled in the art that: Figure 7 The structure shown does not constitute a limitation on the noise reduction device 1000 , and the noise reduction device 1000 may include fewer or more components than shown in the figure, or combine some components, or arrange the components differently.

[0069] In the above embodiments, all or part of the embodiments may be implemented by software, hardware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0070] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, a process or function according to an embodiment of the present invention is generated in whole or in part. The computer device may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website, a computer, a server, or a data center to another website, a computer, a server, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or a data center that includes one or more available media integrated therein. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive Solid State Disk (SSD)), etc.

[0071] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0072] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the unit is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0073] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0074] In addition, each functional unit in each embodiment of the present application can be integrated into one processing unit, or each unit can exist separately, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of software functional units.

[0075] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or all or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only storage medium (ROM, Read-Only Memory), random access storage medium (RAM, Random Access Memory), disk or optical disk, and other media that can store program codes.

[0076] In the above embodiment, by obtaining a number of source audios and responding to the user's setting operation to allow the user to set the types of the number of source audios in different scene modes and generate setting instructions, in response to the setting instructions, the properties of the number of source audios in different scene modes are set to obtain a number of sample audios, and the number of sample audios are fed into a preset training model to learn and obtain a noise cancellation model. After picking up sounds from various directions of the current surrounding environment of the noise reduction device to obtain a number of audios to be processed, the noise cancellation model is used to identify the properties of various types of audios to be processed, so as to perform noise reduction on each of the number of audios to be processed, so as to adapt to different scenes and perform noise reduction in different noise reduction methods, thereby realizing intelligent noise reduction.

[0077] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

[0078] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0079] The above examples are only preferred embodiments of the present application, and certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application still fall within the scope covered by the present application.

Claims

1. An audio processing method for a noise reduction device, characterized in that: The audio processing method of the noise reduction device comprises: Acquire a plurality of source audios, each source audio comprising a type of audio; In response to a setting operation by a user, a setting interface including the plurality of source audios is displayed, so that the user can set the types of the plurality of source audios in different scene modes and generate a setting instruction; In response to the setting instruction, the attributes of the plurality of source audios in different scene modes are set to obtain a plurality of sample audios, wherein the sample audios include one or more scene tags and one or more attribute tags, each scene mode corresponds to a scene tag, and the attribute tags include noise and non-noise; Feeding the plurality of sample audios into a preset training model to learn and obtain a noise elimination model; Picking up sounds from various directions of the current surrounding environment of the noise reduction device to obtain a plurality of audios to be processed, each of which includes multiple types of audios; Using the noise cancellation model to identify the properties of various types of audio to be processed; and Audio processing is performed according to the properties of various types of audio in the plurality of audios to be processed, so as to reduce noise for each of the plurality of audios to be processed.

2. The audio processing method of the noise reduction device according to claim 1, characterized in that: The audio processing includes sound elimination processing and sound enhancement processing, wherein: The sound elimination process includes: Determining the locations of the plurality of audios to be processed; and Outputting audio in the opposite direction to cancel out one or more types of audio marked as noise; The sound enhancement process includes: One or more types of audio that are marked as non-noise are amplified.

3. The audio processing method of the noise reduction device according to claim 1, characterized in that: The noise elimination model includes an audio recognition model and a noise reduction model. The audio recognition model is used to identify the scenes of the plurality of audios to be processed according to the scene tags; the noise reduction model is used to match the plurality of audios to be processed with corresponding noise reduction strategies according to the scenes of the plurality of audios to be processed, and perform noise reduction processing on the corresponding audios to be processed according to the corresponding noise reduction strategies. Different audio scenes correspond to different noise reduction strategies.

4. The audio processing method of the noise reduction device according to claim 3, characterized in that: The scenarios include warning transparency scenarios, directional transparency scenarios, human voice transparency scenarios and original sound transparency scenarios. The noise reduction strategy corresponding to the warning transparency scenario is to transparently transmit the audio of the warning sound in the audio to be processed; the noise reduction strategy corresponding to the directional transparency scenario is to transparently transmit the audio of the human voice in a specific direction in the audio to be processed; the noise reduction strategy corresponding to the human voice transparency scenario is to transparently transmit the audio of the human voice; and the noise reduction strategy corresponding to the original sound transparency scenario is to transparently transmit the audio of the sounds of nature.

5. The audio processing method of the noise reduction device according to claim 4, characterized in that: The transparent transmission of the audio of the warning sound in the audio to be processed specifically includes sound enhancement processing on the audio of the warning sound; the transparent transmission of the audio of the human voice in a specific direction in the audio to be processed specifically includes sound elimination processing on the audio that is not the human voice in the specific direction; the transparent transmission of the audio of the human voice specifically includes sound enhancement processing on the audio of the human voice; the transparent transmission of the audio of the natural sounds specifically includes sound elimination processing on the audio that is not the natural sounds, or sound elimination processing on the audio that is not the natural sounds and the audio of part of the natural sounds.

6. The audio processing method of the noise reduction device according to claim 1, characterized in that: The audio processing method of the noise reduction device also includes: Receive feedback information from user input; According to the feedback information input by the user, the noise elimination model is modified or noise reduction is performed on the audio to be processed.

7. An audio processing device for a noise reduction device, characterized in that: The noise reduction device includes a sound sensor array, wherein the sound sensor array includes a plurality of sound sensors for acquiring audio in different directions; The audio processing device comprises: An audio acquisition module, used for acquiring a plurality of source audios, each of which includes a type of audio; A setting module is used to respond to a setting operation of a user, display a setting interface including the plurality of source audios, so that the user can set the types of the plurality of source audios in different scene modes and generate setting instructions; and in response to the setting instructions, set the attributes of the plurality of source audios in different scene modes to obtain a plurality of sample audios, wherein the sample audios include one or more scene tags and one or more attribute flags, each scene mode corresponds to a scene tag, and the attribute flags include noise and non-noise; A model training module, used for feeding the plurality of sample audios into a preset training model to learn and obtain a noise elimination model; An audio pickup module is used to pick up sounds from various directions of the current surrounding environment of the noise reduction device to obtain a plurality of audios to be processed, each of which includes multiple types of audios; the plurality of audios to be processed are input into the noise elimination model for recognition to obtain properties of the various types of audios of the audios to be processed; and The audio noise reduction module is used to perform audio processing according to the properties of various types of audio in the plurality of audios to be processed, so as to perform noise reduction on each of the plurality of audios to be processed.

8. The audio processing device of the noise reduction device according to claim 7, characterized in that: The noise elimination model includes an audio recognition model and a noise reduction model. The audio recognition model is used to identify the scenes of the plurality of audios to be processed according to the scene tags; the noise reduction model is used to match the plurality of audios to be processed with corresponding noise reduction strategies according to the scenes of the plurality of audios to be processed, and perform noise reduction processing on the corresponding audios to be processed according to the corresponding noise reduction strategies. Different audio scenes correspond to different noise reduction strategies.

9. The audio processing device of the noise reduction device according to claim 7, characterized in that: The audio processing device of the noise reduction device further includes a feedback module, and the feedback module includes: An information feedback unit, used to receive feedback information input by a user; and A correction unit is used to correct the noise elimination model or perform noise reduction on the audio to be processed according to the feedback information input by the user.

10. A noise reduction device, characterized in that: The noise reduction device comprises: A sound sensor array, comprising a plurality of sound sensors for acquiring audio in different directions; and The main control device includes: a memory for storing a computer program; and A processor is used to execute the computer program to implement the audio processing method of the noise reduction device according to any one of claims 1 to 6.

Images