Directional noise reduction method and device, storage medium and electronic equipment
By classifying sound information, extracting feature and periodic analysis, and combining with the instructions of the target device, directional noise reduction is achieved, solving the problem that the immersive experience device cannot reduce directional noise, resulting in poor sound effect experience, and improving the user experience.
Patent Information
- Application Number
- CN202411375799.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-29
- Publication Date
- 2025-02-07
AI Technical Summary
Immersive experience devices cannot be directed to cancel noise, resulting in poor user sound experience.
By performing classification processing, feature extraction and periodic feature analysis on the input sound information, the integrity of the sound signal is determined, and the category identification is sent to the target device according to the target threshold, and the cancellation or retention instructions are obtained to achieve directional noise reduction.
It realizes directional cancellation of specific sounds in ambient sound information, improves the user's sound effect experience, and solves the problem that immersive experience devices cannot be directional noise reduction.
Smart Images
Figure CN119400195A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computers, and in particular to a method, device, storage medium and electronic device for directional noise reduction. Background Art
[0002] With the advancement of technology, the functions and forms of electronic devices such as AR / VR are also rapidly iterating and developing. People have higher and higher requirements for the interactivity and experience of product equipment, which in turn requires continuous advancement of technology. The sound experience for VR, AR and other devices is also getting higher and higher. When users do not want to hear a specific sound from the environment, the existing VR, AR and other devices cannot perform directional noise reduction, which will result in a poor sound experience for users. Summary of the invention
[0003] The present application provides a method, device, storage medium and electronic device for directional noise reduction to solve the technical problem that an immersive experience device cannot perform directional noise reduction, resulting in a poor sound experience.
[0004] In a first aspect, the present application provides a method for directional noise reduction, comprising: classifying an input sound information to obtain at least one sound signal and a category identifier corresponding to each of the sound signals; performing feature extraction on each of the sound signals to obtain feature information corresponding to each of the sound signals, and performing periodic feature analysis on each of the sound signals based on the feature information to obtain feature analysis results for each of the sound signals; when the feature analysis result of the sound signal indicates that the sound signal is a non-periodic sound signal, using the non-periodic sound signal as a first sound signal, determining the integrity of the first sound signal based on the feature information of the first sound signal, and when the integrity of the first sound signal is greater than or equal to a target threshold, sending a first category identifier corresponding to the first sound signal to a target device, so that the target device returns an elimination instruction or a retention instruction for the first sound signal; when receiving an elimination instruction for the first sound signal returned by the target device, eliminating the first sound signal in the sound information and outputting the noise-reduced sound information.
[0005] In a second aspect, the present application provides a device for directional noise reduction, comprising: a classification module, used to classify an input sound information to obtain at least one sound signal and a category identifier corresponding to each of the above sound signals; an extraction module, used to extract features from each of the above sound signals to obtain feature information corresponding to each of the above sound signals, and perform periodic feature analysis on each of the above sound signals based on the feature information to obtain feature analysis results for each of the above sound signals; a first sending module, used to, when the feature analysis result of the above sound signal indicates that the above sound signal is a non-periodic sound signal, use the above-mentioned non-periodic sound signal as the first sound signal, determine the integrity of the above-mentioned first sound signal based on the feature information of the above-mentioned first sound signal, and when the integrity of the above-mentioned first sound signal is greater than or equal to a target threshold, send the first category identifier corresponding to the above-mentioned first sound signal to a target device, so that the above-mentioned target device returns an elimination instruction or a retention instruction for the above-mentioned first sound signal; a first elimination module, used to, when receiving an elimination instruction for the above-mentioned first sound signal returned by the above-mentioned target device, eliminate the above-mentioned first sound signal in the above-mentioned sound information and output the noise-reduced sound information.
[0006] As an optional example, the above-mentioned device also includes: a second sending module, which is used to perform periodic characteristic analysis on each of the above-mentioned sound signals according to the above-mentioned characteristic information to obtain the characteristic analysis results of each of the above-mentioned sound signals. When the characteristic analysis results of the above-mentioned sound signals indicate that the above-mentioned sound signals are periodic sound signals, the above-mentioned periodic sound signal is used as the second sound signal, and the second category identifier corresponding to the above-mentioned second sound signal is sent to the above-mentioned target device, so that the above-mentioned target device returns an elimination instruction or a retention instruction for the above-mentioned second sound signal; a second elimination module, which is used to eliminate the above-mentioned second sound signal in the above-mentioned sound information and output the noise-reduced sound information when receiving the elimination instruction of the above-mentioned second sound signal returned by the above-mentioned target device.
[0007] As an optional example, the extraction module includes: a processing unit, used to take each of the sound signals as a current sound signal, and perform the following operations on the current sound signal: extracting current feature information of the current sound signal according to the current category identifier of the current sound signal, wherein the current feature information is a feature associated with the current category identifier of the current sound signal, and when the current category identifier is a human voice, the current feature information includes timbre features, loudness features and tone features.
[0008] As an optional example, the first sending module includes: a matching unit, used to perform feature matching on the feature information of the first sound signal with the feature information of each sound signal in the sound database to obtain a matching degree between the first sound signal and each sound signal in the sound database; and a determination unit, used to determine the maximum matching degree among all matching degrees as the completeness of the first sound signal.
[0009] As an optional example, the above-mentioned device also includes: a saving module, which is used to save the above-mentioned first sound signal and its characteristic information in the sound database after determining the integrity of the above-mentioned first sound signal according to the characteristic information of the above-mentioned first sound signal, if the above-mentioned integrity is less than the above-mentioned target threshold.
[0010] As an optional example, the first elimination module includes: a first generation unit, which is used to, upon receiving an elimination instruction for the first sound signal returned by the target device, refer to the sound signal corresponding to the maximum match in the sound database, perform time-frequency analysis on the first sound signal, and generate a first inverted signal of the first sound signal; a first elimination unit, which is used to eliminate the first sound signal in the sound information through the first inverted signal, and output the noise-reduced sound information.
[0011] As an optional example, the second elimination module includes: an acquisition unit for acquiring the amplitude and phase of each cycle of the second sound signal; a second generation unit for generating a second inverted signal of the second sound signal, wherein the second inverted signal is equal to the amplitude of each cycle of the second sound signal, and the second inverted signal is opposite to the phase of each cycle of the second sound signal; a second elimination unit for eliminating the second sound signal in the sound information through the second inverted signal, and outputting the noise-reduced sound information.
[0012] In a third aspect, the present application provides a storage medium storing a computer program, wherein the computer program executes the above-mentioned directional noise reduction method when executed by a processor.
[0013] In a fourth aspect, the present application further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the above-mentioned directional noise reduction method through the computer program.
[0014] In an embodiment of the present application, a classification process is adopted for a segment of input sound information to obtain at least one sound signal and a category identifier corresponding to each of the above sound signals; feature extraction is performed on each of the above sound signals to obtain feature information corresponding to each of the above sound signals, and periodic feature analysis is performed on each of the above sound signals based on the above feature information to obtain feature analysis results for each of the above sound signals; when the feature analysis result of the above sound signal indicates that the above sound signal is a non-periodic sound signal, the above non-periodic sound signal is used as the first sound signal, the integrity of the above first sound signal is determined based on the feature information of the above first sound signal, and when the integrity of the above first sound signal is greater than or equal to the target threshold, the corresponding A first category identifier is sent to a target device so that the target device returns an elimination instruction or a retention instruction for the first sound signal; upon receiving the elimination instruction for the first sound signal returned by the target device, the first sound signal in the sound information is eliminated, and the noise-reduced sound information is output. In the above method, the characteristic information of each sound signal in the ambient sound information can be continuously learned and the characteristic information of the sound signal can be recorded. When it is necessary to eliminate a specific sound in the ambient sound information, the sound directional elimination mode is enabled to directionally eliminate a certain sound in the ambient sound information, thereby achieving the purpose of directional noise elimination and improving the user's sound experience, thereby solving the technical problem that the immersive experience device cannot directionally reduce noise, resulting in a poor sound experience. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0016] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0017] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.
[0018] Figure 1 is a flow chart of an optional directional noise reduction method according to an embodiment of the present application;
[0019] Figure 2is an overall implementation flow chart of an optional directional noise reduction method according to an embodiment of the present application;
[0020] Figure 3 is a flowchart of processing a non-periodic sound signal according to an optional directional noise reduction method of an embodiment of the present application;
[0021] Figure 4 is a schematic structural diagram of an optional directional noise reduction device according to an embodiment of the present application;
[0022] Figure 5 is a schematic diagram of an optional electronic device according to an embodiment of the present application. DETAILED DESCRIPTION
[0023] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.
[0024] The disclosure below provides many different embodiments or examples to realize the different structures of the present application. In order to simplify the disclosure of the present application, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present application. In addition, the present application can repeat reference numbers and / or letters in different examples. This repetition is for the purpose of simplification and clarity, and does not itself indicate the relationship between the various embodiments and / or settings discussed.
[0025] According to a first aspect of an embodiment of the present application, a method for directional noise reduction is provided. Optionally, as follows: Figure 1 As shown, the above method includes:
[0026] S102, classifying the input sound information to obtain at least one sound signal and a category identifier corresponding to each sound signal;
[0027] S104, extracting features from each sound signal to obtain feature information corresponding to each sound signal, and performing period feature analysis on each sound signal according to the feature information to obtain feature analysis results for each sound signal;
[0028] S106, when the characteristic analysis result of the sound signal indicates that the sound signal is a non-periodic sound signal, taking the non-periodic sound signal as the first sound signal, determining the integrity of the first sound signal according to the characteristic information of the first sound signal, and when the integrity of the first sound signal is greater than or equal to the target threshold, sending a first category identifier corresponding to the first sound signal to the target device, so that the target device returns a cancellation instruction or a retention instruction of the first sound signal;
[0029] S108, when receiving the instruction to eliminate the first sound signal returned by the target device, eliminate the first sound signal in the sound information and output the sound information after noise reduction.
[0030] Optionally, with the advancement of technology, the functions and forms of electronic devices such as AR / VR are also rapidly iterating and developing. People have higher and higher requirements for the interactivity and experience of product equipment. As a result, the continuous advancement of technology is required, and the sound experience for VR, AR and other devices is also getting higher and higher. When the user does not want to hear a specific sound from the environment, the existing VR, AR and other devices cannot perform directional noise reduction, which will lead to poor user sound experience. Therefore, in this embodiment, a first directional noise reduction method is proposed. Specifically, first, a piece of sound information collected is pre-processed. The sound information is the environmental sound information collected by the MIC (microphone) on the device, which contains at least one sound signal. The sound signal here can be background noise, human voice, music, the sound of vehicles driving on the road, the sound of roadside construction, etc. The sound information is analyzed and classified, and at least one sound signal contained therein is identified, and a corresponding category identifier is assigned to each sound signal. Through classification processing, different types of sound signals can be distinguished. Then, feature extraction is performed on each sound signal to extract feature information related to each sound signal. The characteristic information includes: timbre characteristics: describing the quality or timbre of the sound, whether it is a sound made by a person, an animal, or other types of sounds, which should be distinguished and recorded; loudness characteristics: describing the loudness or volume of the sound. If it is a human voice, AI should be used to learn and record the volume of the human voice. The same is true for other sounds; tone characteristics: refers to some special properties of the sound. For example, for people speaking, some people like to add a certain tone particle when saying a certain sentence. Such characteristics should also be processed and recorded. The same is true for other types of sounds. Certain features need to be recorded. Through feature extraction, the properties of each sound signal can be better characterized, which is convenient for subsequent analysis and processing. Then, based on the extracted characteristic information, a periodic feature analysis is performed on each sound signal to determine whether the sound signal is periodic. Periodic sound signals usually include continuous and stable audio signals, such as some mechanical noises or background music, while non-periodic sound signals include irregular and intermittent sound signals, such as human voices or environmental noise. When the periodic feature analysis results show that a sound signal is non-periodic, it is used as the first sound signal. Based on the characteristic information of the signal, its integrity is further determined. The integrity can measure the integrity and quality of the sound signal, such as whether the signal is missing, damaged or truncated. If the integrity of the first sound signal is greater than or equal to the preset target threshold, its category identifier is sent to the target device. The target device can be a device for sound processing that can return the corresponding processing signal, such as a mobile phone, computer, smart watch and other devices. Based on the category identifier, the target device returns an elimination instruction or a retention instruction for the sound signal, and the user decides whether to retain or eliminate a specific non-periodic sound signal.When the elimination instruction is received from the target device, the first sound signal is eliminated from the sound information, and the sound information after noise reduction processing is output, thereby effectively removing unnecessary non-periodic noise, thereby improving the quality and comprehensibility of the sound signal, reducing the interference of background noise on the main signal, and improving the user experience in scenarios such as voice communication and recording.
[0031] Optionally, in this embodiment, the characteristic information of each sound signal in the ambient sound information can be continuously learned and recorded. When it is necessary to eliminate a specific sound in the ambient sound information, the sound directional elimination mode is enabled to directionally eliminate a certain sound in the ambient sound information, thereby achieving the purpose of directional noise elimination and improving the user's sound experience, thereby solving the technical problem that the immersive experience device cannot directionally reduce noise, resulting in a poor sound experience.
[0032] As an optional example, after performing periodic feature analysis on each sound signal according to the feature information to obtain a feature analysis result of each sound signal, the method further includes:
[0033] When the characteristic analysis result of the sound signal indicates that the sound signal is a periodic sound signal, the periodic sound signal is used as the second sound signal, and the second category identifier corresponding to the second sound signal is sent to the target device, so that the target device returns a cancellation instruction or a retention instruction of the second sound signal;
[0034] When receiving an instruction to eliminate the second sound signal returned by the target device, the second sound signal in the sound information is eliminated, and the sound information after noise reduction is output.
[0035] Optionally, in this embodiment, when the periodic feature analysis result indicates that a sound signal is non-periodic, it is used as the second sound signal, and its category identifier is sent to the target device. The target device returns an elimination instruction or a retention instruction for the sound signal based on the category identifier, and the user decides whether to retain or eliminate a specific periodic sound signal. When the elimination instruction returned by the target device is received, the second sound signal is eliminated from the sound information, and the sound information after noise reduction processing is output, thereby effectively removing unnecessary periodic noise, thereby improving the quality and comprehensibility of the sound signal, reducing the interference of background noise on the main signal, and improving the user experience in scenarios such as voice communication and recording.
[0036] As an optional example, feature extraction is performed on each sound signal to obtain feature information corresponding to each sound signal, including:
[0037] Take each sound signal as the current sound signal and perform the following operations on the current sound signal:
[0038] According to the current category identifier of the current sound signal, current feature information of the current sound signal is extracted, wherein the current feature information is a feature associated with the current category identifier of the current sound signal. When the current category identifier is a human voice, the current feature information includes a timbre feature, a loudness feature, and a tone feature.
[0039] Optionally, in this embodiment, feature extraction is performed on each sound signal, and for each sound signal, the type of the signal is determined by its category identifier. For example, human voice, background noise, music, etc. After identifying that the category identifier of a sound signal is "human voice", feature information associated with the category is further extracted. The extraction basis of these feature information is the category of the sound signal. Different categories of sound signals will have different feature information. For human voice, the main feature information includes the following points: timbre feature: timbre is a unique attribute of human voice, which is used to distinguish the voices of different people. The timbre feature can be characterized by parameters such as the frequency distribution and harmonic structure of the sound. Loudness feature: Loudness is the strength of human voice, which is usually related to the amplitude of the sound. Tone feature: Tone is the emotional expression of speech, which is expressed through intonation, speaking speed, stress, etc. After extracting the human voice features (timbre, loudness, tone) of the sound signal, these features are combined to further analyze the sound signal. For example, the speaker can be identified by analyzing the timbre characteristics, and the speaker's emotions and voice environment can be identified by the loudness and tone characteristics. All these feature information together help the system process and classify the sound signal more accurately. By combining multiple features such as timbre, loudness, and tone, the system can more accurately distinguish different sound signals, especially in the processing of human voices, and can perform noise reduction processing based on the speaker's characteristics and emotions.
[0040] As an optional example, determining the integrity of the first sound signal according to the characteristic information of the first sound signal includes:
[0041] Performing feature matching on the feature information of the first sound signal and the feature information of each sound signal in the sound database to obtain a matching degree between the first sound signal and each sound signal in the sound database;
[0042] The maximum matching degree among all matching degrees is determined as the integrity degree of the first sound signal.
[0043] Optionally, in order to achieve feature matching between the feature information of the target sound signal and the feature information of each sound signal in the sound database, and determine the integrity of each sound signal accordingly, in this embodiment, the feature information of each sound signal is matched with the feature information of each sound signal in the sound database. The sound database is continuously learned by AI (Artificial Intelligence), and the collected sound signals are continuously improved and preserved, and finally a relatively complete database is formed. A matching algorithm (such as Euclidean distance, cosine similarity or other appropriate feature matching methods) is used to calculate the matching degree between each sound signal and each sound signal in the database. Among all the calculated matching degrees, the maximum matching degree is selected as the integrity of the sound signal. This maximum matching degree represents the similarity between the sound signal and the closest sound signal in the database. If the integrity of the sound signal is greater than or equal to the preset target threshold, it is considered that the sound signal is complete and matches the closest sound signal in the database.
[0044] As an optional example, after determining the integrity of the first sound signal according to the characteristic information of the first sound signal, the method further includes:
[0045] When the completeness is less than the target threshold, the first sound signal and its characteristic information are stored in the sound database.
[0046] Optionally, in this embodiment, if the completeness of a sound signal is less than a preset target threshold, the sound signal is considered to be incomplete, and there is no sound signal matching the sound information in the database, then AI technology is used to learn the characteristic information of the sound signal, in this way the sound signal is made into a three-dimensional record and saved in the sound database.
[0047] Optionally, in this embodiment, if the integrity of the sound signal is greater than or equal to a preset first threshold value, the first threshold value may be 70%, and according to the characteristic information of the sound signal, the sound database is queried to find the corresponding sound data, and the sound signal is used to update or improve the corresponding sound data in the sound database. This includes correcting, supplementing or replacing existing data to improve the accuracy and integrity of the sound data in the sound database, thereby continuously improving and preserving the collected sound signals through continuous learning of AI to continuously improve the sound database.
[0048] As an optional example, in the case of receiving an instruction to eliminate the first sound signal returned by the target device, eliminating the first sound signal in the sound information, and outputting the sound information after noise reduction includes:
[0049] When receiving a first sound signal elimination instruction returned by the target device, refer to the sound signal corresponding to the maximum matching degree in the sound database, perform time-frequency analysis on the first sound signal, and generate a first inverted signal of the first sound signal;
[0050] The first sound signal in the sound information is eliminated through the first inverted signal, and the sound information after noise reduction is output.
[0051] Optionally, in this embodiment, when the elimination instruction returned by the target device is received, it means that the current first sound signal needs to be eliminated. The target device determines that the sound signal is unnecessary noise or interference signal according to its algorithm, so it needs to process the signal. The reference sound database is compared with the current first sound signal, and the sound signal closest to it in the database is found to find the signal with the maximum matching degree. The signal with the highest matching degree will be used as a reference signal for subsequent time-frequency analysis and anti-phase signal generation. According to the reference sound signal and the current first sound signal, time-frequency analysis is performed, and the time-frequency analysis decomposes the sound signal in detail in two dimensions of time and frequency. Through time-frequency analysis, the frequency distribution of the first sound signal and its change in time can be accurately understood, and then the first anti-phase signal is generated. The first anti-phase signal is opposite to the original signal in frequency and phase, so it can be used to offset the original signal. Once the first anti-phase signal of the first sound signal is generated, the effect of eliminating the sound signal is achieved by superimposing the first anti-phase signal with the original first sound signal, and the sound information after noise reduction processing is output. By generating an anti-phase signal and directly superimposing it on the original signal, it is possible to quickly and effectively remove unnecessary noise and improve the clarity and quality of the sound information.
[0052] As an optional example, in the case of receiving an instruction to eliminate the second sound signal returned by the target device, eliminating the second sound signal in the sound information, and outputting the sound information after noise reduction includes:
[0053] Acquire the amplitude and phase of each cycle of the second sound signal;
[0054] Generate a second inverted signal of the second sound signal, wherein the amplitude of the second inverted signal is equal to that of the second sound signal in each cycle, and the phase of the second inverted signal is opposite to that of the second sound signal in each cycle;
[0055] The second sound signal in the sound information is eliminated through the second inverted signal, and the sound information after noise reduction is output.
[0056] Optionally, in this embodiment, the second sound signal is subjected to periodic analysis to obtain the amplitude and phase of each period of the sound signal. In each period, the corresponding amplitude is calculated. The amplitude indicates the strength of the sound signal and is the maximum amplitude of the sound waveform in the period. The phase indicates the starting point of each periodic signal in time. In order to generate an anti-phase signal, it is necessary to accurately obtain the phase of each period and prepare for the phase reversal in the subsequent steps. After completing the extraction of the amplitude and phase, the system needs to generate a second anti-phase signal of the second sound signal. The generation rule of the second anti-phase signal is: equal amplitude: the amplitude of each period of the anti-phase signal is the same as the amplitude of the original second sound signal. Opposite phase: the phase of each period of the anti-phase signal is opposite to the phase of the original signal (phase reversal 180 degrees). After the second anti-phase signal is generated, the second anti-phase signal is superimposed with the original sound information to achieve the purpose of eliminating the second sound signal. Since the anti-phase signal is completely opposite in phase to the original signal, when the two signals are superimposed, their amplitudes cancel each other out, thereby eliminating the second sound signal. After the inverted signal is processed, the second sound signal is effectively eliminated, the remaining sound information no longer contains the noise signal, and the sound information after noise reduction is output. By carefully analyzing and inversely processing the periodic amplitude and phase of the second sound signal, noise can be eliminated more accurately and errors can be reduced.
[0057] In conjunction with an example for illustration, the present application relates to a method for directional sound noise reduction based on AI active learning technology, which can continuously learn the characteristics of sound information in the environment, record the characteristics of the sound, continuously improve the characteristics of a certain sound through AI learning, and save the characteristics of the sound information to the cloud database. When it is necessary to eliminate a specific sound in the environment, the AI intelligent noise reduction (directional sound elimination) mode is enabled to directionally eliminate a certain sound in the environment. Different from the traditional noise reduction mode, this method is based on AI active learning technology, which can continuously learn, improve and save the characteristics of a certain type of sound information in the environment. When the user does not want to hear a certain sound from the environment, the noise of a certain sound information can be selectively reduced. The continuous learning and improvement of AI technology will generate a relatively complete database for a certain sound, and save the data to the cloud, enriching the sound database while not occupying local memory.
[0058] The specific overall implementation process is as follows Figure 2 As shown:
[0059] 1. Preprocessing of sound signals: Perform basic preliminary processing on the sound information picked up by the MIC on the device. For example, use AI technology to analyze the sound information collected by the MIC and make simple classifications. For example, distinguish between human voices, music, the sound of vehicles driving on the road, and the sound of roadside construction. The purpose is to analyze the collected sounds with certain characteristics and make brief classifications to form independent sound information for subsequent processing.
[0060] 2. Extraction of different types of sound signals: After AI analysis and simple classification, the sound signals are classified and extracted to obtain sound information with certain characteristics. For example, for human speech, the voices of different people can be distinguished; for music, the specific piece of music can be distinguished.
[0061] 3. Determine whether the sound signal is periodic: Make a preliminary judgment and split the extracted sound signal to determine whether the sound signal is periodic or non-periodic, in preparation for the next step of processing. The detailed processing processes of periodic and non-periodic signals are as follows:
[0062] 1. Processing of periodic sound signals:
[0063] If it is a periodic sound signal, the sound signal will be presented on the APP, and the user can choose whether to eliminate the periodic signal in a directional manner. If directional elimination is selected, a sound signal with the same amplitude and opposite phase as each period of the periodic signal is generated according to the characteristics of the periodic signal to eliminate the sound in a directional manner.
[0064] 2. Processing of non-periodic sound signals:
[0065] as follows Figure 3 As shown in the figure, this is the processing flow for non-periodic sound signals:
[0066] For non-periodic signals, the above flowchart is used to explain in detail. The features that need to be learned and recorded include but are not limited to the following aspects:
[0067] (1) Sound timbre: Whether it is a sound made by a person, an animal, or other types of sounds, they must be distinguished and recorded.
[0068] (2) Loudness of sound: If it is the sound of a person speaking, AI should be used to learn and record the volume of the person’s voice. The same applies to other sounds.
[0069] (3) Other characteristics of sound: This mainly refers to some special properties of sound. For example, when people speak, some people like to add certain modal particles when they say a certain sentence. Such characteristics should also be processed and recorded. The same is true for other types of sounds, and certain characteristics need to be recorded.
[0070] If the sound collected is someone's voice, AI technology can be used to learn the timbre characteristics, sound loudness, speaking style, etc. of the person's voice. In this way, a three-dimensional record and preservation of the person's speech characteristics can be made for the purpose of improving the sound database. If it is other non-periodic sound signals, the same method can be used for processing. Through continuous learning by AI, and continuous improvement and preservation of the collected sound signals, a relatively complete database for a certain specific sound is eventually formed. When the completeness of a certain sound information stored in this database meets the set threshold, it can be used. There is no clear standard for the size of the threshold. The principle of setting this threshold is to match a certain sound information with certain characteristics stored in the database with the actual corresponding sound in reality. (For example: if a person's voice is stored in the database, the threshold is set to 90%. When the sound collected by MIC in reality is analyzed and processed, and the similarity with the sound in the database reaches 90%, then the two are considered to be matched.)
[0071] When the integrity of a sound signal meets the preset threshold, the sound information can be displayed in the APP, and the user can decide whether to eliminate a specific sound in a directional manner in the APP, thereby achieving the purpose of directional sound elimination. If directional elimination is selected, the sound information collected in real time needs to be used to generate the corresponding opposite sound signal using the sound information stored in the database to offset it, so that the purpose of eliminating specific non-periodic sound signals can be achieved.
[0072] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.
[0073] According to another aspect of the embodiment of the present application, a directional noise reduction device is also provided, such as Figure 4 As shown, including:
[0074] The classification module 402 is used to classify the input sound information to obtain at least one sound signal and a category identifier corresponding to each sound signal;
[0075] The extraction module 404 is used to extract features from each sound signal to obtain feature information corresponding to each sound signal, and to perform period feature analysis on each sound signal according to the feature information to obtain feature analysis results for each sound signal;
[0076] A first sending module 406 is used to, when the characteristic analysis result of the sound signal indicates that the sound signal is a non-periodic sound signal, use the non-periodic sound signal as the first sound signal, determine the integrity of the first sound signal according to the characteristic information of the first sound signal, and when the integrity of the first sound signal is greater than or equal to the target threshold, send a first category identifier corresponding to the first sound signal to the target device, so that the target device returns a cancellation instruction or a retention instruction of the first sound signal;
[0077] The first elimination module 408 is used to eliminate the first sound signal in the sound information and output the sound information after noise reduction when receiving the elimination instruction of the first sound signal returned by the target device.
[0078] It should be noted that the classification module 402 in this embodiment can be used to execute step S102 in the embodiment of the present application, the extraction module 404 in this embodiment can be used to execute step S104 in the embodiment of the present application, the first sending module 406 in this embodiment can be used to execute step S106 in the embodiment of the present application, and the first elimination module 408 in this embodiment can be used to execute step S108 in the embodiment of the present application.
[0079] As an optional example, the above device further includes:
[0080] A second sending module is used to perform periodic characteristic analysis on each sound signal according to the characteristic information to obtain the characteristic analysis results of each sound signal, and when the characteristic analysis results of the sound signal indicate that the sound signal is a periodic sound signal, use the periodic sound signal as the second sound signal, and send the second category identifier corresponding to the second sound signal to the target device, so that the target device returns a cancellation instruction or a retention instruction for the second sound signal;
[0081] The second elimination module is used to eliminate the second sound signal in the sound information and output the sound information after noise reduction when receiving the elimination instruction of the second sound signal returned by the target device.
[0082] As an optional example, the extraction module includes:
[0083] The processing unit is used to take each sound signal as a current sound signal and perform the following operations on the current sound signal:
[0084] According to the current category identifier of the current sound signal, current feature information of the current sound signal is extracted, wherein the current feature information is a feature associated with the current category identifier of the current sound signal. When the current category identifier is a human voice, the current feature information includes a timbre feature, a loudness feature, and a tone feature.
[0085] As an optional example, the first sending module includes:
[0086] a matching unit, configured to perform feature matching between feature information of the first sound signal and feature information of each sound signal in the sound database, and obtain a matching degree between the first sound signal and each sound signal in the sound database;
[0087] The determining unit is configured to determine the maximum matching degree among all matching degrees as the integrity of the first sound signal.
[0088] As an optional example, the above device further includes:
[0089] The storage module is used to store the first sound signal and its characteristic information in a sound database after determining the integrity of the first sound signal according to the characteristic information of the first sound signal, if the integrity is less than a target threshold.
[0090] As an optional example, the first elimination module includes:
[0091] A first generating unit is configured to, upon receiving a cancellation instruction of the first sound signal returned by the target device, refer to the sound signal corresponding to the maximum matching degree in the sound database, perform time-frequency analysis on the first sound signal, and generate a first inverted signal of the first sound signal;
[0092] The first elimination unit is used to eliminate the first sound signal in the sound information through the first inverted signal, and output the sound information after noise reduction.
[0093] As an optional example, the second elimination module includes:
[0094] An acquisition unit, used to acquire the amplitude and phase of each cycle of the second sound signal;
[0095] A second generating unit, configured to generate a second inverted signal of the second sound signal, wherein the amplitude of the second inverted signal is equal to that of the second sound signal in each cycle, and the phase of the second inverted signal is opposite to that of the second sound signal in each cycle;
[0096] The second elimination unit is used to eliminate the second sound signal in the sound information through the second inverted signal, and output the sound information after noise reduction.
[0097] For other examples of this embodiment, please refer to the above examples and will not be repeated here.
[0098] Figure 5 is a schematic diagram of an optional electronic device according to an embodiment of the present application, such as Figure 5 As shown, it includes a processor 502, a communication interface 504, a memory 506 and a communication bus 508, wherein the processor 502, the communication interface 504 and the memory 506 communicate with each other through the communication bus 508, wherein,
[0099] Memory 506, used for storing computer programs;
[0100] The processor 502 is used to execute the computer program stored in the memory 506 to implement the following steps:
[0101] Classify the input sound information to obtain at least one sound signal and a category identifier corresponding to each sound signal;
[0102] Extract features from each sound signal to obtain feature information corresponding to each sound signal, and perform periodic feature analysis on each sound signal according to the feature information to obtain feature analysis results for each sound signal;
[0103] When the characteristic analysis result of the sound signal indicates that the sound signal is a non-periodic sound signal, the non-periodic sound signal is used as the first sound signal, the integrity of the first sound signal is determined according to the characteristic information of the first sound signal, and when the integrity of the first sound signal is greater than or equal to the target threshold, a first category identifier corresponding to the first sound signal is sent to the target device, so that the target device returns a cancellation instruction or a retention instruction of the first sound signal;
[0104] When receiving a first sound signal elimination instruction returned by the target device, the first sound signal in the sound information is eliminated, and the sound information after noise reduction is output.
[0105] Optionally, in this embodiment, the communication bus may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industrial Standard Architecture) bus. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5The communication interface is used for communication between the above electronic device and other devices.
[0106] The memory may include RAM, or may include non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.
[0107] As an example, the memory 506 may include, but is not limited to, the classification module 402, the extraction module 404, the first sending module 406, and the first elimination module 408 in the directional noise reduction device. In addition, other module units in the directional noise reduction device may also be included but are not limited to, which will not be repeated in this example.
[0108] The above-mentioned processor can be a general-purpose processor, which can include but not be limited to: CPU (Central Processing Unit), NP (Network Processor), etc.; it can also be DSP (Digital Signal Processing), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
[0109] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments, and this embodiment will not be described in detail here.
[0110] It can be understood by those skilled in the art that Figure 5 The structure shown is for illustration only. The device for implementing the above-mentioned directional noise reduction method may be a terminal device, which may be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (Mobile Internet Devices, MID), a PAD, and other terminal devices. Figure 5 The structure of the electronic device is not limited. For example, the electronic device may also include Figure 5 More or fewer components (such as network interfaces, display devices, etc.) shown in, or having Figure 5 Different configurations shown.
[0111] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, which can include: a flash drive, ROM, RAM, a magnetic disk or an optical disk, etc.
[0112] According to another aspect of the embodiments of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program executes the steps in the above-mentioned directional noise reduction method when executed by a processor.
[0113] Optionally, in this embodiment, a person of ordinary skill in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing hardware related to the terminal device through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.
[0114] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.
[0115] If the integrated units in the above embodiments are implemented in the form of software functional units and sold or used as independent products, they can be stored in the above computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling one or more computer devices (which may be personal computers, servers, or network devices, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application.
[0116] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.
[0117] In the several embodiments provided in the present application, it should be understood that the disclosed client can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.
[0118] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0119] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0120] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.
Claims
1. A method for directional noise reduction, characterized in that: include: Classify the input sound information to obtain at least one sound signal and a category identifier corresponding to each sound signal; Performing feature extraction on each of the sound signals to obtain feature information corresponding to each of the sound signals, and performing period feature analysis on each of the sound signals according to the feature information to obtain feature analysis results of each of the sound signals; When the characteristic analysis result of the sound signal indicates that the sound signal is a non-periodic sound signal, use the non-periodic sound signal as a first sound signal, determine the integrity of the first sound signal according to the characteristic information of the first sound signal, and when the integrity of the first sound signal is greater than or equal to a target threshold, send a first category identifier corresponding to the first sound signal to a target device, so that the target device returns a cancellation instruction or a retention instruction for the first sound signal; When receiving the instruction to eliminate the first sound signal returned by the target device, the first sound signal in the sound information is eliminated, and the sound information after noise reduction is output.
2. The method according to claim 1, characterized in that After performing periodic feature analysis on each of the sound signals according to the feature information to obtain a feature analysis result of each of the sound signals, the method further includes: When the characteristic analysis result of the sound signal indicates that the sound signal is a periodic sound signal, taking the periodic sound signal as a second sound signal, and sending a second category identifier corresponding to the second sound signal to the target device, so that the target device returns a cancellation instruction or a retention instruction of the second sound signal; When receiving the instruction to eliminate the second sound signal returned by the target device, the second sound signal in the sound information is eliminated, and the sound information after noise reduction is output.
3. The method according to claim 1, characterized in that The feature extraction of each of the sound signals to obtain feature information corresponding to each of the sound signals comprises: Take each of the sound signals as a current sound signal, and perform the following operations on the current sound signal: According to the current category identifier of the current sound signal, current feature information of the current sound signal is extracted, wherein the current feature information is a feature associated with the current category identifier of the current sound signal. When the current category identifier is a human voice, the current feature information includes a timbre feature, a loudness feature, and a tone feature.
4. The method according to claim 1, characterized in that: Determining the integrity of the first sound signal according to the characteristic information of the first sound signal includes: Performing feature matching on the feature information of the first sound signal and the feature information of each sound signal in the sound database to obtain a matching degree between the first sound signal and each sound signal in the sound database; The maximum matching degree among all matching degrees is determined as the integrity degree of the first sound signal.
5. The method according to claim 1, characterized in that After determining the integrity of the first sound signal according to the characteristic information of the first sound signal, the method further includes: When the completeness is less than the target threshold, the first sound signal and its characteristic information are stored in a sound database.
6. The method according to claim 4, characterized in that The step of, upon receiving the instruction to eliminate the first sound signal returned by the target device, eliminating the first sound signal in the sound information and outputting the sound information after noise reduction comprises: When receiving the elimination instruction of the first sound signal returned by the target device, refer to the sound signal corresponding to the maximum matching degree in the sound database, perform time-frequency analysis on the first sound signal, and generate a first inverted signal of the first sound signal; The first sound signal in the sound information is eliminated by the first inverted signal, and the sound information after noise reduction is output.
7. The method according to claim 2, characterized in that The step of, upon receiving the instruction to eliminate the second sound signal returned by the target device, eliminating the second sound signal in the sound information and outputting the sound information after noise reduction comprises: Acquire the amplitude and phase of each cycle of the second sound signal; Generate a second inverted signal of the second sound signal, wherein the amplitude of the second inverted signal is equal to that of the second sound signal in each cycle, and the phase of the second inverted signal is opposite to that of the second sound signal in each cycle; The second sound signal in the sound information is eliminated by the second inverted signal, and the sound information after noise reduction is output.
8. A directional noise reduction device, characterized in that: include: A classification module, used to classify a segment of input sound information to obtain at least one sound signal and a category identifier corresponding to each of the sound signals; An extraction module, used to perform feature extraction on each of the sound signals to obtain feature information corresponding to each of the sound signals, and perform period feature analysis on each of the sound signals according to the feature information to obtain feature analysis results of each of the sound signals; a first sending module, configured to, if a feature analysis result of the sound signal indicates that the sound signal is a non-periodic sound signal, use the non-periodic sound signal as a first sound signal, determine the integrity of the first sound signal according to feature information of the first sound signal, and, if the integrity of the first sound signal is greater than or equal to a target threshold, send a first category identifier corresponding to the first sound signal to a target device, so that the target device returns a cancellation instruction or a retention instruction for the first sound signal; The first elimination module is used to eliminate the first sound signal in the sound information and output the sound information after noise reduction when receiving the elimination instruction of the first sound signal returned by the target device.
9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is executed.
10. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 7 through the computer program.