Sound source localization methods, devices, and products based on incremental learning and imbalance correction

Abstract

This invention provides a sound source localization method, apparatus, and product based on incremental learning and imbalance correction. The method includes: acquiring a dataset for a new task, a parameter-frozen feature extractor, and regularization parameters; enhancing the tail category of the new task dataset; initializing an autocorrelation matrix, a cross-correlation matrix, and a category count based on the number of categories in the new task dataset; iterating through each sample in the new task dataset, updating the autocorrelation matrix, cross-correlation matrix, and category count; calculating the category weight for each category and calculating the Gini coefficient describing the category distribution; adaptively adjusting the regularization parameters based on the Gini coefficient; and solving for and outputting the optimal weight matrix based on the adjusted regularization parameters and aggregating the global autocorrelation matrix and global cross-correlation matrix of different categories. The optimal weight matrix is ​​used to update the sound source localization model. This invention addresses the problem of intra-task and inter-task imbalance in sound source localization, improving the sound source localization effect.

Description

Technical Field

[0001] This invention relates to the field of sound source localization technology, specifically to a sound source localization method, device, and product based on incremental learning and imbalance correction. Background Technology

[0002] Sound source localization (SSL) aims to estimate the direction of arrival (DoA) of a sound source.

[0003] Traditional analysis methods, such as Generalized Cross-Correlation with Phase Transform (GCC-PHAT), Multiple Signal Classification (MUSIC), and Steered Response Power with Phase Transform (SRP-PHAT), can provide closed-form solutions under controlled conditions, but their effectiveness deteriorates significantly in noisy and echo-filled environments. In recent years, breakthroughs have been made in the application of deep learning technology, which can be used to enhance traditional methods and achieve end-to-end sound source localization learning, with particularly outstanding performance when large-scale labeled datasets are available.

[0004] However, in practice, well-labeled sound source localization data is scarce, and this data is usually added gradually. For example, in the field of robotic hearing, the annotation of DoA (Domain of Information) for robots is continuously accumulated and improved over time. In this incremental environment, a core challenge is catastrophic forgetting, that is, the model suddenly loses previously acquired knowledge when learning a new task. To address this, class-incremental learning (CIL) methods are needed, which can integrate new classes while retaining old knowledge.

[0005] Existing CIL methods are mainly classified into three categories: regularization-based methods, which preserve information by constraining weight updates; replay-based methods, which perform knowledge distillation by storing typical samples; and analytical methods, which improve computational efficiency by utilizing closed-form solutions. Building upon this foundation, the paper "Analytic class incremental learning for sound source localization with privacy protection" introduced the first CIL framework for SSL, but CIL methods still face challenges from complex real-world conditions, such as duplicate class occurrences and class imbalance.

[0006] These limitations become particularly pronounced in the Generalized Continuous Incremental Learning (GCIL) scenario for sound source localization—a scenario facing a fundamental challenge: the long-tailed class distribution of DoA data. Specifically, this manifests in two forms: intra-task imbalance and inter-task imbalance. Intra-task imbalance occurs within a task, where certain dominant directions are overrepresented in the training data, while a few directions are severely lacking. For example, smart speakers are typically placed at the front of a room, resulting in an overrepresentation of sound source data near the 0° direction, while sound source data from the sides (e.g., 90°, 270°) or rear (180°) are very scarce. This creates a long-tailed distribution, where the model becomes very good at predicting dominant directions but poor at predicting a few directions. Inter-task imbalance occurs between different tasks, where the class distributions are uneven or overlapping, leading to dynamic biases and exacerbating the model's forgetting of old knowledge. For example, training a sound source localization model for a smart speaker might begin with training on an office environment dataset containing common sound source directions within the office. After a period of time, the goal is to enable the model to work in an in-vehicle environment, so a new in-vehicle environment dataset is used for further training. Because the model tends to learn the features of the current task's data, it may forget the knowledge learned from the office data when training with the in-vehicle data. These two imbalances combined make generalized, continuous incremental learning for sound source localization exceptionally difficult, especially given privacy restrictions that prohibit storing past data. Overcoming the intra-task and inter-task imbalances in sound source localization is a pressing technical problem that needs to be solved. Summary of the Invention

[0007] The primary objective of this invention is to provide a sound source localization method based on incremental learning and imbalance correction, which can solve the technical problem of intra-task and inter-task imbalance in sound source localization.

[0008] A second objective of this invention is to provide a computer device for implementing the above-described sound source localization method based on incremental learning and imbalance correction.

[0009] A third objective of this invention is to provide a computer program product that implements the above-described sound source localization method based on incremental learning and imbalance correction.

[0010] To achieve the aforementioned first objective, this invention provides a sound source localization method based on incremental learning and imbalance correction, comprising the following steps: acquiring audio signals corresponding to different tasks collected by a microphone array; processing the audio signals through a sound source localization model and outputting the direction of arrival corresponding to the audio signals; the training process of the sound source localization model includes: acquiring a dataset for a new task, a feature extractor with frozen parameters, and regularization parameters; initializing the autocorrelation matrix, cross-correlation matrix, and class count according to the number of classes in the dataset for the new task; traversing each sample in the dataset for the new task, including: extracting the feature vector of the current sample in the dataset for the new task through the feature extractor, obtaining the true class index of the current sample, updating the autocorrelation matrix according to the feature vector, updating the cross-correlation matrix according to the feature vector and the corresponding target vector, and updating the class count according to the true class index; calculating the class weight of each class and calculating the Gini coefficient describing the class distribution; adaptively adjusting the regularization parameters according to the Gini coefficient; solving and outputting the optimal weight matrix based on the adjusted regularization parameters and the aggregated global autocorrelation matrix and global cross-correlation matrix of different classes, the optimal weight matrix being used to update the sound source localization model.

[0011] A further approach is that the training process of the sound source localization model includes supplementing the samples of the tail category in the dataset of the new task. The supplementation process includes: matching the statistical features of the tail category with the samples of the rich category by shifting and scaling, and adding Gaussian noise to form new samples of the tail category.

[0012] A further approach is to maximize the number of samples in the enriched category, and then determine the number of new samples by multiplying the number of samples in the enriched category by the set data augmentation rate and subtracting the number of samples in the tail category.

[0013] A further proposed approach is to set the data augmentation rate at 0.5.

[0014] A further approach is to use a regularization parameter table. according to Received, among which and To set a baseline value, Gini coefficient, pi is the proportion of the number of samples in the i-th DoA category to the total number of samples in the current task, pj is the proportion of the number of samples in the j-th DoA category to the total number of samples in the current task, and n represents the number of categories. Indicates a specific task.

[0015] A further approach is to set a baseline value, It is 100. The value is 2.

[0016] A further approach is to represent the dataset for the new task as follows: ,in, Indicates a specific task. Indicates task Total number of samples Indicates from GCC-PHAT characteristics of microphone arrays, This represents the feature dimension of each microphone pair. It is a one-hot encoded tag representing the true DoA (value range [0°, 360°)). Based on The generated Gaussian distribution labels have peaks at the corresponding true DoA and standard deviations. .

[0017] As can be seen from the above scheme, this invention utilizes the statistical characteristics of GCC-PHAT peaks to synthesize tail category samples, which alleviates the imbalance within the task while maintaining statistical consistency and privacy. At the same time, this invention adopts a regularization parameter adjustment mechanism based on the Gini coefficient to achieve task adaptive regularization, solving the bias caused by data skew and overlap in cross-task incremental learning. The sound source localization model trained by this invention can simultaneously solve the problem of imbalance within and between tasks, thus improving the sound source localization effect.

[0018] To achieve the second objective mentioned above, the present invention provides a computer device comprising a processor and a memory, wherein: the memory stores a computer program, and when the computer program is executed by the processor, it implements a sound source localization method based on incremental learning and imbalance correction.

[0019] To achieve the third objective mentioned above, the present invention provides a computer program product, including computer instructions, wherein: when the computer instructions are executed by a processor, they implement the aforementioned sound source localization method based on incremental learning and imbalance correction. Attached Figure Description

[0020] Figure 1 This is a flowchart of an embodiment of the sound source localization method based on incremental learning and imbalance correction of the present invention.

[0021] Figure 2 This is a training framework diagram of the sound source localization model in an embodiment of the sound source localization method based on incremental learning and imbalance correction of the present invention.

[0022] Figure 3 This is a flowchart illustrating the operation of the GCC-PHAT feature enhancement unit in an embodiment of the sound source localization method based on incremental learning and imbalance correction of the present invention.

[0023] Figure 4 This is a flowchart illustrating the generation of new samples for the tail category in an embodiment of the sound source localization method based on incremental learning and imbalance correction of the present invention.

[0024] Figure 5 This is a flowchart illustrating the model update process for training a new task in an embodiment of the sound source localization method based on incremental learning and imbalance correction of the present invention.

[0025] Figure 6 This is a schematic diagram illustrating the catastrophic forgetting that occurs after the baseline model learns multiple tasks in an embodiment of the sound source localization method based on incremental learning and imbalance correction of the present invention.

[0026] Figure 7 This is a schematic diagram illustrating how the sound source localization model overcomes catastrophic forgetting after learning multiple tasks in an embodiment of the sound source localization method based on incremental learning and imbalance correction of the present invention.

[0027] The present invention will be further described below with reference to the accompanying drawings and embodiments. Detailed Implementation

[0028] The sound source localization method based on incremental learning and imbalance correction of the present invention is based on samples of GCC-PHAT peak synthesis tail category, and supports dynamic updates to adapt to cross-task imbalance changes through task adaptive regularization.

[0029] Example of a sound source localization method based on incremental learning and imbalance correction:

[0030] This embodiment is implemented by executing a computer program. See also... Figure 1 Specifically, it includes the following steps:

[0031] S11: Acquire the first audio signal corresponding to the first task collected by the microphone array.

[0032] S12: Process the first audio signal through a preset sound source localization model and output the first direction of arrival corresponding to the first audio signal.

[0033] S13: Acquire the second audio signal corresponding to the second task collected by the microphone array.

[0034] S14: Process the second audio signal through a preset sound source localization model and output the second direction of arrival corresponding to the second audio signal.

[0035] In steps S11 and S13 above, the first task and the second task are tasks in different acoustic environments. The noise and / or room impulse response are different in different acoustic environments. For example, the first task can be a sound source localization task in an office environment, and the second task can be a sound source localization task in a vehicle environment. Because the sound source localization model has been trained on office environment data and vehicle environment data successively through the analytical incremental learning and imbalance correction method of this invention, it can accurately output the corresponding direction of arrival based on the audio signal collected by the microphone array in both conference room environment and vehicle environment, and can also give accurate direction of arrival for sound sources from different directions in the same acoustic environment.

[0036] The number of tasks that the sound source localization model can handle in different acoustic environments can be obtained by training it using the analytical incremental learning and imbalance correction method of this invention, according to actual needs.

[0037] The aforementioned sound source localization model can run on a cloud platform, achieving distributed computing processing by receiving sound source localization requests sent by local clients. Alternatively, it can run on human-computer interaction devices (such as smartphones and smart home devices) to achieve sound source localization.

[0038] See Figure 2 The training framework for the sound source localization model includes a GCC-PHAT feature enhancement module and an adaptive dynamic imbalance corrector module designed specifically for GCC-PHAT. Training is performed based on the input imbalanced audio stream. The imbalanced audio stream includes training data from different tasks, and the different categories of tasks exhibit long-tailed distributions. The GCC-PHAT feature enhancement module includes a GCC-PHAT feature enhancement unit and an adaptive weight unit. The GCC-PHAT feature enhancement unit is used to enhance the model's training data, while the adaptive weight unit is used to estimate the optimal weight matrix. The adaptive dynamic imbalance corrector module includes a parameter-freezing-based multilayer perceptron (MLP) feature extractor and a DoA classifier, which transforms GCC-PHAT features into a probability distribution of DoA.

[0039] See Figure 3 The GCC-PHAT feature enhancement unit addresses the issue of limited sample numbers for tail categories in a task. It utilizes head category samples and sequentially performs peak alignment, amplitude calibration, and Gaussian noise addition to generate supplementary samples. These supplementary samples expand the sample number for tail categories, mitigating the imbalance problem within the task under long-tailed distributions. Further details are provided below. Figure 2 and Figure 3 This section details the training process of the sound source localization model.

[0040] The sound source localization problem is modeled as a generalized continuous incremental learning (GCIL) problem: data is input in the form of a sequence of tasks, each task containing DoA classes of new and old sound sources with an imbalanced distribution. Specifically, Indicates the first The training dataset for each task. , Indicates task The total number of samples, of which, Indicates from GCC-PHAT characteristics of microphone arrays, This represents the feature dimension of each microphone pair. It is a one-hot encoded tag representing the true DoA (value range [0°, 360°)). Based on The generated Gaussian distribution labels have peaks at the corresponding true DoA and standard deviations. . The categories in the model follow a long-tail distribution, with a maximum of 60 categories per task. The trained sound source localization model is represented as follows: Minimize the cumulative loss of the task without reviewing previous data. The cumulative loss is expressed as... ,in This represents the binary cross-entropy loss, which solves the problems of catastrophic forgetting and imbalance within tasks without requiring the storage of instances.

[0041] GCC-PHAT estimates the direction of a sound source by using the time delay between microphones. The signals acquired by the two microphones are represented by a Fourier transform as follows: and The generalized cross-correlation function is defined as follows: ,in, This represents time delay (lag). Previous experience has shown that the peaks in GCC-PHAT play a decisive role in sound source localization performance because these peaks carry crucial time delay information. To alleviate the intra-task imbalance problem under long-tailed distributions, this invention designs a domain-specific data augmentation method that manipulates these peaks while maintaining their statistical structure.

[0042] Specifically, for each task eigenvectors Classified as There are segments (one segment per microphone pair), and the feature dimension of each segment is . Regarding the task Each category Calculate the peak position of each segment. With amplitude The mean and variance of . Let . The number of samples in the category with the most samples. For any number of samples, the data augmentation rate is... The category is generated by the GCC-PHAT feature enhancement unit. A new dataset was generated. The GCC-PHAT feature enhancement unit moves and rescales the peak values ​​of the head class features to match the statistical features of the tail class, while injecting low-frequency noise to enhance diversity. It can balance tail features while keeping the correlation between microphones from being distorted.

[0043] With the mission For example, task Category The category with the largest number of samples is called the rich category, and it needs to be based on the rich category. Tail category Generate new samples. The input is a sample from rich categories. The original feature vector is represented as Tail category Statistical characteristics The output is a category belonging to the tail. new samples See Figure 4 The specific process includes the following steps:

[0044] S21: Create a zero vector with the same dimensions as the original feature vector.

[0045] Wherein, it is represented as: .

[0046] S22: Set k=1.

[0047] S23: Calculate the positional difference between the tail category and the rich category on the k-th segment.

[0048] The location difference, i.e., the peak location difference, is expressed as: .

[0049] S24: Extract the sub-vector corresponding to the k-th segment from the original feature vector.

[0050] Where, it is represented as .

[0051] S25: Circularly shift the extracted sub-vectors according to their positional differences.

[0052] Among them, based on location differences For the extracted sub-vectors A circular shift (e.g., to the right or left) is represented as: .

[0053] S26: Scale the amplitude of all values ​​in the shifted segment to match the amplitude of the statistical features.

[0054] Where, it is represented as .

[0055] S27: Add Gaussian noise.

[0056] In this process, a mean of 0 and a standard deviation of 0 are added to the processed fragment. Gaussian noise, The size is proportional to the maximum value of the segment (usually 5% of the maximum value).

[0057] S28: Fill in the position of the kth segment corresponding to the new sample.

[0058] S29: Determine if k is equal to the number of segments.

[0059] If yes, continue to step S30; otherwise, execute step S31, assign the value k=k+1, and then return to step S23.

[0060] S30: Returns the final generated new sample.

[0061] The above process can be represented by the following pseudocode:

[0062]

[0063] Therefore, for tail category Based on rich categories A new sample was generated. .

[0064] In the adaptive dynamic imbalance corrector module, the input pass and Mapped to ,in, It is a three-layer MLP that includes a batch normalization layer, a ReLU layer, and dropout. It has a linear layer with dropout. Feature extractor Training is only performed on the first task; for subsequent tasks, The parameters are frozen, only updated. .

[0065] We set up ADIR for GCIL. Unlike traditional CIL where classes are mutually exclusive across different tasks, GCIL allows classes to repeat, which requires storing the autocorrelation matrix and cross-correlation matrix separately for each class. ADIR and adaptive weighting units are referenced in the paper "Air: Analytic Imbalance Rectifier for Continuous Learning".

[0066] For each task Given frozen MLP features and Gaussian distribution tags The adaptive weighting unit estimates the optimal weight matrix in the following way: ,in, For category Category weights, It is an adaptive regularization parameter used to enhance stability. Specifically, , , , This represents the calculated Gini coefficient (for...). (i.e., when there are 60 DoA categories) , where pi is the proportion of samples in the i-th DoA category to the total number of samples in the current task, pj is the proportion of samples in the j-th DoA category to the total number of samples in the current task, and n represents the number of categories. The imbalance in the category distribution is quantified by calculating the sum of the absolute values ​​of the differences in the proportions of all category pairs. The definition of closed-form update is as follows: ,in, and , This is an identity matrix. To handle the issue of duplicate classes in GCIL, the system maintains a separate matrix for each class. and The parameters are updated using recursive least squares.

[0067] This section details how to update a model for a new task. The input is the training dataset for the new task. A frozen (non-updated) pre-trained feature extractor and setting baseline values and Based on this calculation, the optimal weight matrix is ​​obtained. This optimal weight matrix is ​​used as the parameters of the new classification layer for prediction of the new task. See also Figure 5 Specifically, it includes the following steps:

[0068] S41: Initialization.

[0069] Among them, 360 autocorrelation matrices (corresponding to 360 categories) were initialized. Each matrix is ​​1000x1000 in size and initialized to 0. These matrices are used to sum the feature vectors of all samples within the same class. autocorrelation Initialize 360 ​​cross-correlation matrices Each matrix is ​​1000x360 in size and initialized to 0. This matrix is ​​used to accumulate eigenvectors. With the target vector Cross-correlation Initialize a category count array containing 360 elements. All values ​​are initially set to 0, and are used to record the number of samples appearing in each category.

[0070] S42: Determine if the traversal is complete.

[0071] Specifically, the autocorrelation matrix initialized in S1 is updated by iterating through each sample. Cross-correlation matrix and category count array If all samples have been traversed, proceed to step S48; if the traversal is not yet complete, continue to step S43.

[0072] S43: Extract feature vectors.

[0073] Among them, the current sample Input to feature extractor In this process, a 1000-dimensional feature vector is obtained. .

[0074] S44: Get the category index.

[0075] Among them, the current sample Find its true category index using the argmax function. .

[0076] S45: Update the autocorrelation matrix.

[0077] Among them, the feature vector of the current sample is calculated. and its transpose outer product Accumulate to the corresponding category autocorrelation matrix middle.

[0078] S46: Update the cross-correlation matrix.

[0079] Wherein, the feature vector of the current sample is calculated. transpose of the target vector outer product Accumulate to the corresponding category Cross-correlation matrix middle.

[0080] S47: Update category count.

[0081] Among them, the categories Count Add 1.

[0082] S48: Calculate the prior probability of the category.

[0083] Specifically, a class weight (prior probability) is calculated for each occurrence of a class, which is used for subsequent weighted averaging. For categories greater than 0, calculate = .

[0084] S49: Calculate the Gini coefficient.

[0085] Among them, according to Calculate the Gini coefficient. The closer the Gini coefficient is to 1, the more uneven the distribution of the number of samples in different categories (i.e., some categories are dominant). The closer the Gini coefficient is to 0, the more balanced the distribution of the number of samples in different categories.

[0086] S50: Adaptively adjusts the regularization parameter.

[0087] Specifically, based on the distribution revealed by the Gini coefficient, the regularization is dynamically adjusted to enhance robustness. This is done according to the formula... calculate According to different The coefficients adjust the regularization strength. When the sample distribution is relatively uniform, i.e. When the value is less than 0.5, a uniform distribution is inherently less prone to overfitting and does not require strong regularization, thus reducing the regularization strength; when the sample distribution is non-uniform, i.e. When the value is greater than 0.5, uneven distribution can easily lead to the model overfitting to multiple sample classes, so it is necessary to strengthen the regularization strength. When the value is 0.5, the regularization strength remains at the baseline level. equal to the baseline value .

[0088] S51: Aggregate the global correlation matrix.

[0089] The statistical information from 360 categories is aggregated into two global matrices. A weighted average global autocorrelation matrix is ​​then calculated. , pass Calculated. The weighted average global cross-correlation matrix is ​​calculated. , Through calculation We have obtained these two global matrices, which contain the "knowledge" learned from the current task data.

[0090] S52: Solve for and return the optimal weight matrix.

[0091] Among them, by calculating the closed-form solution Obtain the optimal weight matrix .

[0092] The above process can also be represented by the following pseudocode:

[0093]

[0094] This enables the sound source localization model to be updated based on the dataset corresponding to the new task.

[0095] The sound source localization model obtained based on the above training process was evaluated on the SSLR dataset. The SSLR dataset consists of 4-channel audio recorded at a 48kHz sampling rate, with spatial annotation information. According to the GCIL settings, the SSLR dataset is divided into 10 consecutive tasks, labeled as follows: , Each task contains a maximum of 60 disjoint DoA categories, with a resolution maintained at 1°. Task 1 introduces 60 new categories, while tasks 2 through 9 each add 30 new categories and replay 30 categories from previous tasks to simulate a persistent source. Task 10 covers all remaining categories.

[0096] For the input data, GCC-PHAT features are extracted from the 170-millisecond audio segment. Each microphone array pair corresponds to 51 delay coefficients. Evaluation uses one-hot coded tags. The target signal was Gaussian smoothed during training. Test set Includes only tasks The category, and in The model is evaluated on accumulated data to track forgetting.

[0097] To simulate the dynamics of long-tail distributions in the real world, each task follows an exponentially decaying distribution, and each task has a specific degree of imbalance. Specifically, the decay rate starts from... Gradually increase to The number of categories is determined by the formula. Sure, Let represent the number of samples of the c-th category in the t-th task. This design gradually exacerbates the imbalance between different tasks and uses a varying Gini coefficient to characterize the long-tail distribution dynamics present in real-world sound source localization.

[0098] The experiment evaluated performance using three metrics: Mean Absolute Error (MAE), Accuracy (ACC), and Backward Transfer (BWT). Specifically, MAE measures the average angular error between the predicted direction of arrival and the true direction. ACC is the percentage of predicted values ​​deviating from the true direction within a 5° evaluation range. BWT quantifies the model's ability to forget information. , Indicates the task after training. The BWT is calculated using the ACC. A positive BWT value indicates that the new task helps solve the old task, while a negative BWT value indicates that the knowledge of solving the old task has been forgotten, and the larger the negative value, the more serious the forgetting.

[0099] The sound source localization model was compared with four CIL baseline models: LwF, iCaRL, ACIL, and GACL. All these models were adapted within the same SSL-GCIL framework. Furthermore, the experiments reported lower bounds using a simple multilayer perceptron (MLP) model trained only on the initial task (without any incremental adaptive adjustments) and upper bounds using a joint training method. To ensure fairness across all tasks with access to the full dataset, all baseline models employed the same multilayer perceptron (MLP) feature extractor pre-trained on the first task and frozen in subsequent tasks. For Task 1, all parameters were optimized using an Adam optimizer with a learning rate of 0.001 and a binary cross-entropy loss function. In subsequent tasks, the feature extractor... Keeping the classifier weights fixed It was updated according to the same training protocol.

[0100] Table 1 presents the comprehensive evaluation results under different signal-to-noise ratio conditions. In a clean data environment, the sound source localization model (ours) of this invention performs best: the mean absolute error (MAE) is 5.29°, the mean accuracy (ACC) is 88.98%, and the BWT value is positive at 1.56, fully demonstrating a balance between effective knowledge retention and prevention of catastrophic forgetting. Compared to the ACIL baseline model, the performance of this invention is superior: MAE is reduced by 1.20°, ACC is improved by 3.14%, and the BWT value is optimized by 3.30. The joint training upper limit (MAE of 4.34° and ACC of 93.45%) represents the performance limit when all data can be acquired simultaneously, and this invention can gradually approach this ideal value by only accessing incremental data.

[0101] Table 1. Performance comparison of baseline models implemented by different methods under SNR conditions

[0102]

[0103] As noise levels increase, the performance of all methods declines, but this invention maintains a relative advantage. At a signal-to-noise ratio of 20 dB, the MAE is 7.31° and the ACC is 84.93%, with this invention outperforming ACIL by 0.88% and 3.19%, respectively. Even in harsh noise environments of -10 dB, this invention still stands out among various incremental learning methods with a highest ACC of 15.97%, demonstrating superior noise resistance.

[0104] The BWT metric reveals the remarkable resistance of the sound source localization model of this invention to catastrophic forgetting. While all baseline methods exhibit negative BWT values, this invention maintains positive BWT values ​​across all noise levels, even at a signal-to-noise ratio of -10 dB, maintaining a value of 0.60. This confirms that the adaptive regularization and enhancement strategies of this invention effectively preserve previously learned knowledge.

[0105] Figure 6 This demonstrates the serious forgetting problem inherent in the baseline method. Figure 7 The stability of the invention in different tasks was also demonstrated.

[0106] Table 2 presents the results of the ablation experiments, which confirm that both the GCC-PHAT feature enhancement module and the adaptive dynamic imbalance corrector module make significant contributions. The adaptive dynamic imbalance corrector module alone improves the BWT from -17.65 to 1.36, while adding the GCC-PHAT feature enhancement module further improves the ACC from 82.36% to 88.98%. This combined approach achieves the best balance between accuracy and stability. This invention consistently outperforms all baselines in most scenarios, demonstrating its superior robustness to incremental learning challenges and environmental noise.

[0107] Table 2. Ablation Experiment Results

[0108]

[0109] Computer device embodiment:

[0110] The computer device in this embodiment includes a processor and a memory. The memory stores a computer program, and when the processor executes the computer program, it implements the various steps of the above-described embodiment of the sound source localization method based on incremental learning and imbalance correction.

[0111] A computer device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that a computer device may include more or fewer components, or a combination of certain components, or different components; for example, a computer device may also include input / output devices, network access devices, buses, etc.

[0112] For example, a processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microcontroller or any conventional processor. The processor is the control center of a computer device, connecting all parts of the computer device through various interfaces and lines.

[0113] The memory can be used to store computer programs and / or modules. The controller implements various functions of the computer device by running or executing the computer programs and / or modules stored in the memory, and by accessing data stored in the memory. For example, the memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (e.g., sound receiving function, sound-to-text function, etc.), etc.; the data storage area may store data created based on the use of the mobile phone (e.g., audio data, text data, etc.). In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, RAM, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0114] If the modules integrated into the computer device in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, implementing all or part of the processes of the sound source localization method embodiments based on incremental learning and imbalance correction can also be accomplished by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a controller, it can implement the steps of the sound source localization method embodiments based on incremental learning and imbalance correction described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The storage medium can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content contained in computer-readable media may be appropriately added to or subtracted from the requirements of legislation and patent practice in a jurisdiction. For example, in some jurisdictions, computer-readable media may not include electrical carrier signals and telecommunication signals, in accordance with legislation and patent practice.

[0115] Example of a computer program product:

[0116] The computer program product of this embodiment includes computer instructions stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the various steps of the above-described embodiment of the sound source localization method based on incremental learning and imbalance correction.

[0117] In summary, this invention addresses both intra-task and inter-task data imbalance. For intra-task imbalance, it generates tail-class samples using GCC-PHAT peak statistical features, maintaining inter-microphone correlation without relying on external data. For inter-task imbalance, it provides guidance based on task-specific statistics (such as the Gini coefficient), effectively mitigating cross-task bias and knowledge forgetting. This invention also features a lightweight adaptive supervised learning framework specifically designed for GCIL, improving the performance of sound source localization models.

[0118] Finally, it should be emphasized that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A sound source positioning method based on incremental learning and imbalance correction, characterized in that, The method comprises the following steps: obtaining audio signals corresponding to different tasks collected by a microphone array; processing the audio signals through a sound source positioning model to output a direction of arrival corresponding to the audio signals; the training process of the sound source positioning model comprises: obtaining a data set of a new task, a parameter-frozen feature extractor, and a regularization parameter; initializing a self-correlation matrix, a cross-correlation matrix, and a category count according to the number of categories of the data set of the new task; traversing each sample of the data set of the new task, comprising: extracting a feature vector of a current sample of the data set of the new task through the feature extractor, obtaining a real category index of the current sample, updating the self-correlation matrix according to the feature vector, updating the cross-correlation matrix according to the feature vector and a corresponding target vector, and updating the category count according to the real category index; calculating a category weight of each category and a Gini coefficient describing a category distribution; adaptively adjusting the regularization parameter according to the Gini coefficient; solving an optimal weight matrix according to the adjusted regularization parameter and a global self-correlation matrix and a global cross-correlation matrix aggregated from different categories, the optimal weight matrix being used to update the sound source positioning model.

2. The sound source positioning method based on incremental learning and imbalance correction according to claim 1, wherein: the training process of the sound source positioning model comprises supplementing samples of tail categories of the data set of the new task, and the supplementing process comprises: matching statistical features of the tail categories through shifting and scaling according to samples of rich categories, and adding Gaussian noise to form new samples of the tail categories.

3. The sound source positioning method based on incremental learning and imbalance correction according to claim 2, wherein: the number of samples of the rich categories is the largest, and the number of the new samples is determined according to the number of samples of the rich categories multiplied by a set data enhancement rate and then subtracted by the number of samples of the tail categories.

4. The sound source positioning method based on incremental learning and imbalance correction according to claim 3, wherein: the data enhancement rate is 0.

5.

5. The sound source positioning method based on incremental learning and imbalance correction according to claim 1, wherein: the regularization parameter According to is obtained, wherein and is a set reference value, is a Gini coefficient, , pi is the proportion of the number of samples of the i-th DoA category to the total number of samples of the current task, pj is the proportion of the number of samples of the j-th DoA category to the total number of samples of the current task, and n represents the number of categories, indicates a specific task.

6. The sound source positioning method based on incremental learning and imbalance correction according to claim 5, wherein: The set reference value is, 100, 2.

7. The sound source positioning method based on incremental learning and imbalance correction according to claim 1, wherein: The dataset of the new task is represented as where, represents a specific task, represents the task of the total number of samples, represents the features from GCC-PHAT features of the microphone array, represents the feature dimension of each pair of microphones, is a one-hot encoding label representing the true DoA value range [0°, 360°), is a Gaussian distribution label generated based on with the peak located at the corresponding true DoA and the standard deviation .

8. A computer device comprising a processor and a memory, wherein: the memory stores a computer program, and the computer program is executed by the processor to implement the sound source positioning method based on incremental learning and imbalance correction according to any one of claims 1 to 7.

9. A computer program product comprising computer instructions, wherein: the computer instructions are executed by a processor to implement the sound source positioning method based on incremental learning and imbalance correction according to any one of claims 1 to 7.

Images