A performance analysis method of a graph-based Transformer automatic speech recognition model

Abstract

The application relates to a performance analysis method of a graph-based Transformer automatic speech recognition model, and belongs to the field of artificial intelligence and speech recognition. The method comprises the following steps: obtaining a Transformer automatic speech recognition model; obtaining audio data; inputting the audio data into the Transformer automatic speech recognition model, obtaining weight matrices of multiple attention heads of each layer in the model through forward propagation, and extracting word texts output by the model; performing average processing on the weight matrices of each attention head within a given time to obtain artificial neural activities of the attention head; performing correlation calculation on the artificial neural activities of the attention head by using a Pearson correlation coefficient to obtain a correlation coefficient, constructing a functional connection matrix based on the correlation coefficient; calculating graph theory parameters of the functional connection matrix; calculating a word error rate of the output word texts; and analyzing the performance of the Transformer automatic speech recognition model based on the graph theory parameters and the word error rate. The method provides a basis for analyzing the performance of the Transformer automatic speech recognition model.

Description

Technical Field

[0001] This invention relates to a performance analysis method for a graph theory-based Transformer automatic speech recognition model, belonging to the field of artificial intelligence and speech recognition. Background Technology

[0002] The Transformer is a deep learning model architecture proposed by Vaswani et al. in 2017, initially used for Natural Language Processing (NLP) tasks, particularly machine translation. Due to its superior performance and highly scalable design, the Transformer has quickly become a mainstream model architecture for many NLP tasks and is widely used in speech recognition. However, a deep understanding of the interactions between attention heads in the multi-head attention mechanism within the model, especially how they affect model performance, remains lacking. Existing research mainly focuses on improving the performance of Automatic Speech Recognition (ASR) models by modifying the model's structure, while systematic research on the relationships between attention heads within the model and their impact on model performance is relatively limited. Summary of the Invention

[0003] The technical problem to be solved by this invention is:

[0004] To overcome the shortcomings of existing technologies, this invention provides a performance analysis method for Transformer automatic speech recognition models based on graph theory. The aim is to reveal the interaction relationships between attention heads and how these relationships affect the overall performance of Transformer automatic speech recognition models, particularly their impact on word error rate (WER).

[0005] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0006] A performance analysis method for a graph theory-based Transformer automatic speech recognition model, characterized by comprising:

[0007] Obtain a Transformer automatic speech recognition model, which adopts a multi-layer structure and contains multiple attention heads;

[0008] Acquire audio data and preprocess the audio data;

[0009] The audio data is input into the Transformer automatic speech recognition model, and the weight matrix of multiple attention heads in each layer of the model is obtained through forward propagation. The word text output by the model is then extracted.

[0010] The weight matrix of each attention head is averaged over a given time period to obtain the artificial neural activity of that attention head.

[0011] The correlation coefficient is obtained by calculating the correlation of the artificial neural activity of the attention head using the Pearson correlation coefficient, and a functional connectivity matrix is ​​constructed based on the correlation coefficient; the graph theory parameters of the functional connectivity matrix are then calculated.

[0012] Calculate the word error rate of the output word text;

[0013] Based on the graph theory parameters and the word error rate, the performance of the Transformer automatic speech recognition model is analyzed.

[0014] A further technical solution of the present invention: preprocessing the audio data, including:

[0015] Each audio data segment is divided into short segments of 10 to 20 seconds in length;

[0016] Each audio segment is downsampled to standard call audio quality.

[0017] A further technical solution of the present invention: the construction of the functional connectivity matrix based on the correlation coefficient includes:

[0018] The correlation coefficient of each pair of attention heads is filled into the functional connectivity matrix. Each element of the functional connectivity matrix represents the functional connectivity strength between the two attention heads. The functional connectivity matrix is ​​regarded as a weighted undirected graph, where the nodes of the undirected graph represent attention heads, and the weights of the edges of the undirected graph represent the functional connectivity strength between them.

[0019] A further technical solution of the present invention: calculating the graph theory parameters of the functional connectivity matrix, wherein the graph theory parameters include: clustering coefficient, shortest path length, small-world property, and node degree distribution;

[0020] The formula for calculating the clustering coefficient is:

[0021]

[0022] Among them, e v k is the number of edges that exist between the neighbors of node v. v Let v be the degree of node v, and N be the number of nodes in the network;

[0023] The formula for calculating the shortest path length is:

[0024]

[0025] Where, d ij This represents the shortest path distance from node i to node j;

[0026] The formula for calculating small-world properties:

[0027]

[0028] Among them, C 实 C represents the average clustering coefficient of the original network. 随机 L represents the average clustering coefficient of a random network with the same number of nodes and edges as the original network. 实 L represents the average shortest path length of the original network. 随机 This represents the average shortest path length of a random network with the same number of nodes and edges as the original network.

[0029] Formula for calculating node degree distribution:

[0030]

[0031] Where, N k This represents the number of nodes with degree k.

[0032] A further technical solution of the present invention: the word error rate of the output word text is specifically obtained by calculating the edit distance between the output word text and the reference text.

[0033] A further technical solution of the present invention: the analysis of the performance of the Transformer automatic speech recognition model based on the graph theory parameters and the word error rate specifically includes:

[0034] Input multiple audio data points to obtain multiple graph theory parameters and word error rates;

[0035] We used linear regression analysis to study the correlation between graph theory parameters and word error rate.

[0036] A computer system is characterized by comprising: one or more processors, and a computer-readable storage medium for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the method described above.

[0037] A computer-readable storage medium is characterized by storing computer-executable instructions, which, when executed, are used to implement the above-described method.

[0038] A computer program product is characterized by including computer-executable instructions, which, when executed, are used to implement the above-described method.

[0039] The beneficial effects of this invention are as follows:

[0040] This invention provides a performance analysis method for Transformer-based automatic speech recognition models based on graph theory. By introducing graph theory analysis, it conducts an in-depth study of the operating mechanism of multiple attention heads in Transformer-based automatic speech recognition models, resulting in the following significant effects:

[0041] 1. A negative correlation exists between the small-world properties of the model's internal functional network and the word error rate: By constructing a functional connectivity matrix between attention heads and performing graph theory analysis, this invention reveals a negative correlation between the small-world properties of the model's internal functional network and the word error rate. This provides a new approach for optimizing the model's structure and parameter selection, contributing to improved accuracy and performance of automatic speech recognition models.

[0042] 2. High Correlation Among High-Level Attention Heads: This invention provides an in-depth understanding of the internal working mechanism of automatic speech recognition models through the calculation and analysis of graph theory parameters (such as clustering coefficients and small-world attribute values). It was found that the correlation between high-level attention heads is higher than that between low-level attention heads. This indicates that high-level attention heads may play an important role in the model processing. High-level attention heads are the group of attention heads that contribute most significantly to the model's audio recognition. This can provide a scientific basis for the improvement and application of the model.

[0043] 3. Provides a general analysis framework: The method of this invention is not only applicable to automatic speech recognition models, but can also be extended to other computer vision deep learning models that use multi-head attention mechanisms. It can explore the interaction of attention heads in different tasks and provide a unified analysis framework for improving model performance.

[0044] 4. Reduced development and debugging costs: This invention reveals the linear relationship between the functional connectivity network of the attention head and model performance. It provides a powerful tool for the development and debugging of automatic speech recognition models, helping researchers to more efficiently locate and optimize key parts of the model, and reduce development and debugging time and costs. Attached Figure Description

[0045] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts.

[0046] Figure 1 This is a schematic diagram illustrating the performance analysis method of the Transformer automatic speech recognition model based on graph theory according to the present invention.

[0047] Figure 2 This is a flowchart of the method for constructing the functional connection matrix of the Transformer model according to the present invention.

[0048] Figure 3 This is a schematic diagram of the functional connection matrix of the present invention.

[0049] Figure 4 This is a scatter plot showing the relationship between the small-world properties of the functional network and the word error rate of this invention. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0051] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0052] Inspired by the analysis of brain functional connectivity networks in functional magnetic resonance imaging (fMRI) studies, this invention proposes a method to construct a functional connectivity matrix at the attention head level by analyzing the multi-head attention mechanism in Transformer models. Further graph theory analysis is then performed to investigate the relationship between graph theory parameters and the performance of automatic speech recognition models. This analytical method not only helps to understand the internal working mechanism of Transformer automatic speech recognition models and improves their interpretability, but also provides a theoretical basis for model optimization.

[0053] The objectives of this invention are twofold: First, to optimize the performance of automatic speech recognition models: By studying the relationship between graph theory parameters and word error rates, this invention provides a theoretical basis for optimizing the structure and parameter selection of automatic speech recognition models, thereby improving the model's accuracy and robustness. Second, to provide model interpretability: This invention provides interpretability for the decision-making process of automatic speech recognition models through graph theory analysis, which helps to identify the attention heads that contribute most significantly to model performance.

[0054] This invention provides a graph theory-based performance analysis method for Transformer automatic speech recognition models. By constructing a functional connectivity matrix between attention heads of the Transformer automatic speech recognition model and performing graph theory analysis on the functional connectivity matrix, the method reveals the relationship between the functional network structure of the attention heads of the Transformer automatic speech recognition model and the model performance.

[0055] like Figure 1 As shown, the present invention includes the following steps:

[0056] Step 1: Model Selection: Select an automatic speech recognition model with a Transformer architecture. These models employ a multi-layered structure, incorporating multiple attention heads, and are trained and fine-tuned based on audio data. Their main characteristic is the ability to effectively extract rich features from raw speech signals and learn on unlabeled speech data through unsupervised pre-training, achieving excellent performance in downstream tasks with only a small amount of labeled data.

[0057] Step 2: Data Processing: For audio data processing, we normalized the audio files. Specifically, to ensure the model could process the data efficiently, we divided each audio segment into short clips of 10 to 20 seconds. This length was chosen for two reasons: first, they capture enough contextual information to correctly recognize speech; second, shorter clips reduce computational resource consumption, allowing the model to process large amounts of data within a reasonable timeframe. Next, we downsampled each audio segment, reducing the sampling rate to 8kHz (standard call audio quality). This also helps reduce the computational load on the model while having minimal impact on speech recognition accuracy.

[0058] Step 3: Attention Weight Matrix Extraction: After setting up the model, select test audio samples and obtain the multi-head attention weight matrix of each layer in the model through forward propagation. Finally, output the text of the words recognized by the model for word error rate calculation. The weight matrix represents how the model distributes attention among different attention heads when processing a given speech input.

[0059] Step 4: Extraction of Artificial Neural Activity from the Model: For each attention head, an attention head weight matrix with a length equal to the audio duration is generated during audio processing. This attention head weight matrix represents the degree of attention the attention head pays to the current time and the times before and after it at each moment. The artificial neural activity of the attention head is obtained by averaging all the weights of the attention head weight matrix within 1 second.

[0060] Step 5: Functional Connectivity Matrix Construction: The correlation coefficient of the artificial neural activities of the attention heads is calculated using the Pearson correlation coefficient. The correlation coefficient of each pair of attention heads is then filled into the functional connectivity matrix. Each element of the functional connectivity matrix represents the functional connectivity strength between the two attention heads. This functional connectivity matrix can be viewed as a weighted undirected graph, where the nodes of the undirected graph represent attention heads, and the weights of the edges in the undirected graph represent the functional connectivity strength between them.

[0061] Step 6: Graph Theory Parameter Calculation:

[0062] Calculate the following parameters for the function connectivity matrix of all models:

[0063] Clustering Coefficient: Calculated for each node (i.e., attention head), describing the density of connections between its neighboring nodes. A high clustering coefficient usually indicates that these attention heads have formed a tight cluster structure in the network. For node v, the clustering coefficient C... v It can be represented as:

[0064]

[0065] Among them, e v k is the number of edges that exist between the neighbors of node v. v Let v be the degree of node v (i.e., the number of nodes connected to it).

[0066] The average clustering coefficient C of the entire network is the average of the clustering coefficients of all nodes:

[0067]

[0068] Where N is the number of nodes in the network.

[0069] Shortest Path Length: The shortest path length of a given node quantifies the average distance or routing efficiency between that node and all other nodes in the network. For a given node i, its shortest path length is the average distance between it and all other nodes:

[0070]

[0071] Where, d ij This represents the shortest path distance from node i to node j.

[0072] The average shortest path length L of the network is:

[0073]

[0074] Small-World Property: This property evaluates the overall small-world characteristics of a network, including a comparison of average path length and clustering coefficient. Small-world networks have unique advantages in information propagation and processing efficiency. The evaluation of the small-world property typically includes two main metrics: average path length and clustering coefficient. A network is defined as a small-world network if it satisfies the following:

[0075]

[0076] Among them, C 实 C represents the average clustering coefficient of the original network. 随机 L represents the average clustering coefficient of a random network with the same number of nodes and edges as the original network. 实 L represents the average shortest path length of the original network. 随机 This represents the average shortest path length of a random network with the same number of nodes and edges as the original network.

[0077] Node Degree Distribution: Analyzing the degree distribution of each node in a network helps understand the network's connectivity and information propagation characteristics. The node degree k is the number of connections a node has to other nodes in the network. The node degree distribution describes the probability distribution of node degrees in the network. The degree distribution can be represented as:

[0078]

[0079] Where, N k This represents the number of nodes with degree k, where N is the total number of nodes in the network.

[0080] Step 7: Performance Evaluation: For automatic speech recognition models, a common performance evaluation method is the word error rate. Calculate the word error rate for the word recognition text output by each model.

[0081] Step 8: Performance-related analysis:

[0082] After obtaining the word error rate and graph theory parameters of the model, we studied the relationship between the word error rate and the graph theory parameters to reveal the impact of the attention head network structure on the model performance.

[0083] To enable those skilled in the art to better understand the present invention, the present invention will be described in detail below with reference to specific embodiments.

[0084] The specific steps described above are as follows:

[0085] Step 1: Model Selection

[0086] This invention uses the publicly available pre-trained wav2vec 2.0 model. This model is a self-supervised learning model developed by Meta (formerly Facebook AI) specifically for speech recognition tasks. The wav2vec 2.0 model has a multi-layered structure, containing 12 Transformer encoding layers, each with 12 multi-head attention mechanisms, for a total of 144 attention heads. Each attention head outputs an attention weight matrix during the model's forward propagation. This matrix represents the attention allocation of the model to different positions in the input speech sequence when processing the speech input.

[0087] The reason for choosing the pre-trained wav2vec 2.0 model is that it has demonstrated excellent performance in a variety of speech recognition tasks, and the public availability of the pre-trained model can reduce the time and computational resources required to train our model.

[0088] The architecture of the wav2vec 2.0 model mainly consists of three parts: feature extractor, quantizer, and context network.

[0089] ① Feature Extractor

[0090] The feature extractor is a fundamental component of the wav2vec 2.0 model, responsible for transforming the raw audio signal into a feature representation suitable for model processing. In the wav2vec 2.0 model, the feature extractor employs a convolutional neural network (CNN) structure, extracting local features of the audio signal through convolution operations. These local features contain temporal information and local structure of the audio, providing rich information for subsequent quantizers and context networks.

[0091] ②. Quantizer

[0092] The quantizer is a key component of the wav2vec 2.0 model, transforming the continuous feature representation output by the feature extractor into a discrete latent representation. This discretization process enables the model to learn the intrinsic structure of the audio signal, thereby achieving efficient encoding of the audio signal. In the wav2vec 2.0 model, the quantizer uses a Gumbel-Softmax distribution for discretization, achieving soft discretization of the continuous feature representation.

[0093] ③. Contextual Network

[0094] The context network is another important component of the wav2vec 2.0 model, responsible for capturing the contextual information of the audio signal. By computing discrete latent representations using a self-attention mechanism, the context network learns the global structure and semantic content of the audio signal. This global information is crucial for downstream tasks because it helps the model better understand the intrinsic meaning of the audio signal.

[0095] The wav2vec 2.0 model, as a powerful audio pre-trained model, has achieved significant performance improvements in tasks such as speech recognition and audio classification. By pre-training on large-scale unlabeled audio data, the wav2vec 2.0 model can provide rich feature representations for downstream tasks, thereby achieving better performance. In practical applications, the performance of the Transformer automatic speech recognition model demonstrates the enormous potential of self-supervised learning in the field of audio processing.

[0096] Model runtime environment

[0097] ① Server configuration:

[0098] Processor: Intel Xeon Platinum 8280*2, 28 cores, 56 threads, 2.7GHz clock speed, 38.5MB cache; Memory: 24*64GB DDR4 2933MHz RECC memory, total 1.5TB.

[0099] Hard drives: 2 x 960GB SSDs + 6 x 12TB SAS HDDs

[0100] Graphics cards: 2*RTX2080Ti

[0101] CUDA version: 12.2

[0102] ② Environment configuration:

[0103] Python: 3.8

[0104] PyTorch: 2.4.0

[0105] Tourch: 2.4.0

[0106] Tourchaudio: 2.4.0

[0107] Step 2: Audio Data Processing

[0108] In terms of audio data processing, the original audio files are normalized to improve the efficiency and accuracy of the model when processing this data. Specifically, this invention divides each audio segment into short clips of 10 to 20 seconds in length. This length was carefully chosen based on two main reasons:

[0109] First, such segments are long enough to capture sufficient contextual information, ensuring that the speech recognition model can accurately understand and recognize continuous speech content. In speech recognition, the natural flow of language and contextual information are crucial. By selecting segments of 10 to 20 seconds, the model can capture the overall sentence structure and important semantic information within the speech, which helps improve the accuracy of speech recognition.

[0110] Secondly, segmenting audio into relatively short segments can effectively reduce the consumption of computing resources and improve processing efficiency. Long audio files can be costly to process, increasing computational burden and slowing down model processing, making it difficult to meet the demands of real-time or large-scale data processing. By breaking audio into shorter segments, the model can perform batch processing more flexibly, processing large amounts of data within a reasonable timeframe and improving the overall efficiency of both training and inference phases.

[0111] After audio segmentation, this invention downsamples each audio segment, reducing the sampling rate to 8kHz (standard call audio quality). The purpose of downsampling is to further reduce the amount of data, thereby reducing computational complexity and storage requirements. In speech recognition, an 8kHz sampling rate is widely used in telephone and communication applications; such a rate is sufficient to preserve key speech features without significantly affecting the model's recognition accuracy. Simultaneously, downsampling helps reduce redundant information and high-frequency noise in the audio, simplifying data complexity and allowing the model to focus more on extracting and analyzing speech features.

[0112] Furthermore, the combination of reducing the sampling rate and shortening audio segments makes the data processing pipeline smoother, not only improving the speed of data preprocessing but also reducing the reliance on hardware during model training. Overall, these measures provide better processing efficiency and lower computational costs for automatic speech recognition systems.

[0113] Step 3: Attention Weight Extraction

[0114] Attention weight extraction is one of the core steps of this method. By tracing the forward propagation process of the model, we can obtain the multi-head attention weight matrices of each layer in the model. These weight matrices reflect how the model allocates attention across different time steps (i.e., different phonemes or morphemes) when processing speech input. This attention mechanism helps the model focus on the most relevant parts of the input, thereby more accurately recognizing words in the speech. Finally, the model outputs the recognized text, which is used to calculate the word error rate. The word error rate is a commonly used metric to measure the performance of an automatic speech recognition system, representing the degree of difference between the recognized words and the original reference text.

[0115] Step 4: Constructing the Functional Connection Matrix

[0116] Next, we construct a functional connectivity matrix representing the relationships between attention heads. The specific method is as follows: For each attention head, we fix one sample in the batch (i.e., an audio segment) and calculate the average of its attention weights over the time dimension to obtain the artificial neural activity of that attention head. By calculating the Pearson correlation coefficient between the artificial neural activities of each pair of attention heads and taking its absolute value, we obtain the value of each element of the functional connectivity matrix.

[0117] like Figure 3 As shown, the model outputs an attention weight matrix when processing audio. For a 1-second audio clip, the model generates a 50-frame attention weight matrix. Inputting audio slices sequentially into the model yields attention weight matrices of corresponding slice lengths. We average the values ​​within each 50x50 frame square on the diagonal of this matrix to obtain the attention head sequence. For each pair of attention head sequences, we calculate the Pearson correlation and use the correlation coefficient as the functional connectivity strength, thus constructing the model's functional connectivity matrix, which is the numerical representation of the functional network. A single value in the sequence is derived from the average of all values ​​within a red box in the figure.

[0118] Step 5: Functional Network Construction

[0119] For the functional connectivity matrix, we can use brain network research methods from neuroscience to construct the model's functional network. We can view this matrix as a weighted undirected graph, with attention heads as nodes in the graph, and the strength of the association between them representing the weight of the edges. The functional network allows for a more intuitive observation of how the model interacts at different levels and between different attention heads, and it can be used to analyze the dynamic cooperative activities between attention heads.

[0120] Step 6: Graph Theory Parameter Calculation

[0121] To further analyze the functional network connectivity between the model's attention heads, we computed several graph theory parameters of the model's functional network:

[0122] Clustering coefficient: Used to measure the local community structure of each node. A high clustering coefficient indicates that these attention heads are more likely to form close subgroups, meaning that there is a high degree of interdependence between attention heads at a certain layer of the model.

[0123] Shortest path length: This represents the average length of the shortest path from one node to another in a network. Shorter path lengths mean that information can travel faster in the network, which is important for the real-time performance and efficiency of speech recognition.

[0124] Small-world property: This is assessed by combining average path length and clustering coefficients to determine if a network possesses small-world characteristics. Small-world networks exhibit efficient information transmission, which is a crucial feature in the functional network analysis of neural networks.

[0125] Node degree distribution: This describes the number of connections each node has, helping us understand the network's connectivity and information propagation characteristics. If some nodes have particularly high degrees, it means they play a more important role in the network and may be key nodes for information transmission.

[0126] 8. Performance Evaluation

[0127] During the performance evaluation phase, we measure the performance of the automatic speech recognition model by calculating the word error rate (BER) of the model's output text. The BER is obtained by calculating the edit distance (including insertion, deletion, and replacement operations) between the output text and the reference text. A lower BER indicates higher accuracy in speech recognition. We average the BER from multiple audio samples to provide a global performance metric for the model.

[0128] 9. Performance Correlation Analysis

[0129] Finally, we used linear regression analysis to study the correlation between graph theory parameters and word error rate, such as... Figure 4 As shown, by plotting a scatter plot of the small-world properties of the functional network and the word error rate (WER), a linear relationship was found between the small-world properties of the model's functional network and the WER. Each point in the plot represents the word error rate output by the model when processing an audio segment, in relation to the small-world properties of the functional network constructed using our method. This analysis helps us determine the relationship between graph theory features (such as high clustering coefficients or short path lengths) and the model's speech recognition performance. Further significance tests were used to verify the statistical significance of these relationships, ensuring the reliability and stability of the analysis results. The results of this part of the research can provide important insights for future Transformer model design and optimization, helping researchers understand how to improve model performance by changing the network structure.

[0130] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the scope of the technology disclosed in the present invention, and such modifications or substitutions should all be covered within the scope of protection of the present invention.

Claims

1. A performance analysis method for a graph theory-based Transformer automatic speech recognition model, characterized in that, include: Obtain a Transformer automatic speech recognition model, which adopts a multi-layer structure and contains multiple attention heads; Acquire audio data and preprocess the audio data; The audio data is input into the Transformer automatic speech recognition model, and the weight matrix of multiple attention heads in each layer of the model is obtained through forward propagation. The word text output by the model is then extracted. The weight matrix of each attention head is averaged over a given time period to obtain the artificial neural activity of that attention head. The correlation coefficient is obtained by using the Pearson correlation coefficient to calculate the correlation of the artificial neural activity of the attention head, and a functional connectivity matrix is ​​constructed based on the correlation coefficient. Calculate the graph theory parameters of the functional connectivity matrix; Calculate the word error rate of the output word text; Based on the graph theory parameters and the word error rate, the performance of the Transformer automatic speech recognition model is analyzed.

2. The performance analysis method for a graph theory-based Transformer automatic speech recognition model according to claim 1, characterized in that, Preprocessing the audio data includes: Each audio data segment is divided into short segments of 10 to 20 seconds in length; Each audio segment is downsampled to standard call audio quality.

3. The performance analysis method for a graph-theory-based Transformer automatic speech recognition model according to claim 1, characterized in that, The construction of the functional connectivity matrix based on the correlation coefficient includes: The correlation coefficient of each pair of attention heads is filled into the functional connectivity matrix. Each element of the functional connectivity matrix represents the functional connectivity strength between the two attention heads. The functional connectivity matrix is ​​regarded as a weighted undirected graph, where the nodes of the undirected graph represent attention heads, and the weights of the edges of the undirected graph represent the functional connectivity strength between them.

4. The performance analysis method for a graph theory-based Transformer automatic speech recognition model according to claim 1, characterized in that, Calculate the graph theory parameters of the functional connectivity matrix, including: clustering coefficient, shortest path length, small-world property, and node degree distribution; The formula for calculating the clustering coefficient is as follows: Among them, e v k is the number of edges that exist between the neighbors of node v. v Let v be the degree of node v, and N be the number of nodes in the network; The formula for calculating the shortest path length is: Where, d ij This represents the shortest path distance from node i to node j; The formula for calculating small-world properties: Among them, C 实 C represents the average clustering coefficient of the original network. 随机 L represents the average clustering coefficient of a random network with the same number of nodes and edges as the original network. 实 L represents the average shortest path length of the original network. 随机 This represents the average shortest path length of a random network with the same number of nodes and edges as the original network. Formula for calculating node degree distribution: Where, N k This represents the number of nodes with degree k.

5. The performance analysis method for a graph theory-based Transformer automatic speech recognition model according to claim 1, characterized in that, The word error rate of the output word text is calculated by calculating the edit distance between the output word text and the reference text.

6. The performance analysis method for a graph theory-based Transformer automatic speech recognition model according to claim 1, characterized in that, The performance analysis of the Transformer automatic speech recognition model based on the graph theory parameters and the word error rate is as follows: Input multiple audio data points to obtain multiple graph theory parameters and word error rates; We used linear regression analysis to study the correlation between graph theory parameters and word error rate.

7. A computer system, characterized in that... include: One or more processors, a computer-readable storage medium for storing one or more programs, wherein, when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the method of claim 1.

8. A computer-readable storage medium, characterized in that... The device stores computer-executable instructions, which, when executed, are used to implement the method of claim 1.

9. A computer program product, characterized in that... It includes computer-executable instructions, which, when executed, are used to implement the method of claim 1.

Images