Performance analysis method of Transform automatic speech recognition model based on graph theory

Abstract

The invention relates to a performance analysis method of a Transform automatic speech recognition model based on a graph theory, and belongs to the field of artificial intelligence and speech recognition. Comprising the following steps: acquiring a Transform automatic speech recognition model, and acquiring audio data; inputting the audio data into a Transform automatic speech recognition model, obtaining a weight matrix of a plurality of attention heads of each layer in the model through forward propagation, and extracting a word text output by the model; averaging the weight matrix of each attention head in a given time to obtain the artificial neural activity of the attention head; performing correlation calculation on the artificial neural activity of the attention head by using a Pearson's correlation coefficient to obtain a correlation coefficient, and constructing a functional connection matrix based on the correlation coefficient; calculating graph theory parameters of the function connection matrix; calculating a word error rate of the output word text; and analyzing the performance of the Transform automatic speech recognition model based on the graph theory parameters and the word error rate. According to the method, a basis is provided for performance analysis of the Transform automatic speech recognition model.

Description

Technical Field

[0001] The invention relates to a performance analysis method of a Transformer automatic speech recognition model based on graph theory, and belongs to the field of artificial intelligence and speech recognition. Background Art

[0002] Transformer is a deep learning model architecture proposed by Vaswani et al. in 2017. It was originally used for natural language processing (NLP) tasks, especially machine translation. Due to its excellent performance and highly scalable design, Transformer has quickly become the mainstream model architecture for many natural language processing tasks and has been widely used in the field of speech recognition. However, there is still a lack of in-depth understanding of the interactions between attention heads in the multi-head attention mechanism within the model, especially how they affect model performance. Existing research mainly focuses on improving the performance of automatic speech recognition (ASR) models by improving the structure of the model, while there is less systematic research on the relationship between attention heads within the model and their impact on model performance. Summary of the invention

[0003] The technical problems to be solved by the present invention are:

[0004] In order to avoid the shortcomings of the prior art, the present invention provides a performance analysis method for a Transformer automatic speech recognition model based on graph theory, aiming to reveal the interaction between attention heads and how these relationships affect the overall performance of the Transformer automatic speech recognition model, especially the impact on the word error rate (WER).

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0006] A performance analysis method for a Transformer automatic speech recognition model based on graph theory, characterized by comprising:

[0007] Obtain a Transformer automatic speech recognition model, wherein the Transformer automatic speech recognition model adopts a multi-layer structure and includes multiple attention heads;

[0008] Acquire audio data, and preprocess the audio data;

[0009] Input the audio data into the Transformer automatic speech recognition model, obtain the weight matrix of multiple attention heads of each layer in the model through forward propagation, and extract the word text output by the model;

[0010] The weight matrix of each attention head is averaged within a given time to obtain the artificial neural activity of the attention head;

[0011] Using the Pearson correlation coefficient to calculate the correlation of the artificial neural activities of the attention head to obtain a correlation coefficient, and constructing a functional connection matrix based on the correlation coefficient; calculating graph theory parameters of the functional connection matrix;

[0012] Calculating a word error rate of the output word text;

[0013] Based on the graph theory parameters and the word error rate, the performance of the Transformer automatic speech recognition model is analyzed.

[0014] A further technical solution of the present invention is to pre-process the audio data, comprising:

[0015] Split each audio data into short segments of 10 to 20 seconds in length;

[0016] Downsample each audio segment to standard call audio quality.

[0017] A further technical solution of the present invention: constructing a functional connectivity matrix based on correlation coefficients comprises:

[0018] The correlation coefficient of each pair of attention heads is filled into the functional connection matrix, and each element of the functional connection matrix represents the functional connection strength between two attention heads; the functional connection matrix is ​​regarded as a weighted undirected graph, the nodes of the undirected graph represent the attention heads, and the weights of the edges of the undirected graph represent the functional connection strength between them.

[0019] A further technical solution of the present invention is: calculating graph theory parameters of the functional connectivity matrix, the graph theory parameters including: clustering coefficient, shortest path length, small world property, node degree distribution;

[0020] The calculation formula of clustering coefficient is:

[0021]

[0022] Among them, e v is the number of edges between the neighbors of node v, k v is the degree of node v, N is the number of nodes in the network;

[0023] The formula for calculating the shortest path length is:

[0024]

[0025] Among them, d ij represents the shortest path distance from node i to node j;

[0026] The calculation formula of small world attribute is:

[0027]

[0028] Among them, C 实 represents the average clustering coefficient of the original network, C 随机 represents the average clustering coefficient of random networks with the same number of nodes and edges as the original network, L 实 represents the average shortest path length of the original network, L 随机 represents the average shortest path length of a random network with the same number of nodes and edges as the original network;

[0029] The calculation formula of node degree distribution is:

[0030]

[0031] Among them, N k Represents the number of nodes with degree k.

[0032] A further technical solution of the present invention: the word error rate of the output word text is calculated, specifically, by calculating the edit distance between the output word text and a reference text.

[0033] A further technical solution of the present invention is: analyzing the performance of the Transformer automatic speech recognition model based on the graph theory parameters and the word error rate, specifically:

[0034] Input multiple audio data to obtain multiple graph theory parameters and word error rates;

[0035] Multiple graph theory parameters and word error rate were analyzed by linear regression to study the correlation between graph theory parameters and word error rate.

[0036] A computer system, characterized in that it comprises: one or more processors, and a computer-readable storage medium for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the above-mentioned method.

[0037] A computer-readable storage medium, characterized in that it stores computer-executable instructions, which are used to implement the above method when executed.

[0038] A computer program product, characterized by comprising computer executable instructions, wherein the instructions are used to implement the above method when executed.

[0039] The beneficial effects of the present invention are:

[0040] The present invention provides a performance analysis method of a Transformer automatic speech recognition model based on graph theory. By introducing a graph theory analysis method, the operation mechanism of multiple attention heads in the Transformer-based automatic speech recognition model is deeply studied, and the following significant effects are produced:

[0041] 1. There is a negative correlation between the small-world property of the model's internal functional network and the word error rate: By constructing a functional connection matrix between attention heads and performing graph theory analysis on it, the present invention reveals the negative correlation between the small-world property of the model's internal functional network and the word error rate. This provides a new idea for optimizing the model's structure and parameter selection, and helps improve the accuracy and performance of the automatic speech recognition model.

[0042] 2. High correlation between high-level attention heads of the model: The present invention provides an in-depth understanding of the internal working mechanism of the automatic speech recognition model through the calculation and analysis of graph theory parameters (such as clustering coefficient, small-world attribute value, etc.), and finds that the correlation between high-level attention heads of the model is higher than the correlation between low-level attention heads, which indicates that high-level attention heads may have important functions in the model processing process. High-level attention heads are the attention head group that contributes most significantly to the model's help in identifying audio, which can provide a scientific basis for the improvement and application of the model.

[0043] 3. Provide a general analysis framework: The method of the present invention is not only applicable to automatic speech recognition models, but can also be extended to other computer vision deep learning models that use multi-head attention mechanisms. It can explore the interaction of attention heads in different tasks and provide a unified analysis framework for improving model performance.

[0044] 4. Reduce development and debugging costs: By revealing the linear relationship between the functional connection network of the attention head and the model performance, this invention provides a powerful tool for the development and debugging of automatic speech recognition models, helping researchers to more efficiently locate and optimize the key parts of the model and reduce the time and cost of development and debugging. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The drawings are only for the purpose of illustrating particular embodiments and are not to be considered limiting of the present invention. Like reference symbols denote like components throughout the drawings.

[0046] Figure 1 The present invention is a flowchart of a performance analysis method of a Transformer automatic speech recognition model based on graph theory.

[0047] Figure 2 This is a flow chart of the method for constructing the functional connection matrix of the Transformer model of the present invention.

[0048] Figure 3 Schematic diagram of the functional connection matrix of the present invention.

[0049] Figure 4 It is a scatter plot between the small-world property of the functional network of the present invention and the word error rate. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0051] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0052] Inspired by the analysis of brain functional connectivity networks in functional magnetic resonance imaging (fMRI) research, this paper proposes a method that analyzes the multi-head attention mechanism in the Transformer model, constructs a functional connectivity matrix at the attention head level, and further conducts graph theory analysis to study the relationship between graph theory parameters and the performance of the automatic speech recognition model. This analysis method can not only help understand the internal working mechanism of the Transformer automatic speech recognition model and improve the interpretability of the model, but also provide a theoretical basis for model optimization.

[0053] The purpose of the present invention is: first, to optimize the performance of the automatic speech recognition model: by studying the relationship between graph theory parameters and word error rate, a theoretical basis is provided for optimizing the structure and parameter selection of the automatic speech recognition model, thereby improving the accuracy and robustness of the model. Second, to provide model interpretability: the present invention provides interpretability for the decision-making process of the automatic speech recognition model through graph theory analysis, which helps to determine the attention heads that contribute most significantly to the model performance.

[0054] The present invention provides a performance analysis method for a Transformer automatic speech recognition model based on graph theory. The method constructs a functional connection matrix between attention heads of the Transformer automatic speech recognition model and performs graph theory analysis on the functional connection matrix to reveal the relationship between the functional network structure of the attention heads of the Transformer automatic speech recognition model and the model performance.

[0055] like Figure 1 As shown, the present invention comprises the following steps:

[0056] Step 1: Model selection: Select an automatic speech recognition model with a Transformer architecture. This type of model uses a multi-layer structure, contains multiple attention heads, and is trained and fine-tuned based on audio data. Its main feature is that it can effectively extract rich features from raw speech signals and learn on unlabeled speech data through unsupervised pre-training, achieving excellent performance in downstream tasks with only a small amount of labeled data.

[0057] Step 2: Data Processing: For audio data processing, we normalized the audio files. Specifically, to ensure that the model can process data efficiently, we split each audio segment into short segments of 10 to 20 seconds in length. This length was chosen for two reasons: first, they can capture enough contextual information to correctly recognize speech; second, shorter segments can reduce the use of computing resources, allowing the model to process large amounts of data in a reasonable time. After that, we downsampled each audio segment and reduced the sampling rate to 8kHz (standard call audio quality), which also helped reduce the computational load of the model while having little impact on the accuracy of speech recognition.

[0058] Step 3: Attention weight matrix extraction: After the model is arranged, select the test audio sample, obtain the multi-head attention weight matrix of each layer in the model through forward propagation, and finally output the word text recognized by the model for word error rate calculation. The weight matrix represents how the model allocates attention between different attention heads when processing a given speech input.

[0059] Step 4: Extraction of model artificial neural activity: For each attention head, an attention head weight matrix with a length equal to the audio time length will be generated during the audio processing process. The attention head weight matrix represents the degree of attention of the attention head to the current moment and the previous and subsequent moments at each moment. All weights of the attention head weight matrix within 1S are averaged to obtain the artificial neural activity of the attention head.

[0060] Step 5: Functional connection matrix construction: Use the Pearson correlation coefficient to calculate the correlation of the artificial neural activities of the attention heads, and fill the correlation coefficient of each pair of attention heads into the functional connection matrix. Each element of the functional connection matrix represents the functional connection strength between the two attention heads. This functional connection matrix can be regarded as a weighted undirected graph, where the nodes of the undirected graph represent the attention heads, and the weights of the edges of the undirected graph represent the functional connection strength between them.

[0061] Step 6: Calculation of graph theory parameters:

[0062] The following parameters were calculated for the functional connectivity matrix of all models:

[0063] Clustering Coefficient: Calculate the clustering coefficient of each node (i.e., attention head), which describes the connection density between its neighboring nodes. A high clustering coefficient usually means that these attention heads form a tight community structure in the network. For the clustering coefficient C of node v v It can be expressed as:

[0064]

[0065] Among them, e v is the number of edges between the neighbors of node v, k v is the degree of node v (i.e. the number of nodes connected to it).

[0066] The average clustering coefficient C of the entire network is the average value of the clustering coefficients of all nodes:

[0067]

[0068] Where N is the number of nodes in the network.

[0069] Shortest Path Length: The shortest path length of a given node quantifies the average distance or routing efficiency between that node and all other nodes in the network. For a given node i, its shortest path length is the average distance between it and all other nodes:

[0070]

[0071] Among them, d ij Represents the shortest path distance from node i to node j.

[0072] The average shortest path length L of the network is:

[0073]

[0074] Small-World Property: Evaluates the overall small-world properties of the network, including the comparison of average path length and clustering coefficient. Small-world networks have unique advantages in information dissemination and processing efficiency. The evaluation of small-world properties usually includes two main indicators: average path length and clustering coefficient. When defining a network as a small-world network, it should meet the following requirements:

[0075]

[0076] Among them, C 实 represents the average clustering coefficient of the original network, C 随机 represents the average clustering coefficient of random networks with the same number of nodes and edges as the original network, L 实 represents the average shortest path length of the original network, L 随机 Represents the average shortest path length of a random network with the same number of nodes and edges as the original network.

[0077] Node Degree Distribution: Analyze the degree distribution of each node in the network to understand the connectivity and information propagation characteristics of the network. The node degree k is the number of connections between a node and other nodes in the network. The node degree distribution describes the probability distribution of node degrees in the network. The degree distribution can be expressed as:

[0078]

[0079] Among them, N k represents the number of nodes with degree k, and N is the total number of nodes in the network.

[0080] Step 7: Performance evaluation: For automatic speech recognition models, a common performance evaluation method is word error rate. The word error rate is calculated for each word recognition text output by the model.

[0081] Step 8: Performance correlation analysis:

[0082] After obtaining the word error rate and graph theory parameters of the model, the relationship between the word error rate and the graph theory parameters is studied to reveal the impact of the attention head network structure on the model performance.

[0083] In order to enable those skilled in the art to better understand the present invention, the present invention is described in detail below in conjunction with specific embodiments.

[0084] The above steps are as follows:

[0085] Step 1: Model selection

[0086] The present invention chooses to use the publicly available pre-trained wav2vec 2.0 model. This model is a self-supervised learning model developed by Meta (formerly Facebook) AI, specifically for speech recognition tasks. The wav2vec 2.0 model has a multi-layer structure, which includes 12 Transformer encoding layers, each layer has 12 multi-head attention mechanisms, and a total of 144 attention heads. Each attention head outputs an attention weight matrix during the forward propagation of the model. This matrix represents the model's attention allocation to different positions in the input speech sequence when processing speech input.

[0087] The reason for choosing the pre-trained wav2vec 2.0 model is that it has demonstrated excellent performance in a variety of speech recognition tasks, and the public availability of the pre-trained model can reduce the time and computing resources required to train our model.

[0088] The architecture of the wav2vec 2.0 model mainly consists of three parts: feature extractor, quantizer, and context network.

[0089] ①. Feature Extractor

[0090] The feature extractor is the basic component of the wav2vec 2.0 model, which is responsible for converting the original audio signal into a feature representation suitable for model processing. In the wav2vec 2.0 model, the feature extractor uses the structure of a convolutional neural network (CNN) to extract local features of the audio by performing convolution operations on the audio signal. These local features contain the timing information and local structure of the audio, providing rich information for the subsequent quantizer and context network.

[0091] ②Quantizer

[0092] The quantizer is a key component of the wav2vec 2.0 model, which converts the continuous feature representation output by the feature extractor into a discrete potential representation. This discretization process enables the model to learn the intrinsic structure of the audio signal, thereby achieving effective encoding of the audio signal. In the wav2vec 2.0 model, the quantizer uses the Gumbel-Softmax distribution for discretization operations, achieving soft discretization of continuous feature representations.

[0093] 3. Contextual Network

[0094] The context network is another important component of the wav2vec 2.0 model, which is responsible for capturing the contextual information of the audio signal. By calculating the self-attention mechanism on the discrete potential representation, the context network is able to learn the global structure and semantic content of the audio signal. This global information is crucial for downstream tasks because it helps the model better understand the intrinsic meaning of the audio signal.

[0095] As a powerful audio pre-training model, the wav2vec 2.0 model has achieved significant performance improvements in tasks such as speech recognition and audio classification. By pre-training on large-scale unlabeled audio data, the wav2vec 2.0 model can provide rich feature representations for downstream tasks, thereby achieving better performance. In practical applications, the performance of the Transformer automatic speech recognition model demonstrates the great potential of self-supervised learning in the field of audio processing.

[0096] Model operating environment

[0097] ①Server configuration:

[0098] Processor: Intel Xeon Platinum 8280*2, 28 cores and 56 threads, 2.7GHz main frequency, 38.5MB cache Memory: 24*64G DDR4 2933MHz RECC memory, a total of 1.5TB

[0099] Hard disk: 2*960G solid state hard disk + 6*12T SAS hard disk

[0100] Graphics card: 2*RTX2080Ti

[0101] CUDA version: 12.2

[0102] ②Environment configuration:

[0103] Python: 3.8

[0104] Pytorch: 2.4.0

[0105] Tourch:2.4.0

[0106] Tourchaudio:2.4.0

[0107] Step 2: Audio Data Processing

[0108] In terms of audio data processing, the original audio files are normalized to improve the efficiency and accuracy of the model in processing these data. Specifically, the present invention divides each audio segment into short segments with a length of 10 to 20 seconds. The choice of this length is carefully weighed and based on the following two main reasons:

[0109] First, such a segment length is long enough to capture enough contextual information to ensure that the speech recognition model can accurately understand and recognize the continuous speech content. In speech recognition, the natural flow of language and contextual information are very important. By selecting segments of 10 to 20 seconds, the model is able to capture the overall sentence structure and important semantic information in the speech, which helps improve the accuracy of speech recognition.

[0110] Secondly, splitting the audio into relatively short segments can effectively reduce the use of computing resources and improve processing efficiency. For long audio files, the processing cost may be high, which not only increases the computing burden, but also causes the model processing speed to slow down, making it difficult to meet the needs of real-time or large-scale data processing. By splitting the audio into shorter segments, the model can perform batch processing more flexibly, thereby processing large amounts of data within a reasonable time frame and improving the overall efficiency of the training and inference stages.

[0111] After audio segmentation, the present invention downsamples each audio segment and reduces the sampling rate to 8kHz (standard call audio quality). The purpose of downsampling is to further reduce the amount of data, thereby reducing computational complexity and storage requirements. In speech recognition, a sampling rate of 8kHz is widely used in telephone and communication applications. Such a sampling rate is sufficient to maintain the key feature information of speech and will not significantly affect the recognition accuracy of the model. At the same time, downsampling can help reduce redundant information and high-frequency noise in the audio, thereby simplifying the complexity of the data and allowing the model to focus more on the extraction and analysis of speech features.

[0112] In addition, the combination of lowering the sampling rate and shortening the audio segments makes the data processing pipeline smoother, which not only improves the speed of data preprocessing, but also reduces the dependence on hardware during model training. These measures provide better processing efficiency and lower computing costs for automatic speech recognition systems as a whole.

[0113] Step 3: Attention Weight Extraction

[0114] Attention weight extraction is one of the core steps of this method. By tracing the forward propagation process of the model, we can obtain the multi-head attention weight matrices of each layer in the model. These weight matrices reflect how the model allocates attention between different time steps (i.e., different phonemes or morphemes) when processing speech input. This attention mechanism can help the model focus on the most relevant parts of the input, thereby more accurately recognizing words in the speech. Finally, the model outputs the recognized text, which will be used to calculate the word error rate. The word error rate is a common indicator for measuring the performance of automatic speech recognition systems, indicating the degree of difference between the recognized words and the original reference text.

[0115] Step 4: Functional connectivity matrix construction

[0116] Next, we construct a functional connectivity matrix that represents the relationship between attention heads. The specific method is as follows: for each attention head, we fix a sample in the batch (i.e., an audio clip) and calculate the average of its attention weights in the time dimension to obtain the artificial neural activity of the attention head. By calculating the Pearson correlation coefficient of the artificial neural activity between each pair of attention heads and taking its absolute value, we obtain the value of each element of the functional connectivity matrix.

[0117] like Figure 3 As shown in the figure, the model will output an attention weight matrix when processing audio. For 1 second of audio, the model will generate an attention weight matrix for 50 frames. The audio slices are input into the model in sequence to obtain the attention weight matrix of the corresponding slice length. We average the values ​​in the square matrix of every 50 frames x 50 frames on the diagonal of the matrix, that is, we get the attention head sequence. We calculate the Pearson correlation for each pair of attention head sequences and use the correlation coefficient as the functional connection strength to construct the functional connection matrix of the model, which is a numerical expression of the functional network. The single value of the sequence comes from the average of all values ​​in a red box in the figure.

[0118] Step 5: Functional network construction

[0119] For the functional connectivity matrix, we can use the brain network research method in the field of neuroscience to construct the functional network of the model. We regard the matrix as a weighted undirected graph, and the attention heads as nodes in the graph. The strength of the association between them represents the weight of the edge. The functional network can more intuitively observe the interaction between different levels and different attention heads of the model, and can use the functional network to analyze the dynamic collaborative activities between attention heads.

[0120] Step 6: Graph Theory Parameter Calculation

[0121] To further analyze the functional network connections between the model’s attention heads, we calculated multiple graph-theoretic parameters of the model’s functional network:

[0122] Clustering coefficient: It is used to measure the local community structure of each node. A high clustering coefficient means that these attention heads tend to form tight subgroups, which means that there is a high interdependence between the attention heads in a certain layer of the model.

[0123] Shortest path length: Indicates the average length of the shortest path from one node to another in the network. A shorter path length means that information can be transmitted more quickly in the network, which is important for the real-time performance and efficiency of speech recognition.

[0124] Small-world property: The combination of average path length and clustering coefficient is used to evaluate whether the network has small-world properties. Small-world networks have efficient information transmission characteristics, which is also a very important feature in the functional network analysis of neural networks.

[0125] Node degree distribution: used to describe the number of connections of each node, helping us understand the connectivity and information dissemination characteristics of the network. If the degree of some nodes is particularly high, it means that they play a more important role in the network and may be key nodes for information transmission.

[0126] 8. Performance Evaluation

[0127] During the performance evaluation phase, we measure the performance of the automatic speech recognition model by calculating the word error rate of the model output text. The word error rate is obtained by calculating the edit distance between the output text and the reference text (including insertion, deletion and substitution operations). The lower the word error rate, the more accurate the model recognizes the speech. We average the word error rates of multiple audio samples to provide a global model performance indicator.

[0128] 9. Performance correlation analysis

[0129] Finally, we investigate the correlation between graph theoretic parameters and word error rate by linear regression analysis, e.g. Figure 4 As shown in the figure, by drawing a scatter plot between the small-world property of the functional network and the word error rate, it is found that there is a linear relationship between the small-world property of the model's functional network and the word error rate WER. Each point in the figure represents the word error rate output by the model when processing a piece of audio and the small-world property of the functional network constructed according to our method. This analysis can help us determine the relationship between graph theory features (such as high clustering coefficient or short path length) and the speech recognition performance of the model. Further significance tests are used to verify the statistical significance of these relationships to ensure the reliability and stability of the analysis results. The results of this part of the study can provide important inspiration for future Transformer model design and optimization, helping researchers understand how to improve model performance by changing the network structure.

[0130] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should be included in the protection scope of the present invention.

Claims

1. A performance analysis method for a Transformer automatic speech recognition model based on graph theory, characterized in that: include: Obtain a Transformer automatic speech recognition model, wherein the Transformer automatic speech recognition model adopts a multi-layer structure and includes multiple attention heads; Acquire audio data, and preprocess the audio data; Input the audio data into the Transformer automatic speech recognition model, obtain the weight matrix of multiple attention heads of each layer in the model through forward propagation, and extract the word text output by the model; The weight matrix of each attention head is averaged within a given time to obtain the artificial neural activity of the attention head; Using the Pearson correlation coefficient to calculate the correlation of the artificial neural activities of the attention head to obtain a correlation coefficient, and constructing a functional connection matrix based on the correlation coefficient; Calculating graph theory parameters of the functional connectivity matrix; Calculating a word error rate of the output word text; Based on the graph theory parameters and the word error rate, the performance of the Transformer automatic speech recognition model is analyzed.

2. According to claim 1, a performance analysis method of a Transformer automatic speech recognition model based on graph theory is characterized in that: Preprocessing the audio data includes: Split each audio data into short segments of 10 to 20 seconds in length; Downsample each audio segment to standard call audio quality.

3. According to claim 1, a performance analysis method of a Transformer automatic speech recognition model based on graph theory is characterized in that: The constructing of a functional connection matrix based on the correlation coefficient comprises: The correlation coefficient of each pair of attention heads is filled into the functional connection matrix, and each element of the functional connection matrix represents the functional connection strength between two attention heads; the functional connection matrix is ​​regarded as a weighted undirected graph, the nodes of the undirected graph represent the attention heads, and the weights of the edges of the undirected graph represent the functional connection strength between them.

4. According to claim 1, a performance analysis method of a Transformer automatic speech recognition model based on graph theory is characterized in that: Calculating graph theory parameters of the functional connectivity matrix, the graph theory parameters including: clustering coefficient, shortest path length, small world property, node degree distribution; The calculation formula of clustering coefficient is: Among them, e v is the number of edges between the neighbors of node v, k v is the degree of node v, N is the number of nodes in the network; The formula for calculating the shortest path length is: Among them, d ij represents the shortest path distance from node i to node j; The calculation formula of small world attribute is: Among them, C 实 represents the average clustering coefficient of the original network, C 随机 represents the average clustering coefficient of random networks with the same number of nodes and edges as the original network, L 实 represents the average shortest path length of the original network, L 随机 represents the average shortest path length of a random network with the same number of nodes and edges as the original network; The calculation formula of node degree distribution is: Among them, N k Represents the number of nodes with degree k.

5. According to claim 1, a performance analysis method of a Transformer automatic speech recognition model based on graph theory is characterized in that: The word error rate of the output word text is calculated, specifically, by calculating the edit distance between the output word text and a reference text.

6. The performance analysis method of the Transformer automatic speech recognition model based on graph theory according to claim 1 is characterized in that: The performance of the Transformer automatic speech recognition model is analyzed based on the graph theory parameters and the word error rate, specifically: Input multiple audio data to obtain multiple graph theory parameters and word error rates; Multiple graph theory parameters and word error rate were analyzed by linear regression to study the correlation between graph theory parameters and word error rate.

7. A computer system, characterized in that include: One or more processors, and a computer-readable storage medium for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors are enabled to implement the method of claim 1.

8. A computer-readable storage medium, characterized in that Computer executable instructions are stored, and when the instructions are executed, they are used to implement the method of claim 1.

9. A computer program product, characterized in that The invention comprises computer executable instructions, which are used to implement the method of claim 1 when being executed.

Images