Method and device for constructing hybrid dynamic audio quality evaluation model based on Mangban structure

Abstract

The invention provides a construction method and device for a hybrid dynamic audio quality evaluation model based on a Mangban structure, and the method comprises the following steps: obtaining multiple groups of long audio data as training samples, and constructing an audio quality evaluation frame which is composed of a frequency spectrum conversion module, a hybrid dynamic attention module and a classification module; a spectrum conversion module converts the long audio data into a local feature sequence; the mixed dynamic attention module sequentially processes each local feature vector in the local feature sequence to obtain a mixed attention result; and inputting the mixed attention result into a classification module to obtain an audio quality evaluation result. According to the scheme, a light-weight model is constructed by combining the Munban method with a linear attention mechanism, and capturing of local details and global information is considered in a hybrid dynamic attention module, so that the overall performance of audio quality evaluation is greatly improved, and the method has high efficiency and accuracy of evaluating a long audio sequence.

Description

Technical Field

[0001] The present application relates to the field of audio processing technology, and in particular to a method and device for constructing a hybrid dynamic audio quality evaluation model based on a Mamba structure. Background Art

[0002] With the rapid development of audio processing technology, especially the widespread application of deep learning technology, audio quality evaluation has become a research field that has attracted much attention. Traditional audio quality evaluation methods usually rely on manually designed features or frequency domain analysis based on Fourier transform. However, these methods often show certain limitations when dealing with complex and changeable audio scenes, and it is difficult to meet the requirements of high precision and high efficiency, especially in the quality evaluation of singing audio.

[0003] Therefore, how to process complex audio sequences efficiently and accurately to improve the overall performance of audio quality evaluation is a problem that needs to be solved urgently in the existing technology. Summary of the invention

[0004] The embodiment of the present application provides a method and device for constructing a hybrid dynamic audio quality evaluation model based on the Mamba structure. A lightweight model is constructed by combining the Mamba method with the linear attention mechanism. The capture of local details and global information is taken into account in the hybrid dynamic attention module, thereby greatly improving the overall performance of audio quality evaluation and achieving both high efficiency and accuracy in evaluating long audio sequences.

[0005] In a first aspect, an embodiment of the present application provides a method for constructing a hybrid dynamic audio quality evaluation model based on a Mamba structure, the method comprising:

[0006] Acquire multiple groups of long audio data as training samples and construct an audio quality evaluation framework, wherein the audio quality evaluation framework consists of a spectrum conversion module, a hybrid dynamic attention module, and a classification module;

[0007] The spectrum conversion module converts the long audio data into a logarithmic amplitude spectrum, and then divides the logarithmic amplitude spectrum into a plurality of local feature vectors of equal size to form a local feature sequence;

[0008] The hybrid dynamic attention module processes each local feature vector in the local feature sequence sequentially. The dynamic attention module includes a first branch, a second branch and a linear branch. The local feature vector is input into the first branch and the second branch respectively. The local feature vector is linearly mapped by a linear unit in the first branch to obtain a first branch feature. The second branch includes a linear unit, a dynamic local convolution structure and a dynamic global attention structure. The local feature vector is linearly mapped by a linear unit in the second branch to obtain a second linear feature. The second linear feature is averagely split based on the channel dimension of the second linear feature to obtain a first split result and a second split result. The dynamic local convolution structure assigns a dynamic weight to the first split result to obtain a dynamic local convolution result. The second split result is dynamically calculated in the dynamic global attention structure to obtain a dynamic global attention result. The dynamic local convolution result and the dynamic global attention result are spliced ​​to obtain a second branch feature. The first branch feature and the second branch feature are Hadamard-producted and input into the linear branch to obtain a hybrid attention result.

[0009] The mixed attention result is input into the classification module to obtain an audio quality evaluation result, a loss function is constructed based on the audio quality evaluation result, and the parameters of the audio quality evaluation framework are adjusted based on the loss function to obtain an audio quality evaluation model.

[0010] In a second aspect, an embodiment of the present application provides a hybrid dynamic audio quality evaluation method based on a Mamba structure, the method comprising:

[0011] The long audio data to be evaluated is obtained, and the long audio data to be evaluated is input into the constructed audio quality evaluation model to obtain an audio quality evaluation result.

[0012] In a third aspect, an embodiment of the present application provides a device for constructing a hybrid dynamic audio quality evaluation model based on a Mamba structure, the device comprising:

[0013] A construction module is used to obtain multiple groups of long audio data as training samples and construct an audio quality evaluation framework, wherein the audio quality evaluation framework consists of a spectrum conversion module, a hybrid dynamic attention module, and a classification module;

[0014] A spectrum conversion module, used for converting the long audio data into a logarithmic amplitude spectrum, and then dividing the logarithmic amplitude spectrum into a plurality of local feature vectors of equal size to form a local feature sequence;

[0015] A hybrid dynamic attention module, used for sequentially processing each local feature vector in a local feature sequence, the dynamic attention module includes a first branch, a second branch and a linear branch, the local feature vector is input into the first branch and the second branch respectively, the local feature vector is linearly mapped by a linear unit in the first branch to obtain a first branch feature, the second branch includes a linear unit, a dynamic local convolution structure and a dynamic global attention structure, the local feature vector is linearly mapped by a linear unit in the second branch to obtain a second linear feature, the second linear feature is averagely split based on the channel dimension of the second linear feature to obtain a first split result and a second split result, the dynamic local convolution structure assigns a dynamic weight to the first split result to obtain a dynamic local convolution result, the second split result is dynamically calculated in the dynamic global attention structure to obtain a dynamic global attention result, the dynamic local convolution result is spliced ​​with the dynamic global attention result to obtain a second branch feature, the first branch feature and the second branch feature are Hadamard-producted and input into the linear branch to obtain a hybrid attention result;

[0016] A classification module is used to input the mixed attention result into the classification module to obtain an audio quality evaluation result, construct a loss function based on the audio quality evaluation result, and adjust the parameters of the audio quality evaluation framework based on the loss function to obtain an audio quality evaluation model.

[0017] In a fourth aspect, an embodiment of the present application provides an electronic device, comprising a memory and a processor, characterized in that a computer program is stored in the memory, and the processor is configured to run the computer program to execute a method for constructing a hybrid dynamic audio quality evaluation model based on a Mamba structure or a hybrid dynamic audio quality evaluation method based on a Mamba structure.

[0018] In a fifth aspect, an embodiment of the present application provides a readable storage medium, in which a computer program is stored. The computer program includes a program code for controlling a process to execute a process, and the process includes a method for constructing a hybrid dynamic audio quality evaluation model based on a Mamba structure or a hybrid dynamic audio quality evaluation method based on a Mamba structure.

[0019] The main contributions and innovations of the present invention are as follows:

[0020] The embodiment of the present application constructs a hybrid dynamic attention module based on the Mamba method. The linear attention mechanism is optimized in the hybrid dynamic attention module, which improves the performance of the hybrid dynamic attention module in learning in context, and can effectively process long sequence data and reduce the occurrence of overfitting. In the hybrid attention module, a convolution kernel is generated by combining dynamic weights and static parameters, and the input features are flexibly adjusted through the convolution kernel. In the dynamic global attention structure, the attention matrix is ​​lightweight by dividing and aggregating the attention horizontally and vertically, thereby improving the efficiency and accuracy of audio quality evaluation.

[0021] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0023] Figure 1 It is a flowchart of a method for constructing a hybrid dynamic audio quality evaluation model based on a Mamba structure according to an embodiment of the present application;

[0024] Figure 2 It is a structural schematic diagram of a method for constructing a hybrid dynamic audio quality evaluation model based on a Mamba structure according to an embodiment of the present application;

[0025] Figure 3 is a structural schematic diagram of a dynamic local convolution structure according to an embodiment of the present application;

[0026] Figure 4 This is a structural diagram of a dynamic global attention structure according to an embodiment of the present application.

[0027] Figure 5 It is a structural block diagram of a device for constructing a hybrid dynamic audio quality evaluation model based on a Mamba structure according to an embodiment of the present application;

[0028] Figure 6 It is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0029] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with one or more embodiments of this specification. Instead, they are merely examples of devices and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0030] It should be noted that: in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; and multiple steps described in this specification may be combined into a single step for description in other embodiments.

[0031] Embodiment 1

[0032] The embodiment of the present application provides a method for constructing a hybrid dynamic audio quality evaluation model based on the Mamba structure. A lightweight model is constructed by combining the Mamba method with the linear attention mechanism. The capture of local details and global information is taken into account in the hybrid dynamic attention module, thereby greatly improving the overall performance of audio quality evaluation and achieving both high efficiency and accuracy in evaluating long audio sequences. Specifically, reference Figure 1 as well as Figure 2 , the method comprising:

[0033] Acquire multiple groups of long audio data as training samples and construct an audio quality evaluation framework, wherein the audio quality evaluation framework consists of a spectrum conversion module, a hybrid dynamic attention module, and a classification module;

[0034] The spectrum conversion module converts the long audio data into a logarithmic amplitude spectrum, and then divides the logarithmic amplitude spectrum into a plurality of local feature vectors of equal size to form a local feature sequence;

[0035] The hybrid dynamic attention module processes each local feature vector in the local feature sequence sequentially. The dynamic attention module includes a first branch, a second branch and a linear branch. The local feature vector is input into the first branch and the second branch respectively. The local feature vector is linearly mapped by a linear unit in the first branch to obtain a first branch feature. The second branch includes a linear unit, a dynamic local convolution structure and a dynamic global attention structure. The local feature vector is linearly mapped by a linear unit in the second branch to obtain a second linear feature. The second linear feature is averagely split based on the channel dimension of the second linear feature to obtain a first split result and a second split result. The dynamic local convolution structure assigns a dynamic weight to the first split result to obtain a dynamic local convolution result. The second split result is dynamically calculated in the dynamic global attention structure to obtain a dynamic global attention result. The dynamic local convolution result and the dynamic global attention result are spliced ​​to obtain a second branch feature. The first branch feature and the second branch feature are Hadamard-producted and input into the linear branch to obtain a hybrid attention result.

[0036] The mixed attention result is input into the classification module to obtain an audio quality evaluation result, a loss function is constructed based on the audio quality evaluation result, and the parameters of the audio quality evaluation framework are adjusted based on the loss function to obtain an audio quality evaluation model.

[0037] In the step of "the spectrum conversion module converts the long audio data into a logarithmic amplitude spectrum", the long audio data is subjected to Fourier transform to obtain audio frequency data, the audio frequency data is first mapped to a Mel frequency scale to obtain Mel frequency data, and then the Mel frequency data is converted into a logarithmic amplitude spectrum.

[0038] Specifically, the purpose of converting the long audio data into a logarithmic amplitude spectrum is to obtain a frequency spectrum diagram close to human ear perception, so that the evaluation result of the long audio data is more accurate.

[0039] In the step of "dividing the logarithmic amplitude spectrum into a plurality of local feature vectors of equal size to form a local feature sequence", the logarithmic amplitude spectrum is first divided into a plurality of local feature regions of equal size, and then each local feature region is reshaped into a local feature vector of equal size.

[0040] Specifically, the size of the local region is set manually. For example, when the size of the local region is W×H, the logarithmic amplitude spectrum is first divided into a series of local feature regions of size W×H, and each region is reshaped into a local feature vector X of size WH.

[0041] In this solution, the hybrid dynamic attention module is constructed by optimizing the linear attention mechanism based on the Mamba method, and is used to further extract features from the local feature vector X.

[0042] Specifically, as an emerging technology, the linear attention mechanism can accurately extract key features in audio while maintaining linear computational complexity. Compared with the traditional self-attention mechanism, linear attention still shows excellent performance under limited computing resources. This scheme uses the Mamba method to further optimize the linear attention mechanism. By organically combining it with deep neural networks, a lightweight and efficient hybrid dynamic attention module is formed, which greatly improves the overall performance of audio quality evaluation and has both high efficiency and accuracy in evaluating long audio sequences.

[0043] In this scheme, the local feature sequence is sequentially input into the hybrid dynamic attention module, and the hybrid dynamic attention module processes the local feature vectors one by one.

[0044] In this scheme, the structure of the linear unit of the first branch is the same as that of the second branch. The linear unit is composed of a linear layer connected to an activation function. The linear layer first maps the local feature vector to a high-dimensional space to obtain a high-dimensional local feature, and the high-dimensional local feature is then activated to obtain the first branch feature or the second linear feature.

[0045] Specifically, the linear unit in the first branch obtains the first branch feature, and the linear unit in the second branch obtains the second linear feature.

[0046] In some specific embodiments, the activation function used in the linear unit is the SiLU activation function, which is used to enhance the nonlinear features in the high-dimensional local features.

[0047] Specifically, the first branch and the second branch are parallel. After the local feature vector of size WH is respectively input into the linear unit in the parallel first branch and the linear unit in the second branch, the size of the obtained first branch feature and the second linear feature is C×WH, where C is the number of channels.

[0048] In the step of "obtaining a first splitting result and a second splitting result after evenly splitting the second linear feature based on the channel dimension of the second linear feature", the channel dimension of the first splitting result is the same as that of the second splitting result, and the sum of the channel dimensions of the first splitting result and the second splitting result is equal to the channel dimension of the second linear feature.

[0049] Exemplarily, if the size of the second linear feature is C×WH, the size of the first splitting result and the second splitting result obtained by evenly splitting the second linear feature with the channel dimension C is C / 2×WH.

[0050] Specifically, the first splitting result and the second splitting result contain feature information of different parts of the second linear feature, so that the feature information of different parts is processed through subsequent dynamic local convolution structure and dynamic global attention structure.

[0051] In the step of "the dynamic local convolution structure assigns a dynamic weight to the first split result to obtain a dynamic local convolution result", the structural schematic diagram of the dynamic local convolution structure is as follows Figure 3 As shown, in the dynamic local convolution structure, the first split result is subjected to a depth-separable convolution and a Hadamard product with the first split result to obtain a depth-separable convolution feature, the depth-separable convolution feature is sequentially passed through an adaptive pooling layer and a convolution layer to obtain a dynamic weight, a static learnable parameter is set, the dynamic weight is added to the static learnable parameter to obtain a convolution kernel, and then a convolution operation is performed on the first split result based on the convolution kernel to obtain a dynamic local convolution result.

[0052] Specifically, the spatial feature response can be adjusted by performing a depth-separable convolution on the first split result and then performing a Hadamard product with the first split result, and then a dynamic weight W of size C / 2×S×P×P is generated through an adaptive pooling operation and a convolution operation. d , where P×P is the size of the convolution kernel, followed by the dynamic weight W d With a static learnable parameter W of size C / 2×S×P×P l Add along dimension S to obtain a convolution kernel of size C / 2×P×P, and use the convolution kernel as the first split result F 1，1 The convolution kernel of the local modeling is convolved on the first split result to obtain a dynamic local convolution result of size C / 2×WH The formula is as follows:

[0053] F temp =ReLU6(DWConv(F 1，1 ))⊙F 1，1

[0054] W d =Conv(AdaptPool(F temp ), P)

[0055]

[0056] Among them, ReLU6(·) is the activation function, DWConv(·) represents the depthwise separable convolution, ⊙ is the Hadamard product, Conv(·) is the convolution operation, AdaptPool(·, P) represents the adaptive pooling operation, Convolve(·, W) represents the convolution operation with W as the convolution kernel, and F temp Denotes the depth-separable convolution feature, W d represents the dynamic weight, Represents the dynamic local convolution result.

[0057] In the step of "performing dynamic attention calculation on the second split result in the dynamic global attention structure to obtain a dynamic global attention result", the structural schematic diagram of the dynamic global attention structure is as follows: Figure 4 As shown, the second split result is linearly expanded to obtain a query vector, a key vector and a value vector, the query vector is horizontally blocked and vertically blocked to obtain a horizontally aggregated query component and a vertically aggregated query component, the key vector is horizontally blocked and vertically blocked to obtain a horizontally aggregated key component and a vertically aggregated key component, self-attention is calculated based on the horizontally aggregated query component, the horizontally aggregated key component and the value vector to obtain the horizontal axial aggregated attention, self-attention is calculated based on the vertically aggregated query component, the vertically aggregated key component and the value vector to obtain the vertical axial aggregated attention, the query vector and the value vector are Hadamard-producted to obtain supplementary local features, the horizontal axial aggregated attention, the vertical axial aggregated attention and the supplementary local features are feature added to obtain a dynamic global attention result.

[0058] Furthermore, the query vector is divided into multiple blocks along the horizontal axis and the average value between the multiple blocks is calculated to obtain the horizontal aggregate query component, and the key vector is divided into multiple blocks along the horizontal axis and the average value between the multiple blocks is calculated to obtain the horizontal aggregate key component.

[0059] Furthermore, the query vector is divided into multiple blocks along the vertical axis and the average value between the multiple blocks is calculated to obtain the vertical aggregate query component, and the key vector is divided into multiple blocks along the vertical axis and the average value between the multiple blocks is calculated to obtain the vertical aggregate key component.

[0060] Specifically, the second split result is linearly expanded to obtain a query vector Q, a key vector K, and a value vector V of size C / 2×WH, and the query vector Q and the key vector K are axially divided into n blocks in the horizontal and vertical directions respectively, and the average value of the n blocks is calculated to obtain a horizontally aggregated query component Q of size C / 2×Wn. h and the horizontal aggregation key component K h, and a vertical aggregate query component Q of size C / 2×nH w and the vertical aggregation key component K w , then Q h , K h The self-attention calculation with V is used to obtain the horizontal axial aggregation attention of size C / 2×nH, Q w , K w The self-attention calculation is performed with V to obtain the vertical axial aggregation attention of size C / 2×Wn. The Hadamard calculation is performed with Q and V to obtain the supplementary local features to supplement the local inductive bias. Finally, the vertical axial aggregation attention, the horizontal axial aggregation attention and the supplementary local features are added to obtain the dynamic global attention result of size C / 2×WH. The calculation formula is as follows:

[0061] Q h , K h =Mean(Chunk(Q,k),H,n))

[0062] Q w , K w =Mean((Chunk(Q,k),W,n))

[0063]

[0064] Among them, Mean(·) means finding the average, Chunk(·, H, n) means dividing into n parts along dimension H, and SelfAtt(·) means self-attention calculation.

[0065] In the step of “joining the dynamic local convolution result with the dynamic global attention result to obtain a second branch feature”, the dynamic local convolution result with the dynamic global attention result are joined along the channel dimension to obtain a second branch feature.

[0066] Specifically, due to the dynamic local convolution results The dynamic global attention results The number of channels is the same, so the dynamic local convolution result is concatenated The dynamic global attention results Splicing along the channel dimension to obtain the second branch feature of size C×WH

[0067] In the step of "taking a Hadamard product of the first branch feature and the second branch feature and inputting them into the linear branch to obtain a mixed attention result", the first branch feature and the second branch feature are taken into account and then a pixel-by-pixel product is performed in the linear branch, and then mapped through an activation function to obtain a mixed attention result.

[0068] Specifically, the activation function used is the SiLU activation function, and the size of the mixed attention result is C×WH.

[0069] In this scheme, the mixed attention result is input into the classification module for classification to obtain the audio quality evaluation result of the current local feature vector, the predicted audio quality evaluation result and the actual audio quality evaluation result are used to construct the loss function, and the parameters of the audio quality evaluation framework are adjusted based on the loss function to obtain the constructed audio quality evaluation model.

[0070] Embodiment 2

[0071] A hybrid dynamic audio quality evaluation method based on a Mamba structure, comprising:

[0072] The long audio data to be evaluated is obtained, and the long audio data to be evaluated is input into the audio quality evaluation model constructed in the first embodiment to obtain an audio quality evaluation result.

[0073] Embodiment 3

[0074] Based on the same idea, refer to Figure 5 The present application also proposes a construction device for a hybrid dynamic audio quality evaluation model based on a Mamba structure, comprising:

[0075] A construction module is used to obtain multiple groups of long audio data as training samples and construct an audio quality evaluation framework, wherein the audio quality evaluation framework consists of a spectrum conversion module, a hybrid dynamic attention module, and a classification module;

[0076] A spectrum conversion module, used for converting the long audio data into a logarithmic amplitude spectrum, and then dividing the logarithmic amplitude spectrum into a plurality of local feature vectors of equal size to form a local feature sequence;

[0077] A hybrid dynamic attention module, used for sequentially processing each local feature vector in a local feature sequence, the dynamic attention module includes a first branch, a second branch and a linear branch, the local feature vector is input into the first branch and the second branch respectively, the local feature vector is linearly mapped by a linear unit in the first branch to obtain a first branch feature, the second branch includes a linear unit, a dynamic local convolution structure and a dynamic global attention structure, the local feature vector is linearly mapped by a linear unit in the second branch to obtain a second linear feature, the second linear feature is averagely split based on the channel dimension of the second linear feature to obtain a first split result and a second split result, the dynamic local convolution structure assigns a dynamic weight to the first split result to obtain a dynamic local convolution result, the second split result is dynamically calculated in the dynamic global attention structure to obtain a dynamic global attention result, the dynamic local convolution result is spliced ​​with the dynamic global attention result to obtain a second branch feature, the first branch feature and the second branch feature are Hadamard-producted and input into the linear branch to obtain a hybrid attention result;

[0078] A classification module is used to input the mixed attention result into the classification module to obtain an audio quality evaluation result, construct a loss function based on the audio quality evaluation result, and adjust the parameters of the audio quality evaluation framework based on the loss function to obtain an audio quality evaluation model.

[0079] Embodiment 3

[0080] This embodiment also provides an electronic device, referring to Figure 6 , comprises a memory 404 and a processor 402, wherein the memory 404 stores a computer program, and the processor 402 is configured to run the computer program to execute the steps in any of the above method embodiments.

[0081] Specifically, the processor 402 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0082] Among them, the memory 404 may include a large capacity memory 404 for data or instructions. By way of example and not limitation, the memory 404 may include a hard disk drive (HDD), a floppy disk drive, a solid state drive (SSD), a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 404 may include a removable or non-removable (or fixed) medium. Where appropriate, the memory 404 may be inside or outside the data processing device. In a specific embodiment, the memory 404 is a non-volatile memory. In a specific embodiment, the memory 404 includes a read-only memory (ROM) and a random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (Programmable Read-Only Memory, PROM for short), an erasable PROM (Erasable Programmable Read-Only Memory, EPROM for short), an electrically erasable PROM (Electrically Erasable Programmable Read-Only Memory, EEPROM for short), an electrically alterable ROM (Electrically Alterable Read-Only Memory, EAROM for short) or a flash memory (FLASH) or a combination of two or more of these. Under appropriate circumstances, the RAM can be a static random access memory (Static Random-Access Memory, abbreviated as SRAM) or a dynamic random access memory (Dynamic Random Access Memory, abbreviated as DRAM), wherein the DRAM can be a fast page mode dynamic random access memory 404 (Fast Page Mode Dynamic Random Access Memory, abbreviated as FPMDRAM), an extended data output dynamic random access memory (Extended Date Out Dynamic Random Access Memory, abbreviated as EDODRAM), a synchronous dynamic random access memory (Synchronous Dynamic Random-Access Memory, abbreviated as SDRAM), etc.

[0083] The memory 404 may be used to store or cache various data files required for processing and / or communication, as well as possible computer program instructions executed by the processor 402 .

[0084] The processor 402 reads and executes the computer program instructions stored in the memory 404 to implement any one of the methods for constructing a hybrid dynamic audio quality evaluation model based on the Mamba structure in the above embodiments.

[0085] Optionally, the electronic device may further include a transmission device 406 and an input / output device 408 , wherein the transmission device 406 is connected to the processor 402 , and the input / output device 408 is connected to the processor 402 .

[0086] The transmission device 406 can be used to receive or send data via a network. The specific examples of the above-mentioned network may include a wired or wireless network provided by a communication provider of the electronic device. In one example, the transmission device includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 406 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0087] The input / output device 408 is used to input or output information. In this embodiment, the input information may be long audio data, etc., and the output information may be audio quality evaluation results, etc.

[0088] Optionally, in this embodiment, the processor 402 may be configured to perform the following steps through a computer program:

[0089] Acquire multiple groups of long audio data as training samples and construct an audio quality evaluation framework, wherein the audio quality evaluation framework consists of a spectrum conversion module, a hybrid dynamic attention module, and a classification module;

[0090] The spectrum conversion module converts the long audio data into a logarithmic amplitude spectrum, and then divides the logarithmic amplitude spectrum into a plurality of local feature vectors of equal size to form a local feature sequence;

[0091] The hybrid dynamic attention module processes each local feature vector in the local feature sequence sequentially. The dynamic attention module includes a first branch, a second branch and a linear branch. The local feature vector is input into the first branch and the second branch respectively. The local feature vector is linearly mapped by a linear unit in the first branch to obtain a first branch feature. The second branch includes a linear unit, a dynamic local convolution structure and a dynamic global attention structure. The local feature vector is linearly mapped by a linear unit in the second branch to obtain a second linear feature. The second linear feature is averagely split based on the channel dimension of the second linear feature to obtain a first split result and a second split result. The dynamic local convolution structure assigns a dynamic weight to the first split result to obtain a dynamic local convolution result. The second split result is dynamically calculated in the dynamic global attention structure to obtain a dynamic global attention result. The dynamic local convolution result and the dynamic global attention result are spliced ​​to obtain a second branch feature. The first branch feature and the second branch feature are Hadamard-producted and input into the linear branch to obtain a hybrid attention result.

[0092] The mixed attention result is input into the classification module to obtain an audio quality evaluation result, a loss function is constructed based on the audio quality evaluation result, and the parameters of the audio quality evaluation framework are adjusted based on the loss function to obtain an audio quality evaluation model.

[0093] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0094] In general, various embodiments may be implemented in hardware or dedicated circuits, software, logic, or any combination thereof. Some aspects of the invention may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device, but the invention is not limited thereto. Although various aspects of the invention may be shown and described as block diagrams, flow charts, or using some other graphical representation, it should be understood that, as non-limiting examples, the boxes, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuits or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.

[0095] Embodiments of the present invention may be implemented by computer software that is executable by a data processor of a mobile device, such as in a processor entity, or by hardware, or by a combination of software and hardware. Computer software or programs (also referred to as program products) including software routines, applets and / or macros may be stored in any device-readable data storage medium, and they include program instructions for performing specific tasks. A computer program product may include one or more computer executable components configured to perform an embodiment when the program is run. One or more computer executable components may be at least one software code or a portion thereof. In addition, at this point, it should be noted that, for example, Figure 6 Any block of the logic flow in the program may represent program steps, or interconnected logic circuits, blocks and functions, or a combination of program steps and logic circuits, blocks and functions. The software may be stored on physical media such as memory chips or storage blocks implemented within a processor, magnetic media such as hard disks or floppy disks, and optical media such as, for example, DVDs and data variants thereof, CDs, etc. Physical media are non-transitory media.

[0096] Those skilled in the art should understand that the technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0097] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for constructing a hybrid dynamic audio quality evaluation model based on a Mamba structure, characterized in that: The following steps are involved: Acquire multiple groups of long audio data as training samples and construct an audio quality evaluation framework, wherein the audio quality evaluation framework consists of a spectrum conversion module, a hybrid dynamic attention module, and a classification module; The spectrum conversion module converts the long audio data into a logarithmic amplitude spectrum, and then divides the logarithmic amplitude spectrum into a plurality of local feature vectors of equal size to form a local feature sequence; The hybrid dynamic attention module processes each local feature vector in the local feature sequence sequentially. The dynamic attention module includes a first branch, a second branch and a linear branch. The local feature vector is input into the first branch and the second branch respectively. The local feature vector is linearly mapped by a linear unit in the first branch to obtain a first branch feature. The second branch includes a linear unit, a dynamic local convolution structure and a dynamic global attention structure. The local feature vector is linearly mapped by a linear unit in the second branch to obtain a second linear feature. The second linear feature is averagely split based on the channel dimension of the second linear feature to obtain a first split result and a second split result. The dynamic local convolution structure assigns a dynamic weight to the first split result to obtain a dynamic local convolution result. The second split result is dynamically calculated in the dynamic global attention structure to obtain a dynamic global attention result. The dynamic local convolution result and the dynamic global attention result are spliced ​​to obtain a second branch feature. The first branch feature and the second branch feature are Hadamard-producted and input into the linear branch to obtain a hybrid attention result. The mixed attention result is input into the classification module to obtain an audio quality evaluation result, a loss function is constructed based on the audio quality evaluation result, and the parameters of the audio quality evaluation framework are adjusted based on the loss function to obtain an audio quality evaluation model.

2. The method for constructing a hybrid dynamic audio quality evaluation model based on a Mamba structure according to claim 1, characterized in that: In the step of "dividing the logarithmic amplitude spectrum into a plurality of local feature vectors of equal size to form a local feature sequence", the logarithmic amplitude spectrum is first divided into a plurality of local feature regions of equal size, and then each local feature region is reshaped into a local feature vector of equal size.

3. The method for constructing a hybrid dynamic audio quality evaluation model based on a Mamba structure according to claim 1, characterized in that: In the step of "obtaining a first splitting result and a second splitting result after evenly splitting the second linear feature based on the channel dimension of the second linear feature", the channel dimension of the first splitting result is the same as that of the second splitting result, and the sum of the channel dimensions of the first splitting result and the second splitting result is equal to the channel dimension of the second linear feature.

4. The method for constructing a hybrid dynamic audio quality evaluation model based on a Mamba structure according to claim 1, characterized in that: In the step of "the dynamic local convolution structure assigns dynamic weights to the first splitting result to obtain a dynamic local convolution result", the first splitting result is subjected to a depth-separable convolution in the dynamic local convolution structure, and then subjected to a Hadamard product with the first splitting result to obtain a depth-separable convolution feature, the depth-separable convolution feature is sequentially passed through an adaptive pooling layer and a convolution layer to obtain a dynamic weight, a static learnable parameter is set, the dynamic weight is added to the static learnable parameter to obtain a convolution kernel, and then a convolution operation is performed on the first splitting result based on the convolution kernel to obtain a dynamic local convolution result.

5. The method for constructing a hybrid dynamic audio quality evaluation model based on a Mamba structure according to claim 1, characterized in that: In the step of "performing dynamic attention calculation on the second split result in the dynamic global attention structure to obtain a dynamic global attention result", the second split result is linearly expanded to obtain a query vector, a key vector and a value vector, the query vector is horizontally blocked and vertically blocked to obtain a horizontally aggregated query component and a vertically aggregated query component, the key vector is horizontally blocked and vertically blocked to obtain a horizontally aggregated key component and a vertically aggregated key component, self-attention calculation is performed based on the horizontally aggregated query component, the horizontally aggregated key component and the value vector to obtain a horizontal axially aggregated attention, self-attention calculation is performed based on the vertically aggregated query component, the vertically aggregated key component and the value vector to obtain a vertical axially aggregated attention, the query vector and the value vector are Hadamard-producted to obtain a supplementary local feature, and the horizontal axially aggregated attention, the vertical axially aggregated attention and the supplementary local feature are feature added to obtain a dynamic global attention result.

6. The method for constructing a hybrid dynamic audio quality evaluation model based on a Mamba structure according to claim 5, characterized in that: The query vector is divided into multiple blocks along the horizontal axis and the average value between the multiple blocks is calculated to obtain a horizontal aggregate query component. The key vector is divided into multiple blocks along the horizontal axis and the average value between the multiple blocks is calculated to obtain a horizontal aggregate key component. The query vector is divided into multiple blocks along the vertical axis and the average value between the multiple blocks is calculated to obtain a vertical aggregate query component. The key vector is divided into multiple blocks along the vertical axis and the average value between the multiple blocks is calculated to obtain a vertical aggregate key component.

7. A hybrid dynamic audio quality evaluation method based on Mamba structure, characterized in that: include: Obtain long audio data to be evaluated, and input the long audio data to be evaluated into the audio quality evaluation model constructed in claim 1 to obtain an audio quality evaluation result.

8. A device for constructing a hybrid dynamic audio quality evaluation model based on a Mamba structure, characterized in that: include: A construction module is used to obtain multiple groups of long audio data as training samples and construct an audio quality evaluation framework, wherein the audio quality evaluation framework consists of a spectrum conversion module, a hybrid dynamic attention module, and a classification module; A spectrum conversion module, used for converting the long audio data into a logarithmic amplitude spectrum, and then dividing the logarithmic amplitude spectrum into a plurality of local feature vectors of equal size to form a local feature sequence; A hybrid dynamic attention module, used for sequentially processing each local feature vector in a local feature sequence, the dynamic attention module includes a first branch, a second branch and a linear branch, the local feature vector is input into the first branch and the second branch respectively, the local feature vector is linearly mapped by a linear unit in the first branch to obtain a first branch feature, the second branch includes a linear unit, a dynamic local convolution structure and a dynamic global attention structure, the local feature vector is linearly mapped by a linear unit in the second branch to obtain a second linear feature, the second linear feature is averagely split based on the channel dimension of the second linear feature to obtain a first split result and a second split result, the dynamic local convolution structure assigns a dynamic weight to the first split result to obtain a dynamic local convolution result, the second split result is dynamically calculated in the dynamic global attention structure to obtain a dynamic global attention result, the dynamic local convolution result is spliced ​​with the dynamic global attention result to obtain a second branch feature, the first branch feature and the second branch feature are Hadamard-producted and input into the linear branch to obtain a hybrid attention result; A classification module is used to input the mixed attention result into the classification module to obtain an audio quality evaluation result, construct a loss function based on the audio quality evaluation result, and adjust the parameters of the audio quality evaluation framework based on the loss function to obtain an audio quality evaluation model.

9. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to execute a method for constructing a hybrid dynamic audio quality evaluation model based on a Mamba structure as described in any one of claims 1 to 6 or a hybrid dynamic audio quality evaluation method based on a Mamba structure as described in claim 7.

10. A readable storage medium, characterized in that: A computer program is stored in the readable storage medium, and the computer program includes a program code for controlling a process to execute a process, and the process includes a method for constructing a hybrid dynamic audio quality evaluation model based on a Mamba structure according to any one of claims 1-6 or a hybrid dynamic audio quality evaluation method based on a Mamba structure according to claim 7.

Images