A deep synthetic speech detection method based on vocoder trace fingerprint comparison

Abstract

This invention belongs to the field of audio detection technology, and specifically relates to a deep synthesized speech detection method based on vocoder trace fingerprint comparison. The deep synthesized speech detection method includes the following steps: acquiring the speech to be detected; inputting the speech to be detected into a pre-trained vocoder to obtain synthesized audio; extracting waveform features from the speech to be detected and the synthesized audio; and calculating the authenticity of the speech to be detected based on the waveform features of the speech to be detected and the waveform features of the synthesized audio. The deep synthesized speech detection method provided by this invention, from the perspective of synthesized audio, targets the vocoder, a crucial module in deep synthesis, and performs synthesized audio forgery detection based on vocoder trace fingerprints. It has strong interpretability, and simultaneously, by extracting neural network features from the waveforms of the original audio and the synthesized audio, it can accurately capture the deep synthesis traces of the target vocoder, achieving high accuracy.

Description

Technical Field

[0001] This invention relates to the field of audio detection technology, and in particular to a deep synthesized speech detection method based on vocoder trace fingerprint comparison. Background Technology

[0002] Speech is one of the important forms of modern communication and information exchange. With the continuous development of digital speech technology, deep speech synthesis technology has become one of the key technologies for realizing the automation of speech information and is widely used in speech conversion, deep speech synthesis, music generation, and other fields. However, with the widespread application of deep speech synthesis technology, some malicious actors are using it for online fraud, voice impersonation, and other activities, seriously threatening network security and social stability. Therefore, how to accurately and efficiently detect deep synthesized speech has important practical significance and application value. Traditional synthetic audio detection lacks theoretical research on the differences between real voice and deep synthesized voice, and cannot clearly explain the differences between the two at the feature and signal levels. Therefore, its interpretability and robustness are poor. Summary of the Invention

[0003] In view of this, embodiments of the present invention provide a deep synthesized speech detection method based on vocoder trace fingerprint comparison, in order to solve the technical problems of poor interpretability and robustness of traditional synthesized audio detection methods.

[0004] In a first aspect, embodiments of the present invention provide a deep synthesized speech detection method based on vocoder trace fingerprint comparison, comprising the following steps: acquiring speech to be detected; inputting the speech to be detected into a pre-trained vocoder to obtain synthesized audio; extracting waveform features of the speech to be detected and the synthesized audio; and calculating the authenticity result of the speech to be detected based on the waveform features of the speech to be detected and the waveform features of the synthesized audio.

[0005] Preferably, after acquiring the speech to be detected, the method further includes the following steps: preprocessing the speech to be detected, the preprocessing including the following methods: digitization of the speech to be detected, endpoint detection, pre-emphasis, windowing and framing.

[0006] Preferably, calculating the authenticity result of the speech to be detected based on the waveform features of the speech to be detected and the waveform features of the synthesized audio includes the following steps: calculating the similarity between the waveform features of the speech to be detected and the waveform features of the synthesized audio; if the similarity is higher than a preset threshold, the output result is true; if the similarity is lower than the preset threshold, the output result is false.

[0007] Preferably, training the vocoder includes the following steps: acquiring an audio dataset for training the vocoder; preprocessing the audio data; selecting a recurrent neural network (RNN) model, long short-term memory (LSTM), or gated recurrent unit (GRU) to extract features from the dataset to obtain audio features; constructing a vocoder model; inputting the preprocessed audio dataset and the audio features into the vocoder model for training; and validating the trained vocoder model using a preset validation dataset. If the preset requirements are met, a trained vocoder is obtained.

[0008] Preferably, when extracting waveform features of the speech to be detected and the synthesized audio, the Rawnet structure is used as the network infrastructure.

[0009] Preferably, calculating the authenticity result of the speech to be detected based on the waveform features of the speech to be detected and the waveform features of the synthesized audio includes the following steps: calculating the distance between the waveform features of the speech to be detected and the waveform features of the synthesized audio; if the distance is greater than a preset distance threshold, the output result is false; if the distance is less than the preset distance threshold, the output result is true; the distance between the waveform features is one of Euclidean distance, Manhattan distance or Minkowski distance.

[0010] Secondly, embodiments of the present invention provide a detection device for deep synthesized speech, comprising:

[0011] Audio acquisition module: used to acquire the speech to be detected;

[0012] Analysis of the deep synthesis module: used to generate synthesized audio based on the speech to be detected;

[0013] The comparison network module is used to extract waveform features of the speech to be detected and the synthesized audio.

[0014] Classifier module: used to calculate the authenticity of the speech to be detected based on the waveform features of the speech to be detected and the waveform features of the synthesized audio.

[0015] Preferably, the classifier module employs one or more of the following network structures: convolutional neural network, recurrent neural network, attention mechanism, etc.

[0016] Thirdly, embodiments of the present invention provide a readable storage medium having computer program instructions stored thereon, wherein the computer program instructions, when executed, implement the steps of the method described above.

[0017] Fourthly, embodiments of the present invention provide a program product including computer program instructions, which, when executed, implement the steps of the method described above.

[0018] Compared with existing technologies, the deep synthesized speech detection method based on vocoder trace fingerprint comparison provided by this invention has the following beneficial effects: The deep synthesized speech detection method based on vocoder trace fingerprint comparison provided by this invention includes the following steps: acquiring the speech to be detected; inputting the speech to be detected into a pre-trained vocoder to obtain synthesized audio; extracting waveform features of the speech to be detected and the synthesized audio; calculating the authenticity result of the speech to be detected based on the waveform features of the speech to be detected and the waveform features of the synthesized audio. Traditional synthesized audio detection lacks theoretical research on the differences between real sound and deep synthesized sound, and cannot clearly explain the differences between the two from the feature and signal levels. Therefore, its interpretability and robustness are poor. Understandably, compared with traditional methods, this invention has the following advantages: strong interpretability; this method focuses on synthesized audio from the perspective of synthesized audio, targeting the vocoder, a very important module in deep synthesis, to perform synthesized audio forgery detection based on vocoder trace fingerprints. high accuracy; this method uses deep learning methods, and by comparing neural networks, it simultaneously extracts neural network features from the waveforms of the original audio and the synthesized audio, which can accurately capture the deep synthesis traces of the target vocoder, resulting in high accuracy. High robustness: This method is based on vocoder traces, so the more types of vocoders used during training, the stronger the robustness of this method. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments of the present invention will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort, and these are all within the protection scope of the present invention.

[0020] Figure 1 A flowchart illustrating the deep synthesized speech detection method provided in the first embodiment of the present invention. Figure 1 .

[0021] Figure 2 A flowchart illustrating the deep synthesized speech detection method provided in the first embodiment of the present invention. Figure 2 .

[0022] Figure 3 This is a schematic diagram of the structure of the deep synthesized speech detection device provided in the second embodiment of the present invention.

[0023] Figure 4 This is a schematic diagram of the structure of the program product provided in the fourth embodiment of the present invention.

[0024] Figure 5 This is a schematic diagram of the structure of an electronic device provided in the fifth embodiment of the present invention.

[0025] Explanation of icon numbers:

[0026] 1. Deep synthesized speech detection method; 2. Deep synthesized speech detection device; 3. Program product; 4. Electronic device;

[0027] 20. Audio acquisition module; 21. Analysis and synthesis module; 22. Comparison network module; 23. Classifier module; 30. Computer program instructions; 40. Processor; 41. Memory; 42. Bus; 43. Communication interface. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0029] Please combine Figure 1 and Figure 2 The first embodiment of the present invention provides a deep synthesized speech detection method 1 based on vocoder trace fingerprint comparison, used to detect deep synthesized speech, including the following steps:

[0030] Acquire the speech to be detected;

[0031] The speech to be detected is input into a pre-trained vocoder to obtain synthesized audio;

[0032] Extract waveform features from the detected speech and synthesized audio;

[0033] The authenticity of the detected speech is calculated based on the waveform features of the speech and the waveform features of the synthesized audio.

[0034] Understandably, current synthetic audio detection methods rely on acoustic features and modifications to the model structure. From a deep synthesis perspective, the vocoder is a crucial component. Most of the best-performing synthetic audio models are based on an acoustic model coupled with a vocoder framework. Therefore, any forged audio constructed using a vocoder will retain vocoder traces. Detecting synthetic audio by focusing on this aspect offers greater interpretability and reliability. However, traditional detection methods haven't addressed this aspect. Therefore, this invention analyzes the detected audio through deep synthesis steps to generate a corresponding vocoder-reconstructed audio. By comparing the differences between the original and reconstructed audio—the traces left by the vocoder—it determines whether the detected audio is synthetic. Building a method based on this principle results in strong interpretability. Regarding robustness, since there are multiple types of vocoders, the model can be expanded to analyze the types of vocoders used in deep synthesis, enhancing the model's generalization and robustness.

[0035] Furthermore, after acquiring the speech to be detected, the following steps are also included: preprocessing the speech to be detected, which includes the following methods: digitization of the speech to be detected, endpoint detection, pre-emphasis, windowing, and framing. Understandably, the purpose of preprocessing the speech to be detected is to:

[0036] Digitalization: The process of converting analog speech signals into digital signals, through sampling and quantization, transforming continuous analog signals into discrete digital samples. Digitalization enables speech signals to be processed and stored in computers.

[0037] Endpoint detection: Endpoint detection is used to determine the start and end points of a speech signal. By identifying active and inactive segments in the speech signal, endpoint detection helps determine the valid portions of the speech signal for subsequent processing and analysis.

[0038] Pre-emphasis: Pre-emphasis is a filtering operation designed to highlight the energy of the high-frequency components of a speech signal in order to improve the signal-to-noise ratio. Pre-emphasis can be achieved by processing the speech signal with a filter (usually a first-order high-pass filter) to reduce the amplitude of the low-frequency components and enhance the amplitude of the high-frequency components.

[0039] Windowing: Windowing is used to reduce the impact of signal boundaries and extract fixed-length speech frames. Typically, window functions (such as Hamming windows, rectangular windows, etc.) are used to segment the speech signal, and windowing is applied to each speech frame to smooth signal boundaries and reduce spectral leakage.

[0040] Framing: Dividing a speech signal into short time segments, each segment is called a frame. By framing the speech signal, the continuous signal in the time domain can be converted into a discrete signal in the frequency domain, so that the spectral characteristics of the speech signal remain basically stable in each frame, which facilitates spectral analysis and feature extraction.

[0041] The purpose of these steps is to preprocess the speech signal so that subsequent tasks such as feature extraction, speech recognition, and deep speech synthesis can better process the speech signal and extract useful information.

[0042] In some embodiments, calculating the authenticity of the detected speech based on the waveform features of the detected speech and the waveform features of the synthesized audio includes the following steps:

[0043] The similarity between the waveform features of the speech to be detected and the waveform features of the synthesized audio is calculated. If the similarity is higher than a preset threshold, the output result is true; if the similarity is lower than the preset threshold, the output result is false.

[0044] In some embodiments, training a vocoder includes the following steps:

[0045] Data Collection: Acquire audio datasets for training the vocoder. This data can include real human speech data, deep speech synthesis data, or other suitable audio samples. Ensure the diversity and representativeness of the dataset so that the vocoder can learn audio features of different types and styles.

[0046] Data preprocessing: The audio data is preprocessed for training purposes. This may include audio trimming, sample rate adjustment, audio normalization, and noise removal. Ensure the format and features of the audio data match the input requirements of the deep learning model.

[0047] Model Selection: Recurrent Neural Network (RNN), Long Short-Term Memory (LSTM), or Gated Recurrent Unit (GRU) models are selected to extract features from the dataset, obtaining audio features. These models can model the temporal dependencies of audio sequences. Commonly used audio features include Mel-frequency cepstral coefficients (MFCC) and Mel-frequency cepstral frequencies (MFFC). These features can capture the spectral information and temporal characteristics of audio.

[0048] Model Building: Construct the vocoder model, including defining its architecture and hierarchical structure. This may involve stacking multiple RNN layers, adding convolutional layers, or fully connected layers. Ensure the number and complexity of the model's parameters are appropriate to avoid overfitting or underfitting.

[0049] Model Training: The vocoder model is trained using the preprocessed audio dataset and audio features. During training, a loss function (such as mean squared error or cross-entropy) and an optimization algorithm (such as stochastic gradient descent) need to be defined to minimize the gap between the model's predictions and the actual target.

[0050] Model Evaluation: The trained vocoder model is evaluated and validated using a pre-defined validation dataset. If it meets the pre-defined requirements, the trained vocoder is obtained. Model performance can be evaluated by calculating accuracy, loss function values, or other metrics.

[0051] Model tuning: Based on the evaluation results, the model can be tuned by trying to adjust the model architecture, hyperparameters, or data preprocessing methods to improve the model's performance.

[0052] In some embodiments, the Rawnet architecture is used as the network infrastructure when extracting waveform features of the speech to be detected and the synthesized audio.

[0053] Understandably, RawNet is a neural network architecture for audio processing, primarily used for speech recognition tasks. Its design goal is to learn feature representations directly from raw audio data without the need for traditional acoustic feature extraction processes (such as MFCC).

[0054] Specifically, the Rawnet architecture, as a network infrastructure, has the following applications:

[0055] Input representation: The input to RawNet is the raw audio waveform, i.e., audio data without any preprocessing. The audio waveform is a one-dimensional time-series signal.

[0056] 1D Convolutional Layers: RawNet uses 1D convolutional layers to extract features from audio waveforms. These 1D convolutional layers perform convolution operations in the time dimension to capture local features in the audio waveform. These convolutional kernels can learn short-term speech features in the audio.

[0057] Activation functions: After a 1D convolutional layer, activation functions such as ReLU can be used to introduce nonlinear transformations.

[0058] Pooling layers: RawNet typically uses pooling layers after 1D convolutional layers to reduce the dimensionality of features. Pooling operations can extract the most salient features from audio waveforms.

[0059] Stacked convolutional and pooling layers: RawNet can stack multiple 1D convolutional and pooling layers to gradually extract higher-level audio features.

[0060] Global average pooling layer: In the last layer of RawNet, a global average pooling layer is usually used to pool the features from the entire time series to obtain a fixed-length feature vector.

[0061] Output layer: Depending on the specific task requirements, the output layer of the comparison network module can be one or more neurons. For speech recognition tasks, the output layer is usually a softmax layer used to classify different audio categories.

[0062] RawNet learns richer audio features by directly processing raw audio data, avoiding information loss during traditional feature extraction. This end-to-end approach makes the model more flexible and adaptable to different types of audio tasks. Furthermore, RawNet's structure can be flexibly adjusted and extended according to specific needs.

[0063] In some embodiments, calculating the authenticity of the detected speech based on the waveform features of the detected speech and the waveform features of the synthesized audio includes the following steps:

[0064] Calculate the distance between the waveform features of the speech to be detected and the waveform features of the synthesized audio. If the distance is greater than a preset distance threshold, the output result is false; if the distance is less than the preset distance threshold, the output result is true.

[0065] The distance between waveform features is one of Euclidean distance, Manhattan distance, or Minkowski distance.

[0066] In summary, during the training phase, a specific vocoder is used to analyze and deeply synthesize the dataset, resulting in analyzed and synthesized audio corresponding to each training audio. The original audio and the analyzed and synthesized audio are fed into two neural networks, which are alignment networks with identical structures. The networks output the corresponding audio features and calculate the distance, then calculate the loss based on the distance. Each audio in the training set contains labels for both real and synthesized audio. If the label is real speech, the aim is to increase the alignment distance between the features extracted from the original audio and the analyzed and synthesized audio; conversely, if the label is deeply synthesized speech, the aim is to decrease the alignment distance between the features extracted from the original audio and the analyzed and synthesized audio. During the detection phase, the input detection audio is analyzed and deep synthesized to obtain the corresponding audio. The two audios are input into the alignment network, features are extracted, and the distance is calculated. Whether the result value is greater than a set threshold determines whether it is synthesized audio.

[0067] Please see Figure 3 The second embodiment of the present invention also provides a deep synthesized speech detection device 2 for implementing the above-described deep synthesized speech detection method 1. The deep synthesized speech detection device includes:

[0068] Audio acquisition module 20: used to acquire the speech to be detected;

[0069] Analysis and synthesis module 21: used to generate synthesized audio based on the speech to be detected;

[0070] Comparison network module 22: used to extract waveform features of the speech to be detected and the synthesized audio;

[0071] Classifier module 23: used to calculate the authenticity of the speech to be detected based on the waveform features of the speech to be detected and the waveform features of the synthesized audio.

[0072] Preferably, the classifier module employs one or more of the following network structures: convolutional neural network, recurrent neural network, attention mechanism, etc.

[0073] It should be noted that although several modules or units for executing the process have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0074] Furthermore, although the steps of the method in this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or a step may be broken down into multiple steps.

[0075] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, mobile terminal, or network device, etc.) to execute the method according to the embodiments of this disclosure.

[0076] The third embodiment of the present invention also provides a readable storage medium having computer program instructions stored thereon, which, when executed, implement the steps of the method described above.

[0077] In some possible embodiments, various aspects of this disclosure may also be implemented as a program product comprising program code that, when the program product is run on a terminal device, causes the terminal device to perform the steps described in the "Detection Method of Deep Synthetic Speech" section of this specification according to various exemplary embodiments of this disclosure.

[0078] Please see Figure 4 The fourth embodiment of the present invention also provides a program product 3, which includes computer program instructions 30, which, when executed, implement the steps of the method described above.

[0079] Please see Figure 5 The fifth embodiment of the present invention also provides an electronic device 4, specifically, the electronic device 4 includes a processor 40 and a memory 41; the memory 41 stores a computer program, and the computer program executes the method of any of the above embodiments when run by the processor.

[0080] Furthermore, the processor 40, the communication interface 43, and the memory 41 are connected via a bus 42; the processor 40 is used to execute executable modules, such as computer programs, stored in the memory 41.

[0081] The memory 41 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this system network element and at least one other network element is achieved through at least one communication interface 43 (which can be wired or wireless), such as the Internet, wide area network, local area network, metropolitan area network, etc.

[0082] Bus 42 can be an ISA bus, PCI bus, or EISA bus, etc. Buses can be divided into address buses, data buses, control buses, etc. For ease of representation, Figure 4 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0083] The memory 41 is used to store the program. After receiving the execution instruction, the processor 40 executes the program. The method executed by the device for the flow process definition disclosed in any of the foregoing embodiments of the present invention can be applied to the processor 40 or implemented by the processor 40.

[0084] Processor 40 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of processor 40 or by instructions in software form. Processor 40 can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this invention can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory 41, and the processor 40 reads the information from memory 41 and, in conjunction with its hardware, completes the steps of the above method.

[0085] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, electronic device, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0086] It should be noted that similar reference numerals and letters in the accompanying drawings indicate similar items. Therefore, once an item is defined in one accompanying drawing, it does not need to be further defined and explained in subsequent accompanying drawings. In addition, the terms "first," "second," "third," etc. are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0087] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this invention, and not to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A deep synthetic speech detection method, characterized in that: Includes the following steps: Acquire the speech to be detected; The speech to be detected is input into a pre-trained vocoder to obtain synthesized audio; Extract waveform features from the speech to be detected and the synthesized audio; The method for calculating the authenticity of the speech to be detected based on the waveform features of the speech to be detected and the waveform features of the synthesized audio includes the following steps: calculating the distance between the waveform features of the speech to be detected and the waveform features of the synthesized audio; if the distance is greater than a preset distance threshold, the output result is false; if the distance is less than the preset distance threshold, the output result is true; the distance between the waveform features is one of Euclidean distance, Manhattan distance or Minkowski distance.

2. The deep synthesized speech detection method as described in claim 1, characterized in that: After acquiring the speech to be detected, the process further includes the following steps: preprocessing the speech to be detected, the preprocessing including the following methods: digitization of the speech to be detected, endpoint detection, pre-emphasis, windowing and framing.

3. The deep synthesized speech detection method as described in claim 1, characterized in that: Training the vocoder includes the following steps: Obtain an audio dataset for training a vocoder, wherein each audio in the audio dataset contains labels for real audio and synthetic audio. If the label is real speech, the intention is to increase the comparison distance between the original audio and the features extracted from the analyzed synthetic audio; conversely, if the label is deep synthetic speech, the intention is to decrease the comparison distance between the original audio and the features extracted from the analyzed synthetic audio. Preprocess the audio data; Select a recurrent neural network (RNN) model, a long short-term memory (LSTM) model, or a gated recurrent unit (GRU) model to extract features from the dataset to obtain audio features; Construct a vocoder model; The preprocessed audio dataset and the audio features are fed into the vocoder model for training; The trained vocoder model is evaluated and validated using a pre-defined validation dataset. If the pre-defined requirements are met, the trained vocoder is obtained.

4. The deep synthesized speech detection method as described in claim 3, characterized in that: When extracting waveform features of the speech to be detected and the synthesized audio, the Rawnet structure is used as the network infrastructure.

5. A deep synthesized speech detection device, used to implement the deep synthesized speech detection method as described in any one of claims 1-4, characterized in that: include: Audio acquisition module: used to acquire the speech to be detected; Analysis of the deep synthesis module: used to generate synthesized audio based on the speech to be detected; The comparison network module is used to extract waveform features of the speech to be detected and the synthesized audio. Classifier module: used to calculate the authenticity of the speech to be detected based on the waveform features of the speech to be detected and the waveform features of the synthesized audio.

6. The deep synthesized speech detection device as described in claim 5, characterized in that: The classifier module employs one or more combinations of network structures such as convolutional neural networks, recurrent neural networks, and attention mechanisms.

7. A readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed, they implement the steps of the method as described in any one of claims 1-4.

8. A program product comprising computer program instructions, characterized in that: When the computer program instructions are executed, they implement the steps of the method as described in any one of claims 1-4.

Images