Audio depth forgery detection method and device, storage medium and computer equipment

Abstract

The invention discloses an audio deep counterfeiting detection method and device, a storage medium and computer equipment. The method comprises the following steps: acquiring a real voice sample, and performing self-supervision training on a pre-trained first coding network and a pre-trained second coding network based on the real voice sample to obtain a style encoder and a language encoder; performing feature extraction on the real voice sample and the deep counterfeit sample through a style encoder and a language encoder to obtain voice features of the real voice sample and voice features of the deep counterfeit sample; performing supervised training on a preset classification model based on the voice features of the real voice sample, the voice features of the deep counterfeit voice sample and a preset sample label to obtain a deep counterfeit detection model; and obtaining to-be-detected voice, and carrying out deep forgery detection on the to-be-detected voice through the style encoder, the language encoder and the deep forgery detection model to obtain a deep forgery detection result of the to-be-detected voice. According to the method, the performance of deep forgery detection can be improved.

Description

Technical Field

[0001] The present application relates to the fields of artificial intelligence and financial technology, and in particular to an audio deep fake detection method, apparatus, storage medium and computer equipment. Background Art

[0002] With the rapid development of artificial intelligence technology, generative models are increasingly being used in various fields, especially in speech synthesis. Generative models can use deep learning algorithms to generate highly realistic imitation speech by analyzing and learning the characteristics of real speech. The rise of this technology has led to an increasing number of publicly available text-to-speech (TTS) or voice conversion (VC) tools, which can synthesize fake speech that is extremely similar to real people with just a few seconds of real speech recording.

[0003] However, the generated fake voice, also known as deep fake audio, can be used for a variety of illegal or fraudulent activities under the control of malicious actors. For example, robot phone scams can be carried out by imitating the voices of celebrities or family members, or using fake voices to illegally access voice-protected bank accounts. These behaviors not only seriously infringe on personal privacy and property security, but also pose a serious threat to social order. In response to the above problems, the existing technology proposes that deep fake detection models based on deep learning can be used for identification.

[0004] At present, existing audio deepfake detection models can usually only perform well for specific fields or specific data sets, and their detection performance often drops significantly when facing different fields or cross-domain data. In addition, most existing detection models operate in a black box manner, that is, the decision-making process and basis of the model are not transparent, and there is a lack of explanation for the model's decision. Therefore, users often find it difficult to understand and accept the detection results, which affects the reliability and effectiveness of the detection tools. Summary of the invention

[0005] In view of this, the embodiments of the present application provide an audio deep fake detection method, apparatus, storage medium and computer device, the main purpose of which is to solve the technical problems that the existing audio deep fake detection methods have large performance differences between different data sets and the model decision process lacks explainability.

[0006] According to one aspect of the present application, a method for detecting deep fake audio is provided, the method comprising:

[0007] Acquire a real speech sample, and based on the real speech sample, perform self-supervisory training on the pre-trained first encoding network and the second encoding network to obtain a style encoder and a language encoder;

[0008] By using the style encoder and the language encoder, feature extraction is performed on the real voice sample and the deep fake sample to obtain voice features of the real voice sample and voice features of the deep fake sample;

[0009] Based on the voice features of the real voice sample and the voice features of the deep fake voice sample and the preset sample labels, supervised training is performed on a preset classification model to obtain a deep fake detection model;

[0010] Acquire the speech to be tested, perform deep fake detection on the speech to be tested through the style encoder, the language encoder and the deep fake detection model, and obtain a deep fake detection result of the speech to be tested.

[0011] According to another aspect of the present application, a device for detecting deep fake audio is provided, the device comprising:

[0012] The coding model training module is used to obtain real speech samples, and based on the real speech samples, perform self-supervisory training on the pre-trained first coding network and the second coding network to obtain a style encoder and a language encoder;

[0013] A speech feature extraction module, used to extract features of the real speech sample and the deep fake sample through the style encoder and the language encoder to obtain speech features of the real speech sample and speech features of the deep fake sample;

[0014] A detection model training module, configured to perform supervised training on a preset classification model based on the voice features of the real voice sample and the voice features of the deep fake voice sample and preset sample labels to obtain a deep fake detection model;

[0015] The deep fake detection module is used to obtain the voice to be tested, and perform deep fake detection on the voice to be tested through the style encoder, language encoder and deep fake detection model to obtain the deep fake detection result of the voice to be tested.

[0016] According to another aspect of the present application, a storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the above-mentioned audio deep fake detection method is implemented.

[0017] According to another aspect of the present application, a computer device is provided, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor implements the above-mentioned audio deep fake detection method when executing the program.

[0018] By means of the above technical scheme, an audio deep fake detection method, device, storage medium and computer equipment are provided in the embodiments of the present application. First, based on real voice samples, the pre-trained first encoding network and the second encoding network are self-supervised trained to obtain a style encoder and a language encoder, and then the trained style encoder and the language encoder are used to extract features of the real voice samples and the deep fake samples respectively to obtain the voice features of the two samples, and then based on the voice features of the two samples and their corresponding sample labels, the preset classification model is supervised trained to obtain a deep fake detection model, and finally the tested voice is deep faked through the trained style encoder, language encoder and deep fake detection model to obtain the deep fake detection result of the tested voice. The above method can learn the dependency between style features and language features in real voice by self-supervising training on real voice samples in the first stage; and can learn the difference between style and language content between false voice and real voice through supervised training in the second stage. Based on this, the style encoder, language encoder and deep fake detection model obtained through two-stage training can perform deep fake detection on the test speech, which can improve the detection performance of the model on various data sets and make the decision-making process of the model more explainable, thereby improving the reliability and effectiveness of the model, and then improving the accuracy of deep fake detection.

[0019] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0021] Figure 1 A schematic diagram of an application environment of an audio deep fake detection method provided by an embodiment of the present application is shown;

[0022] Figure 2 A schematic diagram of a process of detecting deep fake audio provided by an embodiment of the present application is shown;

[0023] Figure 3 A schematic diagram of the structure of an audio deep fake detection model provided in an embodiment of the present application is shown;

[0024] Figure 4 A schematic diagram of the structure of an audio deep fake detection device provided in an embodiment of the present application is shown;

[0025] Figure 5 A schematic diagram of the structure of a computer device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] The audio deep fake detection method provided by the embodiment of the present invention can be applied in the following aspects: Figure 1 In the application environment shown, the client can communicate with the server through the network. Specifically, the server can obtain the voice to be tested through the client, and then perform deep forgery detection on the voice to be tested through the trained style encoder, language encoder and deep forgery detection model to obtain the deep forgery detection result of the voice to be tested, and finally feed back the deep forgery detection result of the voice to be tested to the client. In the present invention, the style encoder, language encoder and deep forgery detection model for deep forgery detection are trained in two stages. The first stage is to perform self-supervisory training on real voice samples to learn the dependency between style features and language features in real voice; the second stage is to perform supervised training on false voice and real voice to learn the difference between style and language content between false voice and real voice. Based on this, the model obtained by two-stage training can perform deep forgery detection on the voice to be tested, which can improve the detection performance of the model on various data sets, and can make the decision process of the model more explainable, thereby improving the reliability and effectiveness of the model, and then improving the accuracy of deep forgery detection. Among them, the client can be but not limited to various personal computers, laptops, smart phones, tablet computers and portable wearable devices. The server can be implemented with an independent server or a server cluster consisting of multiple servers. The present invention is described in detail below through specific embodiments.

[0028] Currently, existing audio deepfake detection models have significant differences in their performance when processing audio data from different sources or styles. Specifically, these models perform well on the data used during training (in-domain data), but their detection effect may drop significantly when faced with data from other sources or styles (out-of-domain data). This lack of generalization ability limits the breadth and accuracy of the models in practical applications. In addition, existing detection models often lack transparency and explainability, that is, their decision-making process is invisible or difficult to understand to users or developers. This black-box nature makes it difficult for users to evaluate the reliability of the model and also limits the application of the model in real-world scenarios where explainable decisions are required.

[0029] In view of the above problems, in one embodiment, Figure 2 As shown in FIG. 1 , a method for detecting deep fake audio is provided, which can be implemented with the help of a trained deep fake audio detection model. The deep fake audio detection model can detect deep fake audio by using the differences in style and language content between fake and real audio. The overall architecture of the model is shown in FIG. Figure 3 As shown, the above model adopts a two-stage training framework. Among them, in the first stage, only real speech samples are self-supervised pre-trained to learn the dependency between style features and language features in real speech. In the second stage, deep fake speech and real speech data sets with labels (i.e. true and false binary labels) are used for supervised training, and the learned dependency features are combined with the original style and language features to train the classifier. Finally, the style encoder, language encoder and classifier obtained through the two-stage training are used to perform deep fake detection on the test speech to obtain the deep fake detection result of the test speech. The above-mentioned audio deep fake detection method is now taken as an example for application to computer devices such as servers, including the following steps:

[0030] Step 101: obtain a real speech sample, and based on the real speech sample, perform self-supervisory training on the pre-trained first encoding network and the second encoding network to obtain a style encoder and a language encoder.

[0031] Specifically, refer to Figure 3In the first stage of training, the style encoder and language encoder can be trained by single-class self-supervised contrastive training. First, a large number of unlabeled real speech samples can be collected, and the diversity of speech samples can be ensured to cover different speakers, different emotional states, and different accents. Then, a pre-trained network, such as a convolutional neural network (CNN) or a long short-term memory network (LSTM), is used to extract the style feature representation and language feature representation of each sample. Furthermore, the extracted style feature representation and language feature representation can be embedded and sent to the compression modules corresponding to the two networks respectively, which can average the features output by the Transformer layer in the network and reduce the high-dimensional feature vector to a lower dimension. For example, reducing the 1024-dimensional original feature representation to 256-dimensional can not only reduce the computational complexity, but also make the extracted features more compact and more discriminative. Finally, a self-contrastive loss function can be defined and minimized to learn the dependency characteristics between the style features and the language features in the real speech samples. Finally, after multiple rounds of iterative training, a trained style encoder and language encoder can be obtained, which can effectively capture and distinguish the style information and language information in the speech.

[0032] In this embodiment, style features refer to features in speech that are related to the speaker's identity, emotion, accent, etc. These features are not directly related to the content of the speech, but reflect the speaker's personal characteristics and expression methods. Language features refer to features in speech that are related to the content of the speech, such as the text content, phoneme information, semantic information, etc. of the speech. For example, the same sentence "hello" will have different timbres, tones, and rhythms when spoken by different people, all of which are style features. Although "hello" and "hello" may have the same style features, their text content and semantic information are different, which are language features.

[0033] This embodiment adopts a single-class self-supervised contrast training strategy, which enables the model to more effectively learn the style features and language features in real voice samples, as well as the dependencies between them, which can help the model to more accurately identify fake voices in subsequent stages. In addition, the training process of the style encoder and the language encoder also enables the model to maintain a good detection effect when facing out-of-domain data, thereby improving the accuracy and reliability of the model.

[0034] Step 102: extract features of the real speech sample and the deep fake sample through a style encoder and a language encoder to obtain speech features of the real speech sample and speech features of the deep fake sample.

[0035] Specifically, refer to Figure 3In the second stage of supervised training, in order to ensure that the dependency between the style features and language features learned in the first stage is not destroyed, the parameters of the style encoder and language encoder can be frozen first. Next, the style encoder and language encoder that have been trained in the first stage can be used to extract features from real speech samples and deep fake samples respectively to obtain the original style features and original language features corresponding to each sample. These features are then sent to the compression module corresponding to the encoder, and the original features are processed by the compression module to achieve feature dimensionality reduction, such as reducing the feature size from 1024 dimensions to 256 dimensions, thereby obtaining compressed style features and language features.

[0036] Furthermore, in order to enhance the representation ability of features and the generalization ability of the model, an attention statistics pooling layer (ASP) and a multi-layer perceptron (MLP) network can be introduced, and the original style features and original language features can be transformed and reduced in dimension. Through the ASP layer, the model can learn which parts of the features are more important for distinguishing real speech from deep fake speech, and the MLP network can further map these important features to a lower dimensional space, and finally reduce the feature dimension to 256 dimensions again. Finally, the low-dimensional features extracted by the ASP and MLP networks can be connected with the compressed dependency features extracted by the encoder obtained by the first stage of training to form a 1024-dimensional comprehensive feature vector as the final speech feature of the sample. In this way, whether it is a real speech sample or a deep fake sample, its corresponding speech features can be generated through the same process, and provide a detection basis for subsequent deep fake detection.

[0037] This embodiment freezes the parameters of the style encoder and the language encoder to ensure the consistency and stability of feature extraction. By introducing the attention statistical pooling layer and the multi-layer perceptron, the expressiveness of the features can be enhanced, so that the model can more accurately identify forged speech. In addition, by connecting the dependency features with the compressed style features and language features, a high-dimensional feature vector can be generated, thereby providing richer and more effective feature information for subsequent deep fake detection tasks.

[0038] Step 103: Based on the speech features of the real speech sample, the speech features of the deep fake speech sample and the preset sample labels, supervised training is performed on the preset classification model to obtain a deep fake detection model.

[0039] Specifically, refer to Figure 3In the supervised training phase, for each sample, including real speech samples and deep fake samples, the dependency features extracted by the style encoder and language encoder obtained by the first stage training can be connected with the subspace representation extracted by the attention statistical pooling layer (ASP) and the multi-layer perceptron (MLP) network in the second stage to form a comprehensive feature vector. This comprehensive feature vector contains rich style and language information and can effectively distinguish real speech from deep fake speech. Then, the comprehensive feature vector of each sample is input into a preset classification head, wherein the structure of the classification head can include two fully connected layers and a dropout layer, which can prevent overfitting and improve the generalization ability of the model. Through the classification head, the model can generate a binary true / false prediction result based on the input comprehensive feature vector, that is, judge whether the sample is real speech or deep fake speech. During the training process, the binary cross entropy loss can be used as the optimization target, and the parameters of the classification head can be adjusted through algorithms such as the back propagation algorithm to minimize the difference between the prediction result and the preset sample label. At the same time, this process will also jointly train the ASP and MLP modules to optimize the performance of these modules in feature extraction. Finally, after sufficient training, a trained deep fake detection model can be obtained. This model has strong generalization ability and high detection accuracy when facing new unknown samples.

[0040] This embodiment can ensure that the various modules in the model can work together and extract discriminative features together by jointly training the attention statistical pooling layer, the multi-layer perceptron and the classification head, so as to accurately classify. In addition, by adopting the dropout technology and the binary cross entropy loss function, the generalization ability and robustness of the model can be further enhanced, and the risk of overfitting can be reduced.

[0041] Step 104: obtain the speech to be tested, and perform deep fake detection on the speech to be tested through a style encoder, a language encoder, and a deep fake detection model to obtain a deep fake detection result of the speech to be tested.

[0042] Specifically, when performing the detection task through the trained model, the speech sample to be tested can be obtained first, and then the trained style encoder and language encoder are used to perform preliminary processing on the speech to be tested to extract the original style features and original language features of the speech to be tested. Subsequently, a series of processing is performed on the extracted features, including compressing the original style features and original language features through a compression module to reduce the feature dimension and improve processing efficiency, and using the attention statistical pooling layer (ASP) and the multi-layer perceptron (MLP) network to transform and reduce the dimension of the two original features, and connecting the compressed features and the transformed features, and finally generating a comprehensive feature vector that can fully reflect the style and language characteristics of the speech to be tested. Finally, this comprehensive feature vector can be input into the classification head of the previously trained deep fake detection model, and the classification head can calculate the probability of whether the speech to be tested is a real voice or a deep fake voice based on the input feature vector, thereby obtaining the deep fake detection result of the speech to be tested.

[0043] This embodiment detects deep fake speech by modeling the style and language mismatch of the fake speech, which can make its performance on out-of-domain datasets better than existing benchmarks, and also perform equally well on in-domain datasets. In addition, the method proposed in this embodiment does not require additional labeled data, nor does it require end-to-end fine-tuning of the pre-trained encoder, avoiding high time and computational costs. In contrast, the prior art often requires labeling style information and fine-tuning the encoder using the target dataset to adapt to the target dataset, which undoubtedly adds additional overhead. In addition, this embodiment not only enriches the feature representation of the model by learning the dependency between style features and language features, but also improves the interpretability of the detection results, thereby making the decision-making process of the model more transparent and easy to understand.

[0044] The audio deepfake detection method provided in this embodiment can learn the dependency between style features and language features in real speech through self-supervised training on real speech samples in the first stage; and can learn the difference between style and language content between fake speech and real speech through supervised training in the second stage. Based on this, the style encoder, language encoder, and deepfake detection model obtained through two-stage training can be used to perform deepfake detection on the speech to be tested, which can improve the detection performance of the model on various data sets and make the decision-making process of the model more explainable, thereby improving the reliability and effectiveness of the model, and further improving the accuracy of deepfake detection.

[0045] In one embodiment, the style encoder and the language encoder can be trained by the following method: first, the style features of the real speech sample are extracted by the pre-trained first encoding network, and the language features of the real speech sample are extracted by the pre-trained second encoding network, and then the style features of the real speech sample and the language features of the real speech sample are compressed respectively to obtain the compressed style features and language features, and then based on the compressed style features and language features, the first encoding network and the second encoding network are iteratively parameter trained by the preset loss function. When the sum of the subspace loss of the first encoding network, the subspace loss of the second encoding network, and the cross-subspace loss between the first encoding network and the second encoding network reaches the preset convergence condition, the trained style encoder and language encoder are obtained.

[0046] Specifically, when training the style encoder and the language encoder, the style features of the real speech sample can be first extracted by using the pre-trained first encoding network, such as Wav2vec-SER, and the language features of the real speech sample can be extracted by using the pre-trained second encoding network, such as Wav2vec-ASR. Among them, both encoding networks are built based on the Transformer architecture, which can effectively capture the time series features in the speech signal. Next, the style features and language features can be compressed by the compression module. Among them, the compression module can process the three-dimensional tensor output by each encoder, and the shape of the three-dimensional tensor is [K, F, T], where K represents the index of the Transformer layer, F represents the feature dimension, and T represents the time step. By averaging the K dimensions (i.e., the outputs of different Transformer layers), the outputs of multiple layers of Transformers can be merged into a single feature vector. The averaging method is to sum the outputs of K Transformer layers and then divide them by the number of layers K, so as to obtain the compressed style features and language features.

[0047] Furthermore, based on the compressed style features and language features, the first encoding network and the second encoding network can be iteratively trained in parameters through a preset loss function. The loss function here is designed to minimize three aspects of loss: the loss within the subspace of the first encoding network, the loss within the subspace of the second encoding network, and the cross-subspace loss between the first encoding network and the second encoding network. Among them, the loss within the subspace is used to ensure that the features learned within each encoding network are consistent and compact, while the cross-subspace loss promotes the learning of the interdependence between style features and language features. When the sum of these three losses reaches the preset range, it is considered that the performance indicators of the model meet specific standards. At this time, the training process ends, and the trained style encoder and language encoder are obtained.

[0048] This embodiment can effectively extract the style features and language features of the speech sample by using the pre-trained encoding network as the basis of the style encoder and language encoder. In addition, by averaging the output of the Transformer layer to compress the features, it is not only possible to reduce the dimension of the features and improve the efficiency of subsequent processing, but also to retain key information in the features to ensure the accuracy and robustness of feature extraction. In addition, by paying attention to the losses in three aspects during the model training process and performing iterative training, it is possible to ensure that the overall performance of the model in extracting features is optimal. The style encoder and language encoder trained in this way can better adapt to the needs of deep fake detection tasks and improve the accuracy and efficiency of deep fake detection.

[0049] In one embodiment, the style features and language features of a real speech sample can be extracted by the following method: cross-comparison and correlation coefficient calculation are performed on different network layers of the first encoding network and the second encoding network to obtain a correlation coefficient matrix between the first encoding network and the second encoding network, and then based on the correlation coefficient matrix, the style extraction network layer of the first encoding network and the language extraction network layer of the second encoding network are identified to separate the style features extracted by the style extraction network layer and the language features extracted by the language extraction network layer from each other, and finally the style features of the real speech sample are extracted by the style extraction network layer of the first encoding network, and the language features of the real speech sample are extracted by the language extraction network layer of the second encoding network.

[0050] Specifically, in the process of training the style encoder and the language encoder, when extracting style features and language features through the two encoders, the different network layers of the pre-trained first encoding network and the second encoding network can first be cross-compared and the correlation coefficients can be calculated to obtain the correlation coefficient matrix between the first encoding network and the second encoding network. Among them, the rows of the correlation coefficient matrix represent the layers of the first encoding network model, and the columns represent the layers of the second encoding network. In this embodiment, the correlation coefficient matrix between the two networks can be calculated by calculating the Spearman rank correlation coefficient values ​​between the different layers of the two encoding networks, thereby quantifying the inter-layer similarity of the two networks. For example, assuming that the first encoding network is Wav2vec-SER and the second encoding network is Wav2vec-ASR, by calculating the correlation coefficient matrix between the two encoding networks, it can be found that the early layers (0-10 layers) of the Wav2vec-SER model have a high correlation with the early layers of the Wav2vec-ASR model, while the late layers of the Wav2vec-SER model have a low correlation with the late layers of the Wav2vec-ASR model. This shows that the early layers of the Wav2vec-SER model are more inclined to encode style information, while the later layers of the Wav2vec-ASR model are more inclined to encode language information. Based on the correlation analysis between layers, the 0-10 layers of the Wav2vec-SER model were finally determined as the style extraction network layer, and the 14-21 layers of the Wav2vec-ASR model were determined as the language extraction network layer to achieve the maximum separation of style features and language features. After the style extraction network layer and the language extraction network layer are determined, the style features and language features of the real speech sample can be extracted respectively through the selected network layers, and then the extracted style features and language features are embedded and sent to the compression module to reduce the feature size from 1024 dimensions to 256 dimensions through the compression module, thereby further reducing the dimension of the feature and improving the efficiency of subsequent processing.

[0051] This embodiment can comprehensively analyze the similarities between different layers of the two models by cross-comparing the network layers of the two models and calculating the correlation coefficient, so as to better understand the learning process and feature representation of the models. In this way, the expression of style features and language features can be separated to the greatest extent, improving the performance and interpretability of the model.

[0052] In one embodiment, the loss function includes the intra-subspace loss of the first encoding network, the intra-subspace loss of the second encoding network, and the cross-subspace loss between the first encoding network and the second encoding network. The intra-subspace loss of the first encoding network is composed of the time average dependency features of the style features extracted by the first encoding network, the intra-subspace loss of the second encoding network is composed of the time average dependency features of the language features extracted by the second encoding network, and the cross-subspace loss between the first encoding network and the second encoding network is composed of the dependency features of the style features and the language features at the time step.

[0053] Specifically, the loss function may include the intra-subspace loss of the first encoding network, the intra-subspace loss of the second encoding network, and the cross-subspace loss between the first encoding network and the second encoding network. Through the above design, the training of the style encoder and the language encoder can be optimized. Specifically, the intra-subspace loss of the first encoding network can be composed of the time-averaged dependency features of the style features extracted by the first encoding network, and is used to reduce the redundancy within the style features; the intra-subspace loss of the second encoding network can be composed of the time-averaged dependency features of the language features extracted by the second encoding network, and is used to reduce the redundancy within the language features; the cross-subspace loss between the first encoding network and the second encoding network can be composed of the dependency features of the style features and the language features at the time step, and is used to measure the distance between the style and language features to ensure that the two can be effectively separated. For example, the loss function can be defined as follows:

[0054] L con =L cross +λL intra

[0055]

[0056] L intra =L style +L linguistics

[0057]

[0058] Among them, L con Denotes the total contrast loss. L cross Represents the cross-subspace loss, which is used to measure the distance between style features and language features. intra Represents the loss in the subspace, which is used to reduce the redundancy within the style features and language features. (f，t) and L (f，t)Denote the dependency features of style features and language features at time step t. Time step t refers to the sampling point of the speech signal in the time dimension. Calculating the dependency features of style and language features at time step t can capture the dynamic relationship between them in the time dimension, thereby more effectively detecting deep fake audio. f and L f They represent the temporal average dependency features of style and language features respectively. λ is a hyperparameter used to balance the cross-subspace loss and the intra-subspace loss. T represents the transposed matrix of the corresponding matrix, and I represents the identity matrix. The identity matrix is ​​a square matrix whose elements on the main diagonal are all 1 and the rest are all 0. F represents the matrix norm, which is used to measure the size of the matrix.

[0059] In this embodiment, in the process of training the first encoding network and the second encoding network, a real speech sample can be used as input, and the style features and language features of the real speech sample can be extracted through the style encoder and the language encoder respectively. Then, the dependency features of these features in time steps are calculated, and iterative optimization is performed according to the above loss function. When the loss function value reaches the preset convergence condition, it can be considered that the model has been trained, and the trained style encoder and language encoder can be obtained at this time.

[0060] This embodiment can capture the dynamic relationship of these features in the time dimension by calculating the dependency characteristics of style features and language features in time steps, and calculating the time average and cross-subspace distance respectively, thereby reducing the redundancy within the subspace. The above method can improve the efficiency and generalization ability of the model, ensure the independence of style features and language features in the representation space, and thus provide strong support for subsequent deep fake detection tasks.

[0061] In one embodiment, the preset classification model includes a first attention statistical pooling layer, a first multi-layer perceptron, a second attention statistical pooling layer, a second multi-layer perceptron, and a classification head. On this basis, the deep fake detection model can be trained by the following method: extracting the speech features of the sample through the style encoder and the language encoder, as well as the first attention statistical pooling layer, the first multi-layer perceptron, the second attention statistical pooling layer, and the second multi-layer perceptron, wherein the sample includes a real speech sample and a deep fake sample; inputting the speech features of the sample into the classification head, and based on the speech features and sample labels of the sample, iteratively jointly training the parameters of the first attention statistical pooling layer, the first multi-layer perceptron, the second attention statistical pooling layer, the second multi-layer perceptron, and the classification head; when the binary cross entropy loss of the classification model reaches the preset convergence condition, stop the training of the classification model to obtain the trained deep fake detection model.

[0062] Specifically, the deep fake detection model can be trained by the following method: First, using the trained style encoder and language encoder, combined with the first attention statistical pooling layer, the first multi-layer perceptron, the second attention statistical pooling layer and the second multi-layer perceptron, the speech features of the sample are extracted, including the speech features of the real speech sample and the speech features of the deep fake sample. In this process, the style features and language features of the sample can be extracted by the style encoder and the language encoder respectively, and then the style features are processed by the first attention statistical pooling layer and the first multi-layer perceptron, and the language features are processed by the second attention statistical pooling layer and the second multi-layer perceptron, and then the processed style features and language features are fused to generate the comprehensive speech features of each sample.

[0063] Furthermore, the comprehensive speech features of the sample can be input into the classification head, wherein the classification head can be composed of two fully connected layers and a dropout layer, and is used to generate binary true / false prediction results. During the training process, the parameters of the first attention statistical pooling layer, the first multilayer perceptron, the second attention statistical pooling layer, the second multilayer perceptron, and the classification head can be iteratively jointly trained based on the speech features of the sample and the sample label (i.e., the true category label of the sample). The goal of the training is to minimize the binary cross entropy loss of the classification model, which can measure the difference between the model's predicted probability and the true label. When the binary cross entropy loss of the classification model reaches a preset range, that is, when the performance index of the model meets a specific standard, the training of the classification model can be stopped, that is, the trained deep fake detection model is obtained.

[0064] This embodiment can effectively capture key information in speech features by setting an attention statistics pooling layer and a multi-layer perceptron in the deep fake detection model, thereby improving the accuracy of detection. Moreover, by jointly training all parameters in the classification model, the model can learn a more robust and generalized feature representation. In addition, by setting a preset range of binary cross entropy loss, it can ensure that the model achieves a good performance level during the training process, thereby obtaining a high-quality deep fake detection model.

[0065] In one embodiment, the speech features of the sample can be extracted by the following method: extracting the style features and language features of the sample through a style encoder and a speech encoder to obtain high-dimensional style features and high-dimensional language features of the sample. Performing dimensionality reduction and transformation processing on the high-dimensional style features through the first attention statistical pooling layer and the first multi-layer perceptron to obtain the transformed style features of the sample, and compressing the high-dimensional style features through the compression module in the style encoder to obtain the compressed style features of the sample. Performing dimensionality reduction and transformation processing on the high-dimensional language features through the second attention statistical pooling layer and the second multi-layer perceptron to obtain the transformed language features of the sample, and compressing the high-dimensional language features through the compression module in the language encoder to obtain the compressed language features of the sample. Connecting the transformed style features, compressed style features, transformed language features and compressed language features of the sample to obtain the speech features of the sample.

[0066] Specifically, in the process of training the deep fake detection model, the speech features of the sample can be extracted by the following method: first, the style features and language features of the sample are extracted by using the style encoder and the language encoder to obtain the high-dimensional style features and high-dimensional language features of the sample. Among them, the extracted high-dimensional features usually have a high dimension, such as 1024 dimensions. Directly using the extracted high-dimensional features to perform classification tasks may lead to excessive model complexity and easy overfitting. Based on this, the high-dimensional style features can be reduced in dimension and transformed by the first attention statistical pooling layer (ASP) and the first multi-layer perceptron (MLP) to obtain the transformed style features of the sample; at the same time, the high-dimensional style features are compressed by the compression module in the style encoder to obtain the compressed style features of the sample. Similarly, the high-dimensional language features are reduced in dimension and transformed by the second attention statistical pooling layer and the second multi-layer perceptron to obtain the transformed language features of the sample; and the high-dimensional language features are compressed by the compression module in the language encoder to obtain the compressed language features of the sample. Finally, the comprehensive speech features of the sample can be obtained by connecting the transformed style features, compressed style features, transformed language features and compressed language features of the sample.

[0067] In this embodiment, refer to Figure 3, the attention statistical pooling layer (ASP) and the multi-layer perceptron (MLP) can reduce the dimension and transform the original style features and language features to facilitate subsequent classification tasks. Among them, the attention statistical pooling layer can learn the importance weights of style features and language features in the time dimension, and perform weighted average of the features according to the weights to obtain a more representative feature vector. The multi-layer perceptron is a fully connected neural network that can perform nonlinear transformations on features to extract deeper feature representations. Through these two modules, the feature dimension can be compressed from 1024 dimensions to 256 dimensions, which not only reduces the computational complexity, but also improves the generalization ability of the model. Furthermore, the purpose of compression is to align with the dependency features extracted by the compression module and to splice the four features. In this embodiment, the model uses two features: subspace features and dependency features. Among them, the subspace features are the original style features and language features, which are obtained after compression by the ASP and MLP modules. The dependency features are the dependency features between the style features and the language features, which are extracted by the compression module. In order to concatenate these two features, the dimension of the subspace feature needs to be compressed to the same dimension as the dependency feature, such as 256. After concatenation, a 1024-dimensional comprehensive speech feature can be obtained.

[0068] This embodiment introduces an attention statistical pooling layer and a multi-layer perceptron to reduce the dimension and transform the original style features and language features, so that a more representative feature representation can be obtained. These features can reduce the complexity of the model and improve the generalization ability of the model. At the same time, by extracting the dependency features between style features and language features through the compression module, and splicing these features with the subspace features, a more comprehensive feature representation can be obtained. The above method can make the deep fake detection task more accurate and robust, and has strong scalability, which can adapt to the needs of different data sets and tasks.

[0069] In one embodiment, the following method can be used to detect deep fakes of the speech to be tested: first, the style features and language features of the speech to be tested are extracted through a style encoder and a speech encoder to obtain high-dimensional style features and high-dimensional language features of the speech to be tested, and then the high-dimensional style features and high-dimensional language features of the speech to be tested are reduced in dimension, transformed, and connected through the first attention statistical pooling layer, the first multi-layer perception, the second attention statistical pooling layer, and the second multi-layer perception in the deep fake detection model to obtain the synthetic features of the speech to be tested, and finally the synthetic features of the speech to be tested are input into the classification head in the deep fake detection model to obtain the deep fake detection result of the speech to be tested.

[0070] Specifically, when a new voice to be tested is obtained, the voice to be tested can be deep forged by the following method: First, the style features and language features of the voice to be tested are extracted using the trained style encoder and language encoder to obtain high-dimensional style features and high-dimensional language features of the voice to be tested. Then, the extracted high-dimensional features are reduced in dimension and transformed. Specifically, the high-dimensional style features can be reduced in dimension and transformed by the first attention statistical pooling layer and the first multi-layer perceptron in the deep forgery detection model to obtain the transformed style features of the voice to be tested, and the high-dimensional style features can be compressed by the compression module in the style encoder to obtain the compressed style features of the voice to be tested. Similarly, the high-dimensional language features are reduced in dimension and transformed by the second attention statistical pooling layer and the second multi-layer perceptron to obtain the transformed language features of the voice to be tested, and the high-dimensional language features are compressed by the compression module in the language encoder to obtain the compressed language features of the voice to be tested. Further, the synthetic features of the voice to be tested are obtained by connecting the transformed style features, compressed style features, transformed language features and compressed language features of the voice to be tested. Finally, the synthesized features of the test speech can be input into the classification head in the deep fake detection model. The classification head can calculate the probability that the test speech is a real speech or a deep fake speech based on the input synthesized feature vector, thereby obtaining the deep fake detection result.

[0071] This embodiment uses a style encoder and a speech encoder to extract the style features and language features of the speech to be tested, and combines modules such as the attention statistical pooling layer and the multi-layer perceptron to reduce the dimension and transform the original style features and language features, so as to obtain more representative synthetic features. The synthetic features can provide comprehensive information support for deep fake detection, so that the model can more accurately identify whether the speech to be tested is real speech or deep fake speech, thereby improving the accuracy and interpretability of deep fake detection.

[0072] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. In addition, the numbers corresponding to the various steps in the above embodiments only serve as identification and are not used as a limitation on the order in which the steps are executed. The order in which the steps are executed in each embodiment can be set according to actual conditions.

[0073] Further, as Figure 2 The specific implementation of the method, the embodiment of the present application provides an audio deep fake detection device, such as Figure 4 As shown, the device comprises:

[0074] The coding model training module 31 can be used to obtain real speech samples, and based on the real speech samples, perform self-supervisory training on the pre-trained first coding network and the second coding network to obtain a style encoder and a language encoder;

[0075] A speech feature extraction module 32, which can be used to extract features of the real speech sample and the deep fake sample through the style encoder and the language encoder to obtain speech features of the real speech sample and speech features of the deep fake sample;

[0076] A detection model training module 33, which can be used to perform supervised training on a preset classification model based on the voice features of the real voice sample and the voice features of the deep fake voice sample and the preset sample labels to obtain a deep fake detection model;

[0077] The deep fake detection module 34 can be used to obtain the voice to be tested, and perform deep fake detection on the voice to be tested through the style encoder, language encoder and deep fake detection model to obtain a deep fake detection result of the voice to be tested.

[0078] In a specific application scenario, the coding model training module 31 can be specifically used to extract the style features of the real speech sample through a pre-trained first coding network, and to extract the language features of the real speech sample through a pre-trained second coding network; compress the style features of the real speech sample and the language features of the real speech sample respectively to obtain compressed style features and language features; based on the compressed style features and language features, the first coding network and the second coding network are iteratively parameter trained through a preset loss function; when the sum of the subspace loss of the first coding network, the subspace loss of the second coding network, and the cross-subspace loss between the first coding network and the second coding network reaches a preset convergence condition, the trained style encoder and language encoder are obtained.

[0079] In a specific application scenario, the coding model training module 31 can be specifically used to cross-compare and calculate correlation coefficients of different network layers of the first coding network and the second coding network to obtain a correlation coefficient matrix between the first coding network and the second coding network; based on the correlation coefficient matrix, identify the style extraction network layer of the first coding network and the language extraction network layer of the second coding network to separate the style features extracted by the style extraction network layer and the language features extracted by the language extraction network layer from each other; extract the style features of the real speech sample through the style extraction network layer of the first coding network, and extract the language features of the real speech sample through the language extraction network layer of the second coding network.

[0080] In a specific application scenario, the loss function includes the intra-subspace loss of the first encoding network, the intra-subspace loss of the second encoding network, and the cross-subspace loss between the first encoding network and the second encoding network, wherein the intra-subspace loss of the first encoding network is composed of the time-averaged dependency features of the style features extracted by the first encoding network; the intra-subspace loss of the second encoding network is composed of the time-averaged dependency features of the language features extracted by the second encoding network; and the cross-subspace loss between the first encoding network and the second encoding network is composed of the dependency features of the style features and the language features at time steps.

[0081] In a specific application scenario, the preset classification model includes a first attention statistical pooling layer, a first multi-layer perceptron, a second attention statistical pooling layer, a second multi-layer perceptron and a classification head; the detection model training module 33 can be specifically used to extract the speech features of the sample through the style encoder and the language encoder, as well as the first attention statistical pooling layer, the first multi-layer perceptron, the second attention statistical pooling layer and the second multi-layer perceptron, wherein the sample includes a real speech sample and a deep fake sample; the speech features of the sample are input into the classification head, and based on the speech features and sample labels of the sample, the parameters of the first attention statistical pooling layer, the first multi-layer perceptron, the second attention statistical pooling layer, the second multi-layer perceptron and the classification head are iteratively jointly trained; when the binary cross entropy loss of the classification model reaches a preset convergence condition, the training of the classification model is stopped to obtain a trained deep fake detection model.

[0082] In a specific application scenario, the speech feature extraction module 32 can also be used to extract the style features and language features of the sample through the style encoder and the speech encoder to obtain the high-dimensional style features and high-dimensional language features of the sample; perform dimensionality reduction and transformation processing on the high-dimensional style features through the first attention statistical pooling layer and the first multi-layer perceptron to obtain the transformed style features of the sample, and perform compression processing on the high-dimensional style features through the compression module in the style encoder to obtain the compressed style features of the sample; perform dimensionality reduction and transformation processing on the high-dimensional language features through the second attention statistical pooling layer and the second multi-layer perceptron to obtain the transformed language features of the sample, and perform compression processing on the high-dimensional language features through the compression module in the language encoder to obtain the compressed language features of the sample; connect the transformed style features, compressed style features, transformed language features and compressed language features of the sample to obtain the speech features of the sample.

[0083] In a specific application scenario, the deep fake detection module 34 can be used to extract the style features and language features of the voice to be tested through the style encoder and the voice encoder to obtain high-dimensional style features and high-dimensional language features of the voice to be tested; through the first attention statistical pooling layer, the first multi-layer perception, the second attention statistical pooling layer and the second multi-layer perception in the deep fake detection model, the high-dimensional style features and high-dimensional language features of the voice to be tested are reduced in dimension, transformed and connected to obtain the synthetic features of the voice to be tested; the synthetic features of the voice to be tested are input into the classification head in the deep fake detection model to obtain the deep fake detection result of the voice to be tested.

[0084] It should be noted that other corresponding descriptions of the functional units involved in the audio deep fake detection device provided in the embodiment of the present application can be referred to Figure 2 The corresponding description in the method will not be repeated here.

[0085] The present application also provides a computer device, such as Figure 5 As shown, the computer device can specifically be a personal computer, a server, a network device, etc. The computer device includes a system bus, a processor, a memory and a communication interface, and can also include an input and output interface and a display device. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store location information. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the steps in each method embodiment are implemented.

[0086] Those skilled in the art will appreciate that the structure of the above-mentioned computer device is only a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components, or combine certain components, or have a different arrangement of components.

[0087] In one embodiment, a computer-readable storage medium is provided. The computer-readable storage medium may be non-volatile or volatile, and stores a computer program thereon. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0088] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0089] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0090] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0091] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for detecting deep fake audio, characterized in that: The method comprises: Acquire a real speech sample, and based on the real speech sample, perform self-supervisory training on the pre-trained first encoding network and the second encoding network to obtain a style encoder and a language encoder; By using the style encoder and the language encoder, feature extraction is performed on the real voice sample and the deep fake sample to obtain voice features of the real voice sample and voice features of the deep fake sample; Based on the voice features of the real voice sample and the voice features of the deep fake voice sample and the preset sample labels, supervised training is performed on a preset classification model to obtain a deep fake detection model; Acquire the speech to be tested, perform deep fake detection on the speech to be tested through the style encoder, the language encoder and the deep fake detection model, and obtain a deep fake detection result of the speech to be tested.

2. The method according to claim 1, characterized in that The method of performing self-supervisory training on the pre-trained first encoding network and the second encoding network based on the real speech sample to obtain a style encoder and a language encoder includes: Extracting style features of the real speech sample through a pre-trained first encoding network, and extracting language features of the real speech sample through a pre-trained second encoding network; Compressing the style features of the real speech sample and the language features of the real speech sample respectively to obtain compressed style features and language features; Based on the compressed style features and language features, performing iterative parameter training on the first encoding network and the second encoding network through a preset loss function; When the sum of the intra-subspace loss of the first encoding network, the intra-subspace loss of the second encoding network, and the cross-subspace loss between the first encoding network and the second encoding network reaches a preset convergence condition, a trained style encoder and language encoder are obtained.

3. The method according to claim 2, characterized in that The method of extracting the style features of the real speech sample by using a pre-trained first encoding network and extracting the language features of the real speech sample by using a pre-trained second encoding network includes: Cross-comparison and correlation coefficient calculation are performed on different network layers of the first coding network and the second coding network to obtain a correlation coefficient matrix between the first coding network and the second coding network; Based on the correlation coefficient matrix, identifying the style extraction network layer of the first encoding network and the language extraction network layer of the second encoding network, so that the style features extracted by the style extraction network layer and the language features extracted by the language extraction network layer are separated from each other; The style features of the real speech sample are extracted through the style extraction network layer of the first encoding network, and the language features of the real speech sample are extracted through the language extraction network layer of the second encoding network.

4. The method according to claim 2, characterized in that: The loss function includes the intra-subspace loss of the first encoding network, the intra-subspace loss of the second encoding network, and the cross-subspace loss between the first encoding network and the second encoding network, wherein: The subspace loss of the first encoding network is composed of the temporal average dependency characteristics of the style features extracted by the first encoding network; The loss in the subspace of the second encoding network is composed of the time average dependency characteristics of the language features extracted by the second encoding network; The cross-subspace loss between the first encoding network and the second encoding network is composed of dependency features between style features and language features at time steps.

5. The method according to claim 1, characterized in that The preset classification model includes a first attention statistical pooling layer, a first multi-layer perceptron, a second attention statistical pooling layer, a second multi-layer perceptron and a classification head; the training method of the deep fake detection model includes: Extracting speech features of samples through the style encoder and the language encoder, the first attention statistical pooling layer, the first multi-layer perceptron, the second attention statistical pooling layer, and the second multi-layer perceptron, wherein the samples include real speech samples and deep fake samples; Inputting the speech features of the sample into the classification head, and iteratively jointly training the parameters of the first attention statistical pooling layer, the first multilayer perceptron, the second attention statistical pooling layer, the second multilayer perceptron, and the classification head based on the speech features of the sample and the sample label; When the binary cross entropy loss of the classification model reaches a preset convergence condition, the training of the classification model is stopped to obtain a trained deep fake detection model.

6. The method according to claim 5, characterized in that The extracting speech features of the sample through the style encoder and the language encoder, the first attention statistical pooling layer, the first multi-layer perceptron, the second attention statistical pooling layer and the second multi-layer perceptron includes: The style features and language features of the sample are extracted by the style encoder and the speech encoder to obtain high-dimensional style features and high-dimensional language features of the sample; Performing dimensionality reduction and transformation processing on the high-dimensional style features through the first attention statistical pooling layer and the first multi-layer perceptron to obtain a transformed style feature of the sample, and compressing the high-dimensional style features through the compression module in the style encoder to obtain a compressed style feature of the sample; Performing dimensionality reduction and transformation processing on the high-dimensional language features through the second attention statistical pooling layer and the second multi-layer perceptron to obtain transformed language features of the sample, and compressing the high-dimensional language features through the compression module in the language encoder to obtain compressed language features of the sample; The transformed style features, compressed style features, transformed language features and compressed language features of the sample are connected to obtain the speech features of the sample.

7. The method according to any one of claims 1 to 6, characterized in that: The deep forgery detection is performed on the speech to be tested by using the style encoder, the language encoder and the deep forgery detection model to obtain a deep forgery detection result of the speech to be tested, including: Extracting the style features and language features of the speech to be tested by the style encoder and the speech encoder to obtain high-dimensional style features and high-dimensional language features of the speech to be tested; Through the first attention statistical pooling layer, the first multi-layer perception, the second attention statistical pooling layer and the second multi-layer perception in the deep fake detection model, the high-dimensional style features and the high-dimensional language features of the speech to be tested are reduced in dimension, transformed and connected to obtain the synthetic features of the speech to be tested; The synthetic features of the speech to be tested are input into the classification head in the deep fake detection model to obtain the deep fake detection result of the speech to be tested.

8. An audio deep fake detection device, characterized in that: The device comprises: The coding model training module is used to obtain real speech samples, and based on the real speech samples, perform self-supervisory training on the pre-trained first coding network and the second coding network to obtain a style encoder and a language encoder; A speech feature extraction module, used to extract features of the real speech sample and the deep fake sample through the style encoder and the language encoder to obtain speech features of the real speech sample and speech features of the deep fake sample; A detection model training module, configured to perform supervised training on a preset classification model based on the voice features of the real voice sample and the voice features of the deep fake voice sample and preset sample labels to obtain a deep fake detection model; The deep fake detection module is used to obtain the voice to be tested, and perform deep fake detection on the voice to be tested through the style encoder, language encoder and deep fake detection model to obtain the deep fake detection result of the voice to be tested.

9. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer device comprising a storage medium, a processor, and a computer program stored in the storage medium and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

Images