A data processing method and device

Abstract

The embodiment of the present application discloses a data processing method and device, wherein the method includes the following steps: obtaining a target audio file; performing audio segmentation on the target audio file to generate at least one audio segment; using a preset feature extraction network model to embed learning of the features of the audio segment to obtain an embedded feature vector of the audio segment; searching the embedded feature vector of the at least one audio segment through an audio retrieval library to generate a retrieval result of the target audio file; and analyzing the originality of the target audio file according to the retrieval result. The present application can improve the efficiency and accuracy of originality recognition of audio files.

Description

Technical Field

[0001] The present application relates to the field of Internet technology, and in particular to a data processing method and device. Background Art

[0002] In terms of original audio protection, currently, it is mainly judged by manual detection and the use of audio similarity. However, in the existing technology, the method based on manual review and identification requires a lot of labor costs and time costs in the massive audio and video environment of Internet products, which is inefficient. At the same time, the existing audio similarity judgment method based on audio clustering mainly uses pitch features, Mel Frequency Cepstral Coefficient Features (MFCC), energy, amplitude or envelope features as audio fingerprints. The accuracy of audio recognition is affected by audio feature extraction and clustering. In the clustering process, some audios are very similar to each other and cannot be well clustered, which will lead to incorrect judgments on the originality of audio files. Summary of the invention

[0003] The embodiments of the present application provide a data processing method and device, which can improve the efficiency and accuracy of originality recognition of audio files.

[0004] On the one hand, an embodiment of the present application provides a data processing method, which may include:

[0005] Get the target audio file;

[0006] Performing audio segmentation on the target audio file to generate at least one audio segment;

[0007] Using a preset feature extraction network model to embed features of the audio clip, so as to obtain an embedded feature vector of the audio clip;

[0008] Retrieving the embedded feature vector of at least one audio clip through an audio retrieval library to generate a retrieval result of a target audio file;

[0009] Analyze the originality of the target audio file based on the retrieval results.

[0010] In a feasible implementation manner, the data processing method further includes:

[0011] Performing feature extraction on at least one original audio file to generate an embedded feature vector for each original audio file;

[0012] The original audio file and the embedded feature vector corresponding to the original audio file are stored in an audio retrieval library.

[0013] In a feasible implementation manner, performing audio segmentation on a target audio file to generate at least one audio segment includes:

[0014] Determine a silent segment in the target audio file, and cut the silent segment to obtain at least one non-silent segment;

[0015] At least one non-silent segment is segmented and filtered according to the segmentation duration to obtain at least one audio segment.

[0016] In a feasible implementation, extracting the embedded feature vector of each audio segment in at least one audio segment includes:

[0017] Performing frame processing on the audio clip, and inputting the frame-processed audio clip into an audio frame processing layer of a feature extraction network model to extract a first feature corresponding to the audio clip;

[0018] Input the first feature into the random pooling layer of the feature extraction network model, obtain the mean vector and the standard deviation vector of the first feature, and generate the second feature according to the mean vector and the standard deviation vector of the first feature;

[0019] The second feature is input into the audio segment processing layer of the feature extraction network model to obtain an embedded feature vector corresponding to the audio segment.

[0020] In a feasible implementation, the search result includes: a hit audio file corresponding to the audio segment, a hit segment corresponding to the audio segment in the hit audio file, and a similarity score between the hit audio file and the audio segment.

[0021] In a feasible implementation, analyzing the originality of the target audio file according to the search results includes:

[0022] Determine similar audio files to the target audio file among the hit audio files of the search results;

[0023] The originality of the target audio file is analyzed based on the similarity scores between the audio segments in the target audio file and similar audio files and the hit segments corresponding to the audio segments in the similar audio files.

[0024] In a feasible implementation, the originality of the target audio file is analyzed according to the similarity scores between the audio segments in the target audio file and the similar audio files and the hit segments corresponding to the audio segments in the similar audio files, including:

[0025] Calculating the average of the similarity scores of at least one audio segment in the target audio file to obtain the average score;

[0026] If the score mean is greater than or equal to the first threshold, the target audio file is determined to be non-original audio;

[0027] If the score mean is less than the first threshold and greater than or equal to the second threshold, the number of hit segments corresponding to the audio segments in the similar audio files is counted; if the ratio of the number to the total amount of audio segments included in the target audio file is greater than or equal to the ratio threshold, the target audio file is determined to be non-original audio; if the ratio of the number to the total amount of audio segments included in the target audio file is less than the ratio threshold, the target audio file is determined to be suspected non-original audio;

[0028] If the score mean is less than the second threshold and greater than or equal to the third threshold, the target audio file is determined to be suspected non-original audio;

[0029] If the average score is less than the third threshold, the target audio file is determined to be original audio.

[0030] In a feasible implementation manner, the data processing method further includes:

[0031] If the target audio file is an original audio file, the target audio file and the embedded feature vector corresponding to the target audio file are stored in an audio retrieval library.

[0032] On the one hand, an embodiment of the present application provides a data processing device, which may include:

[0033] An audio acquisition unit, used to acquire a target audio file;

[0034] An audio segmentation unit, used to perform audio segmentation on a target audio file to generate at least one audio segment;

[0035] A feature extraction unit, used to embed features of the audio segment using a preset feature extraction network model to obtain an embedded feature vector of the audio segment;

[0036] An information retrieval unit, used to retrieve the embedded feature vector of at least one audio segment through an audio retrieval library to generate a retrieval result of a target audio file;

[0037] The audio analysis unit is used to analyze the originality of the target audio file according to the retrieval results.

[0038] In a feasible implementation manner, the data processing device further includes:

[0039] A retrieval library creation unit, used for extracting features from at least one original audio file and generating an embedded feature vector for each original audio file;

[0040] The original audio file and the embedded feature vector corresponding to the original audio file are stored in an audio retrieval library.

[0041] In a feasible implementation manner, the audio segmentation unit is specifically used for:

[0042] Determine a silent segment in the target audio file, and cut the silent segment to obtain at least one non-silent segment;

[0043] At least one non-silent segment is segmented and filtered according to the segmentation duration to obtain at least one audio segment.

[0044] In a feasible implementation manner, the feature extraction unit is specifically used for:

[0045] Performing frame processing on the audio clip, and inputting the frame-processed audio clip into an audio frame processing layer of a feature extraction network model to extract a first feature corresponding to the audio clip;

[0046] Input the first feature into the random pooling layer of the feature extraction network model, obtain the mean vector and the standard deviation vector of the first feature, and generate the second feature according to the mean vector and the standard deviation vector of the first feature;

[0047] The second feature is input into the audio segment processing layer of the feature extraction network model to obtain an embedded feature vector corresponding to the audio segment.

[0048] In a feasible implementation, the search result includes: a hit audio file corresponding to the audio segment, a hit segment corresponding to the audio segment in the hit audio file, and a similarity score between the hit audio file and the audio segment.

[0049] In a feasible implementation manner, the audio analysis unit includes:

[0050] A similar audio determination subunit is used to determine similar audio files of the target audio file among the hit audio files of the search results;

[0051] The audio analysis subunit is used to analyze the originality of the target audio file based on the similarity scores between the audio segments in the target audio file and similar audio files and the hit segments corresponding to the audio segments in the similar audio files.

[0052] In a feasible implementation manner, the audio analysis subunit is specifically used for:

[0053] Calculating the average of the similarity scores of at least one audio segment in the target audio file to obtain the average score;

[0054] If the score mean is greater than or equal to the first threshold, the target audio file is determined to be non-original audio;

[0055] If the score mean is less than the first threshold and greater than or equal to the second threshold, the number of hit segments corresponding to the audio segments in the similar audio files is counted; if the ratio of the number to the total amount of audio segments included in the target audio file is greater than or equal to the ratio threshold, the target audio file is determined to be non-original audio; if the ratio of the number to the total amount of audio segments included in the target audio file is less than the ratio threshold, the target audio file is determined to be suspected non-original audio;

[0056] If the score mean is less than the second threshold and greater than or equal to the third threshold, the target audio file is determined to be suspected non-original audio;

[0057] If the average score is less than the third threshold, the target audio file is determined to be original audio.

[0058] In a feasible implementation manner, the data processing device further includes:

[0059] The audio storage unit is used to store the target audio file and the embedded feature vector corresponding to the target audio file into the audio retrieval library if the target audio file is an original audio file.

[0060] On one hand, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. The computer program is suitable for being loaded by a processor and executing the above-mentioned method steps.

[0061] On the one hand, an embodiment of the present application provides a computer device, including a processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the above-mentioned method steps.

[0062] In one aspect, the present application provides a computer program product or a computer program, wherein the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the above method steps.

[0063] In an embodiment of the present application, a target audio file is obtained, and the target audio file is further segmented to generate at least one audio segment, and the features of the audio segment are embedded and learned using a preset feature extraction network model to obtain an embedded feature vector of the audio segment, and the embedded feature vector of at least one audio segment is retrieved through an audio retrieval library to generate a retrieval result of the target audio file, and finally the originality of the audio file to be processed is analyzed based on the retrieval result. The use of the above embedded feature vector to retrieve audio files avoids the problems of low efficiency of manual review and incorrect judgment of the originality of similar audios, and improves the efficiency and accuracy of originality recognition of audio files. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0065] Figure 1 It is a system architecture diagram of data processing provided by an embodiment of the present application;

[0066] Figure 2 It is a flowchart of a data processing method provided in an embodiment of the present application;

[0067] Figure 3 It is a flowchart of a data processing method provided in an embodiment of the present application;

[0068] Figure 4a This is an example schematic diagram of acquiring an embedded feature vector provided in an embodiment of the present application;

[0069] Figure 4b This is an example schematic diagram of data processing provided by an embodiment of the present application;

[0070] Figure 4c It is an example schematic diagram of an originality analysis provided in an embodiment of the present application;

[0071] Figure 5 is a structural schematic diagram of a data processing device provided in an embodiment of the present application;

[0072] Figure 6 It is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0073] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0074] See also Figure 1, is a system architecture diagram of data processing provided by an embodiment of the present invention. The server 10f establishes a connection with a user terminal cluster through a switch 10e and a communication bus 10d, and the user terminal cluster may include: user terminal 10a, user terminal 10b, ..., user terminal 10c. The database 10g stores an audio retrieval library, which includes multiple original audio files and embedded feature vectors corresponding to the original audio files. The database 10g also stores a feature extraction network model. The server 10f obtains the target audio file through the user terminal, further performs audio segmentation on the target audio file, generates at least one audio segment, and uses the feature extraction network model in the database 10g to embed and learn the features of the audio segment to obtain the embedded feature vector of the audio segment, and then retrieves the embedded feature vector of at least one audio segment through the audio retrieval library in the database 10g to generate the retrieval result of the target audio file, and finally analyzes the originality of the audio file to be processed according to the retrieval result.

[0075] The user terminals involved in the embodiments of the present application include: tablet computers, smart phones, personal computers (PCs), laptop computers, PDAs and other terminal devices.

[0076] Figure 2 A flow chart of a data processing method provided in an embodiment of the present application. Figure 2 As shown, the method of the embodiment of the present application may include the following steps S101-S105.

[0077] S101, obtaining a target audio file.

[0078] Specifically, the data processing method provided in this embodiment can be applied to a data processing device, where the data processing device obtains an audio file to be processed. The data processing device can be Figure 1 In the server 10f, the audio file to be processed is an audio file to be tested for originality, and the formats of the audio file include but are not limited to MP3, MIDI, WMA, etc. The audio file to be processed can also be an audio file extracted from a video file. Further, the data processing device pre-processes the audio file to be processed to generate a target audio file, wherein the pre-processing includes unifying the transcoding, sampling rate, encoding format, etc. of the audio file, format conversion, normalization of the audio channel, and coding modulation, etc. For example, audio files in MP3, MIDI or WMA formats are uniformly transcoded into WAV format, the audio file is encoded using pcm16bit little endian, the sampling rate of the audio file is adjusted to 16k, and the dual-channel audio is normalized to a single-channel audio.

[0079] S102: Perform audio segmentation on the target audio file to generate at least one audio segment.

[0080] Specifically, the data processing device performs audio segmentation on the target audio file to generate at least one audio segment. It is understandable that the data processing device determines the silent segments in the target audio file through voice endpoint detection technology, and cuts the target audio file into multiple non-silent segments at longer silent locations. The duration of the non-silent segments generated by the target audio file is generally longer. In general, a forced segmentation strategy will be further adopted for the non-silent segments to prevent the entire target audio file from failing to detect the voice endpoint under strong background music or noise interference. The specific segmentation strategy is to segment and filter at least one non-silent segment according to the segmentation duration to generate at least one audio segment. By segmenting an audio into multiple audio segments and screening the audio segments, the interference of noise on the feature extraction of the audio segment can be reduced in the subsequent feature extraction process. For example, the non-silent segment is segmented according to a duration of 3 seconds, and at the same time, the segments with a duration of less than three seconds are filtered, and finally multiple audio segments with a duration of 3 seconds are generated. It should be noted that different segmentation strategies can be adopted for different non-silent segments.

[0081] S103, using a preset feature extraction network model to perform embedding learning on the features of the audio clip to obtain an embedded feature vector of the audio clip.

[0082] Specifically, the data processing device obtains the embedded feature vector of each audio clip through the feature extraction network model. The feature extraction network model is a deep neural network, which includes, for example, an audio frame processing layer, a random pooling layer, and an audio segment processing layer. The first few layers of the feature extraction network model are audio frame processing layers, which are feature learning and expression at the frame level. The last few layers of the neural network of the feature extraction network model are audio segment processing layers, which are feature embedding learning at the audio segment level. The random pooling layer is used to calculate the mean and standard deviation of the frame level layer, and the mean and standard deviation vectors are spliced ​​together and input into the audio segment processing layer to realize the mapping of frame level layer features to segment level layer features, thereby obtaining the embedded feature vector of the audio segment. Through the learning of the frame level features of the audio and the embedding learning of the segment level features of the deep neural network, the acquired embedded feature vector can better express the features of the audio, thereby improving the accuracy of the subsequent use of the audio segment to identify whether the target audio file is original. It should be noted that the deep neural network trained with a large amount of audio data can also improve the recognition accuracy of other scenarios (such as re-recording original audio).

[0083] S104: searching the embedded feature vector of at least one audio segment through an audio retrieval library to generate a retrieval result of a target audio file.

[0084] Specifically, the data processing device obtains the hit audio file of the embedded feature vector from the audio retrieval library through the vector distance detection method. The vector distance can be specifically Euclidean distance or cosine distance. For example, the Euclidean distance is used for retrieval, that is, a preset number of audio clips with a smaller Euclidean distance to the embedded feature vector are retrieved from the audio retrieval library, specifically, the audio clip corresponding to the minimum Euclidean distance is retrieved. The audio file where the retrieved audio clip is located can be called a hit audio file, and the hit audio file can be any audio file in the audio retrieval library.

[0085] The above method is used to obtain the hit audio files corresponding to each audio clip in the target audio file, and the retrieval results include all hit audio files corresponding to the target audio file. It should be noted that for each audio clip, multiple hit audio files can be selected according to the same type of vector distance, or multiple groups of hit audio files can be selected using multiple types of vector distances. For example, Euclidean distance and cosine distance are used for retrieval at the same time, and 10 hit audio files are selected for each audio clip according to Euclidean distance, and 10 hit audio files are selected for each audio clip according to cosine distance, that is, 20 hit audio files are selected for each audio clip. Continuing to search through multiple distance judgment methods and synthesizing the retrieval results can reduce the impact of noise on the final results and improve noise resistance.

[0086] S105, analyzing the originality of the target audio file according to the search results.

[0087] Specifically, the search results include the hit audio files corresponding to the audio clips, the hit clips corresponding to the audio clips in the hit audio files, and the similarity scores between the hit audio files and the audio clips. The data processing device counts the hit times of all the hit audio files in the search results, determines the similar audio files of the target audio files in the hit audio files according to the hit times, and further analyzes the originality of the target audio files according to the similarity scores between the audio clips in the target audio files and the similar audio files and the hit clips corresponding to the audio clips in the similar audio files. The originality of the target audio files includes non-original audio, suspected original audio and original audio, wherein the suspected original audio needs further detection to determine its originality, which can be manual detection. For example, on a music software platform, users can publish their own audio creations on the platform. The music software can use the above method to analyze the originality of the audio published by the user. If the audio published by the user is non-original audio, the audio is prohibited from being published on the platform. If the audio published by the user is original audio, the audio is allowed to be published on the platform. If the audio published by the user is suspected original audio, its originality is further determined by manual detection.

[0088] The embodiment of the present application can obtain a target audio file, further perform audio segmentation on the target audio file, generate at least one audio segment, and extract the embedded feature vector of each audio segment in the at least one audio segment, search the embedded feature vector through an audio retrieval library, generate a retrieval result of the target audio file, and finally analyze the originality of the target audio file based on the retrieval result. The embedded feature vector obtained by the above method can more accurately express the audio features of the target audio file. The above embedded feature vector is used to search the audio file, which avoids the problems of low efficiency of manual review and incorrect originality judgment of similar audio, and improves the efficiency and accuracy of originality recognition of audio files.

[0089] Figure 3 A flow chart of a data processing method provided in an embodiment of the present application. Figure 3 As shown, the method of the embodiment of the present application may include the following steps S201-S208.

[0090] S201, extracting features from at least one original audio file to generate an embedded feature vector for each original audio file; storing each original audio file and the embedded feature vector corresponding to the original audio file into an audio retrieval library.

[0091] Specifically, a data processing device obtains multiple original audio files, and obtains an embedded feature vector for each original audio file through a feature extraction network model. Each original audio file may correspond to multiple embedded feature vectors. Each original audio file and the embedded feature vector corresponding to the original audio file are stored in an audio retrieval library. Each audio file in the audio retrieval library has a unique identifier such as a digital number. The audio retrieval library is used to judge the originality of the target audio file.

[0092] S202, obtaining a target audio file.

[0093] S203: Perform audio segmentation on the target audio file to generate at least one audio segment.

[0094] S204, using a preset feature extraction network model to perform embedding learning on the features of the audio segment to obtain an embedded feature vector of the audio segment.

[0095] It should be noted that the description of steps S202-S204 can be found in the above Figure 2 The relevant descriptions of the corresponding embodiments are not repeated here.

[0096] In a specific embodiment, the feature extraction network model can be a neural network model, including convolutional neural networks (CNN), deep neural networks (DNN), recurrent neural networks (RNN), time-delay neural networks (TDNN), etc. Figure 4a , which is a schematic diagram of obtaining an embedded feature vector provided by an embodiment of the present application. Figure 4a As shown in Figure 2, the process of acquiring the embedded feature vector is as follows:

[0097] The target audio segment is processed by framing, that is, the target audio segment is divided into smaller audio frames (such as 30ms per frame), and then the features of each audio frame are extracted. For example, the Mel Frequency Cepstrum Coefficient of each audio frame can be extracted. Further, the features of each extracted audio frame are input into the audio frame processing layer of the feature extraction network model to extract the first feature corresponding to the target audio segment, where the first feature is a frame-level feature; further, the first feature is input into the random pooling layer of the feature extraction network model to calculate the mean vector and standard deviation vector of the first feature, and the mean vector and standard deviation vector of the first feature are concatenated to generate the second feature; further, the second feature is input into the audio segment processing layer of the feature extraction network model to obtain the embedded feature vector corresponding to the target audio segment. Specifically, the output of the second-to-last layer of the feature extraction network model can be used as the embedded feature vector corresponding to the target audio segment.

[0098] S205: Search the embedded feature vector of at least one audio segment through an audio retrieval library to generate a retrieval result of a target audio file.

[0099] It should be noted that the description of step S205 can refer to the above Figure 2 The relevant descriptions of the corresponding embodiments are not repeated here.

[0100] S206: Determine similar audio files to the target audio file from among the hit audio files in the search results.

[0101] Specifically, the data processing device counts the number of hits of all hit audio files in the retrieval results, and determines similar audio files of the target audio file based on the number of hits. For example, the target audio file has 30 audio clips, and each audio clip corresponds to 10 hit audio files, so there are a total of 300 hit audio files (including repeated hit audio files). The number of hits of all hit audio files in the retrieval results is counted, and the hit audio file with the most hits is used as a similar audio file of the target audio file. It should be noted that there can be multiple similar audio files. For example, the two hit audio files with the first and second hit times are selected as similar audio files of the target audio file. Further, the similarity score between each audio clip in the target audio file and the similar audio file and the hit clip corresponding to the audio clip in the similar audio file are determined. The similarity score can use vector distance. For example, if the Euclidean distance is used for retrieval, the similarity score is the Euclidean distance.

[0102] S207, analyzing the originality of the target audio file according to the similarity scores between the audio segments in the target audio file and the similar audio files and the hit segments corresponding to the audio segments in the similar audio files.

[0103] Specifically, the data processing device obtains the average score of the similarity score of each audio segment in the target audio file, and analyzes the originality of the target audio file.

[0104] See also Figure 4b , which is a schematic diagram of a method for analyzing the originality of an audio file provided in an embodiment of the present application, such as Figure 4b As shown, the data processing device obtains the audio file to be processed, and pre-processes the audio file to be processed to generate a target audio file. The pre-processing includes unifying the transcoding, sampling rate, encoding format, etc. of the audio file, as well as format conversion, normalization processing of the audio channel, encoding modulation, etc., and further performs audio segmentation on the target audio file to generate at least one audio segment, and extracts the embedded feature vector of each audio segment in the at least one audio segment through a feature extraction network model, and further obtains the hit audio file of the embedded feature vector from the audio retrieval library through vector distance detection, and counts the hit times of all hit audio files in the retrieval results, and determines similar audio files of the target audio file according to the hit times, as well as the similarity scores of each audio segment in the target audio file and the similar audio files and the hit segments corresponding to the audio segments in the similar audio files, and analyzes the originality of the target audio file according to the similarity scores and the hit segments.

[0105] See also Figure 4c , which is a schematic diagram of an originality analysis provided in an embodiment of the present application, such as Figure 4cAs shown, when the score mean is greater than or equal to a first threshold, the target audio file is determined to be non-original audio, and the first threshold is preset;

[0106] When the score mean is less than the first threshold and greater than or equal to the second threshold, the number of hit segments corresponding to the audio segments in the similar audio files is counted, and the ratio information of the number of audio segments in the target audio file is obtained. If the ratio information is greater than the ratio threshold, it means that the two audio files are similar, and then the target audio file is determined to be non-original audio. If the ratio information is less than the ratio threshold, the target audio file is determined to be suspected non-original audio, and the originality of the target audio file is reviewed. The second threshold and the ratio threshold are preset;

[0107] When the score mean is less than the second threshold and greater than or equal to the third threshold, the target audio file is determined to be suspected non-original audio, and the originality of the target audio file is reviewed, and the third threshold is preset;

[0108] When the average score is less than the third threshold, the target audio file is determined to be original audio.

[0109] S208: If the target audio file is an original audio file, the target audio file and the embedded feature vector corresponding to the target audio file are stored in an audio retrieval library.

[0110] Specifically, when it is determined that the target audio file is an original audio file, the target audio file can be entered into an audio retrieval library for originality analysis of the audio. Specifically, the target audio file and the embedded feature vector corresponding to the target audio file are input into the audio retrieval library. At the same time, an audio number of the target audio file is generated according to the numbering rules of the audio retrieval library. The audio number can carry the time of entry and the ranking of the audio entered into the library. The target audio file can also be hashed and the hash value of the audio file is used as the audio number, and the audio number is used as the audio identifier of the target audio file.

[0111] In an embodiment of the present application, by obtaining a target audio file, further performing audio segmentation on the target audio file, generating at least one audio segment, and extracting an embedded feature vector of each audio segment in the at least one audio segment, the embedded feature vector is searched through an audio search library to generate a search result for the target audio file, and finally the originality of the target audio file is analyzed based on the search result. The use of the above embedded feature vector to search audio files avoids the problems of low efficiency of manual review and incorrect originality judgment of similar audios, and improves the efficiency and accuracy of originality recognition of audio files.

[0112] See also Figure 5, which is a schematic diagram of the structure of a data processing device provided in an embodiment of the present application. The data processing device may be a computer program (including program code) running in a computer device, for example, the data processing device is an application software; the device may be used to execute the corresponding steps in the method provided in an embodiment of the present application. Figure 5 As shown, the data processing device 1 of the embodiment of the present application may include: an audio acquisition unit 11, an audio segmentation unit 12, a feature extraction unit 13, an information retrieval unit 14, and an audio analysis unit 15.

[0113] An audio acquisition unit 11 is used to acquire a target audio file;

[0114] The audio segmentation unit 12 is used to perform audio segmentation on the target audio file to generate at least one audio segment;

[0115] A feature extraction unit 13, configured to use a preset feature extraction network model to perform embedding learning on the features of the audio segment to obtain an embedded feature vector of the audio segment;

[0116] An information retrieval unit 14 is used to retrieve the embedded feature vector of at least one audio segment through an audio retrieval library to generate a retrieval result of a target audio file;

[0117] The audio analysis unit 15 is used to analyze the originality of the target audio file according to the search result.

[0118] See also Figure 5 , the data processing device 1 of the embodiment of the present application may further include: a search library creation unit 16;

[0119] A search library creation unit 16 is used to extract features from at least one original audio file and generate an embedded feature vector for each original audio file;

[0120] The original audio file and the embedded feature vector corresponding to the original audio file are stored in an audio retrieval library.

[0121] In a feasible implementation manner, the audio segmentation unit 12 is specifically used for:

[0122] Determine a silent segment in the target audio file, and cut the silent segment to obtain at least one non-silent segment;

[0123] At least one non-silent segment is segmented and filtered according to the segmentation duration to obtain at least one audio segment.

[0124] In a feasible implementation manner, the feature extraction unit 13 is specifically used for:

[0125] Performing frame processing on the audio clip, and inputting the frame-processed audio clip into an audio frame processing layer of a feature extraction network model to extract a first feature corresponding to the audio clip;

[0126] Input the first feature into the random pooling layer of the feature extraction network model, obtain the mean vector and the standard deviation vector of the first feature, and generate the second feature according to the mean vector and the standard deviation vector of the first feature;

[0127] The second feature is input into the audio segment processing layer of the feature extraction network model to obtain an embedded feature vector corresponding to the audio segment.

[0128] In a feasible implementation, the search result includes: a hit audio file corresponding to the audio segment, a hit segment corresponding to the audio segment in the hit audio file, and a similarity score between the hit audio file and the audio segment.

[0129] See also Figure 5 , the audio analysis unit 15 of the embodiment of the present application may include: a similar audio determination subunit 151 and an audio analysis subunit 152;

[0130] A similar audio determination subunit 151 is used to determine similar audio files to the target audio file among the hit audio files in the search results;

[0131] The audio analysis subunit 152 is used to analyze the originality of the target audio file according to the similarity scores between the audio segments in the target audio file and the similar audio files and the hit segments corresponding to the audio segments in the similar audio files.

[0132] In a feasible implementation manner, the audio analysis subunit 152 is specifically used for:

[0133] Calculating the average of the similarity scores of at least one audio segment in the target audio file to obtain the average score;

[0134] If the score mean is greater than or equal to the first threshold, the target audio file is determined to be non-original audio;

[0135] If the score mean is less than the first threshold and greater than or equal to the second threshold, the number of hit segments corresponding to the audio segments in the similar audio files is counted; if the ratio of the number to the total amount of audio segments included in the target audio file is greater than or equal to the ratio threshold, the target audio file is determined to be non-original audio; if the ratio of the number to the total amount of audio segments included in the target audio file is less than the ratio threshold, the target audio file is determined to be suspected non-original audio;

[0136] If the score mean is less than the second threshold and greater than or equal to the third threshold, the target audio file is determined to be suspected non-original audio;

[0137] If the average score is less than the third threshold, the target audio file is determined to be original audio.

[0138] See also Figure 5 , the data processing device 1 of the embodiment of the present application may further include: an audio storage unit 17;

[0139] The audio storage unit 17 is used to store the target audio file and the embedded feature vector corresponding to the target audio file into the audio retrieval library if the target audio file is an original audio file.

[0140] In an embodiment of the present application, a target audio file is obtained, and the target audio file is further segmented to generate at least one audio segment, and the features of the audio segment are embedded and learned using a preset feature extraction network model to obtain an embedded feature vector of the audio segment, and the embedded feature vector of at least one audio segment is retrieved through an audio retrieval library to generate a retrieval result of the target audio file, and finally the originality of the audio file to be processed is analyzed based on the retrieval result. The use of the above embedded feature vector to retrieve audio files avoids the problems of low efficiency of manual review and incorrect judgment of the originality of similar audio, and improves the efficiency and accuracy of originality recognition of audio files.

[0141] See also Figure 6 , which is a schematic diagram of the structure of a computer device according to an embodiment of the present application. Figure 6 As shown, the computer device 1000 may include: at least one processor 1001, such as a CPU, at least one network interface 1004, a user interface 1003, a memory 1005, and at least one communication bus 1002. The communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), and the user interface 1003 may optionally include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a random access memory (Random Access Memory, RAM) or a non-volatile memory (non-volatile memory, NVM), such as at least one disk storage. The memory 1005 may optionally be at least one storage device located away from the aforementioned processor 1001. As Figure 6 As shown, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a data processing application program.

[0142] exist Figure 6In the computer device 1000 shown in the figure, the network interface 1004 can provide a network communication function, the user interface 1003 is mainly used to provide an input interface for the user; and the processor 1001 can be used to call the data processing application stored in the memory 1005 to implement the above Figure 2-4c The description of the data processing method in any corresponding embodiment will not be repeated here.

[0143] It should be understood that the computer device 1000 described in the embodiment of the present application can execute the above Figure 2-4c The description of the data processing method in any corresponding embodiment can also be performed as described above. Figure 5 The description of the data processing device in the corresponding embodiment will not be repeated here. In addition, the description of the beneficial effects of the same method will not be repeated here.

[0144] In addition, it should be pointed out here that: the embodiment of the present application also provides a computer-readable storage medium, and the computer-readable storage medium stores a computer program executed by the data processing device mentioned above, and the computer program includes program instructions. When the processor executes the program instructions, it can execute the above-mentioned Figure 2-4c The description of the data processing method in any corresponding embodiment will not be repeated here. In addition, the description of the beneficial effects of the same method will not be repeated. For technical details not disclosed in the computer-readable storage medium embodiment involved in this application, please refer to the description of the method embodiment of this application. As an example, program instructions may be deployed to be executed on one computing device, or on multiple computing devices located at one location, or on multiple computing devices distributed at multiple locations and interconnected by a communication network. Multiple computing devices distributed at multiple locations and interconnected by a communication network can constitute a blockchain system.

[0145] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, an NVM, or a RAM, etc.

[0146] The above disclosure is only the preferred embodiment of the present application, which certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A data processing method, characterized in that: include: Get the target audio file; Performing audio segmentation on the target audio file to generate at least one audio segment; Using a preset feature extraction network model to perform embedding learning on the features of the audio clip to obtain an embedded feature vector of the audio clip; Retrieving the embedded feature vector of the at least one audio segment from an audio retrieval library by a vector distance detection method to generate a retrieval result of the target audio file; the retrieval result includes a hit audio file corresponding to the audio segment, a hit segment corresponding to the audio segment in the hit audio file, and a similarity score between the hit audio file and the audio segment; The number of hits of all the hit audio files in the retrieval results is counted, similar audio files of the target audio file are determined in the hit audio files according to the number of hits, and the originality of the target audio file is analyzed according to the similarity scores of the audio segments in the target audio file and the similar audio files and the hit segments corresponding to the audio segments in the similar audio files.

2. The method according to claim 1, characterized in that Also includes: Performing feature extraction on at least one original audio file to generate an embedded feature vector for each original audio file; The original audio file and the embedded feature vector corresponding to the original audio file are stored in an audio retrieval library.

3. The method according to claim 1, characterized in that The step of performing audio segmentation on the target audio file to generate at least one audio segment comprises: Determine a silent segment in the target audio file, and cut the silent segment to obtain at least one non-silent segment; The at least one non-silent segment is segmented and filtered according to the segmentation duration to obtain at least one audio segment.

4. The method according to claim 1, characterized in that The extracting the embedded feature vector of each audio segment in the at least one audio segment comprises: Performing frame processing on the audio clip, and inputting the frame-processed audio clip into an audio frame processing layer of a feature extraction network model to extract a first feature corresponding to the audio clip; Inputting the first feature into a random pooling layer of a feature extraction network model, obtaining a mean vector and a standard deviation vector of the first feature, and generating a second feature according to the mean vector and the standard deviation vector of the first feature; The second feature is input into the audio segment processing layer of the feature extraction network model to obtain an embedded feature vector corresponding to the audio segment.

5. The method according to claim 1, characterized in that The analyzing the originality of the target audio file according to the similarity score between the audio segment in the target audio file and the similar audio file and the hit segment corresponding to the audio segment in the similar audio file comprises: Calculating the mean of the similarity scores of the at least one audio segment in the target audio file to obtain a mean score; If the score average is greater than or equal to a first threshold, the target audio file is determined to be non-original audio; If the score mean is less than the first threshold and greater than or equal to the second threshold, count the number of hit segments corresponding to the audio segments in the similar audio file; if the ratio of the number to the total amount of audio segments included in the target audio file is greater than or equal to the ratio threshold, determine the target audio file as non-original audio; if the ratio of the number to the total amount of audio segments included in the target audio file is less than the ratio threshold, determine the target audio file as suspected non-original audio; If the score mean is less than the second threshold and greater than or equal to the third threshold, the target audio file is determined to be a suspected non-original audio; If the score average is less than a third threshold, the target audio file is determined to be original audio.

6. The method according to claim 1, characterized in that Also includes: If the target audio file is an original audio file, the target audio file and the embedded feature vector corresponding to the target audio file are stored in an audio retrieval library.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor and executing the method according to any one of claims 1 to 6.

8. A computer device, characterized in that: include: A processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the method according to any one of claims 1 to 6.

Images