Voice detection method and device, electronic equipment and storage medium

Abstract

The invention provides a voice detection method and device, electronic equipment and a storage medium, and belongs to the technical field of financial science and technology. The method comprises the following steps: in response to a detection request for target voice of a target object, obtaining reference voice of the target object; performing voiceprint detection on the reference voice and the target voice based on a voiceprint detection model to obtain a voiceprint detection result; if the voiceprint detection result indicates that the voiceprint feature of the target voice is the same as the voiceprint feature of the reference voice, coding the target voice and the reference voice based on a preset alteration detection model to obtain a target voice coding vector of the target voice and a reference voice coding vector of the reference voice; splicing the target voice coding vector and the reference voice coding vector to obtain a voice splicing feature vector; and based on the voice splicing feature vector, performing alteration detection through the alteration detection model to obtain an alteration detection result. According to the invention, the detection accuracy of altered voice can be improved.

Description

Technical Field

[0001] The present application relates to the field of financial technology, and in particular to a voice detection method and device, an electronic device and a storage medium. Background Art

[0002] Voice alteration technology refers to the use of advanced speech synthesis and conversion technology to synthesize or modify audio content to simulate the voice characteristics of a specific person. This technology has a wide range of application scenarios. It can be used to create the voice of virtual characters in multimedia scenarios, and it may also be used maliciously to create false information or conduct improper resource interaction activities. Therefore, in various business scenarios, it is often necessary to detect the spoken voice provided by the user to determine whether the spoken voice is altered.

[0003] At present, the relevant technology often adopts a method of combining feature extraction and classifier to perform speech detection. This method relies to a large extent on the accuracy and comprehensiveness of the extracted audio features. Once the extracted audio features are partially missing, the accuracy of the alteration detection results will be reduced. Summary of the invention

[0004] The main purpose of the embodiments of the present application is to propose a speech detection method and device, an electronic device and a storage medium, aiming to improve the accuracy of detecting altered speech.

[0005] To achieve the above object, a first aspect of an embodiment of the present application provides a speech detection method, the speech detection method comprising:

[0006] In response to a detection request for a target speech of a target object, obtaining a reference speech of the target object;

[0007] Performing voiceprint detection on the reference voice and the target voice based on a preset voiceprint detection model to obtain a voiceprint detection result, wherein the voiceprint detection result is used to indicate whether the voiceprint feature of the target voice is the same as the voiceprint feature of the reference voice;

[0008] If the voiceprint detection result indicates that the voiceprint feature of the target voice is the same as the voiceprint feature of the reference voice, encoding the target voice and the reference voice respectively based on a preset alteration detection model to obtain a target voice encoding vector of the target voice and a reference voice encoding vector of the reference voice;

[0009] Performing concatenation processing on the target speech coding vector and the reference speech coding vector to obtain a speech concatenation feature vector;

[0010] Based on the speech concatenation feature vector, alteration detection is performed through a preset alteration detection model to obtain an alteration detection result, wherein the alteration detection result is used to indicate whether the target speech is altered.

[0011] In some embodiments, the voiceprint detection model includes a feature extraction network and a detection network; the voiceprint detection is performed on the reference voice and the target voice based on the preset voiceprint detection model to obtain the voiceprint detection result, including:

[0012] Extracting features of the reference speech based on the feature extraction network to obtain reference speech voiceprint features;

[0013] Extracting features of the target speech based on the feature extraction network to obtain voiceprint features of the target speech;

[0014] Based on the detection network, a similarity detection is performed on the target speech voiceprint feature and the reference speech voiceprint feature to obtain a feature similarity score;

[0015] Based on the comparison between the feature similarity score and a preset score threshold, a voiceprint detection result is obtained.

[0016] In some embodiments, performing alteration detection based on the speech concatenation feature vector by using a preset alteration detection model to obtain an alteration detection result includes:

[0017] Performing Fourier transform on the speech concatenation feature vector based on the alteration detection model to obtain a speech frequency domain feature vector;

[0018] Performing a first attention calculation on the speech frequency domain feature vector to obtain a first attention feature vector;

[0019] Performing an inverse Fourier transform on the first attention feature vector to obtain a speech time domain feature vector;

[0020] Performing a second attention calculation on the speech frequency domain feature vector to obtain a second attention feature vector;

[0021] The second attention feature vector is subjected to alteration detection based on a preset function to obtain the alteration detection result.

[0022] In some embodiments, performing alteration detection on the second attention feature vector based on a preset function to obtain the alteration detection result includes:

[0023] Performing alteration detection on the second attention feature vector based on a preset function to obtain a speech alteration score;

[0024] Normalizing the speech change score to obtain a speech change probability;

[0025] If it is determined that the voice alteration probability is greater than a preset threshold, the alteration detection result is determined as the target voice being altered.

[0026] In some embodiments, the alteration detection model is trained in the following manner:

[0027] Acquire a positive sample speech pair and a negative sample speech pair of a sample object, wherein the positive sample speech pair is composed of a reference speech segment and a sample real speech segment of the sample object, and the negative sample speech pair is composed of a reference speech segment and a sample altered speech segment of the sample object;

[0028] Inputting the positive sample speech pair into the original detection model for alteration detection to obtain a positive sample alteration detection result, and inputting the negative sample speech pair into the original detection model for alteration detection to obtain a negative sample alteration detection result;

[0029] Calculating a first loss function based on the positive sample alteration detection result and the negative sample alteration detection result;

[0030] Based on the first loss function, the original detection model is trained to obtain the altered detection model.

[0031] In some embodiments, the voiceprint detection model is trained in the following manner:

[0032] Obtain sample speech data of a sample object;

[0033] Extracting features from the sample speech data to obtain sample speech features;

[0034] Performing data augmentation on the sample speech feature to obtain augmented speech feature;

[0035] Performing voiceprint detection on the sample voice feature and the pre-acquired reference voice feature based on the original model to obtain a first voiceprint similarity score;

[0036] Performing voiceprint detection on the augmented speech feature and the reference speech feature based on the original model to obtain a second voiceprint similarity score;

[0037] Based on the first voiceprint similarity score and the second voiceprint similarity score, the original model is trained to obtain the voiceprint detection model.

[0038] In some embodiments, in response to a detection request for a target speech of a target object, obtaining a reference speech of the target object includes:

[0039] Parsing the detection request to obtain a target object identifier of the target object;

[0040] Acquire a preset reference voice database, wherein the reference voice database includes a plurality of reference voices, candidate object identifiers of a plurality of candidate objects, and a correspondence between the candidate object identifiers and the reference voices;

[0041] A voice query is performed in the reference voice database based on the target object identifier, and a reference voice corresponding to the candidate object identifier consistent with the target object identifier is determined as the reference voice.

[0042] To achieve the above object, a second aspect of an embodiment of the present application provides a speech detection device, the speech detection device comprising:

[0043] an acquisition unit, configured to acquire a reference voice of the target object in response to a detection request for the target voice of the target object;

[0044] a first detection unit, configured to perform voiceprint detection on the reference voice and the target voice based on a preset voiceprint detection model to obtain a voiceprint detection result, wherein the voiceprint detection result is used to indicate whether the voiceprint feature of the target voice is the same as the voiceprint feature of the reference voice;

[0045] an encoding unit, configured to encode the target voice and the reference voice based on a preset alteration detection model to obtain a target voice encoding vector of the target voice and a reference voice encoding vector of the reference voice, respectively, if the voiceprint detection result indicates that the voiceprint feature of the target voice is the same as the voiceprint feature of the reference voice;

[0046] A concatenation unit, configured to concatenate the target speech coding vector and the reference speech coding vector to obtain a speech concatenation feature vector;

[0047] The second detection unit is used to perform alteration detection based on the speech concatenation feature vector and a preset alteration detection model to obtain an alteration detection result, wherein the alteration detection result is used to indicate whether the target speech is altered.

[0048] To achieve the above-mentioned purpose, the third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the speech detection method described in the first aspect is implemented.

[0049] To achieve the above objectives, the fourth aspect of an embodiment of the present application proposes a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the speech detection method described in the first aspect.

[0050] The speech detection method and device, electronic device and storage medium proposed in the present application obtain the reference speech of the target object by responding to the detection request of the target speech of the target object, firstly perform voiceprint detection on the reference speech and the target speech based on the preset voiceprint detection model to obtain the voiceprint detection result, wherein the voiceprint detection result is used to indicate whether the voiceprint feature of the target speech is the same as the voiceprint feature of the reference speech, so as to determine whether the target speech belongs to the claimed target object according to the comparison of the voiceprint features between the reference speech and the target speech, so as to realize the detection of the speaking object before the alteration detection. Further, if the voiceprint detection result indicates that the voiceprint feature of the target speech is the same as the voiceprint feature of the reference speech, then the target speech and the reference speech are respectively encoded based on the preset alteration detection model to obtain the target speech encoding vector of the target speech and the reference speech encoding vector of the reference speech, and the target speech encoding vector and the reference speech encoding vector are spliced ​​to obtain the speech splicing feature vector, so that the speech splicing feature vector contains the speech feature information of the target speech and the reference speech at the same time, thereby improving the feature comprehensiveness of the speech splicing feature vector. Finally, based on the speech splicing feature vector, alteration detection is performed through the preset alteration detection model to obtain the alteration detection result. This method takes into account the simultaneous processing of the speech feature information of the target speech and the reference speech in alteration detection, which can improve the detection accuracy of altered speech. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 is a flow chart of a speech detection method provided in an embodiment of the present application;

[0052] Figure 2 yes Figure 1 Flow chart of step S101 in FIG.

[0053] Figure 3 is a flow chart of a training voiceprint detection model according to an embodiment of the present application;

[0054] Figure 4 yes Figure 1 Flow chart of step S102 in FIG.

[0055] Figure 5 This is a flowchart of the training alteration detection model of the embodiment of the present application.

[0056] Figure 6 yes Figure 1 Flow chart of step S105 in FIG.

[0057] Figure 7A-7B Detailed diagram of the implementation of the speech detection method provided in the embodiment of the present application;

[0058] Figure 8 is a structural diagram of a speech detection device provided in an embodiment of the present application;

[0059] Fig. 9 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0061] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0062] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0063] The speech detection method and device, electronic device and storage medium provided in the embodiments of the present application are specifically described through the following embodiments. First, the speech detection method in the embodiments of the present application is described.

[0064] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0065] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0066] The speech detection method provided in the embodiment of the present application relates to the fields of financial technology and artificial intelligence technology. The speech detection method provided in the embodiment of the present application can be applied to a terminal, can be applied to a server side, or can be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or can be configured as a server cluster or a distributed system composed of multiple physical servers, and can also be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the speech detection method, etc., but is not limited to the above forms.

[0067] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0068] It should be noted that in each specific implementation of the present application, when it comes to the need to perform relevant processing based on data related to the identity or characteristics of the object, such as object information, object behavior data, object historical data, and object location information, the permission or consent of the object will be obtained first, and the collection, use, and processing of these data will comply with relevant laws, regulations, and standards. In addition, when the embodiment of the present application needs to obtain the personal information of the object, the separate permission or separate consent of the object will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the separate permission or separate consent of the object, the necessary object-related data for the normal operation of the embodiment of the present application will be obtained.

[0069] Figure 1 is an optional flow chart of the voice detection method provided in the embodiment of the present application. Figure 1 The method may include but is not limited to steps S101 to S105.

[0070] Step S101, in response to a detection request for a target voice of a target object, obtaining a reference voice of the target object;

[0071] Step S102, performing voiceprint detection on the reference voice and the target voice based on a preset voiceprint detection model to obtain a voiceprint detection result;

[0072] Step S103, if the voiceprint detection result indicates that the voiceprint feature of the target voice is the same as the voiceprint feature of the reference voice, the target voice and the reference voice are respectively encoded based on a preset alteration detection model to obtain a target voice encoding vector of the target voice and a reference voice encoding vector of the reference voice;

[0073] Step S104, concatenating the target speech coding vector and the reference speech coding vector to obtain a speech concatenation feature vector;

[0074] Step S105 , based on the speech concatenation feature vector, alteration detection is performed using a preset alteration detection model to obtain an alteration detection result.

[0075] It should be noted that the voiceprint detection result is used to indicate whether the voiceprint features of the target voice are the same as the voiceprint features of the reference voice. The alteration detection result is used to indicate whether the target voice is altered.

[0076] In the steps S101 to S105 shown in the embodiment of the present application, by responding to the detection request for the target voice of the target object, the reference voice of the target object is obtained, and the reference voice and the target voice are firstly detected based on the preset voiceprint detection model to obtain the voiceprint detection result, wherein the voiceprint detection result is used to indicate whether the voiceprint feature of the target voice is the same as the voiceprint feature of the reference voice, so as to determine whether the target voice belongs to the claimed target object according to the comparison of the voiceprint features between the reference voice and the target voice, so as to realize the detection of the speaking object before the alteration detection. Further, if the voiceprint detection result indicates that the voiceprint feature of the target voice is the same as the voiceprint feature of the reference voice, the target voice and the reference voice are respectively encoded based on the preset alteration detection model to obtain the target voice encoding vector of the target voice and the reference voice encoding vector of the reference voice, and the target voice encoding vector and the reference voice encoding vector are spliced ​​to obtain the voice splicing feature vector, so that the voice splicing feature vector contains the voice feature information of the target voice and the reference voice at the same time, thereby improving the feature comprehensiveness of the voice splicing feature vector. Finally, based on the speech splicing feature vector, alteration detection is performed through the preset alteration detection model to obtain the alteration detection result. This method takes into account the simultaneous processing of the speech feature information of the target speech and the reference speech in alteration detection, which can improve the detection accuracy of altered speech.

[0077] In step S101 of some embodiments, in response to a detection request for a target voice of a target object, a reference voice of the target object is acquired.

[0078] The target object refers to the speaker to whom the target speech is claimed to belong.

[0079] The target speech refers to a speech segment formed by a speaker speaking or describing.

[0080] The detection request refers to a request generated when speaker recognition and alteration detection are required for the provided target speech.

[0081] Reference speech refers to speech segments originating from the target subject and used as references in speaker recognition and alteration detection.

[0082] For example, in a certain financial business scenario, when object A handles a certain financial business on a financial platform, it provides a demand description in the form of voice, "I am object A, please open xx financial service for me". At this time, the business end of the financial platform will receive the demand description in the form of voice, and obtain the target voice "I am object A, please open xx financial service for me" according to the demand description. The target object is object A. Then, the business will assemble the target object and the target voice into a detection request and send it to the server. Based on this, the server will judge whether the speaker of the target voice is really object A according to the voice detection method of the embodiment of the present application, and when it is determined that the speaker of the target voice is object A, it will judge whether the target voice is a false voice created by other objects, and not generated by object A.

[0083] See also Figure 2 In some embodiments, step S101 may include but is not limited to steps S201 to S203:

[0084] Step S201, parsing the detection request to obtain a target object identifier of the target object;

[0085] Step S202, obtaining a preset reference speech database;

[0086] Step S203: perform a voice query in a reference voice database based on the target object identifier, and determine the reference voice corresponding to the candidate object identifier that is consistent with the target object identifier as the reference voice.

[0087] Steps S201 to S203 are described in detail below.

[0088] In step S201 of some embodiments, the target object identifier is used to identify the target object to distinguish the target object from other objects.

[0089] In the specific implementation of this embodiment, since the detection request will indicate the speaker to whom the target voice claims to belong, the detection request can be parsed by a script program to obtain a code segment in the detection request for identifying the speaker to which the target voice claims to belong, and the target object identifier of the target object can be extracted from the code segment.

[0090] In step S202 of some embodiments, the reference voice database refers to a storage space for storing the reference voices of various objects that have been registered on various platforms or applications associated with the server. The reference voice database includes multiple reference voices, candidate object identifiers of multiple candidate objects, and a correspondence between the candidate object identifiers and the reference voices. The candidate objects refer to various objects that have been registered, and the reference voices are the voice data provided by each candidate object during registration and verified.

[0091] For example, when a certain financial platform applies the voice detection method of the present application, the server corresponding to the financial platform will maintain a reference voice provided by each object that has legally registered on the financial platform when registering again.

[0092] In the specific implementation of this embodiment, since the reference voice database is maintained and continuously updated by the server, based on this, with authorization, the preset reference voice database can be called from the background of the server, and the benchmark voices of each candidate object stored in the reference voice database can be obtained.

[0093] For example, when constructing a reference voice database, for each candidate object registered, when the candidate object is registered, the candidate object is required to provide a valid voice of more than 10 seconds with clear pronunciation as a reference voice, and the candidate object identifier generated by the candidate object when registering or provided by the candidate object is used as a keyword, and the reference voice is used as a value to construct a key-value pair to form a mapping relationship between the candidate object identifier and the reference voice of each candidate object, and this mapping relationship and the reference voice are stored in the reference voice database. Among them, the candidate object identifier of each candidate object is unique.

[0094] In step S203 of some embodiments, when performing a voice query in the reference voice database based on the target object identifier, the target object identifier is compared with each candidate object identifier in the reference voice database. Then, if the target object identifier is consistent with the candidate object identifier, it indicates that the target object identifier and the candidate object identifier indicate the same speaking object, and therefore, the reference voice corresponding to the candidate object identifier consistent with the target object identifier is determined as the reference voice.

[0095] The advantage of this embodiment is that when a detection request is received, the target object identifier of the target object is first parsed from the detection request, and a voice query is performed in the reference voice database based on the target object identifier. Since there is a mapping relationship between the various benchmark voices and candidate object identifiers in the reference voice database, the candidate object identifier that is consistent with the target object identifier can be found more quickly and accurately, thereby extracting the benchmark voice corresponding to the candidate object identifier that is consistent with the target object identifier as the reference voice. This method can improve the query efficiency and query accuracy of the reference voice.

[0096] See also Figure 3 In some embodiments, the specific process of training the voiceprint detection model may include but is not limited to steps S301 to S306:

[0097] Step S301, obtaining sample speech data of a sample object;

[0098] Step S302, extracting features from the sample speech data to obtain sample speech features;

[0099] Step S303, performing data augmentation on the sample speech features to obtain augmented speech features;

[0100] Step S304, performing voiceprint detection on the sample speech feature and the pre-acquired reference speech feature based on the original model to obtain a first voiceprint similarity score;

[0101] Step S305, performing voiceprint detection on the augmented speech feature and the reference speech feature based on the original model to obtain a second voiceprint similarity score;

[0102] Step S306: Based on the first voiceprint similarity score and the second voiceprint similarity score, the original model is trained to obtain a voiceprint detection model.

[0103] Steps S301 to S306 are described in detail below.

[0104] In step S301 of some embodiments, when obtaining sample voice data of a sample object, the audio or video of the sample object performing singing, speaking or chatting can be obtained with authorization, and a short audio segment can be captured from the obtained audio or video as sample voice data, wherein the voice duration of the sample voice data can be set according to actual training needs without restriction.

[0105] In step S302 of some embodiments, the sample speech feature is used to indicate speech feature information of the sample speech data.

[0106] In the specific implementation of this embodiment, a preset Mel filter group is used to extract features from the sample voice data to capture the fixed-dimensional voice features in the sample voice data, thereby obtaining the sample voice features. Specifically, first, the sample voice data is pre-emphasized and DC removed to subtract a portion of the previous sampling point from the voice signal of the sample voice data to enhance the high-frequency voice part. Then, the pre-emphasized sample voice data is framed using the Hamming window function to divide the sample voice data into multiple shorter voice frame segments. Further, the voice frame segments of each frame are fast Fourier transformed to convert the sample voice data from the time domain to the frequency domain, and the frequency domain features corresponding to each voice frame segment are obtained. Then, the multiple frequency domain features are filtered and logarithmically compressed through the Mel filter group to obtain the audio logarithmic compression result corresponding to the sample voice data. Finally, the audio logarithmic compression result is discretized, and the discrete cosine transform is used to reduce the feature dimension, and the important information of the sample voice data is retained to obtain the sample voice features.

[0107] In step S303 of some embodiments, when data augmentation is performed on the sample speech features, the sample speech features may be noised to obtain noise speech features; the sample speech features may be processed at double speed to change the speech speed information of the sample speech features to obtain double speed speech features; the sample speech features may also be randomly masked to discard part of the feature information to obtain masked speech features, thereby integrating the noise speech features, double speed speech features, and masked speech features into a series of speech features, and using this series of speech features as augmented speech features.

[0108] In step S304 of some embodiments, the first voiceprint similarity score is used to indicate the degree of proximity between the sample speech feature and the reference speech feature in voiceprint information.

[0109] In the specific implementation of this embodiment, first, the voiceprint feature of the sample speech feature is extracted through the original model to obtain the voiceprint feature of the sample speech; and the voiceprint feature of the reference speech feature is extracted through the original model to obtain the voiceprint feature of the reference speech. Then, the cosine similarity of the sample speech voiceprint feature and the reference speech voiceprint feature is calculated at the feature level through the original model to obtain the first voiceprint similarity score.

[0110] The reference speech feature is obtained by extracting the feature of the reference speech of the sample object. The original model can be constructed based on a neural network structure such as a residual network (ResNet) or a time delay neural network (TDNN), without any specific limitation.

[0111] In step S305 of some embodiments, the second voiceprint similarity score is used to indicate the degree of proximity between the augmented speech feature and the reference speech feature in the voiceprint information.

[0112] In the specific implementation of this embodiment, the specific process of step S305 is similar to the above step S304, and will not be described in detail to save space.

[0113] In step S306 of some embodiments, since there are multiple sample objects in the training of the voiceprint detection model, and there are also multiple sample speech features and augmented speech features of the sample objects. Based on this, first, the total number of objects of the sample objects is counted. Then, for each sample object, the total number of features of the sample speech features and augmented speech features of the sample object is counted. Further, for each sample object, the first voiceprint similarity score is compared with the preset score threshold, and the second voiceprint similarity score is compared with the preset score threshold. If the first voiceprint similarity score is greater than the preset score threshold, the sample speech features corresponding to the first voiceprint similarity score are added to the target set, and if the second voiceprint similarity score is greater than the preset score threshold, the augmented speech features corresponding to the second voiceprint similarity score are added to the target set. After the comparison is completed, the total number of sample speech features and augmented speech features in the target set of the sample object is counted as the number of valid features. Then, for each sample object, the number of valid features is divided by the total number of features to obtain the detection accuracy of the original model for a single sample object, and the detection accuracy of multiple sample objects is averaged to obtain the overall detection accuracy of the original model. Finally, the model parameters of the original model are updated according to the overall detection accuracy, and the above steps S301 to S306 are repeated to implement iterative training of the original model until the overall detection accuracy of the original model meets the training requirements in a certain iteration round. The model parameters of the iteration round are used as the final model parameters, and the original model with the final model parameters is used as the trained voiceprint detection model.

[0114] The benefit of this embodiment is that, when training the model, the problem of insufficient data volume of the sample voice data is taken into consideration. For the sample voice features corresponding to the sample voice data, data augmentation is performed by using noise, masking, and multiple speed methods, so that the sample voice features and augmented voice features used for training can be fully obtained, thereby increasing the total number of features. Furthermore, the sample voice features, augmented voice features, and reference voice features are input into the original model together for voiceprint detection. During voiceprint detection, the first voiceprint similarity score and the second voiceprint similarity score are obtained by voiceprint feature extraction and similarity comparison. At the same time, for the first voiceprint similarity score and the second voiceprint similarity score, the threshold comparison method is used to determine the overall detection accuracy of the original model in each iteration round, and the original model is trained according to the overall detection accuracy, which can improve the detection accuracy of the trained voiceprint detection model in voiceprint detection.

[0115] See also Figure 4In some embodiments, the voiceprint detection model includes a feature extraction network and a detection network; step S102 may include but is not limited to steps S401 to S404:

[0116] Step S401, extracting features of the reference speech based on a feature extraction network to obtain reference speech voiceprint features;

[0117] Step S402, extracting features of the target speech based on a feature extraction network to obtain voiceprint features of the target speech;

[0118] Step S403, performing similarity detection on the target speech voiceprint features and the reference speech voiceprint features based on the detection network to obtain a feature similarity score;

[0119] Step S404: obtaining a voiceprint detection result based on a comparison between the feature similarity score and a preset score threshold.

[0120] Steps S401 to S404 are described in detail below.

[0121] In step S401 of some embodiments, the reference speech voiceprint feature is used to indicate the voiceprint information contained in the reference speech, wherein the voiceprint information includes but is not limited to the target object's vocalization habits, vocal tract structure, and speech formants, etc., which can be used to characterize the target object's identity.

[0122] It should be noted that the voiceprint features of different objects often have certain differences.

[0123] In the specific implementation of this embodiment, first, the reference speech is input into the feature extraction network, and the reference speech is mapped from the time domain space to the frequency domain space through the feature extraction network to obtain the reference speech frequency domain features. Then, the feature extraction network is used to extract the voiceprint features of the reference speech frequency domain features to obtain the reference speech voiceprint features.

[0124] In step S402 of some embodiments, the target speech voiceprint feature is used to indicate the voiceprint information contained in the target speech.

[0125] In the specific implementation of this embodiment, the specific process of step S402 is similar to the specific process of step S401, and will not be described in detail to save space.

[0126] In step S403 of some embodiments, the feature similarity score is used to indicate the degree of proximity between the target voiceprint feature and the reference voiceprint feature in voiceprint information. The larger the feature similarity score, the closer the target voiceprint feature and the reference voiceprint feature are in voiceprint information, indicating that the target voice and the reference voice are more likely to come from the same speaker.

[0127] In the specific implementation of this embodiment, the specific process of step S403 is similar to the specific process of step S304 described above, and will not be described in detail to save space.

[0128] In step S404 of some embodiments, the preset score threshold is used to measure whether the feature similarity score reaches the minimum limit for identifying the reference speech and the target speech as coming from the same object.

[0129] In the specific implementation of this embodiment, first, the feature similarity score is compared with the preset score threshold. Then, if the feature similarity score is greater than or equal to the preset score threshold, it indicates that the feature similarity score meets the condition of identifying the reference voice and the target voice as coming from the same object, and therefore, the voiceprint detection result is determined as the voiceprint feature of the target voice is the same as the voiceprint feature of the reference voice, and it is determined that the reference voice and the target voice are both from the target object.

[0130] Similarly, if the feature similarity score is less than the preset score threshold, it indicates that the feature similarity score does not meet the condition for identifying the reference voice and the target voice as coming from the same object. Therefore, the voiceprint detection result is determined as the voiceprint feature of the target voice is different from the voiceprint feature of the reference voice, and it is determined that the target voice is not from the target object. Furthermore, since the target voice is not from the target object, it can be directly determined that the target voice is the altered voice of the target object. Based on this, the alteration detection model is no longer called to continue detection, and the alteration detection result is directly determined as the target voice is altered.

[0131] The benefit of this embodiment is that the target voiceprint features of the target voice and the reference voiceprint features of the reference voice are obtained by using the feature extraction network of the voiceprint detection model, which can improve the accuracy and comprehensiveness of voiceprint extraction. Furthermore, the detection network is used to quantify the degree of proximity between the voiceprint features of the target voice and the voiceprint features of the reference voice, thereby generating a voiceprint detection result based on the relationship between the feature similarity score and the preset score threshold. This method can clearly determine whether the target voice is truly from the target object, improve the accuracy of voiceprint detection of the target voice, and further improve the accuracy of identifying whether the speaker of the target voice is the target object.

[0132] In step S103 of some embodiments, if the voiceprint detection result indicates that the voiceprint features of the target speech are the same as the voiceprint features of the reference speech, the target speech and the reference speech are respectively encoded based on a preset alteration detection model to obtain a target speech encoding vector of the target speech and a reference speech encoding vector of the reference speech.

[0133] The alteration detection model refers to a neural network model that can detect the alteration of speech data. The alteration detection model generally takes speech data as input and outputs whether the speech data is altered speech.

[0134] The target speech coding vector is used to indicate the vector representation form of the target speech in the time space.

[0135] The reference speech coding vector is used to indicate the vector representation form of the reference speech in the time space.

[0136] When this embodiment is specifically implemented, if the voiceprint detection result indicates that the voiceprint features of the target voice are the same as the voiceprint features of the reference voice, it means that the target voice comes from the target object, and it is necessary to further detect whether the target voice is altered. Because when the target voice is altered, there will be a relatively obvious difference between the target voice and the reference voice. Based on this, the target voice and the reference voice are input into the preset alteration detection model together, and the target voice is first encoded by the encoder of the alteration detection model, and the target voice is mapped from the time domain space to the potential time sequence space to obtain the target voice encoding vector of the target voice. Then, the reference voice is encoded by the encoder of the alteration detection model, and the reference voice is mapped from the time domain space to the time sequence space to obtain the reference voice encoding vector of the reference voice. This method can realize the feature mapping of the target voice and the reference voice, so as to detect the alteration of the target voice in the time sequence space, which is conducive to improving the convenience and accuracy of alteration detection.

[0137] In step S104 of some embodiments, the target speech coding vector and the reference speech coding vector are concatenated to obtain a speech concatenation feature vector.

[0138] The speech concatenation feature vector is used to indicate a vector representation formed by fusing the speech information of the target speech and the speech information of the reference speech into a whole.

[0139] When the embodiment is specifically implemented, when the target speech coding vector and the reference speech coding vector are spliced, the target speech coding vector and the reference speech coding vector are spliced ​​into a longer vector feature, so that the speech information of the target speech and the speech information of the reference speech are fused into a whole, thereby obtaining a speech splicing feature vector. This method can realize the use of a speech splicing feature vector to represent the speech information of the target speech and the speech information of the reference speech, and can realize the fusion of multiple speech time sequence information, so that the alteration recognition model can process the target speech and the reference speech at the same time, thereby improving the accuracy of alteration detection using the speech splicing feature vector.

[0140] In an embodiment of the present application, the alteration detection model includes an encoder, a Fourier transform layer, a first attention network, an inverse Fourier transform layer, a second attention network and an output layer. Among them, the encoder is used to obtain the time domain feature representation of the target speech and the reference speech. The Fourier transform layer is used to convert the time domain features of the input alteration detection model into frequency domain features to analyze the frequency components of the speech signal. The first attention network is used to perform deep feature extraction on the output features of the Fourier transform layer in the frequency domain space. The inverse Fourier transform layer is used to convert the output results (frequency domain features) of the second attention network back into time domain features. The second attention network is used to perform deep feature extraction on the output features of the inverse Fourier transform layer in the time domain space. The output layer is used to map the output results of the second attention network to the category probability, so as to determine whether the target speech is altered.

[0141] See also Figure 5 In some embodiments, the specific process of training the alteration detection model may include but is not limited to steps S501 to S504:

[0142] Step S501, obtaining a positive sample speech pair and a negative sample speech pair of a sample object;

[0143] Step S502, inputting the positive sample speech pair into the original detection model for alteration detection to obtain the positive sample alteration detection result, and inputting the negative sample speech pair into the original detection model for alteration detection to obtain the negative sample alteration detection result;

[0144] Step S503, calculating a first loss function based on the positive sample alteration detection result and the negative sample alteration detection result;

[0145] Step S504: Based on the first loss function, the original detection model is trained to obtain a modification detection model.

[0146] Steps S501 to S504 are described in detail below.

[0147] In step S501 of some embodiments, the positive sample speech pair is composed of a reference speech segment of the sample object and a sample real speech segment, and the negative sample speech pair is composed of a reference speech segment of the sample object and a sample altered speech segment.

[0148] When the embodiment is specifically implemented, when obtaining the positive sample voice pair and the negative sample voice pair of the sample object, first, obtain the reference voice segment and the sample real voice segment of the sample object, and the specific process is similar to the above-mentioned step S301. In order to save space, it will not be repeated. Then, construct the sample altered voice segment of the sample object. Further, the reference voice segment of the sample object and the sample real voice segment are paired into a positive sample voice pair, and the reference voice segment of the sample object and the sample altered voice segment are paired into a negative sample voice pair.

[0149] Specifically, when constructing the sample altered speech segment of the sample object, it can be based on a preset speech generation model, directly inputting the pre-accurate text into the speech generation model to perform speech generation, and using the generated speech data as the sample altered speech segment of the sample object. The preset speech generation model can be constructed based on neural network structures such as an end-to-end text-to-speech synthesis model (VIT S), a converter-based text-to-audio model (BARK), and a gradient-based text-to-speech model (GradTTS). It can also be constructed by speech conversion to replace the timbre feature information in the sample real speech segment of the sample object with the timbre feature information of other speaking objects to achieve speech alteration.

[0150] In step S502 of some embodiments, the positive sample alteration detection result is used to indicate whether the sample real speech segment in the positive sample speech pair predicted by the original detection model is altered, and the negative sample alteration detection result is used to indicate whether the sample altered speech segment in the negative sample speech pair predicted by the original detection model is altered.

[0151] The original detection model is a neural network model to be trained. When the original detection model is trained, the trained original detection model is the altered detection model. The original detection model can be constructed based on a neural network structure such as a Fourier network and a graph attention network.

[0152] In the specific implementation of this embodiment, when the positive sample speech pair is input into the original detection model for alteration detection, first, the reference speech segment and the sample real speech segment in the positive sample speech pair are respectively encoded by the original detection model to obtain the reference speech time sequence feature corresponding to the reference speech segment and the sample real speech time sequence feature corresponding to the sample real speech segment, and the reference speech time sequence feature and the sample real speech time sequence feature are spliced ​​into a positive sample speech splicing vector. Further, the positive sample speech splicing vector is Fourier transformed by the original detection model, the positive sample speech splicing vector is mapped from the time domain space to the frequency domain space, the positive sample speech frequency domain feature is obtained, and the positive sample speech frequency domain feature is calculated in the frequency domain space to obtain the positive sample speech frequency domain attention feature. Then, the positive sample speech frequency domain attention feature is Fourier inverse transformed by the original detection model, the positive sample speech frequency domain attention feature is converted back to the time sequence space to obtain the positive sample speech frequency domain feature, and the positive sample speech frequency domain attention feature is calculated in the time domain space to obtain the positive sample speech frequency domain attention feature. Finally, the classification probability of the time-domain attention features of the positive sample speech is calculated through the output layer of the original detection model, and the probability value of the sample real speech segment in the positive sample speech pair is calculated. When the probability value is greater than the preset threshold, the positive sample alteration detection result is determined as the sample real speech segment is altered, and when the probability value is less than or equal to the preset threshold, the positive sample alteration detection result is determined as the sample real speech segment is not altered.

[0153] Similarly, the specific process of inputting the negative sample speech pair into the original detection model for alteration detection to obtain the negative sample alteration detection result is similar to the specific process of inputting the positive sample speech pair into the original detection model for alteration detection to obtain the positive sample alteration detection result. To save space, it will not be repeated.

[0154] In step S503 of some embodiments, the first loss function is used to indicate the overall alteration detection accuracy of the original detection model for the positive sample speech pairs and the negative sample speech pairs of the sample object. When the output result of the first loss function is smaller, it indicates that the difference between the alteration detection result of the original detection model for the positive sample speech pairs and the negative sample speech pairs of the sample object and the actual situation is smaller, and the alteration detection accuracy of the original detection model is higher.

[0155] When this embodiment is implemented, when calculating the first loss function, the cross entropy loss function can be used as the first loss function. Specifically, first, for the negative sample speech pair of the sample object, take the negative logarithm of the probability value corresponding to the negative sample alteration detection result to obtain the negative sample loss function of the negative sample speech pair; then, for the positive sample speech pair of the sample object, take the negative logarithm of 1 minus the probability value corresponding to the negative sample alteration detection result to obtain the positive sample loss function of the positive sample speech pair. Further, the positive sample loss function and the negative sample loss function of a single sample object are averaged to obtain the sub-loss function of the single object. Finally, the sub-loss functions of all sample objects are averaged to obtain the first loss function.

[0156] It should be noted that since the alteration detection result is determined based on the probability value of the input speech segment output by the output layer being considered to be altered, when training the model, the negative sample speech pairs of the sample object are taken as the positive class (the reference label is 1), and the positive sample speech pairs of the sample object are taken as the negative class (the reference label is 0).

[0157] In step S504 of some embodiments, the specific process of training the original detection model based on the first loss function is similar to the above step S306, which will not be described in detail to save space.

[0158] The benefit of this embodiment is that, during model training, the positive sample speech pair and the negative sample speech pair of the sample object are constructed, and by distinguishing the positive sample (real speech pair) and the negative sample (altered speech pair), the model is made to fully learn to distinguish the features of real and altered speech during the training process. Further, the first loss function is constructed according to the positive sample speech pair and the negative sample speech pair respectively predicted by the original detection model for the positive sample speech pair and the negative sample speech pair, and the negative sample altered detection results are constructed. By calculating the first loss function, the model can obtain feedback about its own prediction accuracy in each iteration round clock, and constantly adjust the parameters to reduce the loss, thereby improving the detection accuracy of the altered speech by the altered detection model finally trained. In addition, the model is automated during training, and once the initial model parameters, the positive sample speech pair and the negative sample speech pair of the sample object are set, the model can autonomously learn and distinguish the altered feature information in the spoken speech, which can reduce the need for manual intervention, thereby improving the detection consistency and reliability of the model.

[0159] See also Figure 6 In some embodiments, step S105 includes but is not limited to steps S601 to S605:

[0160] Step S601, performing Fourier transform on the speech concatenation feature vector based on the alteration detection model to obtain a speech frequency domain feature vector;

[0161] Step S602, performing a first attention calculation on the speech frequency domain feature vector to obtain a first attention feature vector;

[0162] Step S603, performing inverse Fourier transform on the first attention feature vector to obtain a speech time domain feature vector;

[0163] Step S604, performing a second attention calculation on the speech frequency domain feature vector to obtain a second attention feature vector;

[0164] Step S605: Perform alteration detection on the second attention feature vector based on a preset function to obtain an alteration detection result.

[0165] Steps S601 to S605 are described in detail below.

[0166] In step S601 of some embodiments, the speech-domain feature vector is used to indicate comprehensive speech information of the target speech and the reference speech in the frequency domain space.

[0167] In the specific implementation of this embodiment, the speech splicing feature vector is Fourier transformed based on the Fourier transform layer of the alteration detection model, and the speech splicing feature vector is mapped from the one-dimensional time domain vector space to the frequency domain space, so as to analyze the frequency components of the target speech and the reference speech, thereby obtaining the speech frequency domain feature vector.

[0168] In step S602 of some embodiments, the first attention feature vector is used to indicate deep feature information of the target speech and the reference speech in the frequency domain space.

[0169] In the specific implementation of this embodiment, the query channel of the second attention network based on the alteration detection model linearly projects the voice frequency domain feature vector to obtain the query vector, the key channel based on the second attention network linearly projects the voice frequency domain feature vector to obtain the key vector, and the value channel of the second attention network linearly projects the voice frequency domain feature vector to obtain the value vector, wherein the projection parameters of the key channel, the query channel and the value channel are different. Further, the query vector and the transposed result of the key vector are multiplied to obtain the product result, and the product result is divided by the square difference of the feature dimension of the key vector to obtain the division result. Finally, the division result is normalized to obtain the attention weight, and the attention weight is multiplied by the value vector to obtain the first attention feature vector.

[0170] In step S603 of some embodiments, an inverse Fourier transform is performed on the first attention feature vector based on an inverse Fourier transform layer, and the first attention feature vector is mapped from the frequency domain space to a one-dimensional time domain vector space, so that the first attention feature vector in the form of frequency domain features is reconverted into time domain features to obtain a speech time domain feature vector, so as to facilitate time series analysis of the speech time domain feature vector in alteration detection.

[0171] In step S604 of some embodiments, the second attention feature vector is used to indicate deep feature information of the target speech and the reference speech in the time domain space.

[0172] In the specific implementation of this embodiment, the specific process of step S604 is similar to the above step S602, and will not be described in detail to save space.

[0173] In step S605 of some embodiments, step S605 specifically includes but is not limited to the following steps:

[0174] Performing a change detection on the second attention feature vector based on a preset function to obtain a speech change score;

[0175] Normalize the phonetic variation scores to obtain the phonetic variation probability;

[0176] If it is determined that the probability of voice alteration is greater than a preset threshold, the alteration detection result is determined as the target voice being altered.

[0177] Among them, the preset function is a classification function such as the softmax function; the speech alteration score is used to indicate the quantitative result of the alteration detection model determining the target speech as altered; the speech alteration probability is used to reflect the possibility that the target speech is altered from a statistical perspective; the preset threshold is used to measure whether the minimum limit for identifying the target speech as altered speech is reached.

[0178] Specifically, first, the second attention feature vector is input into a preset function, and the preset function maps the second attention feature vector to the probability of the alteration category, thereby generating a probability distribution for the target speech and obtaining a speech alteration score, wherein the probability distribution generated by the preset function is a binary probability distribution (used to indicate the probability score that the target speech is altered and the probability score that the target speech is not altered). Next, the speech alteration score is normalized, and the speech alteration score is converted into a probability value with a value range of [0,1], thereby obtaining the speech alteration probability. Finally, the speech alteration probability is compared with the preset threshold; if it is determined that the speech alteration probability is greater than the preset threshold, it indicates that the condition for identifying the target speech as altered speech is met, so the alteration detection result is determined as the target speech is altered. If it is determined that the speech alteration probability is less than or equal to the preset threshold, it indicates that the condition for identifying the target speech as altered speech is not met, so the alteration detection result is determined as the target speech is not altered.

[0179] The benefit of this embodiment is that, in alteration detection, the accuracy of alteration detection for the target voice can be improved by joint analysis of the feature splicing results (voice splicing feature vector) of the target voice and the reference voice in the time domain and the frequency domain, and in-depth feature extraction based on the attention mechanism. At the same time, the introduction of the combination of Fourier transform, inverse Fourier transform and attention mechanism can make the alteration detection model focus more on the high-frequency voice feature data of the target voice and the reference voice, so that the alteration detection model can better pay attention to the difference between the target voice and the reference voice of the same speaking object (target object), so as to better identify the altered part in the target voice, which is conducive to improving the accuracy of alteration detection for the target voice.

[0180] Combine the following Fig. 7A and Figure 7B The implementation details of the speech detection method of the embodiment of the present application are described in detail.

[0181] like Fig. 7A As shown, it is a schematic diagram of the overall implementation process of voiceprint detection and alteration detection for the target voice. Specifically, when the business end has a target voice that needs to be verified, the business end will send a detection request to the server, and provide the target voice that needs to be verified and the object identity information of the speaker associated with the target voice to the server, wherein the speaker is the object to which the target voice claims to belong. Based on this, first, according to the object identity information of the speaker associated with the target voice, the reference voice of the candidate object pointed to by the object identity information is found in the reference voice database, and its specific process is similar to the above steps S201-S203. Then, the target voice and the reference voice are input into the voiceprint detection model together, and the voiceprint detection model performs voiceprint detection on the target voice and the reference voice, and outputs a voiceprint detection result representing whether the voiceprint features of the target voice and the reference voice are the same, and its specific process is similar to the above steps S401-S404. Further, when the voiceprint detection result is no, it indicates that the voiceprint features of the target voice and the reference voice are different, and the target voice does not really come from the associated speaker. When the voiceprint detection result is yes, it indicates that the voiceprint features of the target voice and the reference voice are the same, and the target voice is truly from the associated speaker. It is necessary to further determine whether the target voice is a forged voice. Further, the target voice is detected by the forgery detection model, and the forgery detection result of whether the target voice is forged is output. The specific process is similar to the above steps S601 to S605. To save space, it will not be repeated.

[0182] like Figure 7BAs shown, it is a schematic diagram of the specific implementation process of the alteration detection model for the target voice to be altered. Specifically, first, the target voice and the reference voice are input into the encoder of the alteration detection model, and the target voice and the reference voice are encoded by the encoder respectively to obtain the target voice coding vector of the target voice and the reference voice coding vector of the reference voice. Then, the target voice coding vector and the reference voice coding vector are spliced ​​to obtain the vector splicing result (voice splicing feature vector). Further, the vector splicing result is sequentially input into the Fourier transform layer, the first attention network, the inverse Fourier transform layer, the second attention network and the output layer of the alteration detection model, and finally, the alteration detection result (whether the target voice is altered) of the target voice is output by the output layer, and its specific process is similar to the above-mentioned steps S601-step S605. To save space, it will not be repeated.

[0183] See also Figure 8 The present application also provides a speech detection device that can implement the above-mentioned speech detection method. The speech detection device includes:

[0184] An acquisition unit 801 is used to acquire a reference speech of a target object in response to a detection request for a target speech of a target object;

[0185] The first detection unit 802 is used to perform voiceprint detection on the reference speech and the target speech based on a preset voiceprint detection model to obtain a voiceprint detection result, wherein the voiceprint detection result is used to indicate whether the voiceprint feature of the target speech is the same as the voiceprint feature of the reference speech;

[0186] The encoding unit 803 is configured to encode the target speech and the reference speech based on a preset alteration detection model to obtain a target speech encoding vector of the target speech and a reference speech encoding vector of the reference speech if the voiceprint detection result indicates that the voiceprint feature of the target speech is the same as the voiceprint feature of the reference speech;

[0187] A concatenation unit 804 is used to concatenate the target speech coding vector and the reference speech coding vector to obtain a speech concatenation feature vector;

[0188] The second detection unit 805 is used to perform alteration detection based on the speech concatenation feature vector and a preset alteration detection model to obtain an alteration detection result, wherein the alteration detection result is used to indicate whether the target speech is altered.

[0189] In some embodiments, the acquisition unit 801 is used to:

[0190] Parsing the detection request to obtain the target object identifier of the target object;

[0191] Acquire a preset reference voice database, wherein the reference voice database includes a plurality of reference voices, candidate object identifiers of a plurality of candidate objects, and a correspondence between the candidate object identifiers and the reference voices;

[0192] A voice query is performed in a reference voice database based on the target object identifier, and a reference voice corresponding to a candidate object identifier that is consistent with the target object identifier is determined as a reference voice.

[0193] In some embodiments, the voiceprint detection model includes a feature extraction network and a detection network; the first detection unit 802 is used to:

[0194] Extract features of the reference speech based on the feature extraction network to obtain the reference speech voiceprint features;

[0195] Extract the features of the target speech based on the feature extraction network to obtain the voiceprint features of the target speech;

[0196] Based on the detection network, the target voiceprint features and the reference voiceprint features are tested for similarity to obtain a feature similarity score;

[0197] Based on the comparison between the feature similarity score and the preset score threshold, the voiceprint detection result is obtained.

[0198] In some embodiments, the second detection unit 805 is used to:

[0199] Based on the alteration detection model, Fourier transform is performed on the speech concatenation feature vector to obtain the speech frequency domain feature vector;

[0200] Performing a first attention calculation on the speech frequency domain feature vector to obtain a first attention feature vector;

[0201] Perform inverse Fourier transform on the first attention feature vector to obtain the speech time domain feature vector;

[0202] Performing a second attention calculation on the speech frequency domain feature vector to obtain a second attention feature vector;

[0203] Based on a preset function, the second attention feature vector is subjected to alteration detection to obtain an alteration detection result.

[0204] In some embodiments, performing alteration detection on the second attention feature vector based on a preset function to obtain an alteration detection result includes:

[0205] Performing a change detection on the second attention feature vector based on a preset function to obtain a speech change score;

[0206] Normalize the phonetic variation scores to obtain the phonetic variation probability;

[0207] If it is determined that the probability of voice alteration is greater than a preset threshold, the alteration detection result is determined as the target voice being altered.

[0208] In some embodiments, the alteration detection model is trained by:

[0209] Obtaining a positive sample speech pair and a negative sample speech pair of a sample object, wherein the positive sample speech pair is composed of a reference speech segment of the sample object and a sample real speech segment, and the negative sample speech pair is composed of a reference speech segment of the sample object and a sample altered speech segment;

[0210] Inputting the positive sample speech pair into the original detection model for alteration detection to obtain the positive sample alteration detection result, and inputting the negative sample speech pair into the original detection model for alteration detection to obtain the negative sample alteration detection result;

[0211] Calculating a first loss function based on the positive sample alteration detection result and the negative sample alteration detection result;

[0212] Based on the first loss function, the original detection model is trained to obtain the altered detection model.

[0213] In some embodiments, the voiceprint detection model is trained in the following manner:

[0214] Obtain sample speech data of a sample object;

[0215] Extracting features from the sample speech data to obtain sample speech features;

[0216] Performing data augmentation on the sample speech features to obtain augmented speech features;

[0217] Performing voiceprint detection on the sample voice features and the pre-acquired reference voice features based on the original model to obtain a first voiceprint similarity score;

[0218] Perform voiceprint detection on the augmented voice features and the reference voice features based on the original model to obtain a second voiceprint similarity score;

[0219] Based on the first voiceprint similarity score and the second voiceprint similarity score, the original model is trained to obtain a voiceprint detection model.

[0220] The specific implementation of the speech detection device is substantially the same as the specific implementation of the speech detection method described above, and will not be described in detail herein.

[0221] The embodiment of the present application also provides an electronic device, the electronic device comprising: a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory, and the program is executed by the processor to realize the above-mentioned voice detection method. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0222] See also Fig. 9 , Fig. 9 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0223] The processor 901 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0224] The memory 902 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 902 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 902, and the processor 901 calls and executes the speech detection method of the embodiment of this application;

[0225] Input / output interface 903, used to implement information input and output;

[0226] Communication interface 904, used to realize communication interaction between the device and other devices, which can be realized through wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WI FI, Bluetooth, etc.);

[0227] A bus 905 that transmits information between various components of the device (e.g., the processor 901, the memory 902, the input / output interface 903, and the communication interface 904);

[0228] The processor 901 , the memory 902 , the input / output interface 903 and the communication interface 904 are connected to each other in communication within the device via a bus 905 .

[0229] An embodiment of the present application also provides a computer-readable storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement the above-mentioned speech detection method.

[0230] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0231] The speech detection method, speech detection device, electronic device and computer-readable storage medium provided in the embodiment of the present application obtain the reference speech of the target object by responding to the detection request of the target speech of the target object, firstly perform voiceprint detection on the reference speech and the target speech based on the preset voiceprint detection model to obtain the voiceprint detection result, wherein the voiceprint detection result is used to indicate whether the voiceprint feature of the target speech is the same as the voiceprint feature of the reference speech, thereby determining whether the target speech belongs to the claimed target object according to the comparison of the voiceprint features between the reference speech and the target speech, and realizing the detection of the speaking object before the alteration detection. Further, if the voiceprint detection result indicates that the voiceprint feature of the target speech is the same as the voiceprint feature of the reference speech, then the target speech and the reference speech are respectively encoded based on the preset alteration detection model to obtain the target speech encoding vector of the target speech and the reference speech encoding vector of the reference speech, and the target speech encoding vector and the reference speech encoding vector are spliced ​​to obtain the speech splicing feature vector, so that the speech splicing feature vector contains the speech feature information of the target speech and the reference speech at the same time, thereby improving the feature comprehensiveness of the speech splicing feature vector. Finally, based on the speech splicing feature vector, alteration detection is performed through the preset alteration detection model to obtain the alteration detection result. This method takes into account the simultaneous processing of the speech feature information of the target speech and the reference speech in alteration detection, which can improve the detection accuracy of altered speech.

[0232] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0233] It can be understood by those skilled in the art that Figure 1-6 The technical solutions shown in the figure do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than those shown in the figure, or a combination of certain steps, or different steps.

[0234] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0235] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0236] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0237] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0238] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0239] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0240] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0241] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0242] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A speech detection method, characterized in that: The speech detection method comprises: In response to a detection request for a target speech of a target object, obtaining a reference speech of the target object; Performing voiceprint detection on the reference voice and the target voice based on a preset voiceprint detection model to obtain a voiceprint detection result, wherein the voiceprint detection result is used to indicate whether the voiceprint feature of the target voice is the same as the voiceprint feature of the reference voice; If the voiceprint detection result indicates that the voiceprint feature of the target voice is the same as the voiceprint feature of the reference voice, encoding the target voice and the reference voice respectively based on a preset alteration detection model to obtain a target voice encoding vector of the target voice and a reference voice encoding vector of the reference voice; Performing concatenation processing on the target speech coding vector and the reference speech coding vector to obtain a speech concatenation feature vector; Based on the speech concatenation feature vector, alteration detection is performed through a preset alteration detection model to obtain an alteration detection result, wherein the alteration detection result is used to indicate whether the target speech is altered.

2. The method according to claim 1, characterized in that: The voiceprint detection model includes a feature extraction network and a detection network; the voiceprint detection is performed on the reference voice and the target voice based on the preset voiceprint detection model to obtain the voiceprint detection result, including: Extracting features of the reference speech based on the feature extraction network to obtain reference speech voiceprint features; Extracting features of the target speech based on the feature extraction network to obtain voiceprint features of the target speech; Based on the detection network, a similarity detection is performed on the target speech voiceprint feature and the reference speech voiceprint feature to obtain a feature similarity score; Based on the comparison between the feature similarity score and a preset score threshold, a voiceprint detection result is obtained.

3. The method according to claim 1, characterized in that The method of performing alteration detection based on the speech concatenation feature vector by using a preset alteration detection model to obtain an alteration detection result includes: Performing Fourier transform on the speech concatenation feature vector based on the alteration detection model to obtain a speech frequency domain feature vector; Performing a first attention calculation on the speech frequency domain feature vector to obtain a first attention feature vector; Performing an inverse Fourier transform on the first attention feature vector to obtain a speech time domain feature vector; Performing a second attention calculation on the speech frequency domain feature vector to obtain a second attention feature vector; The second attention feature vector is subjected to alteration detection based on a preset function to obtain the alteration detection result.

4. The method according to claim 3, characterized in that The performing alteration detection on the second attention feature vector based on a preset function to obtain the alteration detection result includes: Performing alteration detection on the second attention feature vector based on a preset function to obtain a speech alteration score; Normalizing the speech change score to obtain a speech change probability; If it is determined that the voice alteration probability is greater than a preset threshold, the alteration detection result is determined as the target voice being altered.

5. The method according to claim 1, characterized in that The alteration detection model is trained in the following way: Acquire a positive sample speech pair and a negative sample speech pair of a sample object, wherein the positive sample speech pair is composed of a reference speech segment and a sample real speech segment of the sample object, and the negative sample speech pair is composed of a reference speech segment and a sample altered speech segment of the sample object; Inputting the positive sample speech pair into the original detection model for alteration detection to obtain a positive sample alteration detection result, and inputting the negative sample speech pair into the original detection model for alteration detection to obtain a negative sample alteration detection result; Calculating a first loss function based on the positive sample alteration detection result and the negative sample alteration detection result; Based on the first loss function, the original detection model is trained to obtain the altered detection model.

6. The method according to claim 1, characterized in that The voiceprint detection model is trained in the following way: Obtain sample speech data of a sample object; Extracting features from the sample speech data to obtain sample speech features; Performing data augmentation on the sample speech feature to obtain augmented speech feature; Performing voiceprint detection on the sample voice feature and the pre-acquired reference voice feature based on the original model to obtain a first voiceprint similarity score; Performing voiceprint detection on the augmented speech feature and the reference speech feature based on the original model to obtain a second voiceprint similarity score; Based on the first voiceprint similarity score and the second voiceprint similarity score, the original model is trained to obtain the voiceprint detection model.

7. The method according to any one of claims 1 to 6, characterized in that: The step of obtaining a reference voice of the target object in response to a detection request for the target voice of the target object comprises: Parsing the detection request to obtain a target object identifier of the target object; Acquire a preset reference voice database, wherein the reference voice database includes a plurality of reference voices, candidate object identifiers of a plurality of candidate objects, and a correspondence between the candidate object identifiers and the reference voices; A voice query is performed in the reference voice database based on the target object identifier, and a reference voice corresponding to the candidate object identifier consistent with the target object identifier is determined as the reference voice.

8. A speech detection device, characterized in that: The speech detection device comprises: an acquisition unit, configured to acquire a reference voice of the target object in response to a detection request for the target voice of the target object; a first detection unit, configured to perform voiceprint detection on the reference voice and the target voice based on a preset voiceprint detection model to obtain a voiceprint detection result, wherein the voiceprint detection result is used to indicate whether the voiceprint feature of the target voice is the same as the voiceprint feature of the reference voice; an encoding unit, configured to encode the target voice and the reference voice based on a preset alteration detection model to obtain a target voice encoding vector of the target voice and a reference voice encoding vector of the reference voice, respectively, if the voiceprint detection result indicates that the voiceprint feature of the target voice is the same as the voiceprint feature of the reference voice; A concatenation unit, configured to concatenate the target speech coding vector and the reference speech coding vector to obtain a speech concatenation feature vector; The second detection unit is used to perform alteration detection based on the speech concatenation feature vector and a preset alteration detection model to obtain an alteration detection result, wherein the alteration detection result is used to indicate whether the target speech is altered.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the speech detection method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the speech detection method according to any one of claims 1 to 7 is implemented.

Images