Voiceprint comparison method and device based on large language model, and readable medium

Abstract

The invention discloses a voiceprint comparison method and device based on a large language model, and a readable medium. The voiceprint comparison method comprises the following steps: acquiring a first voice to be compared and a second voice to be compared which are acquired respectively, and splicing the first voice and the second voice into a combined voice; the combined voice and the prompt words are input into a trained voiceprint comparison model, and the combined voice passes through an audio encoder to obtain voice encoding features; the prompt words pass through a text encoder to obtain text encoding features; inputting the voice coding features into an adapter, and performing dimension conversion on the voice coding features to obtain the voice coding features after dimension conversion; the text coding features and the voice coding features after dimension conversion are spliced and then input into an ontology structure of the improved large language model, output features of the ontology structure of the pre-trained large language model and output features of the LoRA module are added to obtain an output token sequence, the output token sequence passes through a text decoder to obtain a corresponding output text, and the output text is input into the LoRA module. And the robustness and accuracy of the model algorithm on voiceprint discrimination are improved.

Description

Technical Field

[0001] The present invention relates to the field of voiceprint comparison, and in particular to a voiceprint comparison method, device and readable medium based on a large language model. Background Art

[0002] Voiceprint matching technology is used to compare whether two audio recordings belong to the same person. Traditional voiceprint matching methods rely on signal processing technology to first extract acoustic features, then combine methods such as Gaussian mixture models for identity modeling, and use methods such as likelihood ratios to complete voiceprint matching. Due to shortcomings such as weak feature expression capabilities, poor model flexibility, and insufficient anti-interference capabilities, traditional methods are gradually being replaced by deep learning methods. Deep learning methods use neural network models such as x-vector or ECAPA-TDNN for voiceprint matching. However, this method requires the collection of a large amount of labeled speech training data to train the neural network model in order to achieve good recognition results. Otherwise, it is prone to problems with robustness and poor comparison accuracy. In addition, the computational complexity and cost are relatively high.

[0003] With the development of large language model technology, large model technologies based on audio or video have emerged, which can be used to directly process audio or video data, rather than being limited to text data. However, large language model technology is not fully utilized in voiceprint comparison scenarios. Therefore, it is urgent to develop a voiceprint comparison method based on large language models. Summary of the Invention

[0004] The purpose of this application is to propose a voiceprint comparison method, device and readable medium based on a large language model to address the above-mentioned technical problems.

[0005] In a first aspect, the present invention provides a voiceprint comparison method based on a large language model, comprising the following steps:

[0006] Acquire the first speech to be compared and the second speech to be compared that are collected separately and splice them into a merged speech;

[0007] Constructing and training a voiceprint comparison model based on a large language model to obtain a trained voiceprint comparison model. The voiceprint comparison model includes an audio encoder, an adapter, a text encoder, an improved large language model ontology structure, and a text decoder. The improved large language model ontology structure includes a pre-trained large language model ontology structure and a LoRA module set in parallel.

[0008] Obtain a prompt word for judging whether the first voice to be compared and the second voice to be compared are the voices of the same person by merging the voice, input the merged voice and the prompt word into the trained voiceprint comparison model, the merged voice first passes through the audio encoder to obtain voice coding features; the prompt word passes through the text encoder to obtain text coding features; the voice coding features are input into the adapter, the voice coding features are dimensionally converted to obtain voice coding features after dimension conversion; the text coding features and the voice coding features after dimension conversion are spliced and input into the ontology structure of the improved large language model, the output features of the ontology structure of the pre-trained large language model and the output features of the LoRA module are added to obtain an output token sequence, and the output token sequence passes through a text decoder to obtain the corresponding output text.

[0009] Preferably, the audio encoder includes a pre-trained transformer-structured speech encoder or a pre-trained wav2vec model, and the pre-trained transformer-structured speech encoder includes a feature extraction module and an encoding module of a wishper model.

[0010] Preferably, the adapter includes a first convolutional layer, a first ReLU activation function layer, a second convolutional layer and a second ReLU activation function layer connected in sequence.

[0011] Preferably, the first voice to be compared and the second voice to be compared in the merged voice are distinguished by duration; multiple prompt words are set, and one of them is randomly selected each time; the output text includes yes or no, if the output text is yes, it is determined that the first voice to be compared and the second voice to be compared are the voices of the same person, if the output text is no, it is determined that the first voice to be compared and the second voice to be compared are the voices of different people.

[0012] As a preference, the training process of the voiceprint comparison model adopts two supervised trainings, specifically:

[0013] During the first training process, the parameters of the pre-trained large language model's main structure are frozen, and the parameters of the LoRA module, audio encoder, and adapter are adjusted to obtain a trained voiceprint comparison model.

[0014] During the second training process, the voiceprint comparison model after the first training is fine-tuned, and the parameters of the pre-trained large language model, the LoRA module after the first training, the audio encoder and the adapter are adjusted to obtain the trained voiceprint comparison model.

[0015] Preferably, the pre-trained large language model includes a thousand-question large model.

[0016] In a second aspect, the present invention provides a voiceprint comparison device based on a large language model, comprising:

[0017] A speech acquisition module is configured to acquire the first speech to be compared and the second speech to be compared that are collected separately and splice them into a merged speech;

[0018] A model construction module is configured to construct and train a voiceprint comparison model based on a large language model to obtain a trained voiceprint comparison model, wherein the voiceprint comparison model includes an audio encoder, an adapter, a text encoder, an improved large language model ontology structure, and a text decoder, wherein the improved large language model ontology structure includes a pre-trained large language model ontology structure and a LoRA module set in parallel;

[0019] The comparison module is configured to obtain a prompt word for judging whether the first voice to be compared and the second voice to be compared are the voices of the same person by merging the voice, and input the merged voice and the prompt word into the trained voiceprint comparison model. The merged voice first passes through the audio encoder to obtain voice coding features; the prompt word passes through the text encoder to obtain text coding features; the voice coding features are input into the adapter, and the voice coding features are dimensionally converted to obtain voice coding features after dimension conversion; the text coding features and the voice coding features after dimension conversion are spliced and input into the ontology structure of the improved large language model, and the output features of the ontology structure of the pre-trained large language model and the output features of the LoRA module are added to obtain an output token sequence, and the output token sequence passes through a text decoder to obtain the corresponding output text.

[0020] In a third aspect, the present invention provides an electronic device comprising one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by one or more processors, the one or more processors implement the method described in any implementation manner in the first aspect.

[0021] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any implementation manner in the first aspect.

[0022] In a fifth aspect, the present invention provides a computer program product, comprising a computer program, which implements the method described in any implementation manner in the first aspect when the computer program is executed by a processor.

[0023] Compared with the prior art, the present invention has the following beneficial effects:

[0024] (1) The voiceprint comparison method based on a large language model proposed in the present invention combines two different voices to obtain a synthesized voice, which is then input into a voiceprint comparison model based on a large language model in combination with a prompt word. The contextual learning ability of the ontology structure of the pre-trained large language model is utilized to identify whether the two voices in the combined voice are the voices of the same person.

[0025] (2) The voiceprint comparison method based on the large language model proposed in the present invention utilizes the technical advantages of the large language model to implement voiceprint comparison technology, improve the robustness and accuracy of the model algorithm for voiceprint discrimination, and improve the voiceprint comparison effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0027] Figure 1 A flowchart of a voiceprint comparison method based on a large language model according to an embodiment of the present application;

[0028] Figure 2 A schematic diagram of a voiceprint comparison model of a voiceprint comparison method based on a large language model according to an embodiment of the present application;

[0029] Figure 3 A schematic diagram of a voiceprint comparison device based on a large language model according to an embodiment of the present application;

[0030] Figure 4 A schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0031] To make the objectives, technical solutions, and advantages of the present invention more apparent, the present invention will be further described in detail below with reference to the accompanying drawings. It is apparent that the embodiments described are only some, not all, of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without creative effort are intended to fall within the scope of protection of the present invention.

[0032] Figure 1 A voiceprint comparison method based on a large language model provided in an embodiment of the present application is shown, comprising the following steps:

[0033] S1, obtain the first speech to be compared and the second speech to be compared that are collected separately and splice them into a merged speech.

[0034] Specifically, one of the first and second voices to be compared may be the voice of an unknown person, while the other may be the voice of a known person. Therefore, by performing voiceprint comparison on the first and second voices to be compared, it is determined whether the first and second voices to be compared are the voices of the same person. If the first and second voices to be compared are the voices of the same person, it can be determined that the identity of the unknown person is the same as that of the known person. If both the first and second voices to be compared are the voices of unknown persons, voiceprint comparison can also be used to determine whether they are the voices of the same person and classify the voices of the same person.

[0035] S2, construct and train a voiceprint comparison model based on the large language model to obtain a trained voiceprint comparison model. The voiceprint comparison model includes an audio encoder, an adapter, a text encoder, an improved large language model ontology structure and a text decoder. The improved large language model ontology structure includes a pre-trained large language model ontology structure and a LoRA module set in parallel.

[0036] In a specific embodiment, the audio encoder includes a pre-trained transformer-structured speech encoder or a pre-trained wav2vec model, and the pre-trained transformer-structured speech encoder includes a feature extraction module and an encoding module of a wisher model.

[0037] In a specific embodiment, the adapter includes a first convolutional layer, a first ReLU activation function layer, a second convolutional layer, and a second ReLU activation function layer connected in sequence.

[0038] In a specific embodiment, the pre-trained large language model includes a thousand-question large model.

[0039] Specifically, refer to Figure 2The voiceprint comparison model proposed in the embodiment of the present application is composed of an audio encoder, an adapter, a text encoder, an improved large language model's main structure, and a text decoder. The audio encoder is used to encode the merged speech and convert the speech into vector data. The audio encoder can use a pre-trained transformer-based speech encoder or a pre-trained wav2vec model. The pre-trained transformer-based speech encoder can specifically use the feature extraction module and the encoding module of the wishper model, and does not require the decoding module of the wishper model. The merged speech is sequentially passed through the feature extraction module and the encoding module of the pre-trained wishper model or the pre-trained wav2vec model to obtain a corresponding vector representation. The vector representation is the speech coding feature, and then the speech coding feature needs to be input into the adapter to convert the speech coding feature into a format that can be accepted by the main structure of the pre-trained large language model. The main step is to perform dimensional conversion on the speech coding feature to obtain the dimensional converted speech coding feature, so that the dimension of the dimensional converted speech coding feature is the same as the dimension of the text coding feature obtained after the prompt word passes through the text encoder. For example, if the dimension of the speech coding feature is 96*N, while the text coding feature and the pre-trained large language model's main structure require a dimension of 112*N, the 96 dimension needs to be converted to 112, where N represents the data length. Therefore, the converted speech coding feature and text coding feature can be directly concatenated and fed simultaneously into the LoRA module in the improved large language model's main structure and the pre-trained large language model's main structure. The output features of the pre-trained large language model's main structure and the output features of the LoRA module are added together to obtain the output features of the improved large language model's main structure, i.e., the output token sequence. The output token sequence is then decoded by the text decoder to obtain the corresponding output text.

[0040] In one embodiment, the improved large language model's ontology structure is to set up a LoRA module in parallel on the basis of the traditional large language model's ontology structure, that is, a low-rank matrix is added to the side of the pre-trained large language model's ontology structure, and the LoRA module is used as an intermediate component in the training process of the large language model. During training, the parameters of the large language model's ontology structure are frozen, and only the parameters of the low-rank matrix of the LORA module are adjusted, thereby effectively reducing the amount of training. The LORA module is an existing structure, and its details are not repeated here. The text encoder and text decoder use traditional rule-based word segmenters, and both do not belong to neural networks, so they do not participate in training. In one example, the pre-trained large language model can adopt a thousand-question large model, such as Qwen2. The ontology structure of the large language model includes an embedding layer, a transformer structure, and an output layer. The ontology structure of the large language model itself has the ability to learn context, so it has the potential to judge the difference between the front and back of a synthetic speech.

[0041] Before training, you need to construct a prompt word to specify the task type. In the embodiment of this application, the prompt word is used to determine whether the first and second voices to be compared are the same person by merging the voices. As an example, you can construct 3 to 5 prompt words and randomly select one each time you use it. By randomly selecting prompt words, you can enrich the form of prompt words and reduce the model's reliance on the expression of prompt words.

[0042] In a specific embodiment, the training process of the voiceprint comparison model adopts two supervised trainings, specifically:

[0043] During the first training process, the parameters of the pre-trained large language model's main structure are frozen, and the parameters of the LoRA module, audio encoder, and adapter are adjusted to obtain a trained voiceprint comparison model.

[0044] During the second training process, the voiceprint comparison model after the first training is fine-tuned, and the parameters of the pre-trained large language model, the LoRA module after the first training, the audio encoder and the adapter are adjusted to obtain the trained voiceprint comparison model.

[0045] Specifically, the two supervised trainings used by the voiceprint comparison model in the embodiment of the present application can be trained using the same dataset, which includes the merged speech and its corresponding label, where the label is yes or no, corresponding to whether the two segments of speech in the merged speech are the speech of the same person or the two segments of speech in the merged speech are the speech of different people. The loss function used in the training process is the cross-entropy loss function, which is used to compare the difference between the label and the output text. The training process is as follows:

[0046] Step 1: Freeze the parameters of the pre-trained large language model's main structure and train the LoRA module, adapter, and audio encoder;

[0047] Step 2: Fine-tune the overall model, including training the LoRA module, adapter, audio encoder, and the pre-trained large language model's ontology structure;

[0048] The second step is to fully train the model and further improve the model recognition effect.

[0049] S3, obtain a prompt word for judging whether the first voice to be compared and the second voice to be compared are the voices of the same person by merging the voice, input the merged voice and the prompt word into the trained voiceprint comparison model, the merged voice first passes through the audio encoder to obtain voice coding features; the prompt word passes through the text encoder to obtain text coding features; the voice coding features are input into the adapter, the voice coding features are dimensionally converted to obtain voice coding features after dimension conversion; the text coding features and the voice coding features after dimension conversion are spliced and input into the ontology structure of the improved large language model, the output features of the ontology structure of the pre-trained large language model and the output features of the LoRA module are added to obtain an output token sequence, and the output token sequence passes through a text decoder to obtain the corresponding output text.

[0050] In a specific embodiment, the first voice to be compared and the second voice to be compared in the merged voice are distinguished by duration; multiple prompt words are set, and one of them is randomly selected each time; the output text includes yes or no, if the output text is yes, it is determined that the first voice to be compared and the second voice to be compared are the voices of the same person, if the output text is no, it is determined that the first voice to be compared and the second voice to be compared are the voices of different people.

[0051] Specifically, the trained voiceprint comparison model is deployed and applied. For the first voice A to be compared and the second voice B to be compared, they are spliced together into a merged voice, and any prompt word used in training is used as the input prompt word. At the same time, the merged voice and the prompt word are input into the trained voiceprint comparison model. According to the <yes> or <no> in the output result of the trained voiceprint comparison model, if the output is <yes>, it is considered that the two voices are the voices of the same person. If the output is <no>, it is judged that they are the voices of different people.

[0052] It should be noted that the first voice and the second voice in the embodiments of the present application are distinguished by duration. In one example, the first voice and the second voice can each be 4 seconds.

[0053] Further references Figure 3As an implementation of the methods shown in the above figures, this application provides an embodiment of a voiceprint comparison device based on a large language model. Figure 1 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0054] The embodiment of the present application provides a voiceprint comparison device based on a large language model, comprising:

[0055] The speech acquisition module 1 is configured to acquire the first speech to be compared and the second speech to be compared that are collected separately and splice them into a combined speech;

[0056] Model construction module 2 is configured to construct and train a voiceprint comparison model based on a large language model to obtain a trained voiceprint comparison model, wherein the voiceprint comparison model includes an audio encoder, an adapter, a text encoder, an improved large language model ontology structure, and a text decoder. The improved large language model ontology structure includes a pre-trained large language model ontology structure and a LoRA module set in parallel;

[0057] The comparison module 3 is configured to obtain a prompt word for judging whether the first voice to be compared and the second voice to be compared are the voices of the same person by merging the voice, and input the merged voice and the prompt word into the trained voiceprint comparison model. The merged voice first passes through the audio encoder to obtain voice coding features; the prompt word passes through the text encoder to obtain text coding features; the voice coding features are input into the adapter, and the voice coding features are dimensionally converted to obtain voice coding features after dimension conversion; the text coding features and the voice coding features after dimension conversion are spliced and input into the ontology structure of the improved large language model, and the output features of the ontology structure of the pre-trained large language model and the output features of the LoRA module are added to obtain an output token sequence, and the output token sequence passes through a text decoder to obtain the corresponding output text.

[0058] Figure 4 Schematic diagram of the hardware structure of the electronic device provided by the embodiment of the present invention. Figure 4 As shown, the electronic device of this embodiment includes: a processor 401 and a memory 402; wherein the memory 402 is used to store computer-executable instructions; and the processor 401 is used to execute the computer-executable instructions stored in the memory to implement the various steps performed by the electronic device in the above embodiment. For details, please refer to the relevant description of the above method embodiment.

[0059] Optionally, the memory 402 may be independent or integrated with the processor 401 .

[0060] When the memory 402 is independently provided, the electronic device further includes a bus 403 for connecting the memory 402 and the processor 401 .

[0061] An embodiment of the present invention further provides a computer storage medium, in which computer-executable instructions are stored. When the processor 401 executes the computer-executable instructions, the above method is implemented.

[0062] An embodiment of the present invention further provides a computer program product, including a computer program. When the computer program is executed by the processor 401, the above method is implemented.

[0063] In the embodiments provided herein, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the module division is merely a logical functional division. In actual implementation, other division methods may be used. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not implemented. In addition, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or module, which may be electrical, mechanical or other forms.

[0064] Modules described as separate components may or may not be physically separate, and components shown as modules may or may not be physical units, that is, they may be located in one place or distributed across multiple network elements. Some or all of these modules may be selected to implement the solution of this embodiment based on actual needs.

[0065] In addition, the functional modules in various embodiments of the present invention may be integrated into a single processing unit, each module may exist physically separately, or two or more modules may be integrated into a single unit. The units formed by the above modules may be implemented in the form of hardware or hardware plus software functional units.

[0066] The above-mentioned integrated module implemented in the form of a software function module can be stored in a computer-readable storage medium. The above-mentioned software function module is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) or processor 401 to perform some steps of the methods of various embodiments of the present application.

[0067] It should be understood that the processor 401 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), or application-specific integrated circuits (ASICs). A general-purpose processor may be a microprocessor, or the processor 401 may be any conventional processor 401. The steps of the method disclosed in the present invention may be directly implemented by the hardware processor 401, or implemented by a combination of hardware and software modules in the processor 401.

[0068] The memory 402 may include a high-speed RAM memory, and may also include a non-volatile storage NVM, such as at least one disk memory, and may also be a USB flash drive, a mobile hard disk, a read-only memory, a magnetic disk, or an optical disk.

[0069] Bus 403 can be an Industry Standard Architecture (ISA), a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. Bus 403 can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, the bus 403 in the drawings of this application is not limited to only one bus 403 or only one type of bus 403.

[0070] The storage medium may be implemented by any type of volatile or non-volatile memory device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The storage medium may be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0071] An exemplary storage medium is coupled to the processor 401, so that the processor 401 can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be an integral part of the processor 401. The processor 401 and the storage medium can be located in an application-specific integrated circuit (ASIC). Of course, the processor 401 and the storage medium can also exist as discrete components in an electronic device or a host control device.

[0072] Those skilled in the art will appreciate that all or part of the steps in the above-described method embodiments can be implemented using hardware associated with program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0073] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A voiceprint comparison method based on a large language model, characterized in that: The following steps are involved: Acquire the first speech to be compared and the second speech to be compared that are collected separately and splice them into a merged speech; Constructing and training a voiceprint comparison model based on a large language model to obtain a trained voiceprint comparison model, wherein the voiceprint comparison model includes an audio encoder, an adapter, a text encoder, an improved large language model ontology structure, and a text decoder, wherein the improved large language model ontology structure includes a pre-trained large language model ontology structure and a LoRA module set in parallel; Obtain a prompt word for determining whether the first voice to be compared and the second voice to be compared are the voices of the same person through the merged voice, input the merged voice and the prompt word into the trained voiceprint comparison model, the merged voice first passes through the audio encoder to obtain voice coding features; the prompt word passes through the text encoder to obtain text coding features; the voice coding features are input into the adapter, and the voice coding features are dimensionally converted to obtain voice coding features after dimension conversion; the text coding features and the voice coding features after dimension conversion are spliced and input into the ontology structure of the improved large language model, the output features of the ontology structure of the pre-trained large language model and the output features of the LoRA module are added to obtain an output token sequence, and the output token sequence passes through the text decoder to obtain the corresponding output text.

2. The voiceprint comparison method based on a large language model according to claim 1 is characterized in that: The audio encoder includes a pre-trained transformer-structured speech encoder or a pre-trained wav2vec model, and the pre-trained transformer-structured speech encoder includes a feature extraction module and an encoding module of a wisher model.

3. The voiceprint comparison method based on a large language model according to claim 1, characterized in that: The adapter includes a first convolutional layer, a first ReLU activation function layer, a second convolutional layer, and a second ReLU activation function layer connected in sequence.

4. The voiceprint comparison method based on a large language model according to claim 1, characterized in that: The first speech to be compared and the second speech to be compared in the merged speech are distinguished by duration; the prompt words are set to be multiple, and one of them is randomly selected each time; The output text includes yes or no. If the output text is yes, it is determined that the first voice to be compared and the second voice to be compared are the voices of the same person. If the output text is no, it is determined that the first voice to be compared and the second voice to be compared are the voices of different people.

5. The voiceprint comparison method based on a large language model according to claim 1 is characterized in that: The training process of the voiceprint comparison model adopts two supervised trainings, specifically: During the first training process, the parameters of the main structure of the pre-trained large language model are first frozen, and the parameters of the LoRA module, audio encoder and adapter are adjusted to obtain a voiceprint comparison model after one training; During the second training process, the voiceprint comparison model after the first training is fine-tuned, and the parameters of the main body structure of the pre-trained large language model, the LoRA module after the first training, the audio encoder and the adapter are adjusted to obtain the trained voiceprint comparison model.

6. The voiceprint comparison method based on a large language model according to claim 1, characterized in that: The pre-trained large language model includes a thousand-question large model.

7. A voiceprint comparison device based on a large language model, characterized in that: include: A speech acquisition module is configured to acquire the first speech to be compared and the second speech to be compared that are collected separately and splice them into a merged speech; A model construction module is configured to construct and train a voiceprint comparison model based on a large language model to obtain a trained voiceprint comparison model, wherein the voiceprint comparison model includes an audio encoder, an adapter, a text encoder, an improved large language model ontology structure, and a text decoder, wherein the improved large language model ontology structure includes a pre-trained large language model ontology structure and a LoRA module set in parallel; The comparison module is configured to obtain a prompt word for determining whether the first voice to be compared and the second voice to be compared are the voices of the same person through the merged voice, and input the merged voice and the prompt word into the trained voiceprint comparison model. The merged voice first passes through the audio encoder to obtain voice coding features; the prompt word passes through the text encoder to obtain text coding features; the voice coding features are input into the adapter, and the voice coding features are dimensionally converted to obtain voice coding features after dimension conversion; the text coding features and the voice coding features after dimension conversion are spliced and input into the ontology structure of the improved large language model, the output features of the ontology structure of the pre-trained large language model and the output features of the LoRA module are added to obtain an output token sequence, and the output token sequence passes through the text decoder to obtain the corresponding output text.

8. An electronic device comprising: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images