Retrieval dialogue model generation method and device, computer device, and storage medium
By combining pre-trained semantic models and target Gaussian classification layers, the problem of insufficient calibration capability of retrieval-based dialogue models under large datasets is solved, achieving higher accuracy in dialogue results.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- PING AN TECH (SHENZHEN) CO LTD
- Filing Date
- 2022-07-28
- Publication Date
- 2026-07-03
AI Technical Summary
Existing retrieval-based dialogue models have poor calibration capabilities when faced with large datasets, resulting in low accuracy.
By combining a pre-trained semantic model and a target Gaussian classification layer, a retrieval-based dialogue model is generated through feature extraction, semantic analysis, spectral normalization, and Gaussian classification layer training.
This improves the accuracy of semantic information understanding and result classification of dialogue data, thereby enhancing the accuracy of dialogue results.
Smart Images
Figure CN115203373B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of predictive models, and more particularly to a method, apparatus, computer device, and storage medium for generating retrieval-based dialogue models. Background Technology
[0002] Retrieval-based dialogue is a widely used technology. It aims to model the relationship between contextual semantics and candidate answers, ultimately using a matching algorithm to find the best candidate answer and quickly uncover implicit information within the text.
[0003] While existing retrieval-based dialogue models have achieved good performance using deep network modeling, their limited calibration capabilities make them prone to providing answers with a low probability of being correct when faced with large datasets. Summary of the Invention
[0004] Therefore, it is necessary to provide a method, apparatus, computer device, and storage medium for generating retrieval-based dialogue models to address the aforementioned technical problems, thereby solving the problem of low accuracy in retrieval-based dialogue models due to their poor calibration capabilities.
[0005] A method for generating a retrieval-based dialogue model, comprising:
[0006] Obtain a training dataset, which includes initial training data;
[0007] Feature extraction is performed on the initial training data to obtain the initial features of the initial training data;
[0008] The initial features of the initial training data are processed using a pre-trained semantic model to obtain the target features of the initial training data;
[0009] The target features of the initial training data are input into the initial Gaussian classification layer for model training, and the model loss function is obtained. When the model loss function reaches the convergence condition, the target Gaussian classification layer is obtained.
[0010] The retrieval-based dialogue model is generated based on the pre-trained semantic model and the target Gaussian classification layer.
[0011] A dialogue result retrieval method, comprising:
[0012] Obtain dialogue data;
[0013] The dialogue data is input into the aforementioned retrieval-based dialogue model to obtain the dialogue results.
[0014] A retrieval-based dialogue model generation device, comprising:
[0015] The training dataset module is used to obtain the training dataset, which includes the initial training data;
[0016] An initial feature module is used to extract features from the initial training data to obtain the initial features of the initial training data.
[0017] The target feature module is used to process the initial features of the initial training data using a pre-trained semantic model to obtain the target features of the initial training data.
[0018] The target Gaussian classification layer module is used to input the target features of the initial training data into the initial Gaussian classification layer for model training, obtain the model loss function, and obtain the target Gaussian classification layer when the model loss function reaches the convergence condition.
[0019] The retrieval-based dialogue model module is used to generate the retrieval-based dialogue model based on the pre-trained semantic model and the target Gaussian classification layer.
[0020] A dialogue result retrieval device, comprising:
[0021] The dialogue data module is used to acquire dialogue data;
[0022] The dialogue results module is used to input the dialogue data into any of the above-mentioned retrieval-based dialogue models to obtain the dialogue results.
[0023] A computer device includes a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor. When the processor executes the computer-readable instructions, it implements the above-described retrieval-based dialogue model generation method, or when the processor executes the computer-readable instructions, it implements the above-described dialogue result retrieval method.
[0024] One or more readable storage media storing computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the above-described retrieval-based dialogue model generation method, or the processors to implement the above-described dialogue result retrieval method when executing the computer-readable instructions.
[0025] The aforementioned method, apparatus, computer device, and storage medium for generating a retrieval-based dialogue model involve: acquiring a training dataset, including initial training data; extracting features from the initial training data to obtain initial features; processing the initial features of the initial training data using a pre-trained semantic model to obtain target features; inputting the target features of the initial training data into an initial Gaussian classification layer for model training, obtaining a model loss function, and obtaining a target Gaussian classification layer when the model loss function reaches convergence; and generating the retrieval-based dialogue model based on the pre-trained semantic model and the target Gaussian classification layer. This invention's retrieval-based dialogue model combines a pre-trained semantic model and a target Gaussian classification layer, enabling a more comprehensive understanding of the semantic information of the dialogue data and more accurate calibration of the result classification, thereby improving the accuracy of the dialogue results. Attached Figure Description
[0026] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0027] Figure 1 This is a schematic diagram of an application environment for a retrieval-based dialogue model generation method according to an embodiment of the present invention;
[0028] Figure 2 This is a flowchart illustrating a retrieval-based dialogue model generation method according to an embodiment of the present invention;
[0029] Figure 3 This is a schematic diagram of a retrieval-based dialogue model generation device in one embodiment of the present invention;
[0030] Figure 4 This is a schematic diagram of a computer device according to an embodiment of the present invention. Detailed Implementation
[0031] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0032] The retrieval-based dialogue model generation method provided in this embodiment can be applied to, for example... Figure 1In this application environment, the client communicates with the server. Clients include, but are not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented using a standalone server or a server cluster consisting of multiple servers.
[0033] In one embodiment, such as Figure 2 As shown, a retrieval-based dialogue model generation method is provided, which can be applied to... Figure 1 Taking the server-side as an example, the explanation includes the following steps:
[0034] S10. Obtain the training dataset, which includes the initial training data.
[0035] Understandably, the training dataset refers to the data used to train the retrieval-based dialogue model, and this training dataset includes the initial training data. Preferably, the training dataset includes a context dataset and a candidate answer dataset, where the context dataset includes T context training data points and the candidate answer dataset includes N answer training data points, where T and N are positive integers. The initial training data refers to the data containing the T context training data points and any one of the answer training data points.
[0036] S20. Extract features from the initial training data to obtain the initial features of the initial training data.
[0037] Understandably, feature extraction refers to concatenating T contextual training data and any answer training data from the initial training data to obtain the first training data; then, feature extraction is performed on the first training data to obtain the initial features of the initial training data. The initial features refer to the word segmentation features of the initial training data.
[0038] S30. The initial features of the initial training data are processed using a pre-trained semantic model to obtain the target features of the initial training data.
[0039] Understandably, the pre-trained semantic model is used to perform semantic analysis and spectral normalization on the initial features of the initial training data. Preferably, this pre-trained semantic model is a BERT model. The BERT model is a pre-trained language representation model, and its goal is to obtain a representation of the rich semantic information contained in the text by training on a large-scale unlabeled corpus. The target feature refers to the feature containing rich semantic information obtained by performing semantic analysis and spectral normalization on the initial features of the initial training data. Specifically, the semantic analysis layer in the pre-trained semantic model is used to perform semantic analysis on the initial features of the initial training data to obtain the semantic features of the initial training data. Then, the spectral normalization layer in the pre-trained semantic model is used to perform spectral normalization on the semantic features of the initial training data to obtain the target features of the initial training data.
[0040] S40. Input the target features of the initial training data into the initial Gaussian classification layer for model training, obtain the model loss function, and obtain the target Gaussian classification layer when the model loss function reaches the convergence condition.
[0041] Understandably, the target features of the initial training data are used as input to the initial Gaussian classification layer. In this initial Gaussian classification layer, the model is trained based on the target features to obtain the model loss function. The initial Gaussian classification layer where the model loss function converges is then used as the target Gaussian classification layer. Here, the target Gaussian classification layer refers to the trained Gaussian classification layer. Modeling using a Gaussian classification layer can improve the calibration effect of the retrieval-based dialogue model. Specifically, the stochastic Fourier transform algorithm is used to reduce the dimensionality of the target features in the initial training data, obtaining stochastic Fourier features. Then, the Laplace approximation algorithm is used to approximate the stochastic Fourier features, obtaining the Laplace posterior value. Subsequently, the initial parameters of the initial Gaussian classification layer are updated based on the Laplace posterior value until the initial parameters stabilize, resulting in the target Gaussian classification layer.
[0042] S50. Based on the pre-trained semantic model and the target Gaussian classification layer, generate the retrieval-based dialogue model.
[0043] Understandably, the retrieval-based dialogue model comprises a pre-trained semantic model and a target Gaussian classification layer. This model can quickly retrieve dialogue results corresponding to the input dialogue data. Because it combines a pre-trained semantic model and a target Gaussian classification layer, it achieves a more comprehensive understanding of the semantic information in the dialogue data and classifies the data more accurately, thereby improving the accuracy of the dialogue results.
[0044] In steps S10-S50, a training dataset is obtained, including initial training data; features are extracted from the initial training data to obtain initial features; a pre-trained semantic model is used to process the initial features of the initial training data to obtain target features; the target features of the initial training data are input into an initial Gaussian classification layer for model training, and a model loss function is obtained. When the model loss function reaches the convergence condition, a target Gaussian classification layer is obtained; based on the pre-trained semantic model and the target Gaussian classification layer, the retrieval-based dialogue model is generated. The retrieval-based dialogue model of this invention combines a pre-trained semantic model and a target Gaussian classification layer, enabling the retrieval-based dialogue model to understand the semantic information of the dialogue data more fully. Simultaneously, it makes the calibration of the result classification of the dialogue data more accurate, thereby improving the accuracy of the dialogue results.
[0045] Optionally, the training dataset includes a context dataset and a candidate answer dataset, wherein the context dataset includes T context training data and the candidate answer dataset includes N answer training data, where T and N are positive integers;
[0046] In step S20, namely, extracting features from the initial training data to obtain the initial features of the initial training data, the following steps are included:
[0047] S201. Concatenate the T context training data and any one of the answer training data to obtain the first training data;
[0048] S202. Extract features from the first training data to obtain the initial features of the initial training data.
[0049] Understandably, the training dataset includes a context dataset and a candidate answer dataset. The context dataset refers to the dialogue data, which includes T context training data points; that is, the dialogue data includes T context training data points. For example, the context dataset includes (A1, A2, A3, A4), where A1 is one context training data point. The candidate answer dataset refers to the set of candidate answers, which contains T answer training data points; that is, the answer training data are candidate answers. For example, the candidate answer dataset includes (B1, B2, B3), where B1 is one answer training data point. Dialogue data refers to the data generated during a dialogue, such as the dialogue data generated when a customer inquires about a business; in this case, the context training data would be a single sentence from that dialogue. The first training data is obtained by concatenating the T context training data points and any one answer training data point. For example, when the context dataset includes (A1, A2, A3, A4) and the candidate answer dataset includes (B1, B2, B3), the first training data can be (A1, A2, A3, A4, B1), (A1, A2, A3, A4, B2), or (A1, A2, A3, A4, B3). After obtaining the first training data, word segmentation and feature extraction can be performed on the first training data to obtain the initial features of the initial training data. The initial features refer to the word segmentation features of the initial training data.
[0050] In steps S201 and S202, the T sets of context training data and any one of the answer training data are concatenated to obtain first training data; feature extraction is performed on the first training data to obtain the initial features of the initial training data. This invention concatenates dialogue data and any one of the answer training data, and then performs feature extraction to obtain the overall features of the dialogue data and dialogue results, thereby improving the accuracy of the retrieval-based dialogue model.
[0051] Optionally, in step S201, namely, concatenating the T context training data and any one of the answer training data to obtain the first training data, includes:
[0052] S2011. The separating character is set between the T context training data and any of the answer training data to obtain the first concatenated data;
[0053] S2012. Set the start character at the beginning of the first concatenated data to obtain the first training data.
[0054] Understandably, a separator character is an identifier that separates two sentences. A start character is an identifier that marks the beginning of a sentence. Setting the separator character between T context training data and any answer training data clearly distinguishes the dialogue data from the candidate answers, resulting in higher accuracy for the trained retrieval-based dialogue model. Preferably, the separator character can be sep. When the context dataset includes (A1, A2, A3, A4) and the candidate answer dataset includes (B1, B2, B3), the first concatenated data can be (A1, A2, A3, A4, sep, B1). Setting the start character at the beginning of the first concatenated data allows for quick location of the dialogue data's start position, accelerating data processing. Preferably, the start character can be cls. If the first concatenated data is (A1, A2, A3, A4, sep, B1), then the first training data is (cls, A1, A2, A3, A4, sep, B1). Training the model based on the first training data with the separator and start characters set improves the accuracy of the retrieval-based dialogue model.
[0055] Optionally, in step S30, i.e., processing the initial features of the initial training data using a pre-trained semantic model to obtain the target features of the initial training data, the following steps are included:
[0056] S301. Using the semantic analysis layer in the pre-trained semantic model, perform semantic analysis on the initial features of the initial training data to obtain the semantic features of the initial training data.
[0057] S302. Using the spectral normalization layer in the pre-trained semantic model, the semantic features of the initial training data are processed by spectral normalization to obtain the target features of the initial training data.
[0058] Understandably, the semantic analysis layer refers to the processing layer used to perform semantic analysis on the initial features of the initial training data. The spectral normalization layer refers to the processing layer that performs spectral normalization on the semantic features of the initial training data. Spectral normalization can enhance the stability of the pre-trained semantic model during the training process, making the pre-trained semantic model more insensitive to input perturbations, thereby making the training process more stable and easier to converge.
[0059] Optionally, in step S40, i.e., inputting the target features of the initial training data into the initial Gaussian classification layer for model training, obtaining the model loss function, and obtaining the target Gaussian classification layer when the model loss function reaches the convergence condition, the following steps are included:
[0060] S401. Using the stochastic Fourier algorithm, the target features of the initial training data are reduced in dimensionality to obtain stochastic Fourier features.
[0061] S402. Using the Laplace approximation algorithm, the random Fourier features are approximated using Laplace to obtain the Laplace posterior value.
[0062] S403. Update the initial parameters of the initial Gaussian classification layer according to the Laplace posterior value until the initial parameters tend to stabilize, and obtain the target Gaussian classification layer.
[0063] Understandably, the stochastic Fourier transform algorithm can be used to determine the stochastic Fourier features of the target features. Here, the stochastic Fourier features refer to the Gaussian prior values of the target features in the initial training data. The stochastic Fourier transform algorithm reduces the dimensionality of the target features in the initial training data, thus reducing the complexity of data processing. Furthermore, the stochastic Fourier features are approximated using the Laplace approximation algorithm, which estimates the uncertainty of the linear weights of the stochastic Fourier features to obtain the Laplace posterior values. These linear weights serve as the initial parameters for the initial Gaussian classification layer. Then, the initial parameters of the initial Gaussian classification layer are updated based on the Laplace posterior values until they stabilize, resulting in the target Gaussian classification layer.
[0064] In steps S401-S403, the dimensionality of the target features in the initial training data can be reduced using the stochastic Fourier algorithm, thus reducing the complexity of data processing. Simultaneously, the posterior a posteriori are calculated using the Laplace approximation, improving the calibration effect.
[0065] In one embodiment, a dialogue result retrieval method is also provided, comprising the following steps:
[0066] S60, Obtain dialogue data;
[0067] S70. Input the dialogue data into any of the above-mentioned retrieval-based dialogue models to obtain the dialogue results.
[0068] Understandably, after obtaining any of the above-mentioned retrieval-based dialogue models, the dialogue data to be processed is acquired, and the dialogue data is input into the retrieval-based dialogue model to obtain the dialogue result, which is the target answer corresponding to the dialogue data. Because this retrieval-based dialogue model has strong calibration capabilities, the dialogue results retrieved through this retrieval-based dialogue model are more accurate.
[0069] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.
[0070] In one embodiment, a retrieval-based dialogue model generation apparatus is provided, which corresponds one-to-one with the retrieval-based dialogue model generation method described in the above embodiments. For example... Figure 3 As shown, the retrieval-based dialogue model generation device includes a training dataset module 10, an initial feature module 20, a target feature module 30, a target Gaussian classification layer module 40, and a retrieval-based dialogue model module 50. Detailed descriptions of each functional module are as follows:
[0071] Training dataset module 10 is used to acquire a training dataset, which includes initial training data;
[0072] Initial feature module 20 is used to extract features from the initial training data to obtain the initial features of the initial training data;
[0073] The target feature module 30 is used to process the initial features of the initial training data using a pre-trained semantic model to obtain the target features of the initial training data.
[0074] The target Gaussian classification layer module 40 is used to input the target features of the initial training data into the initial Gaussian classification layer for model training, obtain the model loss function, and obtain the target Gaussian classification layer when the model loss function reaches the convergence condition.
[0075] The retrieval-based dialogue model module 50 is used to generate the retrieval-based dialogue model based on the pre-trained semantic model and the target Gaussian classification layer.
[0076] Optionally, the training dataset includes a context dataset and a candidate answer dataset, wherein the context dataset includes T context training data and the candidate answer dataset includes N answer training data, where T and N are positive integers;
[0077] The initial feature module includes:
[0078] The first training data unit is used to concatenate the T context training data and any one of the answer training data to obtain the first training data;
[0079] An initial feature unit is used to extract features from the first training data to obtain the initial features of the initial training data.
[0080] Optionally, the first training data unit includes:
[0081] A separating character setting unit is used to set a separating character between the T context training data and any of the answer training data to obtain the first concatenated data;
[0082] The start character setting unit is used to set the start character at the beginning of the first concatenated data to obtain the first training data.
[0083] Optionally, the target feature module 30 includes:
[0084] The semantic feature unit is used to perform semantic analysis on the initial features of the initial training data using the semantic analysis layer in the pre-trained semantic model, so as to obtain the semantic features of the initial training data.
[0085] The target feature unit is used to perform spectral normalization processing on the semantic features of the initial training data using the spectral normalization layer in the pre-trained semantic model to obtain the target features of the initial training data.
[0086] Optionally, the target Gaussian classification layer module 40 includes:
[0087] The random Fourier feature unit is used to perform dimensionality reduction processing on the target features of the initial training data using the random Fourier algorithm to obtain random Fourier features.
[0088] The Laplace posterior value unit is used in the Laplace approximation algorithm to perform Laplace approximation on the random Fourier features to obtain the Laplace posterior value.
[0089] The target Gaussian classification layer unit is used to update the initial parameters of the initial Gaussian classification layer based on the Laplace posterior value until the initial parameters tend to stabilize, thereby obtaining the target Gaussian classification layer.
[0090] In one embodiment, a dialogue result retrieval device is provided, which corresponds one-to-one with the dialogue result retrieval method in the above embodiments.
[0091] A dialogue result retrieval device, comprising:
[0092] The dialogue data module is used to acquire dialogue data;
[0093] The dialogue results module is used to input the dialogue data into any of the above-mentioned retrieval-based dialogue models to obtain the dialogue results.
[0094] Specific limitations regarding the retrieval-based dialogue model generation device can be found in the limitations of the retrieval-based dialogue model generation method described above, and will not be repeated here. Each module in the aforementioned retrieval-based dialogue model generation device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.
[0095] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 4 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computational and control capabilities. The memory includes a readable storage medium and internal memory. The readable storage medium stores an operating system, computer-readable instructions, and a database. The internal memory provides an environment for the operation of the operating system and computer-readable instructions in the readable storage medium. The database stores data related to the retrieval-based dialogue model generation method. The network interface communicates with external terminals via a network connection. When the computer-readable instructions are executed by the processor, a retrieval-based dialogue model generation method is implemented. The readable storage medium provided in this embodiment includes both non-volatile and volatile readable storage media.
[0096] In one embodiment, a computer device is provided, including a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor, wherein the processor performs the following steps when executing the computer-readable instructions:
[0097] Obtain a training dataset, which includes initial training data;
[0098] Feature extraction is performed on the initial training data to obtain the initial features of the initial training data;
[0099] The initial features of the initial training data are processed using a pre-trained semantic model to obtain the target features of the initial training data;
[0100] The target features of the initial training data are input into the initial Gaussian classification layer for model training, and the model loss function is obtained. When the model loss function reaches the convergence condition, the target Gaussian classification layer is obtained.
[0101] The retrieval-based dialogue model is generated based on the pre-trained semantic model and the target Gaussian classification layer.
[0102] In one embodiment, one or more computer-readable storage media storing computer-readable instructions are provided. The readable storage media provided in this embodiment include non-volatile readable storage media and volatile readable storage media. The readable storage media stores computer-readable instructions, which, when executed by one or more processors, perform the following steps:
[0103] Obtain a training dataset, which includes initial training data;
[0104] Feature extraction is performed on the initial training data to obtain the initial features of the initial training data;
[0105] The initial features of the initial training data are processed using a pre-trained semantic model to obtain the target features of the initial training data;
[0106] The target features of the initial training data are input into the initial Gaussian classification layer for model training, and the model loss function is obtained. When the model loss function reaches the convergence condition, the target Gaussian classification layer is obtained.
[0107] The retrieval-based dialogue model is generated based on the pre-trained semantic model and the target Gaussian classification layer.
[0108] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing related hardware with computer-readable instructions. These computer-readable instructions can be stored in a non-volatile readable storage medium or a volatile readable storage medium. When executed, these computer-readable instructions can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
[0109] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.
[0110] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.
Claims
1. A method for generating a retrieval-based dialogue model, characterized in that, include: Obtain a training dataset, which includes initial training data; Feature extraction is performed on the initial training data to obtain the initial features of the initial training data; The initial features of the initial training data are processed using a pre-trained semantic model to obtain the target features of the initial training data; The target features of the initial training data are input into the initial Gaussian classification layer for model training, and the model loss function is obtained. When the model loss function reaches the convergence condition, the target Gaussian classification layer is obtained. The retrieval-based dialogue model is generated based on the pre-trained semantic model and the target Gaussian classification layer. The process of using a pre-trained semantic model to process the initial features of the initial training data to obtain the target features of the initial training data includes: The semantic analysis layer in the pre-trained semantic model is used to perform semantic analysis on the initial features of the initial training data to obtain the semantic features of the initial training data. The spectral normalization layer in the pre-trained semantic model is used to perform spectral normalization on the semantic features of the initial training data to obtain the target features of the initial training data.
2. The retrieval-based dialogue model generation method as described in claim 1, characterized in that, The training dataset includes a context dataset and a candidate answer dataset. The context dataset includes T context training data points, and the candidate answer dataset includes N answer training data points, where T and N are positive integers. The step of extracting features from the initial training data to obtain initial features of the initial training data includes: The first training data is obtained by concatenating the T context training data and any one of the answer training data. Feature extraction is performed on the first training data to obtain the initial features of the initial training data.
3. The retrieval-based dialogue model generation method as described in claim 2, characterized in that, The first training data is obtained by concatenating the T context training data and any one of the answer training data, including: The separating character is placed between the T context training data and any of the answer training data to obtain the first concatenated data; The first character is set at the beginning of the first concatenated data to obtain the first training data.
4. The retrieval-based dialogue model generation method as described in claim 1, characterized in that, The step of inputting the target features of the initial training data into an initial Gaussian classification layer for model training, obtaining a model loss function, and obtaining a target Gaussian classification layer when the model loss function reaches a convergence condition includes: The target features of the initial training data are reduced in dimensionality using the stochastic Fourier algorithm to obtain stochastic Fourier features. The random Fourier features are approximated using the Laplace approximation algorithm to obtain the Laplace posterior value. The initial parameters of the initial Gaussian classification layer are updated based on the Laplace posterior value until the initial parameters tend to stabilize, thus obtaining the target Gaussian classification layer.
5. A method for retrieving dialogue results, characterized in that, include: Obtain dialogue data; The dialogue data is input into the retrieval-based dialogue model according to any one of claims 1-4 to obtain the dialogue result.
6. A retrieval-based dialogue model generation device, characterized in that, include: The training dataset module is used to obtain the training dataset, which includes the initial training data; An initial feature module is used to extract features from the initial training data to obtain the initial features of the initial training data. The target feature module is used to process the initial features of the initial training data using a pre-trained semantic model to obtain the target features of the initial training data. The target feature module includes: a semantic feature unit, used to perform semantic analysis on the initial features of the initial training data using the semantic analysis layer in the pre-trained semantic model to obtain the semantic features of the initial training data; and a target feature unit, used to perform spectral normalization processing on the semantic features of the initial training data using the spectral normalization layer in the pre-trained semantic model to obtain the target features of the initial training data. The target Gaussian classification layer module is used to input the target features of the initial training data into the initial Gaussian classification layer for model training, obtain the model loss function, and obtain the target Gaussian classification layer when the model loss function reaches the convergence condition. The retrieval-based dialogue model module is used to generate the retrieval-based dialogue model based on the pre-trained semantic model and the target Gaussian classification layer.
7. A dialogue result retrieval device, characterized in that, include: The dialogue data module is used to acquire dialogue data; The dialogue results module is used to input the dialogue data into the retrieval-based dialogue model according to any one of claims 1-4 to obtain the dialogue results.
8. A computer device comprising a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor, characterized in that, When the processor executes the computer-readable instructions, it implements the retrieval-based dialogue model generation method as described in any one of claims 1 to 4, or when the processor executes the computer-readable instructions, it implements the dialogue result retrieval method as described in claim 5.
9. One or more readable storage media storing computer-readable instructions, characterized in that, When the computer-readable instruction is executed by one or more processors, the one or more processors perform the retrieval-based dialogue model generation method as described in any one of claims 1 to 4, or the processor executes the computer-readable instruction to implement the dialogue result retrieval method as described in claim 5.