Human-computer interaction method and device based on interaction large model in vertical class field

Abstract

The invention discloses a man-machine interaction method and device based on a large interaction model in the vertical class field. The method comprises the following steps: constructing an interaction model based on a large language model; corpora in a specific application scene are collected to generate training data, the training data are input into the interaction model for training, and a vertical class field interaction large model corresponding to the specific application scene is generated; wherein training data is processed by the neural network layer to generate one or more intermediate features, the fusion layer determines an intermediate feature distribution value, whether the intermediate features are fused with correction terms or not is determined according to the distance from the intermediate features to the intermediate feature distribution value, and output features of the neural network layer are generated; performing subsequent neural network layer processing on the output features and converting the output features into texts; and interacting with the user by adopting the vertical class field interaction large model. According to the method, the intermediate features can be selectively corrected, interference and damage to the original capability of the large language model caused by correction of all the intermediate features are avoided, and the interaction experience of a user in a specific application scene is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of natural speech processing, in particular to a human-computer interaction method and device based on a vertical field interaction large model. BACKGROUND

[0002] Large language models have super strong understanding capabilities and can generate human-like text responses for task-oriented dialogues and question answering, and are therefore widely used in interactions such as chatting and question answering. Through training of specific scene corpus, large language models can also support the needs of different application scenarios. However, the training process requires a large amount of computing resources and a large amount of data, and the training cost is very high.

[0003] And the corpus data of vertical fields with fewer users (such as education, finance, etc.) is difficult to reach the order of magnitude required for training, and in the training process, all intermediate features of each neural network layer of the large language model need to be corrected, causing interference and damage to the original capabilities of the large language model. The existing pre-trained language model often lacks understanding and mastery of the vertical field, affecting the user's interactive experience in specific application scenarios. SUMMARY

[0004] Therefore, the main purpose of the present application is to provide a human-computer interaction method and device based on a vertical field interaction large model, in order to at least partially solve at least one of the above technical problems.

[0005] In order to solve the above technical problems, the first aspect of the present application provides a human-computer interaction method based on a vertical field interaction large model, which comprises:

[0006] An interaction model is constructed based on a large language model; in the interaction model, each neural network layer of the large language model is respectively associated with a fusion layer and a correction layer;

[0007] Training data is generated by collecting corpus in a specific application scenario;

[0008] The training data is input into the interaction model for training to generate a vertical field interaction large model corresponding to the specific application scenario; specifically, the training data is processed by a neural network layer to generate one or more intermediate features, the fusion layer determines the intermediate feature distribution value of the neural network layer, and determines whether to fuse the intermediate feature with the correction term of the correction layer according to the distance from the intermediate feature to the intermediate feature distribution value, to generate the output feature of the neural network layer; the output feature is processed by a subsequent neural network layer and converted into readable text;

[0009] The vertical field interaction large model is used to interact with the user.

[0010] According to a preferred embodiment of the present application, the intermediate feature distribution value is a statistical value of the intermediate feature, and the statistical value includes an average statistical value and a variance statistical value, and the fusion layer determines the statistical value of the intermediate feature of the neural network layer includes:

[0011] The vector length corresponding to each intermediate feature of the neural network layer is obtained;

[0012] The average value of all the vector lengths is obtained to obtain the average statistical value;

[0013] The variance value of all the vector lengths is obtained to obtain the variance statistical value.

[0014] According to a preferred embodiment of the present application, the distance of each intermediate feature to the intermediate feature distribution value is the ratio of the distance between the vector length of each intermediate feature and the average statistical value to the variance statistical value.

[0015] According to a preferred embodiment of the present application, the fusion of the intermediate feature and the correction term of the correction layer is determined according to the distance of the intermediate feature to the intermediate feature distribution value to generate the output feature of the neural network layer includes:

[0016] When the distance of the intermediate feature to the intermediate feature distribution value is greater than a threshold value, the intermediate feature is fused with the correction term of the correction layer and outputted; when the distance of the intermediate feature to the intermediate feature distribution value is less than or equal to the threshold value, the intermediate feature is directly outputted; and the output feature of the neural network layer is generated.

[0017] According to a preferred embodiment of the present application, the output ends of the neural network layer and the correction layer are connected to the input end of the fusion layer, and the output end of the fusion layer is connected to the input end of the next neural network layer and the next correction layer respectively; the neural network layer processes each output feature of the previous fusion layer to obtain each intermediate feature and input the fusion layer, and the correction layer generates a corresponding correction term according to each output feature of the previous fusion layer and inputs the fusion layer; the fusion layer determines the intermediate feature distribution value according to the intermediate feature, and when the distance of the intermediate feature to the intermediate feature distribution value is greater than a threshold value, the intermediate feature is fused with the corresponding correction term and outputted to the next neural network layer and the next correction layer respectively; when the distance of the intermediate feature to the intermediate feature distribution value is less than or equal to the threshold value, the intermediate feature is directly outputted to the next neural network layer and the next correction layer respectively;

[0018] Or, the output end of the neural network layer is connected with the first input end of the fusion layer, the first output end of the fusion layer is connected with the second input end of the correction layer, the output end of the correction layer is connected with the second input end of the fusion layer, and the second output end of the fusion layer is connected with the input end of the next neural network layer and the first input end of the next correction layer respectively; the neural network layer processes each output feature of the previous fusion layer to obtain each intermediate feature and input the intermediate feature into the fusion layer; the fusion layer determines the intermediate feature distribution value according to the intermediate feature, and when the distance between the intermediate feature and the intermediate feature distribution value is greater than a threshold value, the feature identifier of the intermediate feature is output to the correction layer, the correction layer extracts the target feature corresponding to the feature identifier from the output feature of the previous fusion layer to generate a correction term, and inputs the correction term into the fusion layer; the fusion layer fuses the intermediate feature and the correction term according to the feature identifier and outputs them to the next neural network layer and the next correction layer respectively; when the distance between the intermediate feature and the intermediate feature distribution value is less than or equal to the threshold value, the fusion layer directly outputs the intermediate feature to the next neural network layer and the next correction layer respectively.

[0019] According to a preferred embodiment of the present application, during the training process, the parameters of all neural network layers are fixed, and only the parameters of each correction layer are adjusted.

[0020] To solve the above technical problems, the second aspect of the present application provides a human-computer interaction device based on a vertical field interaction large model, which comprises:

[0021] A construction module is configured to construct an interaction model based on a large language model; in the interaction model, each neural network layer of the large language model is respectively associated with a fusion layer and a correction layer.

[0022] A collection module is configured to collect corpus in a specific application scenario to generate training data.

[0023] A training module is configured to input the training data into the interaction model for training to generate a vertical field interaction large model corresponding to the specific application scenario; specifically, the training data is processed by a neural network layer to generate one or more intermediate features, the fusion layer determines the intermediate feature distribution value of the neural network layer, and determines whether to fuse the intermediate feature and the correction term of the correction layer according to the distance between the intermediate feature and the intermediate feature distribution value to generate the output feature of the neural network layer; the output feature is processed by a subsequent neural network layer and converted into readable text.

[0024] An interaction module is configured to interact with a user by using the vertical field interaction large model.

[0025] According to a preferred embodiment of the present application, the intermediate feature distribution value is a statistical value of the intermediate feature, and the statistical value includes an average statistical value and a variance statistical value; in the training module, the fusion layer determines the statistical value of the intermediate feature of the neural network layer, which includes:

[0026] acquire vector lengths corresponding to all intermediate features of the neural network layer;

[0027] take average values of all vector lengths to obtain average statistical values;

[0028] take variance values of all vector lengths to obtain variance statistical values.

[0029] According to a preferred embodiment of the present application, the distance of each intermediate feature to the intermediate feature distribution value is the ratio of the distance between the vector length of each intermediate feature and the average statistical value to the variance statistical value.

[0030] According to a preferred embodiment of the present application, in the training module, when the distance of the intermediate feature to the intermediate feature distribution value is greater than the threshold value, the intermediate feature is fused with the correction term of the correction layer and outputted; when the distance of the intermediate feature to the intermediate feature distribution value is less than or equal to the threshold value, the intermediate feature is directly outputted; and the output feature of the neural network layer is generated.

[0031] According to a preferred embodiment of the present application, the output ends of the neural network layer and the correction layer are connected with the input end of the fusion layer, and the output end of the fusion layer is connected with the input ends of the next neural network layer and the next correction layer, respectively; then in the training module, the neural network layer processes each output feature of the previous fusion layer to obtain each intermediate feature and input the fusion layer, and the correction layer generates a corresponding correction term according to each output feature of the previous fusion layer and inputs the fusion layer; the fusion layer determines the intermediate feature distribution value according to the intermediate feature, and when the distance of the intermediate feature to the intermediate feature distribution value is greater than the threshold value, the intermediate feature is fused with the corresponding correction term and outputted to the next neural network layer and the next correction layer, respectively; when the distance of the intermediate feature to the intermediate feature distribution value is less than or equal to the threshold value, the intermediate feature is directly outputted to the next neural network layer and the next correction layer, respectively.

[0032] Alternatively, the output of the neural network layer is connected to the first input of the fusion layer, the first output of the fusion layer is connected to the second input of the correction layer, the output of the correction layer is connected to the second input of the fusion layer, and the second output of the fusion layer is connected to the input of the next neural network layer and the first input of the next correction layer, respectively. In the training module, the neural network layer processes each output feature of the previous fusion layer to obtain each intermediate feature and inputs it into the fusion layer. The fusion layer determines the distribution value of the intermediate feature based on the intermediate feature, and when the distance between the intermediate feature and the distribution value of the intermediate feature is greater than a threshold, it outputs the feature identifier of the intermediate feature to the correction layer. The correction layer extracts the target feature corresponding to the feature identifier from the output feature of the previous fusion layer to generate a correction term, and inputs the correction term into the fusion layer. The fusion layer fuses the intermediate feature and the correction term according to the feature identifier and outputs them to the next neural network layer and the next correction layer, respectively. When the distance between the intermediate feature and the distribution value of the intermediate feature is less than or equal to the threshold, the fusion layer directly outputs the intermediate feature to the next neural network layer and the next correction layer, respectively.

[0033] According to a preferred embodiment of the present invention, during the training process, the parameters of all neural network layers are fixed, and only the parameters of each correction layer are adjusted.

[0034] In summary, this invention constructs an interactive model based on a pre-trained large language model. Through a fusion layer, the intermediate feature distribution values ​​of the neural network layer are determined based on the corpus of a specific application scenario. The distance between intermediate features and their distribution values ​​determines whether to fuse the intermediate features with the correction terms of the correction layer, generating the output features of the neural network layer. These output features are then processed by subsequent neural network layers and converted into readable text. This enables the interactive model to possess language understanding and generation capabilities for specific application scenarios, generating a large-scale interactive model for a specific vertical domain. Specifically, the intermediate feature distribution values ​​of the neural network layer represent the average value of all intermediate features in that layer, signifying most of the intermediate features. By comparing each intermediate feature with its distribution, the differences between each feature and most other features can be determined. These differences reflect the familiarity of the interactive model with the intermediate features. Features with low familiarity require correction, while those with high familiarity do not. This allows for selective correction of intermediate features, avoiding interference and damage to the original capabilities of the large language model caused by correcting all intermediate features, and improving the user's interactive experience in specific application scenarios. Attached Figure Description

[0035] To make the technical problems solved by this invention, the technical means adopted, and the technical effects achieved clearer, specific embodiments of this invention will be described in detail below with reference to the accompanying drawings. However, it should be noted that the drawings described below are merely drawings of exemplary embodiments of this invention. Those skilled in the art can obtain drawings of other embodiments based on these drawings without any creative effort.

[0036] Figure 1 This is a flowchart illustrating a human-computer interaction method based on a large interaction model of a vertical domain, provided by an embodiment of the present invention.

[0037] Figures 2a-2b These are schematic diagrams of the structures of two interaction models according to embodiments of the present invention;

[0038] Figure 3 This is a schematic diagram illustrating the modification of intermediate features according to an embodiment of the present invention;

[0039] Figure 4 This is a schematic diagram of the structural framework of a human-computer interaction device based on a large interaction model of a vertical domain, provided by an embodiment of the present invention. Detailed Implementation

[0040] Subject to the inventive concept, the structures, performance, effects or other features described in a particular embodiment may be combined in any suitable manner with one or more other embodiments.

[0041] In the description of specific embodiments, detailed descriptions of structures, performance, effects, or other features are provided to enable those skilled in the art to fully understand the embodiments. However, it is not excluded that those skilled in the art can implement the present invention with technical solutions that do not contain the above-described structures, performance, effects, or other features under specific circumstances. The flowcharts in the accompanying drawings are merely exemplary flowchart demonstrations and do not imply that the solution of the present invention must include all the contents, operations, and steps in the flowcharts, nor do they imply that the execution must be carried out in the order shown in the figures.

[0042] refer to Figure 1 , Figure 1 This is a flowchart illustrating a human-computer interaction method based on a large-scale interaction model of a vertical domain, as provided in an embodiment of the present invention. Figure 1 The method includes:

[0043] S1. Construct an interaction model based on a large language model;

[0044] The interaction model includes a large language model, where each neural network layer is associated with a fusion layer and a correction layer. The large language model is a language model composed of multiple neural network layers containing a large number of parameters, and each neural network layer can be implemented using a Transformer architecture. Optionally, the large language model can be a language model such as chatGPT, llama, or chatGLM. In this embodiment, the large language model is pre-trained on a massive corpus to possess general language understanding and generation capabilities. The fusion layer determines the intermediate feature distribution values ​​of the neural network layers and determines whether to fuse the intermediate features with the correction terms of the correction layer based on the distance between the intermediate features and their distribution values, generating the output features of the neural network layers. These output features are then processed by subsequent neural network layers and converted into readable text, thereby enabling the interaction model to possess language understanding and generation capabilities for specific application scenarios, generating a large-scale interaction model for a specific application scenario within a vertical domain.

[0045] For example, for each intermediate feature output by a neural network layer, the fusion layer determines the distribution value of the intermediate feature of the current neural network layer, and determines whether to fuse the intermediate feature with the correction term of the correction layer based on the distance from the intermediate feature to the distribution value, thereby generating the output feature of the neural network layer. Optionally, a threshold can be preset. When the distance from the intermediate feature to the distribution value is greater than the threshold, the fusion layer fuses the intermediate feature with the correction term of the correction layer and outputs it; when the distance from the intermediate feature to the distribution value is less than or equal to the threshold, the intermediate feature is directly output, thus generating the output feature of the neural network layer.

[0046] In one implementation, taking a large language model containing N neural network layers as an example, such as... Figure 2a In the interactive model, the outputs of the i-th neural network layer and the i-th correction layer are both connected to the input of the i-th fusion layer, and the output of the i-th fusion layer is connected to the inputs of the (i+1)-th neural network layer and the (i+1)-th correction layer, respectively. Without correction layers, all neural network layers are connected sequentially and pre-trained with massive amounts of corpus to possess general language understanding capabilities. Therefore, during the training process in step S3, the parameters of all neural network layers are fixed, and only the parameters of each correction layer are adjusted, thus saving computational resources, parameter storage resources, and a large amount of data. By training the correction and fusion layers with training data, they acquire language understanding capabilities for specific application scenarios, improving the user's interactive experience in specific application scenarios.

[0047] During training, the i-th neural network layer processes the output features of the (i-1)-th fusion layer to obtain the intermediate features of the i-th neural network layer, and inputs the intermediate features of the i-th neural network layer into the i-th fusion layer. The i-th correction layer generates corresponding correction terms based on the output features of the (i-1)-th fusion layer, and inputs the correction terms into the i-th fusion layer. The i-th fusion layer determines the distribution value of the intermediate features based on the intermediate features. When the distance between the intermediate feature of the i-th neural network layer and the distribution value of the intermediate features is greater than a threshold, the intermediate feature is fused with the corresponding correction term and output to the (i+1)-th neural network layer and the (i+1)-th correction layer respectively. When the distance between the intermediate feature of the i-th neural network layer and the distribution value of the intermediate features is less than or equal to the threshold, the intermediate feature is directly output to the (i+1)-th neural network layer and the (i+1)-th correction layer respectively.

[0048] Figure 2a In this process, the correction layer needs to generate correction terms for all intermediate features, requiring significant computational resources. To conserve computational resources, in another implementation, such as... Figure 2b In the interactive model, the fusion layer includes a first input terminal, a first output terminal, a second input terminal, and a second output terminal; the correction layer includes a first input terminal, a second input terminal, and an output terminal; the output terminal of the i-th neural network layer is connected to the first input terminal ai of the i-th fusion layer, the first output terminal bi of the i-th fusion layer is connected to the second input terminal pi of the i-th correction layer, the output terminal qi of the i-th correction layer is connected to the second input terminal ci of the i-th fusion layer, and the second output terminal di of the i-th fusion layer is connected to the input terminal of the (i+1)-th neural network layer and the first input terminal oi+1 of the (i+1)-th correction layer, respectively.

[0049] During training, the i-th neural network layer processes the output features of the (i-1)-th fusion layer to obtain intermediate features of the i-th neural network layer. These intermediate features are then input to the first input of the i-th fusion layer. The i-th fusion layer determines the distribution value of the intermediate features based on these features. When the distance between the intermediate feature of the i-th neural network layer and its distribution value is greater than a threshold, the feature identifier of the intermediate feature is output to the i-th correction layer via the first output bi. The i-th correction layer extracts the target feature corresponding to the feature identifier from the output features of the (i-1)-th fusion layer to generate a correction term. This correction term is input to the second input ci of the i-th fusion layer. The i-th fusion layer fuses the intermediate feature and the correction term based on the feature identifier and outputs them to the (i+1)-th neural network layer and the (i+1)-th correction layer via the second output di. When the distance between the intermediate feature of the i-th neural network layer and its distribution value is less than or equal to a threshold, the i-th fusion layer directly outputs the intermediate feature to the (i+1)-th neural network layer and the (i+1)-th correction layer via the second output di.

[0050] In this embodiment: the intermediate feature distribution value of the i-th neural network layer is used to identify the average value of all intermediate features of the i-th neural network layer. This value can represent most of the intermediate features of the neural network layer. By comparing each intermediate feature with the intermediate feature distribution value, the difference between each intermediate feature and most intermediate features can be determined. Since the interaction model is relatively familiar with most intermediate features, the difference between the two reflects the familiarity of the interaction model with the intermediate features. Intermediate features with low familiarity need to be corrected, while intermediate features with high familiarity do not need to be corrected. In this way, intermediate features can be selectively corrected, avoiding interference and damage to the original capabilities of the large language model by correcting all intermediate features, and improving the user's interactive experience in specific application scenarios. The intermediate feature distribution value of the i-th neural network layer can be determined by the statistical value of the intermediate features of the i-th neural network layer, the average value of all intermediate features of the i-th neural network layer, or the median value of all intermediate features of the i-th neural network layer. This invention does not impose a specific limitation.

[0051] Specifically, the correction layer can employ the lora structure, adapter structure, or prefix structure from Parameter-Efficient Fine-Tuning (PEFT). The fusion layer can fuse the correction terms and corresponding intermediate features by directly adding them or by adding them according to weights.

[0052] S2. Collect corpora from specific application scenarios to generate training data;

[0053] In this embodiment, the application scenario can be any scenario in any field that requires text or voice interaction. Preferably, it is a vertical field, such as law, education, finance, etc. After determining the application scenario, corpus can be directly obtained from the open-source corpus of that application scenario, or text corpus can be collected from publicly available web pages, books, news articles, academic papers, social media, etc.; voice corpus can also be collected from broadcasts, advertisements, etc., and then converted into text corpus.

[0054] Furthermore, to improve training effectiveness, after collecting the corpus, preprocessing such as data cleaning, word segmentation, part-of-speech tagging, and stop word removal can be performed to obtain training data.

[0055] It should be noted that the above is only used to describe the specific implementation details of steps S1 and S2, and does not limit the execution order of steps S1 and S2. Obviously, step S2 can also be executed before step S1.

[0056] S3. Input the training data into the interaction model for training to generate a large-scale interaction model for the vertical domain corresponding to the specific application scenario.

[0057] This step may specifically include: the training data is processed by a neural network layer to generate one or more intermediate features; the fusion layer determines the distribution value of the intermediate features of the neural network layer, and determines whether to fuse the intermediate features with the correction term of the correction layer based on the distance between the intermediate features and the distribution value of the intermediate features, thereby generating the output features of the neural network layer; the output features are then processed by a subsequent neural network layer and converted into readable text.

[0058] The specific processing of training data by each layer in the interactive model during training depends on the specific structure of the interactive model. Step S1 has already described this. Figure 2a and 2b The processing of training data by the two different interactive models with different structures will not be elaborated here.

[0059] The following will refer to Figure 3 Taking the LoRa structure used in the correction layer as an example, the processing procedure for a single intermediate feature during training is described in detail. Specifically: the statistical value of the intermediate feature of the i-th neural network layer is used as the intermediate feature distribution value of that layer. The distance from each intermediate feature of the i-th neural network layer to its statistical value determines whether to fuse the intermediate feature with the correction term of the correction layer to generate the output feature of the i-th neural network layer. The statistical value includes: average statistical value and variance statistical value. The statistical value of the intermediate feature of the i-th neural network layer can be obtained by obtaining the vector lengths corresponding to all intermediate features of the i-th neural network layer; taking the average of all vector lengths to obtain the average statistical value; and taking the variance of all vector lengths to obtain the variance statistical value. The distance from each intermediate feature of the i-th neural network layer to the statistical value is the ratio of the distance between the vector length of each intermediate feature and the average statistical value to the variance statistical value.

[0060] like Figure 3 As shown, assuming the intermediate features of the i-th neural network layer are h0, h1, ..., hm, and m is the number of features (which can be any integer), and the correction terms generated by the i-th correction layer are d0, d1, ..., dm, then the statistical value T of the intermediate features of the i-th neural network layer is:

[0061] T=Tmean_and_Tvariance(|h0|L2,|h1|L2,…,|hm|L2);

[0062] Where: ||L2 represents calculating the vector length, and T includes the mean Tmean of the vector lengths corresponding to h0, h1, ..., hm, and the variance Tvariance of the vector lengths corresponding to h0, h1, ..., hm.

[0063] For a certain intermediate feature h in the i-th neural network layer, its distance L to the statistical value T of the intermediate feature is:

[0064] L=(|h|L2-Tmean) / Tvariance;

[0065] In this example, a threshold S is preset. If L is greater than the threshold S, the correction term d corresponding to h is added to h and output. If L is less than the threshold S, d is not added to h and h is directly output.

[0066] In this way, for the intermediate features h0, h1, ..., hm of the i-th neural network layer, only a partial correction term is added to the intermediate features h, and these intermediate features h with added correction terms are all intermediate features that are far from the statistical values ​​of the intermediate features. The greater distance means that these intermediate features are more inclined to features that have not been seen by the large language model, so it is necessary to correct these intermediate features.

[0067] S4. Interact with users using the aforementioned vertical domain interaction model.

[0068] Taking education as a specific application scenario, the vertical domain interaction model, after the aforementioned training, possesses the ability to understand and generate terminology for various disciplines. Upon receiving user corpora, the model can process the corpora and output responses in voice or text format. These responses can be answers to professional questions, introductions to educational institutions, or notifications of educational information (such as class times and locations), etc. This completes voice or text interaction with users in educational scenarios, saving resources for educational institutions and improving the user experience.

[0069] Figure 4 This invention provides a human-computer interaction device based on a large-scale interaction model of a vertical domain, such as... Figure 4 As shown, the device includes:

[0070] Module 41 is used to construct an interactive model based on a large language model; in the interactive model, each neural network layer of the large language model is associated with a fusion layer and a correction layer respectively.

[0071] The acquisition module 42 is used to acquire training data for generating corpora in specific application scenarios;

[0072] Training module 43 is used to input the training data into the interaction model for training, and generate a large-scale interaction model for the vertical domain corresponding to the specific application scenario; specifically, it includes: the training data is processed by a neural network layer to generate one or more intermediate features; the fusion layer determines the distribution value of the intermediate features of the neural network layer, and determines whether to fuse the intermediate features with the correction term of the correction layer based on the distance between the intermediate features and the distribution value of the intermediate features, thereby generating the output features of the neural network layer; the output features are then processed by a subsequent neural network layer and converted into readable text;

[0073] Interaction module 44 is used to interact with users using the large interaction model of the vertical domain.

[0074] In one specific implementation, the intermediate feature distribution value is a statistical value of the intermediate feature, and the statistical value includes: a mean statistical value and a variance statistical value. Therefore, in the training module, the statistical values ​​of the intermediate features determined by the fusion layer for the neural network layer include:

[0075] Obtain the vector lengths corresponding to all intermediate features of the neural network layer;

[0076] Take the average of all vector lengths to obtain the average statistical value;

[0077] Take the variance values ​​of all vector lengths to obtain the variance statistics.

[0078] The distance from each intermediate feature to the distribution value of the intermediate feature is the ratio of the distance between the vector length of each intermediate feature and the mean statistic to the variance statistic.

[0079] Furthermore, in the training module, when the distance between the intermediate feature and the distribution value of the intermediate feature is greater than a threshold, the intermediate feature is fused with the correction term of the correction layer and output; when the distance between the intermediate feature and the distribution value of the intermediate feature is less than or equal to the threshold, the intermediate feature is directly output; thus generating the output feature of the neural network layer.

[0080] Specifically, the outputs of the neural network layer and the correction layer are both connected to the input of the fusion layer, and the output of the fusion layer is connected to the inputs of the next neural network layer and the next correction layer, respectively. In the training module, the neural network layer processes each output feature of the previous fusion layer to obtain intermediate features, which are then input into the fusion layer. The correction layer generates corresponding correction terms based on each output feature of the previous fusion layer and inputs them into the fusion layer. The fusion layer determines the distribution value of the intermediate features based on the intermediate features. When the distance between the intermediate feature and its distribution value is greater than a threshold, the intermediate feature is fused with the corresponding correction term and output to the next neural network layer and the next correction layer, respectively. When the distance between the intermediate feature and its distribution value is less than or equal to the threshold, the intermediate feature is directly output to the next neural network layer and the next correction layer, respectively.

[0081] Alternatively, the output of the neural network layer is connected to the first input of the fusion layer, the first output of the fusion layer is connected to the second input of the correction layer, the output of the correction layer is connected to the second input of the fusion layer, and the second output of the fusion layer is connected to the input of the next neural network layer and the first input of the next correction layer, respectively. In the training module, the neural network layer processes each output feature of the previous fusion layer to obtain each intermediate feature and inputs it into the fusion layer. The fusion layer determines the distribution value of the intermediate feature based on the intermediate feature, and when the distance between the intermediate feature and the distribution value of the intermediate feature is greater than a threshold, it outputs the feature identifier of the intermediate feature to the correction layer. The correction layer extracts the target feature corresponding to the feature identifier from the output feature of the previous fusion layer to generate a correction term, and inputs the correction term into the fusion layer. The fusion layer fuses the intermediate feature and the correction term according to the feature identifier and outputs them to the next neural network layer and the next correction layer, respectively. When the distance between the intermediate feature and the distribution value of the intermediate feature is less than or equal to the threshold, the fusion layer directly outputs the intermediate feature to the next neural network layer and the next correction layer, respectively.

[0082] During training, the parameters of all neural network layers are fixed, and only the parameters of each correction layer are adjusted.

[0083] Those skilled in the art will understand that the modules in the above-described device embodiments can be distributed throughout the device as described, or they can be modified accordingly and distributed in one or more devices different from the above embodiments. The modules in the above embodiments can be combined into one module, or they can be further divided into multiple sub-modules.

[0084] In summary, the present invention can be implemented by methods, apparatus, electronic devices, or computer-readable media that execute computer programs. In practice, some or all of the functions of the present invention can be implemented using general-purpose data processing devices such as microprocessors or digital signal processors (DSPs).

[0085] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the present invention is not inherently related to any specific computer, virtual device, or electronic device, and various general-purpose devices can also implement the present invention. The above descriptions are merely specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A human-computer interaction method based on a large-scale interaction model of vertical domains, characterized in that, The method includes: An interactive model is constructed based on a large language model; in the interactive model, each neural network layer of the large language model is associated with a fusion layer and a correction layer respectively; Collect corpora from specific application scenarios to generate training data; The training data is input into the interaction model for training to generate a large-scale interaction model for the specific application scenario in the vertical domain. Specifically, the training data is processed by a neural network layer to generate one or more intermediate features. The fusion layer determines the distribution value of the intermediate features of the neural network layer and determines whether to fuse the intermediate features with the correction term of the correction layer based on the distance between the intermediate features and the distribution value of the intermediate features, thereby generating the output features of the neural network layer. The output features are then processed by a subsequent neural network layer and converted into readable text. The interaction model of the aforementioned vertical domain is used to interact with users.

2. The method according to claim 1, characterized in that, The intermediate feature distribution values ​​include: mean statistical value and variance statistical value. Therefore, the statistical values ​​used by the fusion layer to determine the intermediate features of the neural network layer include: Obtain the vector lengths corresponding to all intermediate features of the neural network layer; Take the average of all vector lengths to obtain the average statistical value; Take the variance values ​​of all vector lengths to obtain the variance statistics.

3. The method according to claim 2, characterized in that, The distance from each intermediate feature to the distribution value of the intermediate feature is the ratio of the distance between the vector length of each intermediate feature and the mean statistic to the variance statistic.

4. The method according to claim 1, characterized in that, The step of determining whether to fuse the intermediate feature with the correction term of the correction layer based on the distance from the intermediate feature to the distribution value of the intermediate feature, and generating the output feature of the neural network layer, includes: When the distance between an intermediate feature and its distribution value is greater than a threshold, the intermediate feature is fused with the correction term of the correction layer and output; when the distance between an intermediate feature and its distribution value is less than or equal to a threshold, the intermediate feature is output directly; and the output features of the neural network layer are generated.

5. The method according to any one of claims 1-4, characterized in that, During training, the parameters of all neural network layers are fixed, and only the parameters of each correction layer are adjusted.

6. A human-computer interaction device based on a large-scale interaction model of a vertical domain, characterized in that, The device includes: A building module is used to construct an interactive model based on a large language model; in the interactive model, each neural network layer of the large language model is associated with a fusion layer and a correction layer respectively; The data acquisition module is used to collect corpora from specific application scenarios to generate training data. The training module is used to input the training data into the interaction model for training, generating a large-scale interaction model for the specific application scenario in the vertical domain. Specifically, it includes: the training data being processed by a neural network layer to generate one or more intermediate features; the fusion layer determining the distribution value of the intermediate features in the neural network layer; and determining whether to fuse the intermediate features with the correction term of the correction layer based on the distance between the intermediate features and the distribution value, generating the output features of the neural network layer; and further processing the output features by a subsequent neural network layer and converting them into readable text. The interaction module is used to interact with users using the large interaction model of the vertical domain.

7. The apparatus according to claim 6, characterized in that, The intermediate feature distribution values ​​include: mean statistical value and variance statistical value. Therefore, in the training module, the statistical values ​​used by the fusion layer to determine the intermediate features of the neural network layer include: Obtain the vector lengths corresponding to all intermediate features of the neural network layer; Take the average of all vector lengths to obtain the average statistical value; Take the variance values ​​of all vector lengths to obtain the variance statistics.

8. The apparatus according to claim 7, characterized in that, The distance from each intermediate feature to the distribution value of the intermediate feature is the ratio of the distance between the vector length of each intermediate feature and the mean statistic to the variance statistic.

9. The apparatus according to claim 6, characterized in that, In the training module, when the distance between the intermediate feature and the distribution value of the intermediate feature is greater than the threshold, the intermediate feature is fused with the correction term of the correction layer and output; when the distance between the intermediate feature and the distribution value of the intermediate feature is less than or equal to the threshold, the intermediate feature is directly output; and the output features of the neural network layer are generated.

10. The apparatus according to any one of claims 6-9, characterized in that, During training, the parameters of all neural network layers are fixed, and only the parameters of each correction layer are adjusted.

Images