Method and system for constructing enterprise service field pre-training dialog large language model

Abstract

The application discloses a construction method and system of a pre-training dialogue large language model in the field of enterprise services, and relates to the technical field of artificial intelligence. In the method, a preset base model is subjected to semantic prediction training based on an enterprise information dataset to obtain a first semantic prediction model; the first semantic prediction model is fine-tuned based on a preset instruction set in the field of enterprise services to obtain a second semantic prediction model; the second semantic prediction model is trained based on a preset question and answer score dataset to obtain a reward model; the second semantic prediction model is taken as a policy network of a reinforcement learning algorithm, the reward model is taken as a value network of the reinforcement learning algorithm, and a time difference error is calculated based on the policy network and the value network, and the parameters of the policy network are updated to obtain a pre-training dialogue large language model. The technical scheme provided by the application can improve the accuracy and naturalness of a model in answering questions in the process of constructing a question and answer model in the field of enterprise services.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a construction method and system of a pre-training dialogue large language model in the field of enterprise services. BACKGROUND

[0002] With the continuous development of artificial intelligence technology, machine dialogue systems are also constantly upgrading and improving. Machine dialogue systems can be applied to all aspects of life, and in the complex dialogue scenarios in the field of enterprise services, they are particularly important.

[0003] In the process of constructing a traditional question and answer model in the field of enterprise services, the traditional question and answer model is often constructed based on a predefined knowledge base or rule set, resulting in a lower degree of accuracy and naturalness in answering questions in some complex or special situations. Therefore, how to improve the accuracy and naturalness of the model in answering questions in the process of constructing a question and answer model in the field of enterprise services has become a problem that needs to be solved.

[0004] Therefore, a construction method of a pre-training dialogue large language model in the field of enterprise services is needed to solve the current technical problems. SUMMARY

[0005] The present application provides a construction method and system of a pre-training dialogue large language model in the field of enterprise services, which can improve the accuracy and naturalness of the model in answering questions in the process of constructing a question and answer model in the field of enterprise services.

[0006] In a first aspect, the present application provides a construction method of a pre-training dialogue large language model in the field of enterprise services, which comprises: obtaining an enterprise information dataset; performing semantic prediction training on a preset base model based on the enterprise information dataset to obtain a first semantic prediction model; obtaining a preset instruction set in the field of enterprise services; fine-tuning the first semantic prediction model based on the preset instruction set in the field of enterprise services to obtain a second semantic prediction model; obtaining a preset question and answer score dataset; training the second semantic prediction model based on the preset question and answer score dataset and a pairwise algorithm to obtain a reward model; taking the second semantic prediction model as a policy network of a reinforcement learning algorithm, taking the reward model as a value network of the reinforcement learning algorithm, and calculating a time difference error based on the policy network and the value network; updating the parameters of the policy network based on the time difference error to obtain a pre-training dialogue large language model.

[0007] By adopting the technical scheme, the first semantic prediction model is obtained by performing semantic prediction training on the preset base model based on the enterprise information dataset, so as to improve the semantic understanding and semantic generation capabilities of the model, thereby enabling the obtained first semantic prediction model to understand the problem sentence input by the user and correspondingly generate an answer sentence; the second semantic prediction model is obtained by fine-tuning the first semantic prediction model based on the preset instruction set of the enterprise service field, so as to improve the diversity of answers of the second semantic prediction model, that is, to enable the second semantic prediction model to generate a variety of answer sentences; the reward model is obtained by training the second semantic prediction model based on the preset question and answer score dataset and the pairwise algorithm, and the second semantic prediction model is trained based on the reinforcement learning algorithm, so that the finally obtained pre-training dialogue large language model can improve its answering capability in continuous learning, and can automatically adjust the score of the answer according to the context of the multi-round dialogue, so that the model answers the question more accurately and naturally.

[0008] Optionally, the enterprise information dataset includes a plurality of enterprise-related texts; each of the enterprise-related texts includes a plurality of enterprise-related sentences; the preset base model includes a text masking sub-model and a self-recurrence training sub-model; the semantic prediction training of the preset base model based on the enterprise information dataset to obtain the first semantic prediction model specifically includes: inputting the plurality of enterprise-related sentences one by one into the text masking sub-model to obtain the text span and the first damaged text corresponding to each of the enterprise-related sentences; performing semantic prediction training on the text masking sub-model based on the text span and the first damaged text to obtain a first semantic prediction sub-model; inputting the plurality of enterprise-related texts into the self-recurrence training sub-model to obtain second damaged texts and third damaged texts; performing semantic prediction training on the self-recurrence training sub-model based on the second damaged texts and the third damaged texts to obtain a second semantic prediction sub-model; and obtaining the first semantic prediction model based on the first semantic prediction sub-model and the second semantic prediction sub-model.

[0009] By adopting the technical scheme, the plurality of enterprise-related sentences are input into the text shielding sub-model one by one, the text span and the first damaged text corresponding to each enterprise-related sentence are obtained, the text shielding sub-model is trained based on the text span and the first damaged text, the first semantic prediction sub-model is obtained, so that the first semantic prediction sub-model has stronger understanding ability of the semantics of each sentence, the plurality of enterprise-related texts are input into the self-recurrent training sub-model, the second damaged text and the third damaged text are obtained, the self-recurrent training sub-model is trained based on the second damaged text and the third damaged text, the second semantic prediction sub-model is obtained, so that the obtained second semantic prediction sub-model can better understand the meaning of the text and the preceding and following sentences, and the first semantic prediction model can more accurately obtain the meaning of the user input sentence.

[0010] Optionally, the plurality of enterprise-related sentences are input into the text shielding sub-model one by one, and the text span and the first damaged text corresponding to each enterprise-related sentence are obtained, specifically comprising: based on the text shielding sub-model, a first blank area of a preset first proportion is randomly selected from each enterprise-related sentence; the first blank area includes a plurality of blank texts of continuous spans; the text span of the blank text is obtained; the first blank area is shielded from each enterprise-related sentence to obtain the first damaged text.

[0011] Optionally, the text shielding sub-model is trained based on the text span and the damaged text to obtain the first semantic prediction sub-model, specifically comprising: based on the text span and the damaged text, the prediction probability of each blank text is calculated; when the product of the prediction probabilities of a plurality of blank texts reaches the maximum, the model parameters at this time are obtained; based on the model parameters, the parameters of the text shielding sub-model are updated to obtain the first semantic prediction sub-model.

[0012] Optionally, the plurality of enterprise-related texts are input into the self-recurrent training sub-model to obtain the second damaged text and the third damaged text, specifically comprising: based on the self-recurrent training sub-model, a second blank area of a preset proportion range is selected from each enterprise-related text; the second blank area is shielded from the plurality of enterprise-related texts to obtain the second damaged text; based on the self-recurrent training sub-model, a third blank area of a preset second proportion is selected from the plurality of enterprise-related sentences; the third blank area is shielded from the plurality of enterprise-related sentences to obtain the third damaged text.

[0013] Optionally, the obtaining the preset instruction set of the enterprise service field specifically comprises: obtaining an initial instruction set of the enterprise service field; the initial instruction set of the enterprise service field comprises a plurality of initial instruction sequences, each of the initial instruction sequences comprises an initial instruction field, an initial input field and an initial output field; a preset first number of initial instruction sequences are randomly selected from the initial instruction set of the enterprise service field; the preset first number of initial instruction sequences are input into a large language model of a preset architecture to obtain a current continuation instruction sequence; the current continuation instruction sequence comprises a current continuation instruction field, a current continuation input field and a current continuation output field; wherein the verbs corresponding to the current continuation instruction field, the current continuation input field and the current continuation output field are all different from the verbs corresponding to the initial instruction field, the initial input field and the initial output field; a text similarity between the current continuation instruction sequence and the plurality of initial instruction sequences is calculated; it is determined whether the text similarity is greater than a preset similarity threshold; if yes, the current continuation instruction sequence is deleted; if no, the current continuation instruction sequence is stored in the preset instruction set of the enterprise service field; a next continuation instruction sequence is generated, the next continuation instruction sequence is taken as the current continuation instruction sequence, and the step of calculating the text similarity between the current continuation instruction sequence and the plurality of initial instruction sequences is repeatedly executed until the number of the continuation instruction sequences is greater than a preset second number, and the preset instruction set of the enterprise service field is obtained.

[0014] By adopting the above technical solution, the preset first number of initial instruction sequences are input into the large language model of the preset architecture to obtain the current continuation instruction sequence, and the number of the continuation instruction sequences is greater than the preset second number, and the preset instruction set of the enterprise service field is obtained, so that the preset instruction set of the enterprise service field constructed contains more diverse instruction sequences.

[0015] Optionally, the preset instruction set of the enterprise service field is used to fine-tune the first semantic prediction model to obtain a second semantic prediction model, specifically comprising: adjusting the learning rate of the network of the first semantic prediction model to a preset value, and setting the training rounds of the first semantic prediction model to a preset number of rounds; based on the first semantic prediction model, each of the continuation instruction sequences in the preset instruction set of the enterprise service field is traversed, and the parameters of the first semantic prediction model are updated based on a back propagation algorithm; the first semantic prediction model is subjected to the preset number of rounds of parameter update to obtain the second semantic prediction model.

[0016] By adopting the technical scheme, each continuation instruction sequence in the preset instruction set of the enterprise service field is traversed through the first semantic prediction model, and the parameters of the first semantic prediction model are updated based on the back propagation algorithm, so that the generated second semantic prediction model maximizes the diversity of the generated answers.

[0017] In a second aspect of the present application, a system for constructing a pre-trained dialog large language model in an enterprise service field is provided, which comprises an acquisition module, a semantic prediction training module, a model fine-tuning module, a reinforcement learning module, and a processing module. The acquisition module is configured to acquire enterprise information data sets. The semantic prediction training module is configured to perform semantic prediction training on a preset base model based on the enterprise information data sets to obtain a first semantic prediction model. The acquisition module is further configured to acquire a preset instruction set of the enterprise service field. The model fine-tuning module is configured to fine-tune the first semantic prediction model based on the preset instruction set of the enterprise service field to obtain a second semantic prediction model. The acquisition module is further configured to acquire a preset question and answer score data set. The reinforcement learning module is configured to train the second semantic prediction model based on the preset question and answer score data set and a pairwise algorithm to obtain a reward model. The reinforcement learning module is further configured to use the second semantic prediction model as a policy network of a preset reinforcement learning model, use the trained reward model as a value network of the preset reinforcement learning model, and calculate a time difference error based on the policy network and the value network. The processing module is configured to update parameters of the policy network based on the time difference error to obtain a pre-trained dialog large language model.

[0018] In a third aspect of the present application, an electronic device is provided, which comprises a processor, a memory, a user interface, and a network interface. The memory is configured to store instructions. The user interface and the network interface are configured to communicate with other devices. The processor is configured to execute the instructions stored in the memory to enable the electronic device to perform the method according to any one of the first aspect of the present application.

[0019] In a fourth aspect of the present application, a computer-readable storage medium is provided, which stores a computer program capable of being loaded and executed by a processor to perform the method according to any one of the first aspect of the present application.

[0020] In summary, the one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0021] 1. By training the preset base model based on the enterprise information dataset to obtain a first semantic prediction model, the semantic understanding ability and semantic generation ability of the model are improved, so that the first semantic prediction model obtained has the ability to understand the user input question sentence and correspondingly generate an answer sentence; then based on the preset instruction set of the enterprise service field, the first semantic prediction model is fine-tuned to obtain a second semantic prediction model, so as to improve the diversity of the answer of the second semantic prediction model, that is, to make the second semantic prediction model have the ability to generate a variety of answer sentences; then based on the preset question and answer score dataset and the pairwise algorithm, the second semantic prediction model is trained to obtain a reward model, and based on the reinforcement learning algorithm, the second semantic prediction model is trained, so that the finally obtained pre-training dialogue large language model can improve its answering ability in continuous learning, and can automatically adjust the score of the answer according to the context of the multi-round dialogue, so that the model answers the question more accurately and naturally.

[0022] 2. By inputting a plurality of enterprise related sentences into the text shielding sub-model one by one, the text span and the first damaged text corresponding to each enterprise related sentence are obtained; based on the text span and the first damaged text, the semantic prediction training of the text shielding sub-model is carried out to obtain a first semantic prediction sub-model, so that the first semantic prediction sub-model has stronger understanding ability of each sentence semantics; a plurality of enterprise related texts are input into the self-recurrent training sub-model to obtain second damaged text and third damaged text; based on the second damaged text and the third damaged text, the semantic prediction training of the self-recurrent training sub-model is carried out to obtain a second semantic prediction sub-model, so that the obtained second semantic prediction sub-model can better understand the meaning of the text and the preceding and following sentences, and further make the first semantic prediction model more accurately obtain the meaning of the user input sentence.

[0023] 3. By inputting a preset first number of initial instruction sequences into a large language model of a preset architecture, a current continuation instruction sequence is obtained, and until the number of continuation instruction sequences is greater than a preset second number, a preset instruction set of the enterprise service field is obtained, so that the preset instruction set of the enterprise service field constructed contains more diverse instruction sequences. BRIEF DESCRIPTION OF DRAWINGS

[0024] Figure 1 is one of the flowcharts of the method for constructing the enterprise service field pre-training dialogue large language model provided by the embodiments of the present application;

[0025] Figure 2 is another flowchart of the method for constructing the enterprise service field pre-training dialogue large language model provided by the embodiments of the present application;

[0026] Figure 3is a structural schematic diagram of a system for constructing an enterprise service field pre-training dialogue large language model provided by an embodiment of the present application.

[0027] Figure 4 is a structural schematic diagram of an electronic device disclosed by an embodiment of the present application.

[0028] Legend: 1, acquisition module; 2, semantic prediction training module; 3, model fine-tuning module; 4, reinforcement learning module; 5, processing module; 400, electronic device; 401, processor; 402, communication bus; 403, user interface; 404, network interface; 405, memory. DETAILED DESCRIPTION

[0029] In order for those skilled in the art to better understand the technical solutions in the specification, the technical solutions in the specification will be clearly and completely described below in conjunction with the drawings in the embodiments of the specification. Obviously, the described embodiments are only some of the embodiments of the present application, not all.

[0030] In the description of the embodiments of the present application, the words such as "for example" or "for instance" are used to represent an example, illustration or description. Any embodiment or design scheme described as "for example" or "for instance" in the embodiments of the present application should not be interpreted as more preferred or more advantageous than other embodiments or design schemes. Rather, the use of words such as "for example" or "for instance" is intended to present the relevant concept in a specific manner.

[0031] In the description of the embodiments of the present application, the term "a plurality of" means two or more. For example, a plurality of systems means two or more systems, and a plurality of screen terminals means two or more screen terminals. In addition, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly indicating the indicated technical features. Therefore, the features defined with "first" and "second" can explicitly or implicitly include one or more features. The terms "include", "contain", "have" and their variants mean "include but are not limited to", unless otherwise specifically emphasized.

[0032] The present application provides a method for constructing an enterprise service field pre-training dialogue large language model, referring to Figure 1 which shows one of the flow schematic diagrams of the method for constructing an enterprise service field pre-training dialogue large language model provided by an embodiment of the present application. The method comprises steps S11-S18, and the above steps are as follows:

[0033] Step S11: acquiring enterprise information dataset.

[0034] In the above step, the server obtains the enterprise information dataset.

[0035] Specifically, in the technical solution, the server first collects a large amount of enterprise service related data, which includes but is not limited to enterprise registration information, enterprise profile, enterprise patent information, enterprise recruitment, customer consultation, problem solving, customer feedback, etc. The server then preprocesses the enterprise service related data by desensitization; removes web page tag content and invalid description text; segments and sentences the text; calculates the similarity between texts using the word2vec algorithm, filters similar paragraphs, and a series of operations to obtain the enterprise information dataset.

[0036] Step S12: Based on the enterprise information dataset, the base model is trained for semantic prediction to obtain a first semantic prediction model.

[0037] In the above step, the server trains the base model for semantic prediction based on the enterprise information dataset to obtain a first semantic prediction model.

[0038] Specifically, in the technical solution, the base model preferably uses a Transformer model, and the server trains the Transformer model for semantic prediction based on the enterprise information dataset to improve the semantic understanding and semantic generation capabilities of the model, so that the obtained first semantic prediction model has the ability to understand user input question sentences and generate corresponding answer sentences. The step of training the base model for semantic prediction based on the enterprise information dataset to obtain a first semantic prediction model will be described in detail in subsequent embodiments, so it will not be described in detail here.

[0039] Step S13: Obtain a preset instruction set of the enterprise service field.

[0040] In the above step, the server obtains a preset instruction set of the enterprise service field.

[0041] Specifically, in the technical solution, the preset instruction set of the enterprise service field contains a large number and variety of instruction sequences. The steps of constructing the preset instruction set of the enterprise service field will be described in detail in subsequent embodiments, so it will not be described in detail here.

[0042] Step S14: Based on the preset instruction set of the enterprise service field, fine-tune the first semantic prediction model to obtain a second semantic prediction model.

[0043] In the above step, the server fine-tunes the first semantic prediction model based on the preset instruction set of the enterprise service field to obtain a second semantic prediction model.

[0044] Specifically, in the technical solution, the server fine-tunes the first semantic prediction model based on the preset instruction set of the enterprise service field to improve the diversity of the answers of the first semantic prediction model. That is, the obtained second semantic prediction model has the ability to generate a variety of answer sentences. The step of fine-tuning the first semantic prediction model based on the preset instruction set of the enterprise service field to obtain the second semantic prediction model will be described in detail in subsequent embodiments, and therefore will not be described in detail here.

[0045] Step S15: Obtain a preset question and answer score dataset.

[0046] In the above step, the server obtains a preset question and answer score dataset.

[0047] Specifically, in the technical solution, the preset question and answer score dataset is the "question-answer-answer score" dataset annotated by the technical personnel of the present solution, which contains the data set of questions, candidate answers and related answer scores. For each question, at least two candidate answers and corresponding answer scores are included.

[0048] Step S16: Train the second semantic prediction model based on the preset question and answer score dataset and the pairwise algorithm to obtain a reward model.

[0049] In the above step, the server trains the second semantic prediction model based on the preset question and answer score dataset and the pairwise algorithm to obtain a reward model.

[0050] Specifically, in the technical solution, the server first extracts relevant features from questions and answers, which can include words, syntactic structures, contextual information, entity recognition, etc. The purpose of feature extraction is to represent text as a vector form that can be processed by machine learning algorithms. Then use the pairwise algorithm to train the second semantic prediction model. For each question, select a pair of answers from the candidate answer set, compare their quality or relevance, and use the preset question and answer score as a label to guide the training of the model. You can use some evaluation indicators (such as mean square error or cross-entropy) to measure the difference between the predicted value and the label, and use gradient descent and other optimization algorithms to minimize the loss function. After training, use the second semantic prediction model to generate a reward model. The reward model assigns a reward score to each answer based on the semantic relationship between the question and the candidate answer.

[0051] Step S17: Use the second semantic prediction model as the policy network of the reinforcement learning algorithm, use the reward model as the value network of the reinforcement learning algorithm, and calculate the time difference error based on the policy network and the value network.

[0052] In the above step, the server takes the second semantic prediction model as a policy network of the reinforcement learning algorithm, takes the reward model as a value network of the reinforcement learning algorithm, and calculates the time difference error based on the policy network and the value network.

[0053] Specifically, in the technical solution, the server takes the second semantic prediction model as a policy network and interacts with the environment. In each interaction step, an action (e.g., generating a reply) is selected using the policy network according to the current state (e.g., the context of the dialogue). Then, the action is applied to the environment, and the reward signal and the next state (e.g., the user's feedback) are observed. The reward model is taken as a value network to evaluate the value of the current state. According to the predicted value of the reward model, the expected reward of the current state is calculated. According to the expected reward of the current state and the predicted reward value of the action selected by the policy network for the current state, the time difference error is calculated. The time difference error (TD error) is defined as the difference between the expected reward and the actual reward. It represents the difference between the prediction of the policy network for the value of the action in the current state and the evaluation of the value network. The method of calculating the time difference error can be that the policy network calculates the probability distribution of taking the action according to the current state, the value network estimates the score expectation of the state according to the current state and the action, and the time difference error is calculated. By adding the reinforcement learning process, the model can answer questions more naturally, and the model can improve its answering ability in continuous learning, and can automatically adjust the score of the answer according to the context of the multi-round dialogue, so that the model answers questions more accurately and naturally.

[0054] Step S18: updating the parameters of the policy network based on the time difference error to obtain a pre-trained dialogue large language model.

[0055] In the above step, the server updates the parameters of the policy network based on the time difference error to obtain a pre-trained dialogue large language model.

[0056] Specifically, in the technical solution, the server uses the time difference error to update the parameters of the policy network, so that the pre-trained dialogue large language model can quickly learn the optimal question and answer expression mode.

[0057] In one possible implementation, with reference to Figure 2 Fig. 2 shows a flowchart of a method for constructing an enterprise service field pre-trained dialogue large language model according to an embodiment of the present application. Step S12 specifically includes steps S121-S125, which are as follows:

[0058] Step S121: inputting a plurality of enterprise-related sentences one by one into the text screening sub-model to obtain a text span and a first damaged text corresponding to each enterprise-related sentence.

[0059] Specifically, in the technical solution, the enterprise information dataset includes a plurality of enterprise-related texts; each enterprise-related text includes a plurality of enterprise-related sentences; and the preset base model includes a text shielding sub-model and an autoregressive training sub-model. The server inputs the plurality of enterprise-related sentences into the text shielding sub-model one by one to obtain a text span and a first damaged text corresponding to each enterprise-related sentence. The step of obtaining the text span and the first damaged text corresponding to each enterprise-related sentence will be described in detail in subsequent embodiments, and therefore will not be described in detail here.

[0060] Step S122: performing semantic prediction training on the text shielding sub-model based on the text span and the first damaged text to obtain a first semantic prediction sub-model.

[0061] Specifically, in the technical solution, after obtaining the text span and the first damaged text, the server performs semantic prediction training on the text shielding sub-model based on the text span and the first damaged text to obtain a first semantic prediction sub-model. The step of obtaining the first semantic prediction sub-model will be described in detail in subsequent embodiments, and therefore will not be described in detail here.

[0062] Step S123: inputting the plurality of enterprise-related texts into the autoregressive training sub-model to obtain a second damaged text and a third damaged text.

[0063] Specifically, in the technical solution, the server inputs the plurality of enterprise-related texts into the autoregressive training sub-model to obtain a second damaged text and a third damaged text. The step of obtaining the second damaged text and the third damaged text will be described in detail in subsequent embodiments, and therefore will not be described in detail here.

[0064] Step S124: performing semantic prediction training on the autoregressive training sub-model based on the second damaged text and the third damaged text to obtain a second semantic prediction sub-model.

[0065] Specifically, in the technical solution, the server inputs the second damaged text and the third damaged text into the autoregressive training sub-model for semantic prediction training, and the autoregressive training sub-model predicts the content of the second damaged text and the third damaged text. In the forward propagation process of the autoregressive training sub-model, the input data passes through each layer of the neural network, and is calculated layer by layer to finally obtain an output result. In the forward propagation process, the output result obtained is compared with the true label to calculate the cross-entropy loss. The loss is propagated back along the network through the chain rule to calculate the contribution of each parameter to the loss. Starting from the output layer, the gradient of the output layer is calculated according to the gradient of the loss function. Then, the gradient is propagated back to the previous layer, and the gradient of each layer is calculated in turn. According to the gradient of each parameter obtained, the parameters of the autoregressive training sub-model are updated to obtain a second semantic prediction sub-model.

[0066] Step S125: Based on the first semantic prediction sub-model and the second semantic prediction sub-model, obtain the first semantic prediction model.

[0067] Specifically, in this technical solution, the first semantic prediction model includes a first semantic prediction sub-model and a second semantic prediction sub-model.

[0068] In one possible implementation, step S121 specifically includes the following steps: based on the text masking sub-model, randomly select a first blank area with a preset first proportion from each enterprise-related statement; the first blank area includes multiple consecutive spans of blank text; obtain the text span of the blank text; mask the first blank area from each enterprise-related statement to obtain the first damaged text.

[0069] Specifically, in this technical solution, the server, based on a text masking sub-model, randomly selects a first blank area from each enterprise-related statement, representing a preset first proportion. Each enterprise-related statement includes multiple minimum text units. For Chinese statements, the minimum text unit is each Chinese character; for English statements, the minimum text unit is a frequently occurring letter as defined by industry rules. The preset first proportion is preferably 15%, meaning 15% of the minimum text units from each enterprise-related statement are selected as the first blank area. The first blank area includes multiple consecutive spans of blank text, and each consecutive span of blank text includes at least one minimum text unit. For example, a statement x includes 100 minimum text units, i.e., [x1, x2, ..., x...]. i ,…,x 100 From 100 smallest text units, 15 smallest text units are selected as the first blank area. The consecutive blank text spanning the first blank area can be [x2, x3, ..., x7], [x...]. 15 ,x 16 ,…,x 19 ], [x 82 ,…,x 85 The text span of the blank text is obtained by determining the span {s1, s2, ..., s} of each blank text area based on the selected blank area. m In the example above, the spans of the blank text are 6, 5, and 4 respectively. The server then masks the first blank area from each enterprise-related statement, that is, it replaces multiple blank texts in the first blank area with multiple [MASK] characters, resulting in the first corrupted text. Therefore, in the example above, the resulting first corrupted text is [x1, [MASK], x8, ..., x 14 [MASK],x 20 ,…,x 81 [MASK],x 86x 100 ]。

[0070] In a possible implementation, step S122 specifically includes the following steps: based on the text span and the damaged text, a prediction probability of each blank text is calculated; when the product of the prediction probabilities of the multiple blank texts reaches the maximum, the model parameter at this time is obtained; and based on the model parameter, the parameter of the text masking sub-model is updated to obtain the first semantic prediction sub-model.

[0071] Specifically, in the technical solution, the server inputs the text span and the damaged text into the text masking sub-model to calculate the prediction probability of each blank text. The product of the prediction probabilities of the multiple blank texts reaching the maximum is the training target for training the first semantic prediction sub-model, and the training target can be represented by the following formula:

[0072]

[0073] wherein θ is the model parameter; Z m is all possible permutation combinations of the index sequence with the length of m; E is the expectation; p θ is the prediction probability of the blank text; is the span of the i th blank text; X corrupt is the first damaged text; S z<i is which represents that the word of [MASK] that has not been predicted is predicted using the word of [MASK] that has been predicted. It should be noted that, in order to sufficiently capture the mutual dependency between different text spans, the order of the text spans can be randomly arranged and input into the text masking sub-model. When the product of the prediction probabilities of the multiple blank texts reaches the maximum, the model parameter θ at this time is obtained, and the parameter of the text masking sub-model is updated to obtain the first semantic prediction sub-model.

[0074] In a possible implementation, step S123 specifically includes the following steps: based on the autoregressive training sub-model, a second blank area in a preset proportion range is selected from each enterprise-related text; the second blank area is shielded from the multiple enterprise-related texts to obtain a second damaged text; based on the autoregressive training sub-model, a third blank area in a preset second proportion is selected from the multiple enterprise-related sentences; and the third blank area is shielded from the multiple enterprise-related sentences to obtain a third damaged text.

[0075] Specifically, in the technical solution, the server selects a second blank area in a preset proportion range from each enterprise-related text based on an autoregressive training sub-model; the preset proportion range is preferably 50%-100%. The second blank area is shielded from the multiple enterprise-related texts to obtain a second damaged text, that is, 50%-100% of the enterprise-related texts are shielded from the multiple enterprise-related texts to obtain the second damaged text. The third blank area in a preset second proportion is selected from the multiple enterprise-related sentences based on the autoregressive training sub-model; the preset second proportion can be specifically set according to actual conditions, but the third blank area must be a complete enterprise-related sentence. The third blank area is shielded from the multiple enterprise-related sentences to obtain a third damaged text, that is, the complete enterprise-related sentence in the preset second proportion is shielded from the multiple enterprise-related sentences to obtain the third damaged text.

[0076] In a possible implementation, the step S13 specifically includes the following steps:

[0077] An instruction set of an initial enterprise service field is obtained; the instruction set of the initial enterprise service field includes multiple initial instruction sequences, and each initial instruction sequence includes an initial instruction field, an initial input field, and an initial output field.

[0078] Specifically, in the technical solution, the server obtains an instruction set of an initial enterprise service field, which is constructed by the technical personnel of the present solution, wherein the instruction set of the initial enterprise service field includes multiple initial instruction sequences, and each initial instruction sequence includes an initial instruction field (instruction), an initial input field (input), and an initial output field (output).

[0079] A preset first number of initial instruction sequences are randomly selected from the instruction set of the initial enterprise service field.

[0080] Specifically, in the technical solution, the server randomly selects a preset first number of initial instruction sequences from the instruction set of the initial enterprise service field. The preset first number is preferably 3, and the number of selected initial instruction sequences can also be specifically set according to actual conditions.

[0081] The preset first number of initial instruction sequences are input into a large language model of a preset architecture to obtain a current continuation instruction sequence; the current continuation instruction sequence includes a current continuation instruction field, a current continuation input field, and a current continuation output field; wherein the verbs corresponding to the current continuation instruction field, the current continuation input field, and the current continuation output field are all different from the verbs corresponding to the initial instruction field, the initial input field, and the initial output field.

[0082] Specifically, in the technical solution, the server inputs a preset first number of initial instruction sequences into a large language model of a preset architecture, and the large language model generates a current continuation instruction sequence. The large language model of the preset architecture is preferably a large language model based on a GPT-4.0 model architecture. The current continuation instruction field, the current continuation input field, and the current continuation output field in the current continuation instruction sequence generated by the large language model are all different from the verbs corresponding to the initial instruction field, the initial input field, and the initial output field in the initial instruction sequence.

[0083] Calculate the text similarity between the current continuation instruction sequence and the plurality of initial instruction sequences.

[0084] Specifically, in the technical solution, after generating a current continuation instruction sequence, the server calculates the text similarity between the current continuation instruction sequence and the plurality of initial instruction sequences. The method of calculating the text similarity is not limited in the present application.

[0085] Determine whether the text similarity is greater than a preset similarity threshold.

[0086] Specifically, in the technical solution, the server determines whether the text similarity is greater than a preset similarity threshold. The preset similarity threshold is preferably 0.7, and the preset similarity threshold can also be specifically set according to actual conditions.

[0087] If yes, delete the current continuation instruction sequence.

[0088] Specifically, in the technical solution, if the server determines that the text similarity is greater than 0.7, the current continuation instruction sequence is deleted.

[0089] If no, store the current continuation instruction sequence in a preset instruction set of the enterprise service field.

[0090] Specifically, in the technical solution, if the server determines that the text similarity is not greater than 0.7, the current continuation instruction sequence is stored in a preset instruction set of the enterprise service field.

[0091] Generate a next continuation instruction sequence, take the next continuation instruction sequence as the current continuation instruction sequence, and repeat the step of calculating the text similarity between the current continuation instruction sequence and the plurality of initial instruction sequences until the number of continuation instruction sequences is greater than a preset second number, and obtain the instruction set of the preset enterprise service field.

[0092] Specifically, in the technical solution, the server controls the large language model to generate a next continuation instruction sequence, takes the next continuation instruction sequence as a current continuation instruction sequence, and repeatedly performs the step of calculating the text similarity between the current continuation instruction sequence and the plurality of initial instruction sequences until the number of continuation instruction sequences is greater than a preset second number, to obtain an instruction set of a preset enterprise service field. The preset second number is preferably 1 million.

[0093] In a possible implementation, the step S14 specifically includes the following steps: adjusting a learning rate of a network of the first semantic prediction model to a preset value, and setting a training round of the first semantic prediction model to a preset round; based on the first semantic prediction model, traversing each continuation instruction sequence in the instruction set of the preset enterprise service field, and updating parameters of the first semantic prediction model based on a back propagation algorithm; performing parameter updating of the first semantic prediction model for the preset round to obtain a second semantic prediction model.

[0094] Specifically, in the technical solution, the server adjusts a learning rate of a network of the first semantic prediction model to a preset value, which is specifically set based on a data size of the instruction set of the preset enterprise service field. The server sets a training round of the first semantic prediction model to a preset round, which is preferably 5 rounds, so that the model reaches a convergence state after 5 rounds of training. The same preset round can be specifically set according to actual conditions. Then, based on the first semantic prediction model, each continuation instruction sequence in the instruction set of the preset enterprise service field is traversed, and parameters of the first semantic prediction model are updated based on a back propagation algorithm, so that the generated second semantic prediction model maximally improves the diversity of generated answers.

[0095] Reference Figure 3As shown in FIG. 1, which shows a structural schematic diagram of a system for constructing an enterprise service field pre-training dialogue large language model provided by an embodiment of the application. The system comprises: an acquisition module 1, a semantic prediction training module 2, a model fine-tuning module 3, a reinforcement learning module 4, and a processing module 5; the acquisition module 1 is configured to acquire an enterprise information dataset; the semantic prediction training module 2 is configured to perform semantic prediction training on a preset base model based on the enterprise information dataset to obtain a first semantic prediction model; the acquisition module 1 is further configured to acquire a preset instruction set of the enterprise service field; the model fine-tuning module 3 is configured to fine-tune the first semantic prediction model based on the preset instruction set of the enterprise service field to obtain a second semantic prediction model; the acquisition module is further configured to acquire a preset question and answer score dataset; the reinforcement learning module 4 is configured to train the second semantic prediction model based on the preset question and answer score dataset and a pairwise algorithm to obtain a reward model; the reinforcement learning module 4 is further configured to use the second semantic prediction model as a policy network of a preset reinforcement learning model, use the trained reward model as a value network of the preset reinforcement learning model, and calculate a time difference error based on the policy network and the value network; and the processing module 5 is configured to update parameters of the policy network based on the time difference error to obtain a pre-training dialogue large language model.

[0096] It should be noted that the device provided in the above embodiments, when realizing its functions, is only exemplified by the above division of functional modules. In actual applications, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the above-described functions. In addition, the device and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process is detailed in the method embodiments, which will not be described here.

[0097] The application also discloses an electronic device. Referring to Figure 4 , Figure 4 is a structural schematic diagram of an electronic device disclosed by an embodiment of the application. The electronic device 400 can include at least one processor 401, at least one network interface 404, a user interface 403, a memory 405, and at least one communication bus 402.

[0098] The communication bus 402 is configured to realize the connection and communication between the components.

[0099] The user interface 403 can include a display screen (Display) and a camera (Camera). Optionally, the user interface 403 can further include a standard wired interface and a wireless interface.

[0100] The network interface 404 can optionally include a standard wired interface and a wireless interface (such as a WI-FI interface).

[0101] The processor 401 can include one or more processing cores. The processor 401 connects various parts within the server through various interfaces and lines, performs various functions of the server and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 405, and calling data stored in the memory 405. Alternatively, the processor 401 can be implemented in at least one of a hardware form of a digital signal processing (DSP), a field-programmable gate array (FPGA), and a programmable logic array (PLA). The processor 401 can integrate a combination of one or more of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes operating systems, user interfaces, and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; and the modem is used for processing wireless communication. It can be understood that the above-mentioned modem can also not be integrated into the processor 401, but can be realized by a separate chip.

[0102] The memory 405 can include a random access memory (RAM) and can also include a read-only memory (ROM). Optionally, the memory 405 includes a non-transitory computer-readable storage medium. The memory 405 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 405 can include a program storage area and a data storage area, wherein the program storage area can store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area can store data involved in the above-mentioned various method embodiments, etc. The memory 405 can also be at least one storage device located away from the aforementioned processor 401. Referring to Figure 4 The memory 405 as a kind of computer readable storage medium can include an operating system, a network communication module, a user interface module and an application program.

[0103] In Figure 4In the electronic device 400 shown, the user interface 403 is mainly used to provide an interface for user input, and obtain data input by the user; and the processor 401 can be used to call an application stored in the memory 405, and when executed by one or more processors 401, cause the electronic device 400 to perform the method of one or more of the above embodiments. It should be noted that, for the foregoing method embodiments, in order to simply describe, they are all described as a combination of a series of actions, but those skilled in the art should know that the present application is not limited to the order of the actions described, because according to the present application, certain steps can be performed in other order or at the same time. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0104] In the above embodiments, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0105] In several embodiments provided in the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only schematic; the division of units is only a logical function division, and other division manners can be adopted in actual implementation, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some service interfaces, devices or units, and can be electrical or other forms.

[0106] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the present embodiment.

[0107] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically independently, or two or more units can be integrated into one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0108] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable memory. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a memory and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the embodiments of the present application. The aforementioned memory includes: a U disk, a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0109] The above are only exemplary embodiments of the present disclosure, and cannot limit the scope of the present disclosure. That is, any equivalent changes and modifications made in accordance with the teachings of the present disclosure are still within the scope of the present disclosure. Other embodiments of the present disclosure will be readily apparent to those skilled in the art upon considering the specification and practicing the true principles of the present disclosure.

[0110] The present application is intended to cover any variations, uses or adaptive changes of the present disclosure that follow the general principles of the present disclosure and include common knowledge or conventional technical means in the technical field not described in the present disclosure. The specification and examples are only considered as exemplary, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. A method for constructing an enterprise service domain pre-trained dialogue large language model, characterized in that, The method comprises: obtaining an enterprise information dataset, the enterprise information dataset comprising a plurality of enterprise-related texts, each of the enterprise-related texts comprising a plurality of enterprise-related sentences; performing semantic prediction training on a preset base model based on the enterprise information dataset to obtain a first semantic prediction model, the preset base model comprising a text masking submodel and a self-recurrent training submodel; obtaining a preset instruction set of an enterprise service field; fine-tuning the first semantic prediction model based on the preset instruction set of the enterprise service field to obtain a second semantic prediction model; obtaining a preset question and answer score dataset; training the second semantic prediction model based on the preset question and answer score dataset and a pairwise algorithm to obtain a reward model; using the second semantic prediction model as a policy network of a reinforcement learning algorithm and using the reward model as a value network of the reinforcement learning algorithm, and calculating a time difference error based on the policy network and the value network; updating parameters of the policy network based on the time difference error to obtain a pre-trained conversational large language model; wherein the semantic prediction training on the preset base model based on the enterprise information dataset to obtain the first semantic prediction model specifically comprises: inputting the plurality of enterprise-related sentences one by one into the text masking submodel to obtain a text span and a first damaged text corresponding to each of the enterprise-related sentences; performing semantic prediction training on the text masking submodel based on the text span and the first damaged text to obtain a first semantic prediction submodel; inputting the plurality of enterprise-related texts into the self-recurrent training submodel to obtain a second damaged text and a third damaged text; performing semantic prediction training on the self-recurrent training submodel based on the second damaged text and the third damaged text to obtain a second semantic prediction submodel; obtaining the first semantic prediction model based on the first semantic prediction submodel and the second semantic prediction submodel; wherein the inputting the plurality of enterprise-related sentences one by one into the text masking submodel to obtain a text span and a first damaged text corresponding to each of the enterprise-related sentences specifically comprises: randomly selecting a first blank area of a preset first proportion from each of the enterprise-related sentences based on the text masking submodel; the first blank area comprises a plurality of continuous span blank texts; obtaining a text span of the blank text; masking the first blank area from each of the enterprise-related sentences to obtain a first damaged text; wherein the performing semantic prediction training on the text masking submodel based on the text span and the damaged text to obtain a first semantic prediction submodel specifically comprises: calculating a prediction probability of each of the blank texts based on the text span and the damaged text; obtaining model parameters at the time when the product of the prediction probabilities of the plurality of blank texts reaches a maximum; updating parameters of the text masking submodel based on the model parameters to obtain the first semantic prediction submodel; The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps:

2. The method of claim 1, wherein, The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps:

3. The method of claim 2, wherein, The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the 4.A system for constructing an enterprise service domain pre-trained dialogue large language model, characterized in that, The construction method for implementing the enterprise service field pre-training conversational large language model according to claim 1, the system comprises: an acquisition module, a semantic prediction training module, a model fine-tuning module, a reinforcement learning module, and a processing module; The acquisition module is configured to acquire an enterprise information dataset; The semantic prediction training module is configured to perform semantic prediction training on a preset base model based on the enterprise information dataset to obtain a first semantic prediction model; The acquisition module is further configured to acquire a preset instruction set of the enterprise service field; The model fine-tuning module is configured to fine-tune the first semantic prediction model based on the preset instruction set of the enterprise service field to obtain a second semantic prediction model; The acquisition module is further configured to acquire a preset question and answer score dataset; The reinforcement learning module is configured to train the second semantic prediction model based on the preset question and answer score dataset and a pairwise algorithm to obtain a reward model; The reinforcement learning module is further configured to use the second semantic prediction model as a policy network of a preset reinforcement learning model, use the trained reward model as a value network of the preset reinforcement learning model, and calculate a time difference error based on the policy network and the value network; The processing module is configured to update parameters of the policy network based on the time difference error to obtain a pre-training conversational large language model.

5. An electronic device, comprising: The electronic device (400) comprises a processor (401), a memory (405), a user interface (403), and a network interface (404), the memory (405) is configured to store instructions, the user interface (403) and the network interface (404) are configured to communicate with other devices, and the processor (401) is configured to execute the instructions stored in the memory (405) to enable the electronic device (400) to perform the method according to any one of claims 1-3.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions, when the instructions are executed, the method steps according to any one of claims 1-3 are performed.

Images