Model training method, computer, storage medium and program product

Abstract

The invention discloses a model training method, a computer, a storage medium and a program product, and the method comprises the steps: constructing a second prompt text of each evaluation dimension according to a first prompt text, and a first text and a second text of each evaluation dimension; inputting the second prompt text into a first model to obtain a plurality of first vectors output by a hidden layer; determining a second vector of each evaluation dimension according to the plurality of first vectors; and according to the second vector, performing parameter adjustment on the hidden layer of the first model to obtain a second model. Through a mode of constructing the prompt text and directly adjusting hidden layer parameters, the problem of low efficiency of optimizing all parameters of the model based on large-scale annotation data in end-to-end fine tuning is avoided, and the efficiency of the model in reply verbal skill safety control is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a model training method, a computer, a storage medium, and a program product. Background Art

[0002] In business scenarios like intelligent customer service and financial consulting, which rely on large models as their underlying inference engines, the security of response language is crucial. These scenarios provide direct services to users, and the output of illegal content not only poses compliance risks to the company but can also cause substantial harm to user rights. Therefore, it is essential to optimize large models through technical means to avoid inappropriate responses.

[0003] Currently, end-to-end fine-tuning is primarily used to optimize the output of large models in the context of secure conversations by collecting complete samples of secure conversations in scenarios such as customer service. This is achieved through post-training methods such as supervised fine-tuning or reinforcement learning. However, the need to construct large datasets covering the entire business conversation context in real time, as well as to collect, label, and maintain this data, results in low efficiency in controlling the security of response text. Summary of the Invention

[0004] The embodiments of the present application provide a model training method, a computer, a storage medium, and a program product to construct a prompt text to obtain a first vector and adjust the model hidden layer parameters in a directionally manner using a second vector, thereby avoiding the inefficiency of large-scale data collection, labeling, and maintenance in end-to-end fine-tuning, and improving the efficiency of large models in controlling the security of reply text.

[0005] In a first aspect, an embodiment of the present application provides a model training method, the method comprising: Constructing a second prompt text for each evaluation dimension based on the first prompt text and the first text and the second text corresponding to each evaluation dimension of the plurality of evaluation dimensions, wherein the first text is a text that positively evaluates the first evaluation dimension, the second text is a text that negatively evaluates the first evaluation dimension, and the first evaluation dimension is any one of the plurality of evaluation dimensions; Inputting the second prompt text of each evaluation dimension into the first model, obtaining a plurality of first vectors of each evaluation dimension output by the hidden layer of the first model, wherein the first vectors correspond one-to-one to the second prompt text; determining a matrix of the first vectors based on the plurality of the first vectors; Calculating and obtaining a second vector for each evaluation dimension according to the matrix; According to the second vector of each evaluation dimension, parameters of the hidden layer of the first model are adjusted to obtain a second model.

[0006] In a second aspect, an embodiment of the present application provides a model training device, comprising: a construction module, configured to construct a second prompt text for each evaluation dimension based on a first prompt text and a first text and a second text corresponding to each evaluation dimension of a plurality of evaluation dimensions, wherein the first text is a text that positively evaluates the first evaluation dimension, the second text is a text that negatively evaluates the first evaluation dimension, and the first evaluation dimension is any one of the plurality of evaluation dimensions; An acquisition module is used to input the second prompt text of each evaluation dimension into the first model to obtain multiple first vectors of each evaluation dimension output by the hidden layer of the first model, and the first vectors correspond one-to-one to the second prompt text; and is used to determine the matrix of the first vector based on the multiple first vectors; and is used to calculate the second vector of each evaluation dimension based on the matrix; and is used to adjust the parameters of the hidden layer of the first model according to the second vector of each evaluation dimension to obtain the second model.

[0007] In a third aspect, an embodiment of the present application provides a computer, including: A memory, a processor, and an executable program code stored in the memory and executable on the processor, wherein the executable program code is configured to implement part or all of the steps described in any method in the first aspect.

[0008] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a model training program is stored. When the model training program is executed by a processor, some or all of the steps described in any method in the first aspect are implemented.

[0009] In a fifth aspect, embodiments of the present application provide a computer program product, wherein the computer program product includes a computer program operable to cause a computer to perform some or all of the steps described in any method of the first aspect of the embodiments of the present application. The computer program product may be a software installation package.

[0010] In an embodiment of the present application, a second prompt text for each evaluation dimension is first constructed based on a first prompt text and the first text and second text corresponding to each evaluation dimension in a plurality of evaluation dimensions, wherein the first text is a text that positively evaluates the first evaluation dimension, the second text is a text that negatively evaluates the first evaluation dimension, and the first evaluation dimension is any one of the plurality of evaluation dimensions; then, the second prompt text for each evaluation dimension is input into the first model to obtain a plurality of first vectors for each evaluation dimension output by the hidden layer of the first model, and the first vectors correspond one-to-one to the second prompt texts; based on the plurality of first vectors, a matrix of the first vector is determined; based on the matrix, a second vector for each evaluation dimension is calculated; finally, based on the second vector for each evaluation dimension, parameters of the hidden layer of the first model are adjusted to obtain a second model. By constructing prompt text based on positive evaluation text and negative evaluation text for each evaluation dimension, the prompt text is input into the model to obtain the first vector of the hidden layer output of the model, and then the parameters of the model hidden layer are adjusted according to the steering vector determined by multiple first vectors. By constructing prompt text and directly adjusting the hidden layer parameters, the low efficiency problem of optimizing all model parameters based on large-scale labeled data in end-to-end fine-tuning is avoided, and the efficiency of the model in controlling the security of reply words is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the background technology, the drawings required for use in the embodiments of the present application or the background technology will be described below.

[0012] Figure 1 This is a schematic diagram of the architecture of a model training system provided in an embodiment of the present application; Figure 2 This is a flow chart of a model training method provided in an embodiment of the present application; Figure 3 This is a flow chart of a method for constructing a second prompt text for each evaluation dimension provided by an embodiment of the present application; Figure 4 is a flow chart of a method for determining a matrix of a first vector provided in an embodiment of the present application; Figure 5 This is a flow chart of a method for calculating the second vector of each evaluation dimension provided by an embodiment of the present application; Figure 6 This is a flow chart of a method for adjusting parameters of a hidden layer of a first model to obtain a second model, provided in an embodiment of the present application; Figure 7 This is a structural diagram of a model training device provided in an embodiment of the present application; Figure 8 It is a structural diagram of a computer provided in an embodiment of the present application. DETAILED DESCRIPTION

[0013] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative work should fall within the scope of protection of the present invention.

[0014] The terms "first," "second," and "third," etc. in the specification, claims, and drawings of this application are used to distinguish between different objects, not to describe a particular order. In addition, the terms "including," "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or elements is not limited to the listed steps or elements, but may optionally include steps or elements not listed, or may optionally include other steps or elements inherent to the process, method, product, or apparatus.

[0015] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0016] In business scenarios like intelligent customer service and financial consulting, which rely on large models as their underlying inference engines, the security of response language is crucial. These scenarios provide direct services to users, and the output of illegal content not only poses compliance risks to the company but can also cause substantial harm to user rights. Therefore, it is essential to optimize large models through technical means to avoid inappropriate responses.

[0017] Currently, end-to-end fine-tuning is primarily used to optimize the output of large models in the context of secure conversations by collecting complete samples of secure conversations in scenarios such as customer service. This is achieved through post-training methods such as supervised fine-tuning or reinforcement learning. However, the need to construct large datasets covering the entire business conversation context in real time, as well as to collect, label, and maintain this data, results in low efficiency in controlling the security of response text.

[0018] In an embodiment of the present application, a second prompt text for each evaluation dimension is first constructed based on a first prompt text and the first text and second text corresponding to each evaluation dimension in a plurality of evaluation dimensions, wherein the first text is a text that positively evaluates the first evaluation dimension, the second text is a text that negatively evaluates the first evaluation dimension, and the first evaluation dimension is any one of the plurality of evaluation dimensions; then, the second prompt text for each evaluation dimension is input into the first model to obtain a plurality of first vectors for each evaluation dimension output by the hidden layer of the first model, and the first vectors correspond one-to-one to the second prompt texts; based on the plurality of first vectors, a matrix of the first vector is determined; based on the matrix, a second vector for each evaluation dimension is calculated; finally, based on the second vector for each evaluation dimension, parameters of the hidden layer of the first model are adjusted to obtain a second model.

[0019] By constructing prompt text based on positive evaluation text and negative evaluation text for each evaluation dimension, the prompt text is input into the model to obtain the first vector of the hidden layer output of the model, and then the parameters of the model hidden layer are adjusted according to the steering vector determined by multiple first vectors. By constructing prompt text and directly adjusting the hidden layer parameters, the low efficiency problem of optimizing all model parameters based on large-scale labeled data in end-to-end fine-tuning is avoided, and the efficiency of the model in controlling the security of reply words is improved.

[0020] The model training method, computer, storage medium and program product provided in the embodiments of the present application can be applied to Figure 1 In the model training system shown, see Figure 1 , Figure 1 This is a schematic diagram of the architecture of a model training system provided in an embodiment of the present application. The model training system 100 includes a client 101 and a server 102. The client 101 can communicate with the server 102 via a network. The client 101 refers to a device used by a user, such as a smartphone or computer. In this solution, the client 101 provides an interface for users to interact with the model training system 100. Through the client 101, users can interact with the model training system 100. Users can also use the client 101 to set multiple evaluation dimensions for evaluating the security of conversation data.

[0021] Server 102 is a remote computer used to process large amounts of computing tasks and store data. In this solution, server 102 is responsible for first constructing a second prompt text for each evaluation dimension based on a first prompt text and the first and second texts corresponding to each evaluation dimension in a plurality of evaluation dimensions, wherein the first text is a text that positively evaluates the first evaluation dimension, the second text is a text that negatively evaluates the first evaluation dimension, and the first evaluation dimension is any one of the plurality of evaluation dimensions; then, the second prompt text for each evaluation dimension is input into a first model to obtain multiple first vectors for each evaluation dimension output by the hidden layer of the first model, wherein the first vectors correspond one-to-one to the second prompt texts; based on the multiple first vectors, a matrix of the first vector is determined; based on the matrix, a second vector for each evaluation dimension is calculated; and finally, based on the second vector for each evaluation dimension, parameters of the hidden layer of the first model are adjusted to obtain a second model.

[0022] Based on this, the present application provides a model training method, a computer, a storage medium and a program product. The present application is described in detail below with reference to the accompanying drawings.

[0023] See also Figure 2 , Figure 2 This is a flow chart of a model training method provided in an embodiment of the present application. Figure 2 As shown, the model training method includes the following steps: S201 : Construct a second prompt text for each evaluation dimension based on a first prompt text and a first text and a second text corresponding to each evaluation dimension in a plurality of evaluation dimensions.

[0024] Among them, the execution subject of this method can be Figure 1 Server 102 of model training system 100.

[0025] Among them, the first prompt text refers to the text obtained by converting the conversation context data generated by the user in multiple rounds of conversations, the user's current question data, and the business data supporting the reply to the user's question according to a preset format.

[0026] Specifically, the conversation context data generated in multiple rounds of dialogue refers to the interaction content generated between the user and the system in multiple rounds of historical dialogue, including past questions, answers, user intentions, and the logical context of the dialogue. The user's current question data refers to the specific questions or needs raised by the user in the current round of dialogue. The business data that supports responding to user questions refers to professional data such as industry rules, product information, business processes, historical records, etc. that are directly related to the user's current question and are used to assist in generating accurate answers. Business data can be directly obtained from the company's internal database or industry database.

[0027] Among them, evaluation dimensions refer to the key evaluation angles or standards used to measure and evaluate conversation quality, compliance, and security in business scenarios such as intelligent customer service and financial consulting. For example, in the business scenario of financial consulting, conversation evaluation dimensions may include but are not limited to compliance (whether it complies with response specifications), accuracy (whether product information is accurately expressed), security (whether it contains sensitive content, whether it guides users to perform high-risk operations), and friendliness (whether the tone is appropriate).

[0028] The first text is a text that positively evaluates the first evaluation dimension, and the second text is a text that negatively evaluates the first evaluation dimension. The first evaluation dimension can be any one of multiple evaluation dimensions. Specifically, the first text is standard dialogue content that meets the requirements of each evaluation dimension (such as compliant risk warnings and accurate product introductions); the second text is an example of a violation of the evaluation dimension (such as incorrect investment advice and improper user information collection). For each evaluation dimension, multiple first and second texts can be constructed, and there is no limit on the number of first and second texts for each evaluation dimension.

[0029] The first and second texts can be obtained by directly constructing the first and second texts that match each of the multiple pre-set evaluation dimensions. Specifically, since the multiple evaluation dimensions are pre-set, internal business personnel can compile a first text that provides a positive evaluation and a second text that provides a negative evaluation for each evaluation dimension.

[0030] Preferably, the evaluation dimension in this method refers to an assessment perspective used to measure and evaluate conversation security. For example, the evaluation dimension is "correct use of customer service permissions in business scenarios." In this case, the first text constructed is "Please strictly follow business rule definitions, provide a solution, and do not exceed available permissions." The second text constructed is "Please comply with user requirements, try to meet user demands, and flexibly provide solutions."

[0031] In one possible implementation, see Figure 3 , Figure 3 This is a flow chart of a method for constructing a second prompt text for each evaluation dimension provided by an embodiment of the present application, such as Figure 3 As shown, according to the first prompt text, and the first text and the second text corresponding to each evaluation dimension in a plurality of evaluation dimensions, constructing the second prompt text for each evaluation dimension includes the following steps: The following processing is performed on the first text and the second text corresponding to each evaluation dimension: S301, merging the first prompt text and the first text to obtain a third prompt text; S302, merging the first prompt text and the second text to obtain a fourth prompt text; S303, constructing the third prompt text and the fourth prompt text to obtain the second prompt text.

[0032] Among them, the third prompt text is obtained by merging the first prompt text and the first text, and the fourth prompt text is obtained by combining the first prompt text and the second text, and then the second prompt text is constructed according to the third prompt text and the fourth prompt text.

[0033] Specifically, when merging the first prompt text with the first text, or merging the first prompt text and the second text, multiple texts can be directly spliced according to a preset logic, for example, the splicing mode is "first prompt text+separator+evaluation text", so that the positive evaluation and the negative evaluation form a contrast in the same prompt text.

[0034] Among them, the second prompt text is constructed according to the third prompt text and the fourth prompt text, which can adopt a structured splicing or comparative integration manner. Specifically, the structured splicing manner is to splice the third prompt text and the fourth prompt text through a separator, and the comparative integration manner is to embed the two into a preset structured template, and to distinguish the positive evaluation and the negative evaluation by field markers.

[0035] For example, the preset structured template can be as follows: {"context":"first prompt text", "positive_example":"third prompt text", "negative_example":"fourth prompt text"}.

[0036] As can be seen, in this example, the first prompt text is merged with the positive and negative evaluation texts respectively and then integrated into the second prompt text, so as to construct a prompt system containing the same evaluation dimension, which facilitates the model to accurately identify the compliance boundary of the reply according to the prompt text.

[0037] S202, inputting the second prompt text of each evaluation dimension into the first model to obtain a plurality of first vectors of each evaluation dimension of the hidden layer output of the first model.

[0038] Among them, when the second prompt text of each evaluation dimension is input into the first model, after the second prompt text of each evaluation dimension is input into the first model, the model will perform semantic analysis on the text, and its hidden layer will output the corresponding first vector for each second prompt text. These first vectors are numerical representations of the text in the semantic space, which contain the positive and negative evaluation information contained in the second prompt text and the semantic features related to the evaluation dimension, and each first vector corresponds one-to-one to the input second prompt text.

[0039] The second prompt text is in the form of a prompt. For example, the second prompt text may be the following prompt: User question: "How to invest in high-yield products?" The first text: "Investment involves risks and you need to make choices based on your own circumstances. It is recommended that you first understand the product risk level."

[0040] Second text: "Purchase this product to guarantee returns, no losses."

[0041] Among them, when the first text of the user question and the positive reply is input, each hidden layer of the model will output the corresponding positive first vector, which represents the semantic features of the compliant reply; when the second text of the user question and the negative reply is input, each hidden layer of the model will output the corresponding negative first vector, which represents the semantic features of the illegal reply; different first vectors (positive first vector + negative first vector) correspond to two replies to the same user question, which facilitates subsequent comparative analysis of the semantic differences between the two.

[0042] The core function of the first model is to semantically encode the input dialogue data pairs, generating first vector pairs that capture the characteristic differences between positive and negative responses. The first model can be a large pre-trained model (such as the BERT or GPT series) and can include multiple hidden layers. These layers are used to gradually transform the surface symbols (words and sentences) of the input text into deep semantic vectors through nonlinear transformations within a multi-layer neural network. This involves performing layer-by-layer abstract semantic encoding on the input text to generate the first vectors.

[0043] S203: Determine a matrix of the first vector according to the plurality of first vectors.

[0044] Among them, after the second prompt text is input into the first model to obtain multiple first vectors output by the hidden layer of the first model, it is necessary to further obtain a corresponding matrix based on the multiple first vectors.

[0045] Among them, the method of determining the matrix based on multiple first vectors is to directly splice multiple first vectors corresponding to each evaluation dimension to construct a matrix, which uses the first vector as a row element or a column element to form a structured representation of the semantic features under the evaluation dimension.

[0046] In one possible implementation, see Figure 4 , Figure 4 This is a flow chart of a method for determining a matrix of a first vector provided by an embodiment of the present application. Figure 4 As shown, determining the matrix of the first vector according to the plurality of the first vectors includes the following steps: S401, calculating the difference between every two first vectors among a plurality of first vectors to obtain a difference vector between every two first vectors; S402: Concatenate difference vectors between every two first vectors in a plurality of first vectors to obtain a matrix of the first vectors.

[0047] Among them, for multiple first vectors under the same evaluation dimension (these first vectors are generated after different prompt texts corresponding to the dimension are input into the model, and each prompt text contains positive evaluation information and negative evaluation information), the difference between each pair of first vectors is calculated to obtain a difference vector. Specifically, each two first vectors specifically refer to a vector pair consisting of a positive first vector generated by a positive evaluation prompt text and a negative first vector generated by a negative evaluation prompt text under the same evaluation dimension. For example, for a certain evaluation dimension "response safety", the positive first vector corresponding to the positive prompt text and the negative first vector corresponding to the negative prompt text form a pair. By calculating the difference between the two (such as vector subtraction), a difference vector representing the difference between "safe semantics" and "unsafe semantics" is obtained. This vector reflects the directional offset and distance characteristics of positive and negative evaluations in the model semantic space.

[0048] The calculation formula for obtaining the difference vector between every two first vectors is as follows: ; in, is the first positive vector, is the first negative vector, t is the different dialogue dimensions, l represents the lth layer of the model parameter matrix, that is, the lth hidden layer of the model, and i represents the i-th vector of the lth layer.

[0049] For example, the first forward vector of a hidden layer is h1 + =[0.8,0.2,0.5], the first negative vector is h1 - =[0.3,0.6,0.1], at this time the difference vector of the two first vectors is [0.5,-0.4,0.4].

[0050] After calculating the difference vector between every two first vectors, the difference vectors between every two first vectors in the first vectors are concatenated to obtain a matrix of the first vectors. Specifically, each difference vector is used as a row or column of the matrix, and then the matrix is ​​arranged and combined in sequence to obtain a matrix.

[0051] It can be seen that in this example, by calculating the difference vector of every two first vectors and splicing the difference vectors into a matrix, the matrix can be used to structure the storage of the semantic difference features of positive and negative evaluations under the same evaluation dimension, providing a basis for subsequent precise analysis of semantic boundaries and model optimization.

[0052] S204: Calculate and obtain the second vector of each evaluation dimension according to the matrix.

[0053] The matrix contains information about the first vector within the same evaluation dimension (e.g., the semantic differences between positive and negative evaluations). Based on this matrix, specific mathematical operations (such as eigenvalue decomposition and principal component analysis) are used to identify the vector representing the primary direction of change in the data within the matrix, which serves as the second vector. Specifically, the second vector can characterize the primary change or shift in the data. This second vector reflects the key semantic shifts within the corresponding evaluation dimension, providing guidance for subsequent adjustments to model parameters.

[0054] In one possible implementation, see Figure 5 , Figure 5 This is a flow chart of a method for calculating the second vector of each evaluation dimension provided by an embodiment of the present application, such as Figure 5 As shown, according to the matrix, calculating the second vector of each evaluation dimension includes the following steps: S501 , performing singular value decomposition on the matrix to calculate and obtain a second vector for each evaluation dimension.

[0055] Specifically, the matrix obtained in step S402 is recorded as matrix A, and the steps of performing singular value decomposition on matrix A are as follows: Step 1: Calculate A T A. The calculation here directly multiplies matrix A by the transposed matrix of matrix A.

[0056] Step 2: Calculate A T The eigenvalue of A. Among them, the eigenvalue is an important parameter that describes the properties of the matrix. In the singular value decomposition, A T The square root of the eigenvalue of A is the singular value of the matrix A. of The value of is the eigenvalue, and E is the identity matrix. By solving this determinant equation, we get the value of λ, and the repeated roots are recorded by their multiplicity.

[0057] Step 3: Calculate A TEigen vector corresponding to eigen value of A. Wherein, the eigen vector and the eigen value are mutually related, which is used to construct the decomposed matrix in singular value decomposition. For each eigen value λ, solve the homogeneous linear equations , whose non-zero solution x is the eigen vector corresponding to the eigen value. Different eigen values correspond to an eigen vector.

[0058] Step four, the eigen vector calculated in step three and inverse Schmidt orthogonalization. Take three eigen vectors α1, α2, α3 as an example, first let β1=α1; then β2 is orthogonal to β1, which is obtained by subtracting the projection of α2 in the direction of β1 from α2; β3 is orthogonal to β1 and β2, which is obtained by subtracting the projection of α3 in the direction of β1 and β2 from α3. Through this iterative method, a group of eigen vectors is converted into an orthogonal vector group.

[0059] Step five, arrange the eigen values and the corresponding eigen vectors in descending order.

[0060] Step six, construct the orthogonal matrix V composed of the orthogonalized eigen vectors and the diagonal matrix D containing the eigen values, and construct the diagonal matrix ∑ containing the singular values from the diagonal matrix D containing the eigen values. Specifically, the orthogonalized eigen vectors are arranged in columns to form the matrix V; the matrix D is a diagonal matrix, and the elements on the diagonal are sorted eigen values; the matrix ∑ is a diagonal matrix obtained by replacing the eigen values with their square roots (i.e. singular values) based on the matrix D (if A is an m×n matrix, the ∑ matrix is an m×n diagonal matrix, and the elements on the diagonal are singular values, and the other elements are all 0).

[0061] Step seven, construct the column vectors of the orthogonal matrix in singular value decomposition, i.e. the standard orthogonal vectors. Use the following formula: .

[0062] Wherein, , represents the singular value (λi is the eigen value of A T ), is the column vector of V matrix (i.e. the eigen vector of orthogonalized ATA). The calculation is essentially a linear transformation of matrix A and , and then normalized by singular value to obtain the standard orthogonal vector.

[0063] Step eight, the constructed standard orthogonal vector is expanded to m-dimensional vector space according to the method of standard orthogonal basis expansion, to form a new orthogonal matrix U.

[0064] Step nine, according to the above steps, the singular value decomposition result of matrix A is obtained. Specifically, the singular value decomposition formula is as follows: A=U∑VT .

[0065] The meanings of A, U, ∑, and V are explained in the above steps and will not be repeated here.

[0066] Among them, after the singular value decomposition of matrix A is performed through the above steps, the principal component corresponding to the first singular value that meets the preset value selection rules is determined to be the second vector. Among them, in the singular value decomposition, the principal component corresponding to the singular value is essentially a vector representation of the main change direction of the data in the high-dimensional space.

[0067] Among them, the preset value-taking rules can be set by the user in the client 101 of the model training system 100. The preset value-taking rules can be to take the first singular value, the largest singular value, or a singular value greater than a specific threshold, etc. There is no restriction on the preset value-taking rules here.

[0068] Preferably, the preset value rule is the maximum value rule, in which case the first singular value is the maximum singular value, and the principal component corresponding to the maximum singular value is the second vector.

[0069] It can be seen that in this example, the second vector is calculated by performing singular value decomposition on the matrix, which can effectively extract the key semantic change direction of the same evaluation dimension in the matrix and provide directional guidance for subsequent model optimization.

[0070] S205: Adjust parameters of the hidden layer of the first model according to the second vector of each evaluation dimension to obtain a second model.

[0071] Among them, since each hidden layer of the first model can obtain the second vector of each evaluation dimension, the second vector can be used to adjust the parameters of the hidden layer in a targeted manner, thereby optimizing the model.

[0072] It can be seen that in this example, prompt text is constructed according to the positive evaluation text and negative evaluation text of each evaluation dimension, the prompt text is input into the model to obtain the first vector of the hidden layer output of the model, and then the parameters of the model hidden layer are adjusted according to the steering vector determined by multiple first vectors. By constructing prompt text and directly adjusting the hidden layer parameters, the low efficiency problem of optimizing all model parameters based on large-scale labeled data in end-to-end fine-tuning is avoided, and the efficiency of the model in controlling the security of reply words is improved.

[0073] In one possible implementation, see Figure 6 , Figure 6 This is a flow chart of a method for adjusting parameters of a hidden layer of a first model to obtain a second model, as provided in an embodiment of the present application. Figure 6As shown, the second model is obtained by adjusting the hidden layer of the first model according to the second vector of each evaluation dimension, including the following steps: S601, obtaining a third vector by weighted averaging the second vectors of each evaluation dimension in the plurality of evaluation dimensions.

[0074] Since there are multiple evaluation dimensions, each evaluation dimension has a corresponding second vector (these vectors respectively represent the key change direction of the semantics under each dimension). At this time, the second vectors are weighted and averaged, that is, each second vector is given a weight according to the importance of different evaluation dimensions, and then the second vectors are calculated by weighted averaging to obtain a third vector.

[0075] Wherein, the user can set or change the weight of the second vector of each dimension on the client 101 of the model training system 100.

[0076] S602, vector summing the third vector and the vector of the hidden layer of the first model to obtain a second model.

[0077] Wherein, after obtaining the third vector, it is vector summed with the vector of the hidden layer of the first model. Specifically, the hidden layer vector contains the semantic encoding information of the model to the input text, and the vector summing operation (adding the corresponding elements of the third vector and the hidden layer vector) can integrate the comprehensive semantic adjustment direction represented by the third vector into the semantic representation of the hidden layer, guiding the model to perform semantic processing and output adjustment in the subsequent calculation in the direction that meets the requirements of multiple evaluation dimensions.

[0078] As can be seen, in this example, the third vector is obtained by weighted averaging the second vectors of multiple evaluation dimensions, and the third vector is vector summed to the hidden layer, so as to adjust the model, which can make the model optimize the semantic representation by considering the requirements of multiple evaluation dimensions, and improve the comprehensive performance.

[0079] In one possible implementation, the method further includes: inputting a third text into the second model to obtain a fourth text for replying to the third text.

[0080] Wherein, the third text is an input text that needs to be processed by the second model, and the third text is a text asked by the user. The second model is an optimized model obtained by steps S201-S205, and the second model is used to generate a reply that meets the preset standards (such as safety and accuracy) according to the user's question. The fourth text is the reply content generated by the second model based on the third text input.

[0081] For example, the third text input by the user is: "What is the profit of Product A?" The second model optimizes the semantic representation along the direction of the steering vector and generates the fourth text: "Product A has the risk of market fluctuations, and the specific profit cannot be guaranteed. It is recommended to carefully evaluate it based on your own risk tolerance."

[0082] It can be seen that in this example, inputting the third text into the second model optimized by the steering vector to generate the fourth text can make the fourth text used to reply to the third text more compliant and accurate under the constraints of multiple evaluation dimensions, thereby improving the quality of model output.

[0083] See also Figure 7 , Figure 7 This is a structural diagram of a model training device provided in an embodiment of the present application. Figure 7 As shown, the model training device 700 includes: A construction module 701 is configured to construct a second prompt text for each evaluation dimension based on a first prompt text and a first text and a second text corresponding to each evaluation dimension of a plurality of evaluation dimensions, wherein the first text is a text that positively evaluates the first evaluation dimension, the second text is a text that negatively evaluates the first evaluation dimension, and the first evaluation dimension is any one of the plurality of evaluation dimensions; Acquisition module 702 is used to input the second prompt text of each evaluation dimension into the first model to obtain multiple first vectors of each evaluation dimension output by the hidden layer of the first model, and the first vectors correspond one-to-one to the second prompt text; and is used to determine the matrix of the first vector based on the multiple first vectors; and is used to calculate the second vector of each evaluation dimension based on the matrix; and is used to adjust the parameters of the hidden layer of the first model according to the second vector of each evaluation dimension to obtain the second model.

[0084] In one possible implementation, in terms of constructing the second prompt text for each evaluation dimension based on the first prompt text and the first text and the second text corresponding to each evaluation dimension in multiple evaluation dimensions, the construction module 701 is specifically used to: perform the following processing on the first text and the second text corresponding to each evaluation dimension: merge the first prompt text and the first text to obtain a third prompt text; merge the first prompt text and the second text to obtain a fourth prompt text; and construct the second prompt text by combining the third prompt text and the fourth prompt text.

[0085] In one possible implementation, when determining the matrix of the first vector based on multiple first vectors, the acquisition module 702 is specifically used to: calculate the difference between every two first vectors in the multiple first vectors to obtain a difference vector between every two first vectors; and splice the difference vectors between every two first vectors in the multiple first vectors to obtain the matrix of the first vector.

[0086] In a possible implementation, in terms of calculating the second vector of each evaluation dimension based on the matrix, the acquisition module 702 is specifically used to: perform singular value decomposition on the matrix to calculate the second vector of each evaluation dimension.

[0087] In one possible implementation, in terms of adjusting the parameters of the hidden layer of the first model according to the second vector of each evaluation dimension to obtain the second model, the acquisition module 702 is specifically used to: obtain a third vector by performing a weighted average of the second vector of each evaluation dimension in the multiple evaluation dimensions; and perform a vector sum operation on the third vector and the vector of the hidden layer of the first model to obtain the second model.

[0088] In one possible implementation, before constructing the second prompt text for each evaluation dimension based on the first prompt text and the first text and second text corresponding to each evaluation dimension in multiple evaluation dimensions, the construction module 701 is also used to: construct the first text and second text matching each of the evaluation dimensions based on the preset multiple evaluation dimensions.

[0089] In a possible implementation, the acquisition module 702 is further configured to: input the third text into the second model to obtain a fourth text used to reply to the third text.

[0090] It is worth noting that the specific functional implementation of the model training device 700 is shown in the above Figure 2 The description of the model training method shown in FIG. 7 is as follows, for example, the construction module 701 is used to implement the relevant content of executing S201, and the acquisition module 702 is used to implement the relevant content of executing S202-S205. The various units or modules in the model training device 700 can be individually or completely merged into one or several other units or modules to form a structure, or one (or some) of the units or modules can be further divided into multiple functionally smaller units or modules to form a structure, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present invention. The above-mentioned units or modules are divided according to logical functions. In actual applications, the functions of one unit (or module) are implemented by multiple units (or modules), or the functions of multiple units (or modules) are implemented by one unit (or module).

[0091] According to the description of the above method embodiment and related device embodiment, please refer to Figure 8 , Figure 8 It is a structural diagram of a computer provided in an embodiment of the present application. Figure 8 The computer 800 shown includes a processor 801 , a memory 802 , a communication interface 803 , and a bus 804 . The processor 801 , the memory 802 , and the communication interface 803 are communicatively connected to each other via the bus 804 .

[0092] Optionally, the memory 802 is a ROM, a static storage device, a dynamic storage device or a RAM.

[0093] The memory 802 can store executable program codes. When the executable program codes stored in the memory 802 are executed by the processor 801, the processor 801 and the communication interface 803 are used to execute the program codes. Figure 2 The various steps of the model training method of the illustrated embodiment.

[0094] The processor 801 uses a general-purpose CPU, a microprocessor, an application-specific integrated circuit ASIC, a GPU or one or more integrated circuits to execute relevant programs to perform the model training method of the embodiment of the method of the present application.

[0095] Processor 801 can also be an integrated circuit chip with signal processing capabilities. During implementation, each step of the model training method of the present application can be completed by hardware integrated logic circuits in processor 801 or software instructions. Optionally, processor 801 is a general-purpose processor, DSP, ASIC, FPGA or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component. The processor can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor is a microprocessor or any conventional processor. The steps of the method disclosed in conjunction with the embodiments of the present application can be directly implemented as being executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The optional software module is located in random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 802. Processor 801 reads the information in memory 802 and, in combination with its hardware, completes the functions required to be performed by the modules included in a model training device 700 of the embodiment of the present application, or executes the model training method of the method embodiment of the present application.

[0096] The communication interface 803 uses, for example but not limited to, a transceiver or other transceiver-related device.

[0097] The bus 804 may include a path for transmitting information between various components of the computer 800 (eg, the memory 802 , the processor 801 , and the communication interface 803 ).

[0098] It should be noted that although Figure 8 The computer 800 shown only shows a memory, a processor, and a communication interface. However, in the specific implementation process, those skilled in the art should understand that the computer 800 also includes other devices necessary for normal operation. At the same time, according to specific needs, those skilled in the art should understand that the computer 800 may also include hardware devices that implement other additional functions. In addition, those skilled in the art should understand that the computer 800 may also include only the devices necessary to implement the embodiments of the present application, and does not necessarily include Figure 8 All devices shown in .

[0099] An embodiment of the present application provides a computer-readable storage medium, which stores a computer program for electronic data exchange. The computer program includes execution instructions, and the execution instructions are used to execute part or all of the steps of any one of the model training methods described in the above-mentioned model training method embodiments. The above-mentioned computer includes an electronic client device.

[0100] An embodiment of the present application provides a computer program product, wherein the computer program product includes a computer program, and the computer program is operable to enable a computer to perform part or all of the steps of any model training method recorded in the above method embodiments. The computer program product can be a software installation package.

[0101] It should be noted that for any of the aforementioned embodiments of the model training method, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should know that this application is not limited to the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by this application.

[0102] The above is a detailed introduction to the embodiments of the present application. Specific examples are used herein to illustrate the principles and implementation methods of a model training method, computer, storage medium, and program product of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for those skilled in the art, based on the ideas of a model training method, computer, storage medium, and program product of the present application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

[0103] The present application is described with reference to the flowcharts and / or block diagrams of the methods, hardware products, and computer program products of the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable model training device to produce a machine, so that the instructions executed by the processor of the computer or other programmable model training device generate instructions for implementing the process in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0104] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable model training device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device that implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The memory may include a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0105] Although the present application is described herein in conjunction with various embodiments, in the process of implementing the claimed application, those skilled in the art may understand and implement other variations of the disclosed embodiments by reviewing the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude a plurality of components or steps. The fact that certain measures are recited in different dependent claims does not mean that these measures cannot be combined to produce good results.

[0106] Those skilled in the art will understand that all or part of the steps in the various methods of the method embodiments of any of the above-mentioned model training methods can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable memory, which may include: a flash drive, a read-only memory (English: Read-Only Memory, abbreviated as: ROM), a random access memory (English: Random Access Memory, abbreviated as: RAM), a disk or an optical disk, etc.

[0107] It can be understood that any product that is controlled or configured to execute the processing method of the flowchart described in an embodiment of a model training method of the present application, such as the device and computer program product of the above flowchart, falls within the scope of the related products described in this application.

[0108] Obviously, those skilled in the art may make various modifications and variations to the model training method, computer, storage medium, and program product provided in this application without departing from the spirit and scope of this application. Thus, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application is intended to include such modifications and variations.

Claims

1. A model training method, characterized in that: The method comprises: Constructing a second prompt text for each evaluation dimension based on the first prompt text and the first text and the second text corresponding to each evaluation dimension of the plurality of evaluation dimensions, wherein the first text is a text that positively evaluates the first evaluation dimension, the second text is a text that negatively evaluates the first evaluation dimension, and the first evaluation dimension is any one of the plurality of evaluation dimensions; Inputting the second prompt text of each evaluation dimension into the first model, obtaining a plurality of first vectors of each evaluation dimension output by the hidden layer of the first model, wherein the first vectors correspond one-to-one to the second prompt text; determining a matrix of the first vectors based on the plurality of the first vectors; Calculating and obtaining a second vector for each evaluation dimension according to the matrix; According to the second vector of each evaluation dimension, parameters of the hidden layer of the first model are adjusted to obtain a second model.

2. The method according to claim 1, wherein The step of constructing a second prompt text for each evaluation dimension based on the first prompt text and the first text and the second text corresponding to each evaluation dimension in the plurality of evaluation dimensions includes: The following processing is performed on the first text and the second text corresponding to each evaluation dimension: Merging the first prompt text and the first text to obtain a third prompt text; Merging the first prompt text and the second text to obtain a fourth prompt text; The third prompt text and the fourth prompt text are constructed to obtain the second prompt text.

3. The method according to claim 1, wherein The determining a matrix of the first vectors according to the plurality of the first vectors includes: Calculating the difference between every two first vectors in the plurality of first vectors to obtain a difference vector between every two first vectors; The difference vectors between every two of the first vectors in the plurality of first vectors are concatenated to obtain a matrix of the first vectors.

4. The method according to claim 3, wherein The step of calculating the second vector of each evaluation dimension according to the matrix includes: Perform singular value decomposition on the matrix to calculate the second vector of each evaluation dimension.

5. The method according to claim 4, wherein The step of adjusting parameters of a hidden layer of the first model according to the second vector of each evaluation dimension to obtain a second model includes: Obtaining a third vector by performing weighted averaging on the second vector of each evaluation dimension in the plurality of evaluation dimensions; Perform vector summation on the third vector and the vector of the hidden layer of the first model to obtain a second model.

6. The method according to claim 1, wherein Before constructing the second prompt text for each evaluation dimension based on the first prompt text and the first text and the second text corresponding to each evaluation dimension in the plurality of evaluation dimensions, the method further includes: According to a plurality of preset evaluation dimensions, a first text and a second text matching each of the evaluation dimensions are constructed.

7. The method according to any one of claims 1 to 6, wherein: The method further comprises: The third text is input into the second model to obtain a fourth text used to reply to the third text.

8. A computer, characterized in that: include: A memory, a processor, and an executable program code stored in the memory and runnable on the processor, wherein the processor executes the steps of the model training method according to any one of claims 1 to 7 when executing the executable program code.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores executable program code, which includes execution instructions for executing the steps of the model training method according to any one of claims 1 to 7.

10. A computer program product, characterized in that The computer program product comprises a computer program, and the computer program is used to enable a computer to execute the steps of the model training method according to any one of claims 1 to 7.

Images