Training method, device, electronic device and storage medium for content generation model

Abstract

The present disclosure provides a training method, device, electronic device and storage medium for a content generation model. It relates to the field of computer technology. The method includes: when it is necessary to deploy a content generation model of the vertical field it serves on the corresponding edge node of the content generation platform, the center point of the content generation platform obtains the training corpus of the corresponding vertical field from the corresponding edge node, and fine-tunes the general content generation model in combination with the training corpus to obtain a content generation model adapted to the corresponding vertical field, and sends the obtained content generation model to the corresponding edge node to deploy the corresponding content generation model on the corresponding edge node. Thus, by fine-tuning the general content generation model, a content generation model adapted to the corresponding vertical field is obtained, which reduces the resources and costs required to obtain the content generation model of the corresponding vertical field and improves the efficiency and accuracy of the content generation model of the corresponding vertical field.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a training method, device, electronic device, and storage medium for a content generation model. Background Art

[0002] Currently, content generation platforms typically use a general content generation model deployed on a central node to serve content generation requests for each specific vertical field. However, processing content generation requests for each specific vertical field through a central node not only results in a large amount of traffic and a high load submitted to the central node, but also results in poor content generation results for the generated vertical fields. Therefore, in order to serve content generation requests for each specific vertical field, multiple edge nodes can be deployed on the content generation platform, and content generation models corresponding to the corresponding vertical fields can be deployed on the edge nodes. However, training and generating a set of content generation models for each vertical field and starting full training from an initial state requires a lot of resources. Therefore, how to enable edge nodes to obtain the content generation model corresponding to their corresponding vertical fields is a technical problem that urgently needs to be solved. Summary of the Invention

[0003] The present disclosure provides a training method, device, electronic device, and storage medium for a content generation model.

[0004] In the first aspect, the present disclosure provides a training method for a content generation model, the method is applied to a central node in a content generation platform, a general content generation model is deployed on the central node, the content generation platform also includes a plurality of edge nodes, each of the edge nodes serves a different vertical field, the method comprises: for any one of the plurality of edge nodes, obtaining training corpus of a target vertical field from the target edge node, the target vertical field being the vertical field served by the target edge node; obtaining a first low-rank weight matrix and a second low-rank weight matrix obtained by matrix decomposition of a weight matrix to be trained, wherein the first low-rank weight matrix The product of the first low-rank weight matrix and the second low-rank weight matrix is ​​equal to the weight matrix to be trained, and the dimension of the weight matrix to be trained and the weight matrix of the general content generation model are the same; the first low-rank weight matrix and the second low-rank weight matrix are configured into the general content generation model to obtain a first content generation model; according to the training corpus, the first content generation model is trained to obtain a second content generation model under the target vertical field, wherein, during the training of the first content generation model, the weight matrix is ​​frozen; the second content generation model is sent to the target edge node to deploy the second content generation model on the target edge node.

[0005] In the second aspect, the present disclosure provides a training device for a content generation model, the device is applied to a central node in a content generation platform, a general content generation model is deployed on the central node, the content generation platform also includes a plurality of edge nodes, each of the edge nodes serves a different vertical field, the device includes: a first acquisition module, for obtaining training corpus of a target vertical field from any target edge node among the plurality of edge nodes, the target vertical field being the vertical field served by the target edge node; a second acquisition module, for obtaining a first low-rank weight matrix and a second low-rank weight matrix obtained by matrix decomposition of a weight matrix to be trained, wherein the first low-rank weight matrix and The product of the second low-rank weight matrix is ​​equal to the weight matrix to be trained, and the dimension of the weight matrix to be trained and the weight matrix of the general content generation model are the same; a configuration module is used to configure the first low-rank weight matrix and the second low-rank weight matrix into the general content generation model to obtain a first content generation model; a training module is used to train the first content generation model according to the training corpus to obtain a second content generation model in the target vertical field, wherein the weight matrix is ​​frozen during the training of the first content generation model; a sending module is used to send the second content generation model to the target edge node to deploy the second content generation model on the target edge node.

[0006] In a third aspect, the present disclosure provides an electronic device comprising: a processor, and a memory communicatively connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the training method of the content generation model disclosed in an embodiment of the present disclosure.

[0007] In a fourth aspect, the present disclosure provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the training method of the content generation model disclosed in the embodiment of the present disclosure.

[0008] In a fifth aspect, the present disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements the training method of the content generation model disclosed in the embodiment of the present disclosure.

[0009] The technical solutions provided by the embodiments of the present disclosure bring at least the following beneficial effects:

[0010] When a content generation model for the vertical domain it serves needs to be deployed on the corresponding edge nodes of a content generation platform, the center of the content generation platform obtains the training corpus for the corresponding vertical domain from the corresponding edge node, and fine-tunes the general content generation model based on the training corpus to obtain a content generation model adapted for the corresponding vertical domain. The obtained content generation model is then sent to the corresponding edge node for deployment on the corresponding edge node. Thus, by fine-tuning the general content generation model, a content generation model adapted for the corresponding vertical domain is obtained, reducing the resources and cost required to obtain the content generation model for the corresponding vertical domain and improving the efficiency and accuracy of the content generation model for the corresponding vertical domain. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0012] Figure 1 This is a flowchart of a method for training a content generation model according to an exemplary embodiment;

[0013] Figure 2 An example diagram of the relationship between the central node and multiple edge nodes of the content generation platform;

[0014] Figure 3 This is a flowchart of a method for training a content generation model according to another exemplary embodiment;

[0015] Figure 4 1 is a schematic structural diagram of a training device for a content generation model according to an exemplary embodiment;

[0016] Figure 5 1 is a schematic structural diagram of a training device for a content generation model according to another exemplary embodiment;

[0017] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure.

[0018] The above drawings illustrate specific embodiments of the present disclosure, which will be described in more detail below. These drawings and textual descriptions are not intended to limit the scope of the present disclosure in any way, but rather to illustrate the concepts of the present disclosure to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0019] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all possible embodiments consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

[0020] The following detailed description of the technical solution of the present disclosure and how the technical solution of the present disclosure solves the above-mentioned technical problems is provided with specific embodiments. The following specific embodiments may be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments. The embodiments of the present disclosure will be described below in conjunction with the accompanying drawings.

[0021] First combine Figure 1 , an exemplary description is given of the training method of the content generation model provided in the embodiment of the present disclosure.

[0022] Figure 1 The present invention is a flowchart of a method for training a content generation model according to an exemplary embodiment.

[0023] It should be noted that the training method for the content generation model provided in the embodiments of the present disclosure can be executed by a training device for the content generation model, wherein the training device for the content generation model can be implemented by software and / or hardware. The training device for the content generation model can be a central node in the content generation platform, or can be configured in the central node to train the content generation model. The content generation platform can also include multiple edge nodes, each edge node serving a different vertical field.

[0024] It should be noted that the relationship between the central node and multiple edge nodes of the content generation platform in this example is shown in the following figure: Figure 2 shown.

[0025] The central node in this example is used to dynamically allocate resources and schedule computing tasks to execute the training method of the content generation model.

[0026] It can be understood that in this embodiment, a content generation platform can be composed of multiple electronic devices, and a target electronic device is selected from the multiple electronic devices as a central node, and a general content generation model is deployed on the central node, and other electronic devices except the target electronic device among the multiple electronic devices are used as edge nodes to deploy content generation models corresponding to vertical fields on the edge nodes.

[0027] The electronic device may be any device with computing capabilities, such as a server.

[0028] like Figure 1 As shown, the training method of the content generation model includes the following steps:

[0029] Step 101 : For any target edge node among multiple edge nodes, obtain training corpus of a target vertical field from the target edge node, where the target vertical field is the vertical field served by the target edge node.

[0030] In this embodiment, the target edge node refers to any edge node that needs to deploy a content generation model for the vertical field it serves.

[0031] In this embodiment, the target edge node can send a model training request to the central node, where the model training request includes training corpus for the target vertical domain. Correspondingly, the central node can obtain training corpus for the target vertical domain served by the target edge node based on the model training request.

[0032] Among them, the target vertical field in this embodiment can be a small field vertically subdivided from any field. For example, the target vertical field can be any one of the vertical fields such as food tourism, outdoor adventure tourism, cultural tourism, etc. subdivided from the tourism industry. For example, the target vertical field can be any one of the vertical fields such as investment banking, asset management, insurance, etc. subdivided from the financial industry.

[0033] Step 102: Obtain a first low-rank weight matrix and a second low-rank weight matrix obtained by performing matrix decomposition on the weight matrix to be trained, wherein the product of the first low-rank weight matrix and the second low-rank weight matrix is ​​equal to the weight matrix to be trained, and the dimensions of the weight matrix to be trained and the weight matrix of the general content generation model are the same.

[0034] The same dimensions of the weight matrix to be trained and the weight of the general content generation model refer to the same number of columns and rows of the two matrices.

[0035] For example, the weight matrix of a general content generation model is represented as W0∈R d×k At this time, it can be known that the number of rows of the weight matrix W0 of the content generation model is d and the number of columns is k, and the number of rows and columns of the weight matrix ΔW to be trained is also d and k.

[0036] Step 103: configure the first low-rank weight matrix and the second low-rank weight matrix into a universal content generation model to obtain a first content generation model.

[0037] In this example, the first low-rank weight matrix and the second low-rank weight matrix can be initialized using a random distribution, and the initialized first low-rank weight matrix and second low-rank weight matrix can be configured into the general content generation model to obtain the first content generation model.

[0038] For example, ΔW is decomposed into the first low-rank weight matrix B r and the second low-rank weight matrix A r For the first content generation model, the weight matrix corresponding to the first content generation model can be expressed as:

[0039] W0 + ΔW r = W0 + B r A r

[0040] where B r ∈R d×r and A r ∈R d×r , and the rank r << min(d, k). It can be understood that the number of rows of B r is d, and the number of columns is the rank r; the number of rows of A r is the rank r, and the number of columns is k.

[0041] Step 104: According to the training corpus, train the first content generation model to obtain the second content generation model in the target vertical domain. During the training of the first content generation model, the weight matrix is frozen.

[0042] That is, during the training of the first content generation model, the weight matrix of the general content generation model is frozen, and only the first low-rank weight matrix and the second low-rank weight matrix are updated. That is, during the process of obtaining the content generation model in the target vertical domain, in this embodiment, the weight matrix W0 of the general content generation model remains unchanged, and only the first low-rank weight matrix and the second low-rank weight matrix are updated.

[0043] In an embodiment of the present disclosure, the sample conversations in the training corpus can be vectorized, and the vectorized representation is input into the first content generation model to obtain the predicted content generated by the first content generation model. Then, according to the predicted content and the sample reply content corresponding to the sample conversation in the training corpus, the first content generation model is trained until the training end condition is met to obtain the trained first content generation model.

[0044] Among them, the training end condition is pre-set according to the actual situation. For example, the training end condition can be that the number of training times reaches a specified number of training times, or the loss function value of the first content generation model is less than a preset value, or the change of the loss function value tends to be stable, that is, the difference between the loss function values ​​corresponding to two or more adjacent trainings is less than the set value, that is, the loss function value basically no longer changes. This embodiment does not specifically limit the training end condition.

[0045] Step 105: Send the second content generation model to the target edge node to deploy the second content generation model on the target edge node.

[0046] In this embodiment, the central node sends the second content generation model to the target edge node. Correspondingly, the target edge node deploys the second content generation model on itself to process content generation requests belonging to the target vertical field through the deployed second content generation model.

[0047] The training method of the content generation model provided by the embodiment of the present disclosure is that when the content generation model of the vertical field it serves needs to be deployed on the corresponding edge node of the content generation platform, the center point of the content generation platform obtains the training corpus of the corresponding vertical field from the corresponding edge node, and fine-tunes the general content generation model in combination with the training corpus to obtain a content generation model adapted to the corresponding vertical field, and sends the obtained content generation model to the corresponding edge node to deploy the obtained content generation model on the corresponding edge node. Thus, by fine-tuning the general content generation model, a content generation model adapted to the corresponding vertical field is obtained, which reduces the resources and costs required to obtain the content generation model of the corresponding vertical field and improves the efficiency and accuracy of the content generation model of the corresponding vertical field.

[0048] In one embodiment of the present disclosure, in order to improve the effect of model training, there may be multiple first content generation models in this embodiment, and the number of columns of the first low-rank weight matrix and the number of rows of the second low-rank weight matrix in each first content generation model are both of rank r, and the ranks r corresponding to different first content generation models are different. It should be noted that the above-mentioned rank r is pre-set according to demand. That is to say, in this example, different ranks r can be set to obtain multiple first content generation models to be trained, and the multiple second content generation models can be trained separately in combination with the training corpus to obtain multiple trained first content generation models, and the second content generation model under the target vertical field is generated according to the weight matrix of the universal content generation model and the first low-rank weight matrix and the second low-rank weight matrix in each trained first content generation model. In this way, the accuracy of the obtained second target content generation model under the target vertical field can be further improved.

[0049] In order to understand the process clearly, the following Figure 3 This process is described exemplarily.

[0050] Figure 3 The present invention is a flowchart of a method for training a content generation model according to another exemplary embodiment.

[0051] like Figure 3 As shown, this may include:

[0052] Step 301 : training a plurality of first content generation models respectively according to training corpus to obtain a plurality of trained first content generation models.

[0053] Step 302 : Obtain a maximum value among multiple ranks r, and determine an intermediate content generation model corresponding to the maximum value from multiple trained first content generation models.

[0054] Step 303: For each trained first content generation model, determine the attention weight corresponding to the trained first content generation model based on the first low-rank weight matrix and the second low-rank weight matrix in the trained first content generation model and the first low-rank weight matrix and the second low-rank weight matrix in the intermediate content generation model.

[0055] In this embodiment, for each trained first content generation model, according to the first low-rank weight matrix and the second low-rank weight matrix in the trained first content generation model and the first low-rank weight matrix and the second low-rank weight matrix in the intermediate content generation model, a possible implementation method for determining the attention weight corresponding to the trained first content generation model is: for each trained first content generation model, multiply the first low-rank weight matrix and the second low-rank weight matrix in the trained first content generation model to obtain the target weight matrix corresponding to the trained first content generation model; determine the attention weight corresponding to the trained first content generation model according to the target weight matrix corresponding to the intermediate content generation model and the target weight matrix corresponding to the trained first content generation model.

[0056] In this embodiment, in order to accurately determine the attention weight corresponding to the first content generation model after training, a possible implementation method for determining the attention weight corresponding to the first content generation model after training is based on the target weight matrix corresponding to the intermediate content generation model and the target weight matrix corresponding to the first content generation model after training: determine the inner product value between the target weight matrix corresponding to the intermediate content generation model and the target weight matrix corresponding to the first content generation model after training; normalize the inner product value to obtain the attention weight corresponding to the first content generation model after training.

[0057] Among them, it can be understood that the inner product value between the target weight matrix corresponding to the intermediate content generation model and the target weight matrix corresponding to the trained first content generation model is obtained by performing inner product calculation on the target weight matrix corresponding to the intermediate content generation model and the target weight matrix corresponding to the trained first content generation model.

[0058] In step 304 , a weighted summation process is performed on the attention weights and target weight matrices corresponding to the trained first content generation models to obtain a fusion weight matrix in the target vertical field.

[0059] Step 305 : Generate a second content generation model in the target vertical field based on the weight matrix and the fusion weight matrix.

[0060] In one embodiment of the present disclosure, in order to accurately obtain the second content generation model in the target vertical field, the weight matrix and the fusion weight matrix of the general content generation model can be merged to obtain a merged weight matrix, and based on the merged weight matrix, the second target content generation model in the target vertical field is generated.

[0061] In this example, a possible implementation method of merging the weight matrix of the general content generation model and the fusion weight matrix to obtain a merged weight matrix is: adding the weight matrix of the general content generation model and the fusion weight matrix to obtain a merged weight matrix.

[0062] In this embodiment, in order to quickly obtain the second target content generation model under the target vertical field, after obtaining the merged weight matrix, the weight matrix of the general content generation model can be replaced with the merged weight matrix to obtain the second target content generation model under the target vertical field.

[0063] For example, based on r max The target weight matrix of the corresponding intermediate content generation model is expressed as B rmax A rmax , among which, other r n The target weight matrix of the corresponding trained content generation model is expressed as B n A n ,in,

[0064]

[0065] Among them, used to calculate other r n The corresponding target weight matrix B of the trained content generation model n A n With B rmax A rmax The formula for the inner product between is:

[0066]

[0067] Q n Respectively with Q r max Normalize the product of and get the attention weight, that is, the target weight matrix obtained by each different low-rank r training and B rmax A rmax The attention coefficient can be expressed as:

[0068] Attention(ΔW)=softmax(B1A1,B2A2,...,Br max Ar max ) T

[0069] Then, all target weight matrices are weighted and summed with the corresponding attention weights to calculate the fusion weight matrix. The formula for obtaining the fusion weight matrix ΔW1 is:

[0070] ΔW1=Attention(ΔW)×(B1A1, B2A2,..., Br max Ar max ) T

[0071] Finally, the initial matrix W0 of the general content generation model is merged with ΔW1 to obtain a merged weight matrix, and the second content generation model in the target vertical field is obtained based on the merged weight matrix.

[0072] Based on the above description, it can be seen that in this embodiment, by setting different ranks r, multiple first content generation models to be trained are obtained, and multiple second content generation models are trained separately in combination with training corpus to obtain multiple trained first content generation models. The attention weights and target weight matrices corresponding to each trained first content generation model are weighted and summed to obtain a fusion weight matrix under the target vertical field, and the second content generation model under the target vertical field is generated based on the weight matrix and the fusion weight matrix. In this way, the accuracy of the obtained second target content generation model under the target vertical field can be further improved.

[0073] Figure 4 This figure is a schematic diagram of a content generation model training device according to an exemplary embodiment. It should be noted that the content generation model training device is applied to a central node in a content generation platform, on which a general content generation model is deployed. The content generation platform also includes multiple edge nodes, each serving a different vertical domain.

[0074] like Figure 4 As shown, the training device 400 for the content generation model includes: a first acquisition module 401, a second acquisition module 402, a configuration module 403, a training module 404 and a sending module 405, wherein:

[0075] The first acquisition module 401 is configured to acquire, for any target edge node among the plurality of edge nodes, training corpus in a target vertical field from the target edge node, where the target vertical field is the vertical field served by the target edge node.

[0076] The second acquisition module 402 is used to obtain a first low-rank weight matrix and a second low-rank weight matrix obtained by performing matrix decomposition on the weight matrix to be trained, wherein the product of the first low-rank weight matrix and the second low-rank weight matrix is ​​equal to the weight matrix to be trained, and the dimensions of the weight matrix to be trained and the weight matrix of the general content generation model are the same.

[0077] The configuration module 403 is configured to configure the first low-rank weight matrix and the second low-rank weight matrix into the universal content generation model to obtain a first content generation model.

[0078] The training module 404 is used to train the first content generation model based on the training corpus to obtain a second content generation model in the target vertical field, wherein the weight matrix is ​​frozen during the training of the first content generation model.

[0079] The sending module 405 is configured to send the second content generation model to a target edge node, so as to deploy the second content generation model on the target edge node.

[0080] In one embodiment of the present disclosure, there are multiple first content generation models, and the number of columns of the first low-rank weight matrix and the number of rows of the second low-rank weight matrix in each first content generation model are both rank r. The ranks r corresponding to different first content generation models are different, wherein Figure 4 Based on the embodiment shown, Figure 5 As shown, the training module 404 may include:

[0081] A training unit 4041 is configured to train the plurality of first content generation models according to the training corpus to obtain a plurality of trained first content generation models;

[0082] The generating unit 4042 is configured to generate a second content generation model in a target vertical field according to the weight matrix and the first low-rank weight matrix and the second low-rank weight matrix in each trained first content generation model.

[0083] In one embodiment of the present disclosure, the generating unit 4042 includes:

[0084] The first determining subunit 40421 is configured to obtain a maximum value among the multiple ranks r, and determine an intermediate content generation model corresponding to the maximum value from the multiple trained first content generation models;

[0085] A second determining subunit 40422 is configured to determine, for each trained first content generation model, an attention weight corresponding to the trained first content generation model based on the first low-rank weight matrix and the second low-rank weight matrix in the trained first content generation model and the first low-rank weight matrix and the second low-rank weight matrix in the intermediate content generation model;

[0086] The summation processing subunit 40423 is used to perform weighted summation processing on the attention weights and target weight matrices corresponding to the trained first content generation models to obtain a fusion weight matrix in the target vertical field;

[0087] The generating subunit 40424 is configured to generate a second content generation model in the target vertical field according to the weight matrix and the fusion weight matrix.

[0088] In one embodiment of the present disclosure, the second determination subunit 40422 is specifically used to: for each trained first content generation model, multiply the first low-rank weight matrix and the second low-rank weight matrix in the trained first content generation model to obtain the target weight matrix corresponding to the trained first content generation model; determine the attention weight corresponding to the trained first content generation model based on the target weight matrix corresponding to the intermediate content generation model and the target weight matrix corresponding to the trained first content generation model.

[0089] In one embodiment of the present disclosure, the second determination subunit 40422 is specifically used to: determine the inner product value between the target weight matrix corresponding to the intermediate content generation model and the target weight matrix corresponding to the trained first content generation model; normalize the inner product value to obtain the attention weight corresponding to the trained first content generation model.

[0090] In one embodiment of the present disclosure, the generation subunit 40424 is specifically used to: merge the weight matrix and the fusion weight matrix to obtain a merged weight matrix; and generate a second target content generation model under the target vertical field based on the merged weight matrix.

[0091] It should be noted that the aforementioned description of the embodiment of the training method for the content generation model is also applicable to the training device for the content generation model of this embodiment, and will not be repeated here.

[0092] The training device for the content generation model provided by the embodiment of the present disclosure is such that, when the content generation model of the vertical field it serves needs to be deployed on the corresponding edge node of the content generation platform, the center point of the content generation platform obtains the training corpus of the corresponding vertical field from the corresponding edge node, and fine-tunes the general content generation model in combination with the training corpus to obtain a content generation model adapted to the corresponding vertical field, and sends the obtained content generation model to the corresponding edge node to deploy the obtained content generation model on the corresponding edge node. Thus, by fine-tuning the general content generation model, a content generation model adapted to the corresponding vertical field is obtained, which reduces the resources and costs required to obtain the content generation model of the corresponding vertical field and improves the efficiency and accuracy of the content generation model of the corresponding vertical field.

[0093] According to an embodiment of the present disclosure, an electronic device is also provided, including: a processor; and a memory for storing instructions executable by the processor, wherein the processor is configured to: implement the training method of the content generation model disclosed in the embodiment of the present disclosure.

[0094] In order to implement the above embodiment, the embodiment of the present disclosure further proposes a storage medium.

[0095] In which, when the instructions in the storage medium are executed by the processor, the processor is enabled to execute the training method of the content generation model disclosed in the embodiment of the present disclosure.

[0096] In order to implement the above embodiments, the embodiments of the present disclosure also provide a computer program product.

[0097] When the computer program product is executed by a processor of an electronic device, the electronic device is enabled to execute the training method of the content generation model disclosed in the embodiment of the present disclosure.

[0098] Figure 6 The figure is a structural block diagram of an electronic device according to an exemplary embodiment. Figure 6 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0099] like Figure 6As shown, the electronic device 1000 includes a processor 111, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 112 or a program loaded from a memory 116 to a random access memory (RAM) 113. Various programs and data required for the operation of the electronic device 1000 are also stored in the RAM 113. The processor 111, the ROM 112, and the RAM 113 are connected to each other via a bus 114. An input / output (I / O) interface 115 is also connected to the bus 114.

[0100] The following components are connected to the I / O interface 115: a memory 116 including a hard disk, etc.; and a communication part 117 including a network interface card such as a local area network (LAN) card, a modem, etc., which performs communication processing via a network such as the Internet; a drive 118 is also connected to the I / O interface 115 as needed.

[0101] In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, an embodiment of the present disclosure includes a computer program carried on a computer-readable medium, the computer program including program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 117. When the computer program is executed by the processor 111, the above-mentioned functions defined in the method of the present disclosure are performed.

[0102] In an exemplary embodiment, a storage medium including instructions is further provided, such as a memory including instructions, and the instructions can be executed by the processor 111 of the electronic device 1000 to perform the above method. Alternatively, the storage medium can be a non-transitory computer-readable storage medium, for example, a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0103] In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wire, optical cable, RF, etc., or any suitable combination of the foregoing.

[0104] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the following claims.

[0105] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A training method for a content generation model, characterized in that: The method is applied to a central node in a content generation platform, where a general content generation model is deployed. The content generation platform also includes multiple edge nodes, each of which serves a different vertical field. The method includes: For any target edge node among the multiple edge nodes, obtain training corpus of a target vertical field from the target edge node, where the target vertical field is the vertical field served by the target edge node; Obtaining a first low-rank weight matrix and a second low-rank weight matrix obtained by performing matrix decomposition on the weight matrix to be trained, wherein the product of the first low-rank weight matrix and the second low-rank weight matrix is ​​equal to the weight matrix to be trained, and the dimension of the weight matrix to be trained and the weight matrix of the universal content generation model are the same; Configuring the first low-rank weight matrix and the second low-rank weight matrix into the universal content generation model to obtain a first content generation model; Training the first content generation model based on the training corpus to obtain a second content generation model in the target vertical field, wherein the weight matrix is ​​frozen during the training of the first content generation model; The second content generation model is sent to the target edge node to deploy the second content generation model on the target edge node.

2. The method according to claim 1, wherein There are multiple first content generation models, and the number of columns of the first low-rank weight matrix and the number of rows of the second low-rank weight matrix in each of the first content generation models are both of rank r. The ranks r corresponding to different first content generation models are different. The first content generation model is trained according to the training corpus to obtain a second content generation model in the target vertical field, including: training the plurality of first content generation models respectively according to the training corpus to obtain a plurality of trained first content generation models; A second content generation model in the target vertical field is generated according to the weight matrix and the first low-rank weight matrix and the second low-rank weight matrix in each trained first content generation model.

3. The method according to claim 2, wherein Generating a second content generation model in the target vertical field according to the weight matrix and the first low-rank weight matrix and the second low-rank weight matrix in each trained first content generation model includes: Obtaining a maximum value among the multiple ranks r, and determining an intermediate content generation model corresponding to the maximum value from the multiple trained first content generation models; For each trained first content generation model, determining an attention weight corresponding to the trained first content generation model based on the first low-rank weight matrix and the second low-rank weight matrix in the trained first content generation model and the first low-rank weight matrix and the second low-rank weight matrix in the intermediate content generation model; Performing weighted summation processing on the attention weights and target weight matrices corresponding to each of the trained first content generation models to obtain a fusion weight matrix under the target vertical field; A second content generation model in the target vertical field is generated according to the weight matrix and the fusion weight matrix.

4. The method according to claim 3, wherein For each trained first content generation model, determining the attention weight corresponding to the trained first content generation model based on the first low-rank weight matrix and the second low-rank weight matrix in the trained first content generation model and the first low-rank weight matrix and the second low-rank weight matrix in the intermediate content generation model, including: For each trained first content generation model, multiplying the first low-rank weight matrix and the second low-rank weight matrix in the trained first content generation model to obtain a target weight matrix corresponding to the trained first content generation model; The attention weight corresponding to the trained first content generation model is determined according to the target weight matrix corresponding to the intermediate content generation model and the target weight matrix corresponding to the trained first content generation model.

5. The method according to claim 4, wherein The determining, based on the target weight matrix corresponding to the intermediate content generation model and the target weight matrix corresponding to the trained first content generation model, the attention weight corresponding to the trained first content generation model includes: Determining an inner product value between a target weight matrix corresponding to the intermediate content generation model and a target weight matrix corresponding to the trained first content generation model; The inner product value is normalized to obtain an attention weight corresponding to the trained first content generation model.

6. The method according to claim 3, wherein Generating a second content generation model in the target vertical field according to the weight matrix and the fusion weight matrix includes: Merging the weight matrix and the fusion weight matrix to obtain a merged weight matrix; A second target content generation model under the target vertical field is generated according to the merged weight matrix.

7. A training device for a content generation model, characterized in that: The device is applied to a central node in a content generation platform, a general content generation model is deployed on the central node, and the content generation platform also includes multiple edge nodes, each of which serves a different vertical field. The device includes: A first acquisition module is configured to acquire, for any target edge node among the plurality of edge nodes, training corpus in a target vertical field from the target edge node, where the target vertical field is the vertical field served by the target edge node; A second acquisition module is configured to acquire a first low-rank weight matrix and a second low-rank weight matrix obtained by performing matrix decomposition on the weight matrix to be trained, wherein the product of the first low-rank weight matrix and the second low-rank weight matrix is ​​equal to the weight matrix to be trained, and the dimensions of the weight matrix to be trained and the weight matrix of the universal content generation model are the same; a configuration module, configured to configure the first low-rank weight matrix and the second low-rank weight matrix into the universal content generation model to obtain a first content generation model; a training module, configured to train the first content generation model based on the training corpus to obtain a second content generation model in the target vertical field, wherein the weight matrix is ​​frozen during the training of the first content generation model; A sending module is configured to send the second content generation model to the target edge node, so as to deploy the second content generation model on the target edge node.

8. The device according to claim 7, wherein There are multiple first content generation models, and the number of columns of the first low-rank weight matrix and the number of rows of the second low-rank weight matrix in each of the first content generation models are both rank r. The ranks r corresponding to different first content generation models are different. The training module includes: a training unit, configured to train the plurality of first content generation models respectively according to the training corpus to obtain a plurality of trained first content generation models; A generating unit is configured to generate a second content generation model under the target vertical field based on the weight matrix and the first low-rank weight matrix and the second low-rank weight matrix in each trained first content generation model.

9. The device according to claim 8, wherein The generating unit comprises: a first determining subunit, configured to obtain a maximum value among the multiple ranks r, and determine an intermediate content generation model corresponding to the maximum value from the multiple trained first content generation models; A second determining subunit is configured to determine, for each trained first content generation model, an attention weight corresponding to the trained first content generation model based on the first low-rank weight matrix and the second low-rank weight matrix in the trained first content generation model and the first low-rank weight matrix and the second low-rank weight matrix in the intermediate content generation model; A summation processing subunit, configured to perform weighted summation processing on the attention weights and target weight matrices corresponding to each of the trained first content generation models to obtain a fusion weight matrix under the target vertical field; A generating subunit is configured to generate a second content generation model in the target vertical field according to the weight matrix and the fusion weight matrix.

10. The device according to claim 9, wherein The second determining subunit is specifically configured to: For each trained first content generation model, multiplying the first low-rank weight matrix and the second low-rank weight matrix in the trained first content generation model to obtain a target weight matrix corresponding to the trained first content generation model; The attention weight corresponding to the trained first content generation model is determined according to the target weight matrix corresponding to the intermediate content generation model and the target weight matrix corresponding to the trained first content generation model.

11. The device according to claim 10, wherein The second determining subunit is specifically configured to: Determining an inner product value between a target weight matrix corresponding to the intermediate content generation model and a target weight matrix corresponding to the trained first content generation model; The inner product value is normalized to obtain an attention weight corresponding to the trained first content generation model.

12. The device according to claim 9, wherein The generating subunit is specifically used for: Merging the weight matrix and the fusion weight matrix to obtain a merged weight matrix; A second target content generation model under the target vertical field is generated according to the merged weight matrix.

13. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor implements the method according to any one of claims 1 to 6 when executing the computer program.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

15. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 6 when the computer program is executed by a processor.

Images