Multi-task learning model training method, device, electronic device and storage medium

Abstract

The present invention relates to artificial intelligence technology and discloses a multi-task learning model training method, comprising: obtaining a multi-task learning model composed of an encoder connected in parallel with multiple decoders, wherein the decoders correspond one-to-one to task categories; using each decoder to perform weighted processing on the encoder's output features to obtain a label analysis value for each task category; calculating a task loss value based on the label analysis and the task label; and calculating a target loss value using all task loss values; when the target loss value is greater than or equal to a loss threshold, updating the parameters of the multi-task learning model and retraining the updated model; and when the target loss value is less than the loss threshold, training is completed. The present invention also relates to blockchain technology, wherein the task loss value can be stored in a blockchain node. The present invention also proposes a multi-task learning model training device, equipment, and medium. The present invention can improve the efficiency of model training.

Description

Technical Field

[0001] The present invention relates to artificial intelligence technology, and in particular to a multi-task learning model training method, device, electronic device and storage medium. Background Art

[0002] In recent years, with the development of artificial intelligence, multi-task learning jointly learns multiple tasks in one model to achieve better generalization performance and lower memory usage. The combination of the two, namely, multi-task learning model training research, has attracted more and more attention. For example, in the daily business practice of a certain auto insurance company, it is necessary to screen high-quality customers and recommend suitable auto insurance to users. In the past, it was necessary to train two models to solve the two tasks of screening high-quality customers and recommending suitable auto insurance to users. Now, multi-task learning can be used to train one model to solve both tasks.

[0003] However, during the training process of the current multi-task learning model, different tasks affect each other, and it is impossible to focus on the corresponding sub-tasks, resulting in low model training efficiency. Summary of the Invention

[0004] The present invention provides a multi-task learning model training method, device, electronic device and storage medium, the main purpose of which is to improve the efficiency of model training.

[0005] Obtaining a training text data set, wherein each training text data in the training text data set includes task labels corresponding to multiple task categories;

[0006] Obtain a pre-built multi-task learning model, wherein the multi-task learning model is composed of an encoder connected in parallel with multiple decoders, and the decoders correspond one-to-one to the task categories;

[0007] Using the encoder to select training text data in the training text data set for feature extraction to obtain a text feature vector;

[0008] Using the decoder corresponding to each task category to perform feature extraction based on the attention mechanism on the text feature vector, to obtain a label analysis value corresponding to each task category;

[0009] Confirming the true value of the label corresponding to the task category according to the task label corresponding to the task category of the training text data corresponding to the text feature vector;

[0010] Calculate the task loss value between the label analysis value and the label true value corresponding to the task category using the preset loss function corresponding to the task category, and use the task loss values corresponding to all the task categories to calculate and obtain the target loss value;

[0011] When the target loss value is greater than or equal to the loss threshold, updating the parameters of the multi-task learning model, and returning to the step of selecting the training text data in the training text data set by the encoder for feature extraction;

[0012] When the target loss value is less than the loss threshold, the trained multi-task learning model is output.

[0013] Optionally, the step of selecting training text data from the training text dataset by the encoder for feature extraction to obtain a text feature vector includes:

[0014] Selecting any one of the training text data sets to obtain target training text data, and deleting the target training text data in the training text data set to obtain an updated training text data set;

[0015] Extracting user behavior sequence text and user portrait labels from the target training text data;

[0016] Convert each user behavior text in the user behavior sequence into a vector, and combine the converted vectors to obtain a user behavior matrix;

[0017] Convert the user portrait label into a vector to obtain a user portrait vector;

[0018] Convolving the user behavior matrix using a convolutional layer in the encoder to obtain a user behavior vector;

[0019] The user behavior vector is combined with the user portrait vector to obtain the text feature vector.

[0020] Optionally, converting each user behavior text in the user behavior sequence into a vector and combining the converted vectors to obtain a user behavior matrix includes:

[0021] Convert each user behavior text in the user behavior sequence into a vector to obtain a user behavior text vector;

[0022] The user behavior text vectors are combined according to the order of the corresponding user behavior texts in the user behavior sequence to obtain the user behavior matrix.

[0023] Optionally, the step of performing weighted calculation on the text feature vector using the attention mechanism network in the decoder corresponding to the task category to obtain a weighted feature vector corresponding to each task category includes:

[0024] Using the attention mechanism network in the decoder corresponding to the task category to perform weighted calculation on the text feature vector, obtaining a weighted feature vector corresponding to each task category;

[0025] The multi-layer perceptron in the decoder corresponding to the task category is used to perform feature extraction on the weighted feature vector corresponding to the task category to obtain a label analysis value corresponding to the task category.

[0026] Optionally, the step of performing weighted calculation on the text feature vector using the attention mechanism network in the decoder corresponding to the task category to obtain a weighted feature vector corresponding to each task category includes:

[0027] Performing global pooling on the text feature vector using the fully connected layer in the attention mechanism network to obtain a pooled feature vector;

[0028] Obtaining the weights and biases of the fully connected layer in the attention mechanism network, and calculating the pooled feature vector based on a preset activation function and the obtained weights and biases to obtain an attention weight;

[0029] The attention weight and the text feature vector are used to perform weighted calculation to obtain the weighted feature vector.

[0030] Optionally, the step of calculating the target loss value by using the task loss values corresponding to all the task categories includes:

[0031] All the task loss values are added together to obtain the target loss value.

[0032] Optionally, the step of calculating the target loss value by using the task loss values corresponding to all the task categories includes:

[0033] Normalizing the task loss value using a preset normalization standard value to obtain a standard task loss value;

[0034] The target loss value is obtained by summing up all the standard task loss values.

[0035] In order to solve the above problems, the present invention also provides a multi-task learning model training device, which includes:

[0036] A feature extraction module is configured to obtain a training text dataset, wherein each training text data in the training text dataset contains task labels corresponding to multiple task categories; obtain a pre-built multi-task learning model, wherein the multi-task learning model is composed of an encoder connected in parallel with multiple decoders, and the decoders correspond one-to-one to the task categories; use the encoder to select training text data in the training text dataset for feature extraction to obtain a text feature vector; use the decoder corresponding to each task category to perform feature extraction on the text feature vector based on the attention mechanism to obtain a label analysis value corresponding to each task category;

[0037] A loss value calculation module is used to confirm the true value of the label corresponding to the task category according to the task label corresponding to the task category of the training text data corresponding to the text feature vector; calculate the task loss value between the label analysis value corresponding to the task category and the true value of the label using a preset loss function corresponding to the task category, and calculate using the task loss values corresponding to all the task categories to obtain a target loss value;

[0038] The model training module is used to update the parameters of the multi-task learning model when the target loss value is greater than or equal to the loss threshold, return to the step of selecting the training text data in the training text data set using the encoder for feature extraction; and output the trained multi-task learning model when the target loss value is less than the loss threshold.

[0039] In order to solve the above problem, the present invention further provides an electronic device, comprising:

[0040] a memory storing at least one computer program; and

[0041] The processor executes the computer program stored in the memory to implement the multi-task learning model training method described above.

[0042] In order to solve the above problems, the present invention also provides a computer-readable storage medium, which stores at least one computer program, and the at least one computer program is executed by a processor in an electronic device to implement the multi-task learning model training method described above.

[0043] The embodiment of the present invention utilizes a decoder corresponding to each task category to perform feature extraction based on the attention mechanism on the text feature vector to obtain a label analysis value corresponding to each task category; by weighting the text feature vector based on the attention mechanism by the decoder constructed for different tasks, the input features focused on the task corresponding to the text feature vector can be timely screened, the influence of irrelevant features can be removed, the training of each task can be more focused, and the training efficiency of the model can be improved. Therefore, the multi-task learning model training method, device, electronic device and readable storage medium proposed in the embodiment of the present invention improve the efficiency of model training. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 A flowchart of a multi-task learning model training method provided by one embodiment of the present invention;

[0045] Figure 2 A schematic diagram of a module of a multi-task learning model training device provided by one embodiment of the present invention;

[0046] Figure 3 A schematic diagram of the internal structure of an electronic device for implementing a multi-task learning model training method provided by one embodiment of the present invention;

[0047] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0048] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0049] An embodiment of the present invention provides a multi-task learning model training method. The execution subject of the multi-task learning model training method includes but is not limited to at least one of the electronic devices such as a server, a terminal, etc. that can be configured to execute the method provided by the embodiment of the present application. In other words, the multi-task learning model training method can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc. The server can be an independent server or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and basic cloud computing services such as big data and artificial intelligence platforms.

[0050] Reference Figure 1FIG. 1 is a flow chart of a multi-task learning model training method provided by an embodiment of the present invention. In this embodiment of the present invention, the multi-task learning model training method includes:

[0051] S1. Obtain a training text dataset, wherein each training text data in the training text dataset includes task labels corresponding to multiple task categories;

[0052] In the embodiment of the present invention, the purpose of training a multi-task learning model is to train multiple related tasks to help an insurance company identify user levels and recommend suitable insurance types. All of the task categories are interrelated, such as: the multiple tasks trained are user level classification and auto insurance type recommendation, then the task labels corresponding to the user level classification task are respectively high-quality customers or non-high-quality customers; the task label corresponding to the auto insurance type recommendation task is a certain category of auto insurance. The training text data is the user data of the existing users of the insurance company, including: user behavior sequence text and user portrait labels, wherein the user behavior sequence text is a sequence composed of different user behavior texts, wherein the user behavior text is user behavior represented by text, such as "purchase Class A auto insurance"; the user portrait label is a short text representing user characteristics, such as "teacher".

[0053] S2. Obtain a pre-built multi-task learning model, wherein the multi-task learning model is composed of an encoder connected in parallel with multiple decoders, and the decoders correspond to the task categories one by one;

[0054] The multi-task learning model in the embodiment of the present invention is composed of an encoder connected in parallel with decoders corresponding to multiple task categories, wherein the encoder contains network parameters shared by all tasks; further, each task corresponds to a decoder, and the decoder contains task-specific network parameters.

[0055] Optionally, the encoder in the embodiment of the present invention includes a convolutional layer and an embedding layer, and the decoder is composed of an attention mechanism network and a multi-layer perceptron.

[0056] S3, using the encoder to select training text data in the training text data set for feature extraction to obtain a text feature vector;

[0057] The training text data in the embodiment of the present invention consists of user behavior sequences and user portraits.

[0058] Furthermore, in an embodiment of the present invention, the encoder is used to select training text data from the training text dataset for feature extraction to obtain a text feature vector, including:

[0059] Selecting any one of the training text data sets to obtain target training text data, and deleting the target training text data in the training text data set to obtain an updated training text data set;

[0060] Extracting user behavior sequence text and user portrait labels from the target training text data;

[0061] Convert each user behavior text in the user behavior sequence into a vector, and combine the converted vectors to obtain a user behavior matrix;

[0062] Convert the user portrait label into a vector to obtain a user portrait vector;

[0063] Specifically, in the embodiment of the present invention, the Embedding layer in the encoder is used to convert the above text or label into a vector.

[0064] Convolving the user behavior matrix using a convolutional layer in the encoder to obtain a user behavior vector;

[0065] The user behavior vector is combined with the user portrait vector to obtain the text feature vector.

[0066] Furthermore, in the embodiment of the present invention, each user behavior text in the user behavior sequence is converted into a vector, and the converted vectors are combined to obtain a user behavior matrix, including:

[0067] Convert each user behavior text in the user behavior sequence into a vector to obtain a user behavior text vector;

[0068] The user behavior text vectors are combined according to the order of the corresponding user behavior texts in the user behavior sequence to obtain the user behavior matrix.

[0069] S4, using the decoder corresponding to each task category to perform feature extraction based on the attention mechanism on the text feature vector to obtain a label analysis value corresponding to each task category;

[0070] In the implementation of the present invention, since different task categories have different emphases on features learned from the text feature vector, the embodiment of the present invention uses a decoder corresponding to each task category to perform feature extraction based on the attention mechanism on the text feature vector to obtain a label analysis value corresponding to each task category.

[0071] In detail, the embodiment of the present invention uses the decoder corresponding to each task category to perform feature extraction based on the attention mechanism on the text feature vector to obtain the label analysis value corresponding to each task category, including:

[0072] Using the attention mechanism network in the decoder corresponding to the task category to perform weighted calculation on the text feature vector, obtaining a weighted feature vector corresponding to each task category;

[0073] The multi-layer perceptron in the decoder corresponding to the task category is used to perform feature extraction on the weighted feature vector corresponding to the task category to obtain a label analysis value corresponding to the task category.

[0074] Furthermore, in the embodiment of the present invention, the attention mechanism network in the decoder corresponding to the task category is used to perform weighted calculation on the text feature vector to obtain a weighted feature vector corresponding to each task category, including:

[0075] Performing global pooling on the text feature vector using the fully connected layer in the attention mechanism network to obtain a pooled feature vector;

[0076] Obtaining the weights and biases of the fully connected layer in the attention mechanism network, and calculating the pooled feature vector based on a preset activation function and the obtained weights and biases to obtain an attention weight;

[0077] Specifically, an embodiment of the present invention calculates the product of the weight and the pooled feature vector, and sums the calculated product with the bias to obtain an attention parameter; and uses the attention parameter as a function variable parameter of the activation function to calculate the activation function to obtain the attention weight.

[0078] The attention weight and the text feature vector are used to perform weighted calculation to obtain the weighted feature vector.

[0079] Specifically, in an embodiment of the present invention, the attention weight corresponding to the attention mechanism network in the decoder corresponding to each task category is multiplied by the text feature vector to obtain a weighted feature vector of the task category.

[0080] The multi-layer perceptron described in the embodiment of the present invention is a multi-layer neural network composed of multiple layers of fully connected layers connected in series.

[0081] Furthermore, in an embodiment of the present invention, the weighted feature vector corresponding to the task category is input into a multilayer perceptron corresponding to the task category, so that the multilayer perceptron is used to perform high-order feature extraction on the weighted feature vector to obtain a label analysis value corresponding to the task category.

[0082] S5, confirming the true value of the label corresponding to the task category according to the task label corresponding to the task category of the training text data corresponding to the text feature vector;

[0083] In an embodiment of the present invention, the consistency between the label analysis value predicted by the model and the actual corresponding task label is better measured, and the true value of the label corresponding to the task category is confirmed according to the task label corresponding to the task category of the training text data corresponding to the text feature vector.

[0084] For example, if the task label corresponding to the task category is the customer level label, then when the task label is "high-quality customer", the true value of the label corresponding to the task label is 1; when the task label is "non-high-quality customer", the true value of the label corresponding to the task label is 0.

[0085] S6. Calculate the task loss value between the label analysis value and the label true value corresponding to the task category using the preset loss function corresponding to the task category, and calculate using the task loss values corresponding to all the task categories to obtain the target loss value;

[0086] In the embodiment of the present invention, each task category corresponds to a preset loss function, and the loss functions corresponding to each task category can be the same or different.

[0087] Specifically, the loss function described in the embodiment of the present invention includes but is not limited to a logarithmic loss function, a square loss function, an absolute value loss function, a cross entropy loss function, and the like.

[0088] Furthermore, in the embodiment of the present invention, the task loss values corresponding to all the task categories are used to calculate and obtain the target loss value, including:

[0089] All the task loss values are added together to obtain the target loss value.

[0090] In another embodiment of the present invention, the target loss value is obtained by calculating the task loss values corresponding to all the task categories, including:

[0091] Normalizing the task loss value using a preset normalization standard value to obtain a standard task loss value;

[0092] The target loss value is obtained by summing up all the standard task loss values.

[0093] In another embodiment of the present invention, the target loss value is obtained by calculating the task loss values corresponding to all the task categories, including:

[0094] Calculate the proportion of each task loss value in all task loss values to obtain a loss value proportion;

[0095] Specifically, the embodiment of the present invention calculates the ratio of the task loss value to the sum of all task loss values to obtain the loss value proportion.

[0096] Taking the inverse of the loss value ratio of the task loss value to obtain the loss value weight of the task loss value;

[0097] Calculate each task loss value and the corresponding loss value weight to obtain the corresponding standard task loss value.

[0098] For example: the loss value weight of task loss value A is 0.1, and task loss value A is 0.5, then the standard task loss value of task loss value A is 0.5*0.1=0.05.

[0099] In another embodiment of the present invention, the task loss value can be stored in the blockchain node, utilizing the high throughput characteristics of the blockchain node to improve the efficiency of data access.

[0100] S7. Determine whether the target loss value is less than a preset loss threshold;

[0101] In detail, in order to measure whether the training accuracy of the model meets the preset requirements in the embodiment of the present invention, it is determined whether the target loss value is less than the loss threshold.

[0102] S8. When the target loss value is greater than or equal to the loss threshold, update the parameters of the multi-task learning model, and return to the step of selecting the training text data in the training text data set using the encoder to perform feature extraction;

[0103] In this embodiment of the present invention, when the target loss value is greater than or equal to the loss threshold, it indicates that the model accuracy of the multi-task learning model has not met the requirements. Therefore, it is necessary to modify the parameters of the multi-task learning model and continue training. Specifically, this embodiment of the present invention utilizes a gradient descent algorithm to update the model parameters of the multi-task learning model to obtain an updated multi-task learning model.

[0104] Furthermore, in the implementation of the present invention, the parameters of the multi-task learning model are updated, and the step of selecting the training text data in the training text dataset using the encoder is returned to perform feature extraction.

[0105] S9. When the target loss value is less than the loss threshold, output the trained multi-task learning model.

[0106] In the embodiment of the present invention, when the target loss value is less than the loss threshold, the multi-task learning model at this time is output to obtain a trained multi-task learning model.

[0107] like Figure 2 , which is a functional module diagram of the multi-task learning model training device of the present invention.

[0108] The multi-task learning model training device 100 of the present invention can be installed in an electronic device. Depending on the functions implemented, the multi-task learning model training device can include a feature extraction module 101, a loss value calculation module 102, and a model training module 103. The modules described herein can also be referred to as units, which refer to a series of computer program segments that can be executed by an electronic device processor and can perform fixed functions, and are stored in the memory of the electronic device.

[0109] In this embodiment, the functions of each module / unit are as follows:

[0110] The feature extraction module 101 is used to obtain a training text data set, wherein each training text data in the training text data set contains task labels corresponding to multiple task categories; obtain a pre-built multi-task learning model, wherein the multi-task learning model is composed of an encoder connected in parallel with multiple decoders, and the decoders correspond to the task categories one by one; use the encoder to select the training text data in the training text data set for feature extraction to obtain a text feature vector; use the decoder corresponding to each task category to perform feature extraction on the text feature vector based on the attention mechanism to obtain a label analysis value corresponding to each task category;

[0111] The loss value calculation module 102 is used to confirm the true value of the label corresponding to the task category according to the task label corresponding to the task category of the training text data corresponding to the text feature vector; use the preset loss function corresponding to the task category to calculate the task loss value between the label analysis value corresponding to the task category and the true value of the label, and use the task loss values corresponding to all the task categories to calculate and obtain the target loss value;

[0112] The model training module 103 is used to update the parameters of the multi-task learning model when the target loss value is greater than or equal to the loss threshold, and return to the step of selecting the training text data in the training text data set using the encoder for feature extraction; when the target loss value is less than the loss threshold, output the trained multi-task learning model.

[0113] In detail, each module described in the multi-task learning model training device 100 in the embodiment of the present invention adopts the same Figure 1 The multi-task learning model training method described in

[15] is the same technical means and can produce the same technical effects, so I will not go into details here.

[0114] like Figure 3 , which is a structural diagram of an electronic device for implementing a multi-task learning model training method according to the present invention.

[0115] The electronic device may include a processor 10, a memory 11, a communication bus 12 and a communication interface 13, and may also include a computer program stored in the memory 11 and executable on the processor 10, such as a multi-task learning model training program.

[0116] Wherein, the memory 11 includes at least one type of readable storage medium, and the readable storage medium includes a flash memory, a mobile hard disk, a multimedia card, a card-type memory (for example: SD or DX memory, etc.), a magnetic memory, a disk, an optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of an electronic device, such as a mobile hard disk of the electronic device. In other embodiments, the memory 11 can also be an external storage device of an electronic device, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device. Furthermore, the memory 11 can also include both an internal storage unit of the electronic device and an external storage device. The memory 11 can not only be used to store application software and various types of data installed in the electronic device, such as the code of a multi-task learning model training program, but can also be used to temporarily store data that has been output or is to be output.

[0117] In some embodiments, the processor 10 may be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and a combination of various control chips. The processor 10 is the control core (Control Unit) of the electronic device, connecting the various components of the entire electronic device using various interfaces and lines, and executing or executing programs or modules (such as multi-task learning model training programs, etc.) stored in the memory 11, as well as calling data stored in the memory 11, to perform various functions of the electronic device and process data.

[0118] The communication bus 12 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. Such a bus may be classified as an address bus, a data bus, a control bus, etc. The communication bus 12 is configured to enable communication between the memory 11 and at least one processor 10. For ease of illustration, the figure shows only one thick line, but this does not imply that there is only one bus or only one type of bus.

[0119] Figure 3 Only the electronic device with components is shown, and it can be understood by those skilled in the art that Figure 3 The structure shown does not limit the electronic device, and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0120] For example, although not shown, the electronic device may further include a power source (such as a battery) for powering various components. Preferably, the power source may be logically connected to the at least one processor 10 via a power management device, thereby implementing functions such as charging management, discharging management, and power consumption management through the power management device. The power source may further include any components such as one or more DC or AC power sources, a recharging device, a power failure classification circuit, a power converter or inverter, a power status indicator, etc. The electronic device may further include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0121] Optionally, the communication interface 13 may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device and other electronic devices.

[0122] Optionally, the communication interface 13 may further include a user interface, which may be a display or an input unit (such as a keyboard). Optionally, the user interface may also be a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. The display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device and to display a visual user interface.

[0123] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited to this structure.

[0124] The multi-task learning model training program stored in the memory 11 of the electronic device is a combination of multiple computer programs. When running in the processor 10, it can achieve:

[0125] Obtaining a training text data set, wherein each training text data in the training text data set includes task labels corresponding to multiple task categories;

[0126] Obtain a pre-built multi-task learning model, wherein the multi-task learning model is composed of an encoder connected in parallel with multiple decoders, and the decoders correspond one-to-one to the task categories;

[0127] Using the encoder to select training text data in the training text data set for feature extraction to obtain a text feature vector;

[0128] Using the decoder corresponding to each task category to perform feature extraction based on the attention mechanism on the text feature vector, to obtain a label analysis value corresponding to each task category;

[0129] Confirming the true value of the label corresponding to the task category according to the task label corresponding to the task category of the training text data corresponding to the text feature vector;

[0130] Calculate the task loss value between the label analysis value and the label true value corresponding to the task category using the preset loss function corresponding to the task category, and use the task loss values corresponding to all the task categories to calculate and obtain the target loss value;

[0131] When the target loss value is greater than or equal to the loss threshold, updating the parameters of the multi-task learning model, and returning to the step of selecting the training text data in the training text data set by the encoder for feature extraction;

[0132] When the target loss value is less than the loss threshold, the trained multi-task learning model is output.

[0133] Specifically, the specific implementation method of the processor 10 for the above computer program can refer to Figure 1 The description of the relevant steps in the corresponding embodiments will not be repeated here.

[0134] Furthermore, if the module / unit integrated into the electronic device is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. The computer-readable medium can be non-volatile or volatile. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, or read-only memory (ROM).

[0135] An embodiment of the present invention may further provide a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program. When the computer program is executed by a processor of an electronic device, the computer program may implement:

[0136] Obtaining a training text data set, wherein each training text data in the training text data set includes task labels corresponding to multiple task categories;

[0137] Obtain a pre-built multi-task learning model, wherein the multi-task learning model is composed of an encoder connected in parallel with multiple decoders, and the decoders correspond one-to-one to the task categories;

[0138] Using the encoder to select training text data in the training text data set for feature extraction to obtain a text feature vector;

[0139] Using the decoder corresponding to each task category to perform feature extraction based on the attention mechanism on the text feature vector, to obtain a label analysis value corresponding to each task category;

[0140] Confirming the true value of the label corresponding to the task category according to the task label corresponding to the task category of the training text data corresponding to the text feature vector;

[0141] Calculate the task loss value between the label analysis value and the label true value corresponding to the task category using the preset loss function corresponding to the task category, and use the task loss values corresponding to all the task categories to calculate and obtain the target loss value;

[0142] When the target loss value is greater than or equal to the loss threshold, updating the parameters of the multi-task learning model, and returning to the step of selecting the training text data in the training text data set by the encoder for feature extraction;

[0143] When the target loss value is less than the loss threshold, the trained multi-task learning model is output.

[0144] Furthermore, the computer-usable storage medium may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function, etc.; the data storage area may store data created according to the use of the blockchain node, etc.

[0145] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the module division is merely a logical function division, and other division methods may be used in actual implementation.

[0146] The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed across multiple network elements. Some or all of the modules may be selected to achieve the purpose of the solution of this embodiment according to actual needs.

[0147] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to achieve optimal results.

[0148] In addition, the functional modules in various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or hardware plus software functional modules.

[0149] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0150] Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims rather than the foregoing description, and all changes that come within the meaning and range of equivalents of the claims are intended to be embraced therein. Any reference to a figure in a claim should not be construed as limiting the claim to which it relates.

[0151] Blockchain, as used in this article, refers to a novel application model for computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Blockchain is essentially a decentralized database, a series of data blocks generated using cryptographic methods. Each block contains information about a batch of online transactions, used to verify the validity of this information (to prevent counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, the platform product service layer, and the application service layer.

[0152] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a system claim may also be implemented by a single unit or device through software or hardware. Second-order terms are used to indicate names and do not imply any particular order.

[0153] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A multi-task learning model training method, characterized in that: The method comprises: Obtaining a training text data set, wherein each training text data in the training text data set includes task labels corresponding to multiple task categories; Obtain a pre-built multi-task learning model, wherein the multi-task learning model is composed of an encoder connected in parallel with multiple decoders, and the decoders correspond one-to-one to the task categories; Extract target training text data from the training text data set, extract user behavior sequence text and user portrait labels from the target training text data, obtain a user behavior matrix based on the user behavior sequence text, convert the user portrait label into a vector to obtain a user portrait vector, use the convolution layer in the encoder to convolve the user behavior matrix to obtain a user behavior vector, combine the user behavior vector with the user portrait vector to obtain a text feature vector; use the attention mechanism network of the decoder to perform weighted calculation on the text feature vector to obtain a weighted feature vector corresponding to each task category, use the multi-layer perceptron of the decoder to perform feature extraction on the weighted feature vector corresponding to the task category, and obtain a label analysis value corresponding to each task category; Confirming the true value of the label corresponding to the task category according to the task label corresponding to the task category of the training text data corresponding to the text feature vector; Calculate the task loss value between the label analysis value and the label true value corresponding to the task category using the preset loss function corresponding to the task category, and use the task loss values corresponding to all the task categories to calculate and obtain the target loss value; When the target loss value is greater than or equal to the loss threshold, updating the parameters of the multi-task learning model, and returning to the step of selecting the training text data in the training text data set by the encoder for feature extraction; When the target loss value is less than the loss threshold, the trained multi-task learning model is output.

2. The multi-task learning model training method according to claim 1, characterized in that: The step of extracting target training text data from the training text data set, extracting user behavior sequence text and user portrait labels from the target training text data, and obtaining a user behavior matrix based on the user behavior sequence text includes: Selecting any one of the training text data sets to obtain target training text data, and deleting the target training text data in the training text data set to obtain an updated training text data set; Extracting user behavior sequence text and user portrait labels from the target training text data; Each user behavior text in the user behavior sequence is converted into a vector, and the converted vectors are combined to obtain a user behavior matrix.

3. The multi-task learning model training method according to claim 1, characterized in that The step of converting each user behavior text in the user behavior sequence into a vector and combining the converted vectors to obtain a user behavior matrix includes: Convert each user behavior text in the user behavior sequence into a vector to obtain a user behavior text vector; The user behavior text vectors are combined according to the order of the corresponding user behavior texts in the user behavior sequence to obtain the user behavior matrix.

4. The multi-task learning model training method according to claim 1, characterized in that The step of performing weighted calculation on the text feature vector using the attention mechanism network in the decoder corresponding to the task category to obtain a weighted feature vector corresponding to each task category includes: Performing global pooling on the text feature vector using the fully connected layer in the attention mechanism network to obtain a pooled feature vector; Obtaining the weights and biases of the fully connected layer in the attention mechanism network, and calculating the pooled feature vector based on a preset activation function and the obtained weights and biases to obtain an attention weight; The attention weight and the text feature vector are used to perform weighted calculation to obtain the weighted feature vector.

5. The multi-task learning model training method according to claim 1, characterized in that The calculation using the task loss values corresponding to all the task categories to obtain the target loss value includes: All the task loss values are added together to obtain the target loss value.

6. The multi-task learning model training method according to any one of claims 1 to 5, wherein: The calculation using the task loss values corresponding to all the task categories to obtain the target loss value includes: Normalizing the task loss value using a preset normalization standard value to obtain a standard task loss value; The target loss value is obtained by summing up all the standard task loss values.

7. A multi-task learning model training device, characterized in that: include: A feature extraction module is used to obtain a training text data set, wherein each training text data in the training text data set contains task labels corresponding to multiple task categories; obtain a pre-built multi-task learning model, wherein the multi-task learning model is composed of an encoder connected in parallel with multiple decoders, and the decoders correspond to the task categories one by one; extract target training text data from the training text data set, extract user behavior sequence text and user portrait labels from the target training text data, obtain a user behavior matrix based on the user behavior sequence text, convert the user portrait label into a vector to obtain a user portrait vector, use the convolution layer in the encoder to convolve the user behavior matrix to obtain a user behavior vector, combine the user behavior vector with the user portrait vector to obtain a text feature vector; use the attention mechanism network of the decoder to perform weighted calculation on the text feature vector to obtain a weighted feature vector corresponding to each task category, use the multi-layer perceptron of the decoder to perform feature extraction on the weighted feature vector corresponding to the task category, and obtain a label analysis value corresponding to each task category; A loss value calculation module is used to confirm the true value of the label corresponding to the task category according to the task label corresponding to the task category of the training text data corresponding to the text feature vector; calculate the task loss value between the label analysis value corresponding to the task category and the true value of the label using a preset loss function corresponding to the task category, and calculate using the task loss values corresponding to all the task categories to obtain a target loss value; The model training module is used to update the parameters of the multi-task learning model when the target loss value is greater than or equal to the loss threshold, return to the step of selecting the training text data in the training text data set using the encoder for feature extraction; and output the trained multi-task learning model when the target loss value is less than the loss threshold.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively coupled to the at least one processor; The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the multi-task learning model training method as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the multi-task learning model training method according to any one of claims 1 to 6 is implemented.

Images