Data processing method and device, electronic equipment and computer storage medium
By forming the first training set and the second training set and training the target classification model based on these data sets, the problem of artificial intelligence models in the prior art performing poorly in different fields is solved, and the effect of accurately classifying the target field data under a small amount of data is achieved.
Patent Information
- Application Number
- CN202411829906.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-12
- Publication Date
- 2025-04-18
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
In the prior art, artificial intelligence models can only be used to predict data in the same field after training, and a large amount of training data is required to perform well in other fields.
By acquiring the training data sets of at least two fields, each data set includes a first training data of a first amount of data and a second training data of a second amount of data, forming a first training set and a second training set. Then, the first network is trained based on the first training set, an initial cumulative gradient is obtained, and the first weight matrix is determined based on the initial cumulative gradient and the initial weight matrix of the second network. The second network with the weight matrix as the first weight matrix is used as the update network, and the update network is trained through the second training set to obtain the second weight matrix, form a target network for training, and obtain the target classification model.
The target classification model obtained by training based on the first training set, the second training set and the target training set based on a small amount of data is realized, and the pending data in the target field can be accurately classified.
Smart Images
Figure CN119848538A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of model training technology, and in particular to a data processing method, device, electronic device and computer storage medium. Background Art
[0002] Currently, AI models are limited in their application scenarios and have two problems:
[0003] After training with data from a certain field, only the data from that field can be used for prediction. This trained model does not perform well when dealing with data from other fields.
[0004] If you want to train a model in a field to perform better, you currently need a large amount of training data. Summary of the invention
[0005] The technical problem to be solved by the present invention is to provide a data processing method, device, electronic device and computer storage medium, aiming to solve at least one of the above technical problems.
[0006] In the first aspect, the technical solution of the present invention to solve the above technical problem is as follows: a data processing method, the method comprising:
[0007] Obtain data to be processed for the target area;
[0008] According to the pre-trained target classification model, the data to be processed is classified to obtain the classification result of the data to be processed, wherein the training process of the target classification model is:
[0009] Acquire a training data set for each of at least two fields, each of the training data sets including first training data of a first data volume and second training data of a first data volume;
[0010] The first training data corresponding to each field forms a first training set, and the second training data corresponding to each field forms a second training set, wherein the first training set and the second training set both include the same amount of data;
[0011] Training the first network using the first training set to obtain an initial cumulative gradient corresponding to the first network;
[0012] Based on the initial accumulated gradient and the initial weight matrix corresponding to the second network, determine a first weight matrix, use the second network whose weight matrix is the first weight matrix as an update network, and the first network and the second network are networks with the same network structure;
[0013] The update network is trained using the second training set to obtain a second weight matrix corresponding to the update network, and the update network whose weight matrix is the second weight matrix is used as the target network to train according to the target network to obtain the target classification model.
[0014] The beneficial effect of the present invention is that the target classification model is obtained by training based on a first training set, a second training set and a target training set with a small amount of data, and during the training process of the target classification model, an accurate target network can be determined based on the first training set and the second training set, so that the target classification model obtained based on the target network training can accurately classify the data to be processed in the target field.
[0015] Based on the above technical solution, the present invention can also be improved as follows.
[0016] Further, determining a first weight matrix based on the initial accumulated gradient and an initial weight matrix corresponding to the second network includes:
[0017] Based on the initial accumulated gradient, the initial weight matrix corresponding to the second network and the learning rate corresponding to the second network, the first weight matrix is determined by a first formula, wherein the first formula is:
[0018] W1=W-lr*t
[0019] Among them, W1 is the first weight matrix, W is the initial weight matrix, lr is the learning rate, and t is the initial accumulated gradient.
[0020] Furthermore, each piece of data in the first training set corresponds to a label value, and each label value represents a true classification result of the corresponding data; and the first network is trained using the first training set to obtain an initial cumulative gradient corresponding to the first network, including:
[0021] Training the first network using the first training set to obtain a prediction value corresponding to each piece of data in the first training set, each of the prediction values representing a classification result of the corresponding data;
[0022] For each piece of data in the first training set, according to the predicted value and the labeled value of the piece of data, a loss value corresponding to the piece of data is determined by using a crossentropy loss function;
[0023] An initial cumulative gradient corresponding to the first network is determined according to all loss values corresponding to the first training set.
[0024] Furthermore, the first network includes an embedding layer, a multi-head attention mechanism layer, a forward calculation layer and a classification layer connected in sequence, and the first network is trained through the first training set to obtain a prediction value corresponding to each data in the first training set, including:
[0025] Through the embedding layer, each piece of data in the first training set is converted into a multi-dimensional feature vector;
[0026] Through the multi-head attention mechanism layer, feature extraction and attention information fusion processing are performed on the multidimensional feature vector corresponding to each data in the first training set to obtain a processed multidimensional feature vector corresponding to each data;
[0027] Performing nonlinear processing on the processed multidimensional feature vector corresponding to each piece of data in the first training set through the forward computing layer to obtain a target multidimensional feature vector corresponding to each piece of data;
[0028] The target multidimensional feature vector corresponding to each piece of data in the first training set is classified by the classification layer to obtain a prediction value corresponding to each piece of data.
[0029] In a second aspect, in order to solve the above technical problem, the present invention further provides a data processing device, the device comprising:
[0030] An acquisition module is used to acquire the data to be processed for the target field;
[0031] A classification module is used to classify the data to be processed according to a pre-trained target classification model to obtain a classification result of the data to be processed;
[0032] A model training module is used to obtain a training data set for each of at least two fields, each of which includes first training data of a first data volume and second training data of a first data volume; the first training data corresponding to each field forms a first training set, and the second training data corresponding to each field forms a second training set, and the first training set and the second training set both include the same amount of data; through the first training set, a first network is trained to obtain an initial cumulative gradient corresponding to the first network; based on the initial cumulative gradient and the initial weight matrix corresponding to the second network, a first weight matrix is determined, and a second network whose weight matrix is the first weight matrix is used as an update network, and the first network and the second network are networks with the same network structure; through the second training set, the update network is trained to obtain a second weight matrix corresponding to the update network, and the update network whose weight matrix is the second weight matrix is used as a target network, so as to train according to the target network to obtain the target classification model.
[0033] In a third aspect, in order to solve the above-mentioned technical problem, the present invention also provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the data processing method of the present application is implemented when the processor executes the computer program.
[0034] In a fourth aspect, in order to solve the above-mentioned technical problem, the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the data processing method of the present application is implemented.
[0035] Additional aspects and advantages of the present application will be partially given in the following description, which will become apparent from the following description, or will be understood through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for describing the embodiments of the present invention are briefly introduced below.
[0037] Figure 1 A flowchart of a data processing method provided by an embodiment of the present invention;
[0038] Figure 2 A schematic diagram of the structure of a data processing device provided by one embodiment of the present invention;
[0039] Figure 3 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0040] The principles and features of the present invention are described below. The examples given are only used to explain the present invention and are not used to limit the scope of the present invention.
[0041] The technical solution of the present invention and how the technical solution of the present invention solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present invention will be described below in conjunction with the accompanying drawings.
[0042] The solution provided by the embodiment of the present invention can be applied to any application scenario where it is necessary to accurately classify the data to be processed in a certain field. The solution provided by the embodiment of the present invention can be executed by any electronic device, for example, it can be a user's terminal device, including at least one of the following: a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, a smart TV, and a smart car device.
[0043] The embodiment of the present invention provides a possible implementation method, such as Figure 1 As shown, a flowchart of a data processing method is provided. The scheme can be executed by any electronic device, for example, a terminal device, or a terminal device and a server. For the convenience of description, the method provided by the embodiment of the present invention will be described below by taking the terminal device as the execution subject as an example. Figure 1 The method may include the following steps:
[0044] S1, obtain the data to be processed for the target field;
[0045] S2, classifying the data to be processed according to the pre-trained target classification model to obtain the classification result of the data to be processed, wherein the training process of the target classification model is:
[0046] S21, obtaining a training data set for each of at least two fields, each of the training data sets including first training data of a first data volume and second training data of a first data volume;
[0047] S22, forming a first training set with the first training data corresponding to each field, and forming a second training set with the second training data corresponding to each field, wherein the first training set and the second training set both include the same amount of data;
[0048] S23, training the first network using the first training set to obtain an initial cumulative gradient corresponding to the first network;
[0049] S24, determining a first weight matrix based on the initial accumulated gradient and the initial weight matrix corresponding to the second network, and using the second network whose weight matrix is the first weight matrix as an update network, wherein the first network and the second network are networks with the same network structure;
[0050] S25, training the update network through the second training set to obtain the second weight matrix corresponding to the update network, taking the update network whose weight matrix is the second weight matrix as the target network, and training according to the target network to obtain the target classification model.
[0051] The target classification model in the method of the present invention is obtained by training based on a first training set, a second training set and a target training set with a small amount of data, and during the training process of the target classification model, an accurate target network can be determined based on the first training set and the second training set, so that the target classification model obtained based on the target network training can accurately classify the data to be processed in the target field.
[0052] The solution of the present invention is further described below in conjunction with the following specific embodiments. In this embodiment, a data processing method provided may include the following steps:
[0053] S1, obtain the data to be processed for the target field;
[0054] The target field refers to a certain field, such as the medical field, the transportation field, etc.
[0055] S2, classifying the data to be processed according to the pre-trained target classification model to obtain the classification result of the data to be processed, wherein the training process of the target classification model is:
[0056] S21, obtaining a training data set for each of at least two fields, each of the training data sets including first training data of a first data volume and second training data of a first data volume;
[0057] The first data volume is smaller than the set data volume, and the data volume of the training data set corresponding to each field is the same, for example, both include a set number of data.
[0058] S22, forming a first training set with the first training data corresponding to each field, and forming a second training set with the second training data corresponding to each field, wherein the first training set and the second training set both include the same amount of data;
[0059] The first training set includes first training data corresponding to each field, and the second training set includes second training data corresponding to each field.
[0060] As an example, there are training data sets in 10 fields, which are recorded as: data1, data2, data3, data4...data10. The first training data of each field is 10 data items, and the second training data of each field is 10 data items. Then the first training set includes 10 data items in each of all fields, for a total of 100 data items. Similarly, the second training set includes 10 data items in each of all fields, for a total of 100 data items.
[0061] The data volume of the first training set and the second training set are both small-volume training sets, so that the first training set and the second training set both include training data of multiple fields.
[0062] S23, training the first network using the first training set to obtain an initial cumulative gradient corresponding to the first network;
[0063] Among them, one possible implementation method of training the first network through the first training set to obtain the initial cumulative gradient corresponding to the first network is:
[0064] The first network is trained respectively using the training data corresponding to each field in the first training set to obtain the gradient corresponding to each field, and the gradients corresponding to all fields are accumulated and summed to obtain the initial cumulative gradient.
[0065] The gradient corresponding to each field refers to the gradient corresponding to when the first network converges when the first network is trained based on the training data of the field. The convergence of the first network can be judged based on the following method: when the loss value corresponding to the training data corresponding to the field is less than the set loss value, the first network is judged to converge.
[0066] Among them, the initial weight matrix can be a setting matrix.
[0067] Optionally, each piece of data in the first training set corresponds to a label value, and each label value represents a true classification result of the corresponding data; in S23, the first network is trained using the first training set to obtain an initial cumulative gradient corresponding to the first network, including:
[0068] S231, training the first network using the first training set to obtain a prediction value corresponding to each piece of data in the first training set, each of the prediction values representing a classification result of the corresponding data;
[0069] S232, for each piece of data in the first training set, determine the loss value corresponding to the piece of data according to the predicted value and the labeled value of the piece of data by using the crossentropy loss function;
[0070] S233: Determine an initial cumulative gradient corresponding to the first network according to all loss values corresponding to the first training set.
[0071] Optionally, the specific implementation of the above S232 is:
[0072] Among them, the crossentropy loss function can be expressed as:
[0073] H(p,q)=-∑xp(x)logq(x)
[0074] Among them, p(x) is the labeling result corresponding to each data, which can also be called the true label label, q(x) is the predicted value corresponding to each data, and H(p,q) is the loss value corresponding to each data.
[0075] As an example, assuming a three-classification task, the correct label of a piece of data is the first category, then p = [1, 0, 0], and the predicted value corresponding to the data is assumed to be [0.5, 0.4, 0.1]. The cross entropy is calculated as follows:
[0076] H(p=[1,0,0], q=[0.5, 0.4, 0.1])=-(1*log0.5+0*log0.4+0*log0.1)≈0.3
[0077] That is, the loss value corresponding to this piece of data is 0.3.
[0078] Among them, one possible implementation method of the above S233 is:
[0079] According to all loss values corresponding to each field in all loss values corresponding to the first training set, the gradient corresponding to each field in the first training set is determined, and the gradients corresponding to all fields are accumulated and summed to obtain an initial cumulative gradient.
[0080] S24, determining a first weight matrix based on the initial accumulated gradient and the initial weight matrix corresponding to the second network, and using the second network whose weight matrix is the first weight matrix as an update network, wherein the first network and the second network are networks with the same network structure;
[0081] Optionally, in S24, determining a first weight matrix based on the initial accumulated gradient and an initial weight matrix corresponding to the second network includes:
[0082] Based on the initial accumulated gradient, the initial weight matrix corresponding to the second network and the learning rate corresponding to the second network, the first weight matrix is determined by a first formula, wherein the first formula is:
[0083] W1=W-lr*t
[0084] Among them, W1 is the first weight matrix, W is the initial weight matrix, lr is the learning rate, and t is the initial accumulated gradient.
[0085] S25, training the update network through the second training set to obtain the second weight matrix corresponding to the update network, taking the update network whose weight matrix is the second weight matrix as the target network, and training according to the target network to obtain the target classification model.
[0086] The implementation process of training the update network through the second training set to obtain the second weight matrix corresponding to the update network is the same as the implementation method of S24 above, which is specifically expressed as follows:
[0087] W2=W1-lr*t
[0088] Among them, the second weight matrix is W2.
[0089] Optionally, if the at least two fields include the target field, the target network may be trained based on the target training set (the first training set and / or the second training set) to obtain a target classification model.
[0090] Optionally, if the target domain is not included in at least two domains, the target network can be fine-tuned with a small sample based on the training set corresponding to the target domain, that is, based on the method for determining the second weight matrix in the previous article, the target network is trained according to the second weight matrix to obtain a third weight matrix corresponding to the target network, and the target network whose weight matrix is the third weight matrix is used as the final network, and the final network is trained according to the training set corresponding to the target domain to obtain the target classification model.
[0091] Among them, the target network is trained according to the second weight matrix to obtain the third weight matrix corresponding to the target network in the same way as the second weight matrix described above, which will not be repeated here.
[0092] Based on the above method, fine-tuning training on new tasks (training sets corresponding to the target domain) can be achieved using small sample data instead of large sample data, so that the model no longer requires a large number of samples to be well trained (with higher accuracy).
[0093] The difference between this application scheme and the general network training process is:
[0094] First, obtain two initial networks with the same network structure, one is the first network and the other is the second network, and at the same time obtain the initial weight matrix corresponding to each initial network;
[0095] For the first network in two initial networks with the same network structure, the initial cumulative gradient corresponding to the first network can be determined based on the method of determining the initial cumulative gradient in the previous article, but the initial cumulative gradient is not updated to the initial weight matrix corresponding to the second network. Instead, the initial cumulative gradient involved is recorded and used in the subsequent training process of the second network.
[0096] Optionally, the first network includes an embedding layer, a multi-head attention mechanism layer, a forward calculation layer and a classification layer connected in sequence. In S231, the first network is trained using the first training set to obtain a prediction value corresponding to each data in the first training set, including:
[0097] S2311, converting each piece of data in the first training set into a multi-dimensional feature vector through the embedding layer;
[0098] Specifically, the embedding weight matrix corresponding to the embedding layer is matrix multiplied with each data in the first training set (the ID mapped to each data, the data identifier) to obtain the embedding word vector as the embedding matrix representation corresponding to the first training set. In the matrix obtained by the matrix multiplication, the vector dimension is 768 dimensions, that is, each data in the first training set is converted into a multi-dimensional feature vector.
[0099] S2312, performing feature extraction and attention information fusion processing on the multidimensional feature vector corresponding to each piece of data in the first training set through the multi-head attention mechanism layer to obtain a processed multidimensional feature vector corresponding to each piece of data;
[0100] Specifically, the 768-dimensional feature vector output by the embedding layer is subjected to matrix feature extraction by three linear layers respectively, and then matrix multiplication is performed to obtain a 768-dimensional vector representation of each input data (each data in the first training set) that incorporates the attention information, and the processed multidimensional feature vector corresponding to each data is obtained.
[0101] The attention information refers to the features of each piece of data in the first training set that can express the classification target.
[0102] S2313, performing nonlinear processing on the processed multidimensional feature vector corresponding to each piece of data in the first training set through the forward computing layer to obtain a target multidimensional feature vector corresponding to each piece of data;
[0103] Specifically, each output obtained by the multi-head attention mechanism layer passes through two linear layers and is activated by the activation layer to output the final 768-dimensional vector (target multi-dimensional feature vector) representation of each data;
[0104] S2314: Classify the target multidimensional feature vector corresponding to each piece of data in the first training set through the classification layer to obtain a prediction value corresponding to each piece of data.
[0105] In the present application, a verification set can also be generated based on the training data set corresponding to each field. The verification set can be determined in the same way as the first training set, and the amount of data in the verification set is the first amount of data. Based on the verification set, the accuracy of the target classification model can be verified, so that the model parameters can be adjusted based on the verification results to improve the training accuracy.
[0106] Compared with the prior art, the solution of the present invention has the following beneficial effects:
[0107] 1. Generate a first training set and a second training set based on training data from different fields, so that the target classification model trained based on the first training set and the second training set can classify the data to be processed in different fields;
[0108] 2. In the model training method of the present application, the data amount of the first training set and the second training set used are both the first data amount, that is, the small sample data amount. Therefore, the present solution can obtain a target classification model that can accurately perform classification based on the data of the small sample data amount.
[0109] Based on Figure 1 Based on the same principle as the method shown in , the embodiment of the present invention further provides a data processing device 20, such as Figure 2 As shown in , the data processing device 20 may include an acquisition module 210, a classification module 220 and a model training module 230, wherein:
[0110] An acquisition module 210 is used to acquire data to be processed for a target field;
[0111] The classification module 220 is used to classify the data to be processed according to the pre-trained target classification model to obtain the classification result of the data to be processed;
[0112] The model training module 230 is used to obtain a training data set for each of at least two fields, each of which includes first training data of a first data volume and second training data of a first data volume; the first training data corresponding to each field forms a first training set, and the second training data corresponding to each field forms a second training set, and the first training set and the second training set both include the same amount of data; the first training set is used to train the first network to obtain an initial cumulative gradient corresponding to the first network; based on the initial cumulative gradient and the initial weight matrix corresponding to the second network, a first weight matrix is determined, and a second network whose weight matrix is the first weight matrix is used as an update network, and the first network and the second network are networks with the same network structure; the update network is trained through the second training set to obtain a second weight matrix corresponding to the update network, and the update network whose weight matrix is the second weight matrix is used as a target network, so as to train according to the target network to obtain the target classification model.
[0113] Optionally, when determining the first weight matrix based on the initial accumulated gradient and the initial weight matrix corresponding to the second network, the model training module is specifically used to:
[0114] Based on the initial accumulated gradient, the initial weight matrix corresponding to the second network and the learning rate corresponding to the second network, the first weight matrix is determined by a first formula, wherein the first formula is:
[0115] W1=W-lr*t
[0116] Among them, W1 is the first weight matrix, W is the initial weight matrix, lr is the learning rate, and t is the initial accumulated gradient.
[0117] Optionally, each piece of data in the first training set corresponds to a label value, and each of the label values represents a true classification result of the corresponding data; when the model training module trains the first network through the first training set to obtain an initial cumulative gradient corresponding to the first network, it is specifically used to:
[0118] Training the first network using the first training set to obtain a prediction value corresponding to each piece of data in the first training set, each of the prediction values representing a classification result of the corresponding data;
[0119] For each piece of data in the first training set, according to the predicted value and the labeled value of the piece of data, a loss value corresponding to the piece of data is determined by using a crossentropy loss function;
[0120] An initial cumulative gradient corresponding to the first network is determined according to all loss values corresponding to the first training set.
[0121] Optionally, the first network includes an embedding layer, a multi-head attention mechanism layer, a forward calculation layer and a classification layer connected in sequence, and the model training module trains the first network through the first training set to obtain a predicted value corresponding to each data in the first training set, specifically for:
[0122] Through the embedding layer, each piece of data in the first training set is converted into a multi-dimensional feature vector;
[0123] Through the multi-head attention mechanism layer, feature extraction and attention information fusion processing are performed on the multidimensional feature vector corresponding to each data in the first training set to obtain a processed multidimensional feature vector corresponding to each data;
[0124] Performing nonlinear processing on the processed multidimensional feature vector corresponding to each piece of data in the first training set through the forward computing layer to obtain a target multidimensional feature vector corresponding to each piece of data;
[0125] The target multidimensional feature vector corresponding to each piece of data in the first training set is classified by the classification layer to obtain a prediction value corresponding to each piece of data.
[0126] The data processing device of the embodiment of the present invention can execute the data processing method provided by the embodiment of the present invention, and the implementation principle is similar. The actions performed by each module and unit in the data processing device in each embodiment of the present invention correspond to the steps in the data processing method in each embodiment of the present invention. For the detailed functional description of each module of the data processing device, please refer to the description in the corresponding data processing method shown in the previous text, which will not be repeated here.
[0127] The data processing device may be a computer program (including program code) running in a computer device, for example, the data processing device is an application software; the device may be used to execute corresponding steps in the method provided in an embodiment of the present invention.
[0128] In some embodiments, the data processing device provided by the embodiment of the present invention can be implemented in a combination of software and hardware. As an example, the data processing device provided by the embodiment of the present invention can be a processor in the form of a hardware decoding processor, which is programmed to execute the data processing method provided by the embodiment of the present invention. For example, the processor in the form of a hardware decoding processor can adopt one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field programmable gate arrays (FPGAs) or other electronic components.
[0129] In other embodiments, the data processing device provided by the embodiments of the present invention may be implemented in software. Figure 2 A data processing device stored in a memory is shown, which may be software in the form of a program and a plug-in, and includes a series of modules, including an acquisition module 210, a classification module 220 and a model training module 230, for implementing the data processing method provided in an embodiment of the present invention.
[0130] The modules involved in the embodiments of the present invention may be implemented by software or hardware, wherein the name of a module does not limit the module itself in some cases.
[0131] Based on the same principle as the method shown in the embodiments of the present invention, an electronic device is also provided in the embodiments of the present invention, which may include but is not limited to: a processor and a memory; the memory is used to store a computer program; the processor is used to execute the method shown in any embodiment of the present invention by calling the computer program.
[0132] In an alternative embodiment, an electronic device is provided, such as Figure 3 As shown, Figure 3 The electronic device 4000 shown includes: a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, such as through a bus 4002. Optionally, the electronic device 4000 may also include a transceiver 4004, which may be used for data interaction between the electronic device and other electronic devices, such as data transmission and / or data reception. It should be noted that in actual applications, the transceiver 4004 is not limited to one, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present invention.
[0133] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of the present invention. Processor 4001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.
[0134] The bus 4002 may include a path to transmit information between the above components. The bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 4002 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0135] The memory 4003 can be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited to this.
[0136] The memory 4003 is used to store application code (computer program) for executing the solution of the present invention, and the execution is controlled by the processor 4001. The processor 4001 is used to execute the application code stored in the memory 4003 to implement the content shown in the above method embodiment.
[0137] The electronic device may also be a terminal device. Figure 3 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.
[0138] An embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer-readable storage medium is run on a computer, the computer can execute the corresponding contents of the aforementioned method embodiment.
[0139] According to another aspect of the present invention, a computer program product or a computer program is also provided, the computer program product or the computer program comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the methods provided in the implementation modes of the above-mentioned various embodiments.
[0140] Computer program code for performing the operations of the present invention may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).
[0141] It should be understood that the flow charts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the module, the program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0142] The computer-readable storage medium provided by the embodiment of the present invention may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, device, or device.
[0143] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device executes the method shown in the above embodiment.
[0144] The above description is only a preferred embodiment of the present invention and an explanation of the technical principles used. Those skilled in the art should understand that the disclosure scope involved in the present invention is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present invention (but not limited to) by each other.
Claims
1. A data processing method, characterized in that: The following steps are involved: Obtain data to be processed for the target area; According to the pre-trained target classification model, the data to be processed is classified to obtain the classification result of the data to be processed, wherein the training process of the target classification model is: Acquire a training data set for each of at least two fields, each of the training data sets including first training data of a first data volume and second training data of a first data volume; The first training data corresponding to each field forms a first training set, and the second training data corresponding to each field forms a second training set, wherein the first training set and the second training set both include the same amount of data; Training the first network using the first training set to obtain an initial cumulative gradient corresponding to the first network; Based on the initial accumulated gradient and the initial weight matrix corresponding to the second network, determine a first weight matrix, use the second network whose weight matrix is the first weight matrix as an update network, and the first network and the second network are networks with the same network structure; The update network is trained using the second training set to obtain a second weight matrix corresponding to the update network, and the update network whose weight matrix is the second weight matrix is used as the target network to train according to the target network to obtain the target classification model.
2. The method according to claim 1, characterized in that The determining of the first weight matrix based on the initial accumulated gradient and the initial weight matrix corresponding to the second network includes: Based on the initial accumulated gradient, the initial weight matrix corresponding to the second network and the learning rate corresponding to the second network, the first weight matrix is determined by a first formula, wherein the first formula is: W1=W-lr*t Among them, W1 is the first weight matrix, W is the initial weight matrix, lr is the learning rate, and t is the initial accumulated gradient.
3. The method according to claim 1, characterized in that Each piece of data in the first training set corresponds to a label value, and each of the label values represents a true classification result of the corresponding data; The step of training the first network by using the first training set to obtain an initial cumulative gradient corresponding to the first network includes: Training the first network using the first training set to obtain a prediction value corresponding to each piece of data in the first training set, each of the prediction values representing a classification result of the corresponding data; For each piece of data in the first training set, according to the predicted value and the labeled value of the piece of data, a loss value corresponding to the piece of data is determined by using a crossentropy loss function; An initial cumulative gradient corresponding to the first network is determined according to all loss values corresponding to the first training set.
4. The method according to claim 3, characterized in that The first network includes an embedding layer, a multi-head attention mechanism layer, a forward calculation layer and a classification layer connected in sequence. The first network is trained using the first training set to obtain a prediction value corresponding to each data in the first training set, including: Through the embedding layer, each piece of data in the first training set is converted into a multi-dimensional feature vector; Through the multi-head attention mechanism layer, feature extraction and attention information fusion processing are performed on the multidimensional feature vector corresponding to each data in the first training set to obtain a processed multidimensional feature vector corresponding to each data; Performing nonlinear processing on the processed multidimensional feature vector corresponding to each piece of data in the first training set through the forward computing layer to obtain a target multidimensional feature vector corresponding to each piece of data; The target multidimensional feature vector corresponding to each piece of data in the first training set is classified by the classification layer to obtain a prediction value corresponding to each piece of data.
5. A data processing device, characterized in that: include: An acquisition module is used to acquire the data to be processed for the target field; A classification module is used to classify the data to be processed according to a pre-trained target classification model to obtain a classification result of the data to be processed; A model training module, used to obtain a training data set for each of at least two fields, each of the training data sets including first training data of a first data volume and second training data of the first data volume; The first training data corresponding to each field forms a first training set, and the second training data corresponding to each field forms a second training set, wherein the first training set and the second training set both include the same amount of data; Training the first network using the first training set to obtain an initial cumulative gradient corresponding to the first network; Based on the initial accumulated gradient and the initial weight matrix corresponding to the second network, determine a first weight matrix, use the second network whose weight matrix is the first weight matrix as an update network, and the first network and the second network are networks with the same network structure; The update network is trained using the second training set to obtain a second weight matrix corresponding to the update network, and the update network whose weight matrix is the second weight matrix is used as the target network to train according to the target network to obtain the target classification model.
6. The device according to claim 5, characterized in that When the model training module determines the first weight matrix based on the initial accumulated gradient and the initial weight matrix corresponding to the second network, it is specifically used to: Based on the initial accumulated gradient, the initial weight matrix corresponding to the second network and the learning rate corresponding to the second network, the first weight matrix is determined by a first formula, wherein the first formula is: W1=W-lr*t Among them, W1 is the first weight matrix, W is the initial weight matrix, lr is the learning rate, and t is the initial accumulated gradient.
7. The device according to claim 5, characterized in that Each piece of data in the first training set corresponds to a label value, and each label value represents a true classification result of the corresponding data; when the model training module trains the first network through the first training set to obtain the initial cumulative gradient corresponding to the first network, it is specifically used to: Training the first network using the first training set to obtain a prediction value corresponding to each piece of data in the first training set, each of the prediction values representing a classification result of the corresponding data; For each piece of data in the first training set, according to the predicted value and the labeled value of the piece of data, a loss value corresponding to the piece of data is determined by using a crossentropy loss function; An initial cumulative gradient corresponding to the first network is determined according to all loss values corresponding to the first training set.
8. The device according to claim 7, characterized in that The first network includes an embedding layer, a multi-head attention mechanism layer, a forward calculation layer and a classification layer connected in sequence. When the model training module trains the first network through the first training set to obtain a predicted value corresponding to each data in the first training set, it is specifically used to: Through the embedding layer, each piece of data in the first training set is converted into a multi-dimensional feature vector; Through the multi-head attention mechanism layer, feature extraction and attention information fusion processing are performed on the multidimensional feature vector corresponding to each data in the first training set to obtain a processed multidimensional feature vector corresponding to each data; Performing nonlinear processing on the processed multidimensional feature vector corresponding to each piece of data in the first training set through the forward computing layer to obtain a target multidimensional feature vector corresponding to each piece of data; The target multidimensional feature vector corresponding to each piece of data in the first training set is classified by the classification layer to obtain a prediction value corresponding to each piece of data.
9. An electronic device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 4 when executing the computer program.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.