A data processing method and device, computer equipment and readable storage medium

Abstract

This application provides a data processing method, apparatus, computer equipment, and readable storage medium, applicable to scenarios such as cloud technology, artificial intelligence, intelligent transportation, assisted driving, model initialization, efficient low-parameter fine-tuning, and task migration. The method includes: obtaining candidate network models corresponding to N candidate tasks; inputting business sample data corresponding to the target task into the N candidate network models, and outputting business feature vectors from the N candidate network models; determining the model weights corresponding to the N candidate network models based on the business sample data and the business feature vectors output by the N candidate network models; and generating an initial network model corresponding to the target task based on the N candidate network models and their corresponding model weights. This application can improve the accuracy of the initial network model.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a data processing method, apparatus, computer equipment, and readable storage medium. Background Technology

[0002] Current model initialization methods obtain at least two candidate tasks and directly average the candidate network models corresponding to these two tasks to obtain the network model used for initializing the target task (i.e., the initial network model). Understandably, existing model initialization methods initialize from the perspective of the model itself, directly averaging the model parameters of at least two candidate network models. In this case, for different candidate tasks, different candidate network models all make the same contribution to the initialization of the initial network model. However, this ignores the information contained in the task data of the target and candidate tasks. When the task data of at least two candidate tasks differ significantly, the model parameters in at least two candidate network models will also differ significantly. In this case, directly averaging the significantly different model parameters makes it difficult to ensure the accuracy of the initial network model obtained. Summary of the Invention

[0003] This application provides a data processing method, apparatus, computer device, and readable storage medium that can improve the accuracy of the initial network model.

[0004] One embodiment of this application provides a data processing method, including:

[0005] Obtain the candidate network models corresponding to N candidate tasks; N is a positive integer greater than 1.

[0006] The business sample data corresponding to the target task is input into N candidate network models respectively, and the business feature vectors are output by the N candidate network models respectively.

[0007] Based on the business sample data and the business feature vectors output by N candidate network models, the model weights corresponding to the N candidate network models are determined. The model weights corresponding to the N candidate network models are used to characterize the task similarity between the target task and the N candidate tasks.

[0008] Based on the N candidate network models and their respective model weights, an initial network model corresponding to the target task is generated.

[0009] One embodiment of this application provides a data processing apparatus, including:

[0010] The model acquisition module is used to acquire candidate network models corresponding to N candidate tasks; N is a positive integer greater than 1.

[0011] The data input module is used to input the business sample data corresponding to the target task into N candidate network models respectively, and output the business feature vectors through the N candidate network models respectively;

[0012] The weight determination module is used to determine the model weights of the N candidate network models based on the business sample data and the business feature vectors output by the N candidate network models respectively. The model weights of the N candidate network models are used to characterize the task similarity between the target task and the N candidate tasks.

[0013] The model generation module is used to generate an initial network model corresponding to the target task based on N candidate network models and their respective model weights.

[0014] The data input module is specifically used to perform word segmentation on the business sample data corresponding to the target task to obtain the business data word segmentation of the business sample data.

[0015] The data input module is specifically used to obtain the encoded word segmentation vector corresponding to the word segmentation of business data. It extracts features from the encoded word segmentation vector through N candidate network models to obtain the business feature vectors output by the N candidate network models respectively.

[0016] Among them, N candidate network models include candidate network model S p p is a positive integer less than or equal to N;

[0017] The weight determination module includes:

[0018] The first parameter determination unit is used to determine parameters based on business sample data and candidate network models S. p The output business feature vector is used to generate candidate network model S. p The corresponding weight adjustment parameters;

[0019] The weight determination unit is used to determine the weights based on the candidate network model S. p The corresponding weight adjustment parameters and the weight adjustment parameters corresponding to the N candidate network models are used to determine the candidate network model S. p The corresponding model weights.

[0020] The first parameter determination unit is specifically used to perform word segmentation on the business sample tags corresponding to the business sample data to obtain business tag word segmentation of the business sample tags.

[0021] The first parameter determination unit is specifically used to obtain the tag segmentation vector corresponding to the business tag segmentation, and to generate the tag segmentation vector and candidate network model S based on the loss function. p The cross-entropy loss parameter between the output business feature vectors;

[0022] The first parameter determination unit is specifically used to generate candidate network models S based on the cross-entropy loss parameters. p The corresponding weight adjustment parameters.

[0023] Among them, candidate network model S p The number of output business feature vectors is H, where H is a positive integer; the number of tag word segmentation vectors is A, where A is a positive integer;

[0024] The first parameter determination unit is specifically used for determining the candidate network model S. p The output H business feature vectors are normalized respectively to obtain the sample normalized vectors corresponding to the H business feature vectors;

[0025] The first parameter determination unit is specifically used to determine the cross-entropy loss parameter between the tag segmentation vectors and the sample normalization vectors at the same feature position based on the order of A tag segmentation vectors, the order of H sample normalization vectors, and the loss function.

[0026] The first parameter determination unit is specifically used to determine the cross-entropy loss parameter between the sample normalization vectors and the label segmentation vectors at the same feature position if H equals A, based on the order of the H sample normalization vectors, the order of the A label segmentation vectors, and the loss function.

[0027] The first parameter determination unit is specifically used to obtain A sample normalized vectors from H sample normalized vectors if H is greater than A, and to determine the cross-entropy loss parameter between the sample normalized vectors and the label segmentation vectors at the same feature position based on the order of the A sample normalized vectors, the order of the A label segmentation vectors and the loss function.

[0028] The first parameter determination unit is specifically used to obtain H tag segmentation vectors from A tag segmentation vectors if H is less than A, and to determine the cross-entropy loss parameter between the sample normalization vectors and the tag segmentation vectors at the same feature position based on the order of the H sample normalization vectors, the order of the H tag segmentation vectors, and the loss function.

[0029] The first parameter determination unit is specifically used to determine the business sample labels corresponding to the business sample data and the candidate network model S. p The output business feature vector is used to generate candidate network model S. p The candidate model loss during the first backpropagation of parameters;

[0030] The first parameter determination unit is specifically used to determine the candidate network model S. p The candidate model loss during the first backpropagation is used to determine the candidate network model S. p The parameter gradient of the adjustable parameter in the text;

[0031] The first parameter determination unit is specifically used to determine the candidate network model S. p The sum of the gradients of the adjustable parameters in the model is used to determine the candidate network model S. p The corresponding weight adjustment parameters.

[0032] The first parameter determination unit is specifically used to obtain the auxiliary network model corresponding to the target task; the auxiliary network model is obtained by adjusting the parameters of the random initial network model based on the target task.

[0033] The first parameter determination unit is specifically used to input business sample data into the auxiliary network model and output auxiliary feature vectors through the auxiliary network model.

[0034] The first parameter determination unit is specifically used to determine parameters based on auxiliary feature vectors and candidate network models S. p The output business feature vector is used to generate candidate network model S. p The corresponding weight adjustment parameters.

[0035] Among them, candidate network model S p The number of output business feature vectors is H, where H is a positive integer; the number of auxiliary feature vectors is R, where R is a positive integer.

[0036] The first parameter determination unit, specifically, if H equals R, determines the vector divergence between the business feature vectors and auxiliary feature vectors at the same feature position based on the order of the H business feature vectors and the order of the R auxiliary feature vectors. The sum of the vector divergences at each feature position is then used to determine the candidate network model S. p The corresponding weight adjustment parameters;

[0037] The first parameter determination unit is specifically used to: if H is greater than R, obtain R business feature vectors from the H business feature vectors; determine the vector divergence between business feature vectors and auxiliary feature vectors at the same feature position based on the order of the R business feature vectors and the order of the R auxiliary feature vectors; and determine the sum of the vector divergences at each feature position as the candidate network model S. p The corresponding weight adjustment parameters;

[0038] The first parameter determination unit is specifically used to obtain H auxiliary feature vectors from R auxiliary feature vectors if H is less than R. Based on the order of the H business feature vectors and the H auxiliary feature vectors, it determines the vector divergence between the business feature vectors and auxiliary feature vectors at the same feature position. The sum of the vector divergences at each feature position is then used to determine the candidate network model S. p The corresponding weight adjustment parameters.

[0039] The first parameter determination unit is specifically used to perform word segmentation on the business sample tags corresponding to the business sample data to obtain business tag word segmentation of the business sample tags.

[0040] The first parameter determination unit is specifically used to obtain the tag segmentation vector corresponding to the business tag segmentation, and to generate the tag segmentation vector and candidate network model S. p Similarity parameters between the output business feature vectors;

[0041] The first parameter determination unit is specifically used to generate candidate network models S based on similarity parameters. p The corresponding weight adjustment parameters.

[0042] The weight determination unit is specifically used to calculate and process the weight adjustment parameters corresponding to the N candidate network models to obtain the candidate weights corresponding to the N candidate network models.

[0043] The weight determination unit is specifically used to determine the total candidate weight by summing the candidate weights corresponding to the N candidate network models.

[0044] The weight determination unit is specifically used to determine the weights of candidate network models S. p The ratio of the corresponding candidate weights to the total candidate weights is determined as the candidate network model S. p The corresponding model weights.

[0045] Among them, N candidate network models include candidate network model S p p is a positive integer less than or equal to N; candidate network model S p Includes a parameter tuning subnetwork; candidate network model S p The parameter adjustment subnetwork includes adjustable parameters; all N candidate network models include a text processing subnetwork with the same model structure and the same model parameters;

[0046] The model generation module includes:

[0047] The second parameter determination unit is used to determine the candidate network model S. p The parameter information of the adjustable parameters in the parameter adjustment subnetwork and the candidate network model S p The corresponding model weights are used to generate candidate network models S. p The corresponding model adjustment parameters;

[0048] The second parameter determination unit is used to generate the total model adjustment parameters based on the model adjustment parameters corresponding to the N candidate network models respectively.

[0049] Model generation unit, used to generate candidate network models S pThe parameter information of the adjustable parameters in the parameter adjustment subnetwork is adjusted to the total model adjustment parameters. The parameter adjustment subnetwork and the text processing subnetwork after parameter adjustment are determined as the initial network model corresponding to the target task.

[0050] The device also includes:

[0051] The loss determination module is used to input business sample data into the initial network model and output a predicted feature vector through the initial network model.

[0052] The loss determination module is used to obtain the business sample labels corresponding to the business sample data, obtain the label word segmentation vectors corresponding to the business sample labels, and determine the model loss value of the initial network model based on the label word segmentation vectors and the predicted feature vectors.

[0053] The parameter adjustment module is used to adjust the parameters of the initial network model based on the model loss value. When the parameter-adjusted initial network model meets the model convergence condition, the parameter-adjusted initial network model is determined as the target network model. The target network model has the ability to identify business data belonging to the task type of the target task.

[0054] One embodiment of this application provides a computer device, including: a processor and a memory;

[0055] The processor is connected to a memory, which stores a computer program. When the computer program is executed by the processor, it causes the computer device to perform the method provided in the embodiments of this application.

[0056] One aspect of this application provides a computer-readable storage medium storing a computer program adapted to be loaded and executed by a processor, so that a computer device having the processor performs the method provided in this application.

[0057] One embodiment of this application provides a computer program product comprising a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, causing the computer device to perform the method provided in this application embodiment.

[0058] In this embodiment, the computer device can acquire candidate network models corresponding to N candidate tasks, and then input the business sample data corresponding to the target task into the N candidate network models, and output business feature vectors through the N candidate network models. Here, N can be a positive integer greater than 1. Further, the computer device can determine the model weights corresponding to the N candidate network models based on the business sample data and the business feature vectors output by the N candidate network models, and then generate an initial network model corresponding to the target task based on the N candidate network models and their corresponding model weights. The model weights corresponding to the N candidate network models are used to characterize the task similarity between the target task and the N candidate tasks. Therefore, this embodiment, by inputting the business sample data corresponding to the target task into the N candidate network models to determine their corresponding model weights, can fully mine data information and more deeply explore the similarity between the candidate tasks and the target task from the perspective of task data. Therefore, this application proposes an evaluation index (i.e., model weight) to measure the similarity between candidate tasks and target tasks. Based on the model weight, N candidate network models can be weighted. In this way, for different candidate tasks, different candidate network models make different contributions to the initialization of the initial network model. This realizes the method of using the model (i.e., candidate network model) corresponding to the existing task (i.e., candidate task) to initialize the target task, thereby improving the accuracy of the initial network model obtained by initialization. Attached Figure Description

[0059] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0060] Figure 1 This is a schematic diagram of a network architecture provided in an embodiment of this application;

[0061] Figure 2 This is a schematic diagram of a data interaction scenario provided in an embodiment of this application;

[0062] Figure 3 This is a flowchart illustrating a data processing method provided in an embodiment of this application;

[0063] Figure 4 This is a schematic diagram of a scenario for generating an initial network model provided in an embodiment of this application;

[0064] Figure 5 This is a flowchart illustrating a data processing method provided in an embodiment of this application;

[0065] Figure 6 This is a flowchart illustrating a data processing method provided in an embodiment of this application;

[0066] Figure 7 This is a flowchart illustrating a data processing method provided in an embodiment of this application;

[0067] Figure 8 This is a flowchart illustrating a data processing method provided in an embodiment of this application;

[0068] Figure 9 This is a schematic diagram of the structure of a data processing device provided in an embodiment of this application;

[0069] Figure 10 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0070] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0071] It should be understood that Artificial Intelligence (AI) is the theory, methods, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0072] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, as well as machine learning / deep learning, autonomous driving, and intelligent transportation.

[0073] The solutions provided in this application mainly involve artificial intelligence technologies such as Natural Language Processing (NLP) and Machine Learning (ML).

[0074] Natural Language Processing (NLP) is an important area within computer science and artificial intelligence. It studies the theories and methods for enabling effective communication between humans and computers using natural language. NLP is a science that integrates linguistics, computer science, and mathematics. Therefore, research in this field involves natural language—the language people use in daily life—and thus it has a close connection with linguistic research. NLP technologies typically include text processing, semantic understanding, machine translation, question answering, and knowledge graphs.

[0075] Machine learning is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory, among others. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instructional learning.

[0076] For details, please see Figure 1 , Figure 1 This is a schematic diagram of a network architecture provided in an embodiment of this application. Figure 1 As shown, this network architecture may include a server 2000 and a terminal device cluster. The terminal device cluster may specifically include one or more terminal devices; the number of terminal devices in the cluster is not limited here. Figure 1 As shown, the multiple terminal devices may specifically include terminal device 3000a, terminal device 3000b, terminal device 3000c, ..., terminal device 3000n; terminal device 3000a, terminal device 3000b, terminal device 3000c, ..., terminal device 3000n can be directly or indirectly connected to server 2000 via wired or wireless communication, so that each terminal device can interact with server 2000 through the network connection.

[0077] Each terminal device in the terminal device cluster can include: smartphones, tablets, laptops, desktop computers, intelligent voice interaction devices, smart home appliances (e.g., smart TVs), wearable devices, vehicle terminals, aircraft, and other intelligent terminals with data processing capabilities. For ease of understanding, embodiments of this application can... Figure 1 From the multiple terminal devices shown, one terminal device is selected as the target terminal device. For example, in the embodiments of this application, a terminal device can be selected as the target terminal device. Figure 1 The terminal device 3000c shown is used as the target terminal device.

[0078] Among them, Server 2000 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0079] Understandably, pre-trained language models (PLMs) have become the foundational architecture for various NLP tasks, and the scale of these models is increasing. For example, pre-trained language models can be T5 (Text-to-Text Transfer Transformer) models, BERT (Bidirectional Encoder Representations from Transformers) models, and so on. Larger models not only achieve better results on known tasks but also demonstrate the potential to accomplish more complex, unknown tasks. However, larger models also face greater challenges in application. Traditional methods for fine-tuning all parameters of ultra-large-scale pre-trained language models consume significant amounts of GPU (Graphics Processing Unit) computing and storage resources, making the process prohibitively expensive. To address this issue, numerous efficient methods for fine-tuning pre-trained models with few parameters have been proposed, such as Adapter Tuning (Parameter-Efficient Transfer Learning for NLP), LoRA (Lo-Rank Adaptation of Large Language Models), and Prefix Tuning (Prefix-Tuning: Optimizing Continuous Prompts for Generation). Compared to full-parameter fine-tuning methods, these methods significantly reduce the number of parameters to be fine-tuned while ensuring that the performance on downstream tasks is not significantly reduced.

[0080] Understandably, the aforementioned network framework can be applied to task transfer and efficient few-parameter fine-tuning in natural language processing. Task transfer refers to transferring the parameters of a pre-trained model to a new model to aid in its training; the new and old models can be applied to different tasks. Fine-tuning refers to adjusting the parameters of a trained model. Efficient few-parameter fine-tuning (i.e., efficient few-parameter pre-trained model fine-tuning methods) involves inserting small-scale neural modules or adjustable parameters into the model and achieving efficient model adaptation by fine-tuning only this small subset of parameters.

[0081] It should be understood that the data processing method provided in this application embodiment can be executed by a computer device. This computer device can acquire candidate network models corresponding to N candidate tasks, and based on the business sample data corresponding to the target task and the N candidate network models, determine the model weights corresponding to the N candidate network models. Each candidate task corresponds to one candidate network model, and each candidate network model corresponds to one model weight. The model weights can be used to characterize the task similarity between the target task and the candidate tasks. Furthermore, the computer device can generate an initial network model corresponding to the target task based on the N candidate network models and their corresponding model weights. The initial network model can represent the network model obtained by initializing for the target task. Based on the target task, the initial network model can be trained to obtain the target network model.

[0082] The data processing method provided in this application embodiment can be executed by server 2000 (i.e., the computer device mentioned above can be server 2000), or by target terminal device (i.e., the computer device mentioned above can be target terminal device), or by server 2000 and target terminal device jointly.

[0083] It is understandable that the candidate network model corresponding to the candidate task is a model fine-tuned using the full dataset. When the target task has a small amount of data, the computer device can initialize the model for the target task using the model corresponding to the existing task (i.e., the candidate task) (i.e., the candidate network model). The initialized model is the initial network model, which is then trained using the target task. Here, N candidate tasks can be understood as tasks with a large amount of data, and the target task as a task with a small amount of data. When training a random initial network model with a large amount of data, the accuracy of the generated candidate network model can be ensured. However, when training a random initial network model with a small amount of data, it is difficult to ensure the accuracy of the trained model. Therefore, in this embodiment, an initial network model can be obtained by first initializing based on N candidate network models, and then trained using a small amount of data, thereby improving the accuracy of the target network model while improving the accuracy of the initial network model.

[0084] It should be understood that the N candidate tasks and the target task in the embodiments of this application can be the same task or different tasks. For ease of understanding, the embodiments of this application will be described using the example of N candidate tasks and the target task being different tasks. Both the target task and the N candidate tasks are question-answering tasks. The target task can also be called a target question-answering task, and the candidate tasks can also be called candidate question-answering tasks. In other words, the target task and the N candidate tasks can be question-answering tasks belonging to different domains or question-answering tasks belonging to the same domain. For the target task, the business sample data can be the questions in the question-answering task, and the business sample tags can be the answers in the question-answering task. Similarly, the N candidate tasks are also composed of questions and answers, where a question and an answer can constitute a set of question-answering data. For example, the target task can be a common sense question-answering task (i.e., the task type of the target task is common sense), the business sample data can be "what is the speed of light in kilometers per second", and the business sample tag can be "299792.458 kilometers per second".

[0085] For better understanding, please refer to [link / reference]. Figure 2 , Figure 2 This is a schematic diagram illustrating a data interaction scenario provided in an embodiment of this application. For example... Figure 2 The server 20a shown can be the one described above. Figure 1 The server 2000 in the corresponding embodiment, such as Figure 2 The terminal device 20b shown can be the one described above. Figure 1 The target terminal device in the corresponding embodiment, such as Figure 2 The terminal device 20c shown can be the one described above. Figure 1 Any terminal device in the corresponding embodiment, such as Figure 2 The terminal device 20d shown can be the above Figure 1 Any of the terminal devices in the corresponding embodiments. For ease of understanding, this application embodiment uses the data processing method executed by terminal device 20b as an example for illustration. Figure 1 The terminal device 3000a in the corresponding embodiment is terminal device 20c. Figure 1 The terminal device 3000n in the corresponding embodiment is used as an example of terminal device 20d for explanation.

[0086] like Figure 2As shown, terminal device 20c can adjust the parameters of a random initial network model (not shown in the figure) based on candidate task 22a to obtain candidate network model 21a, and then send candidate network model 21a to server 20a, so that server 20a can send candidate network model 21a to terminal device 20b. Similarly, terminal device 20d can adjust the parameters of a random initial network model (not shown in the figure) based on candidate task 22b to obtain candidate network model 21b, and then send candidate network model 21b to server 20a, so that server 20a can send candidate network model 21b to terminal device 20b.

[0087] It is understood that the embodiments of this application assume that the number of candidate network models 21a, ..., candidate network models 21b is N, where N can be a positive integer greater than 1. In addition, the embodiments of this application assume that candidate network models 21a, ..., candidate network models 21b are trained by different terminal devices. Optionally, candidate network models 21a, ..., candidate network models 21b can also be trained by the same terminal device. This application does not limit this.

[0088] like Figure 2 As shown, after receiving N candidate network models (i.e., candidate network models 21a, ..., candidate network model 21b), the terminal device 20b can input the service sample data corresponding to the target task into each of the N candidate network models, and output service feature vectors through each of the N candidate network models. Specifically, the service feature vector output by candidate network model 21a can be service feature vector 23a, and the service feature vector output by candidate network model 21b can be service feature vector 23b.

[0089] Furthermore, such as Figure 2 As shown, terminal device 20b can determine the model weights corresponding to N candidate network models based on business sample data, business feature vectors 23a, ..., business feature vectors 23b. Specifically, the model weight corresponding to candidate network model 21a can be model weight 24a, and the model weight corresponding to candidate network model 21b can be model weight 24b. Model weight 24a is used to characterize the task similarity between the target task and candidate task 22a, and model weight 24b is used to characterize the task similarity between the target task and candidate task 22b.

[0090] Furthermore, such as Figure 2As shown, terminal device 20b can generate an initial network model corresponding to the target task based on N candidate network models and their corresponding model weights (i.e., model weights 24a, ..., model weights 24b). Then, based on the target task, the parameters of the initial network model are adjusted to obtain the target network model. The initial network model is the model initialized for the target task, and the target network model is the model obtained by training the initial network model. The target network model has the ability to identify business data belonging to the task type of the target task.

[0091] Optionally, if the data processing method provided in this application embodiment is executed by server 20a, then as follows Figure 2 After receiving N candidate network models, the server 20a can generate a target network model corresponding to the target task based on the business sample data and the N candidate network models, and then send the target network model to the terminal device 20b so that the terminal device 20b can identify the business data belonging to the task type of the target task through the target network model.

[0092] Therefore, the embodiments of this application can determine the task similarity between the target task and N candidate tasks, and then weight the N candidate network models based on the model weights associated with the task similarity to obtain an initial network model. This allows for model initialization from the perspective of task data; the degree of difference between tasks is reflected in the model weights, and the model parameters in different candidate network models can be fully utilized based on the model weights, thereby improving the accuracy of the initial network model.

[0093] Further, please see Figure 3 , Figure 3 This is a flowchart illustrating a data processing method provided in an embodiment of this application. The method can be executed by a server, a terminal device, or both. The server can be one of the aforementioned... Figure 2 The server 20a in the corresponding embodiment can be the terminal device described above. Figure 2 The corresponding embodiment is terminal device 20b. For ease of understanding, this application embodiment is described using the method executed by the terminal device as an example. The data processing method may include the following steps S101-S104:

[0094] Step S101: Obtain the candidate network models corresponding to the N candidate tasks;

[0095] Here, N can be a positive integer greater than 1. The candidate network model is obtained by adjusting the parameters of a randomly initialized network model based on candidate tasks. N candidate network models are obtained by adjusting the parameters of a randomly initialized network model using N candidate tasks respectively. One candidate task can be used to generate one candidate network model. One candidate task can include multiple sets of data; that is, one candidate question-answering task can include multiple sets of question-answering data. This application does not limit the number of question-answering data sets in each candidate task. The N candidate network models include candidate network model S. p Here, p can be a positive integer less than or equal to N.

[0096] Optionally, N can also be a positive integer. If N equals 1, the terminal device can directly determine the candidate network model as the initial network model corresponding to the target task, and the model weight corresponding to the candidate network model is equal to 1.

[0097] In this context, a stochastically initialized network model refers to an initialized model generated using an efficient few-parameter pre-trained model fine-tuning method. This model can include two sub-networks: a text processing sub-network and an initial parameter tuning sub-network. The model parameters in the text processing sub-network are determined by the model parameters in the pre-trained language model, while the model parameters in the initial parameter tuning sub-network are obtained through random initialization. The model structure of the text processing sub-network is determined by the model structure of the pre-trained language model, and the model structure of the initial parameter tuning sub-network is determined by the few-parameter pre-trained model fine-tuning method. The stochastically initialized network models corresponding to the N candidate network models can be the same or different.

[0098] Step S102: Input the business sample data corresponding to the target task into N candidate network models respectively, and output the business feature vectors through N candidate network models respectively;

[0099] Specifically, the terminal device can perform word segmentation on the business sample data corresponding to the target task to obtain business data word segments. Further, the terminal device can obtain the encoded word segmentation vectors corresponding to the business data word segments, and extract features from the encoded word segmentation vectors using N candidate network models respectively, to obtain business feature vectors output by the N candidate network models. It should be understood that this application embodiment does not limit the model structure of the N candidate network models; therefore, this application embodiment does not limit the specific process of feature extraction.

[0100] It can be understood that the word segmentation process is to recombine a continuous sequence of characters (i.e., business sample data) into a sequence of words (i.e., business data word segmentation) according to certain specifications. For example, when the business sample data is: "What is the height of Mount Qomolangma", the business data word segmentation obtained by performing word segmentation on the business sample data can be: "Zhu", "Mu", "Lang", "Ma", "Feng", "de", "gao", "du", "shi", "duo", "shao", "mi", or the business data word segmentation obtained by performing word segmentation on the business sample data can also be: "Mount Qomolangma", "de", "height", "shi", "duoshao", "mi".

[0101] Optionally, the terminal device can perform word segmentation conversion on the business data word segmentation through a word embedding algorithm (for example, Word2Vec (Word to Vector), Glove, etc.) to obtain a word segmentation vector corresponding to the business data word segmentation; the terminal device can perform position encoding on the text position of the business data word segmentation in the business sample data through a position formula to obtain a position vector corresponding to the business data word segmentation. Further, the terminal device can perform feature fusion on the word segmentation vector and the position vector to obtain an encoded word segmentation vector corresponding to the business data word segmentation. Optionally, the terminal device can also obtain a segmentation vector corresponding to the business data word segmentation, and then perform feature fusion on the word segmentation vector, the position vector, and the segmentation vector to obtain an encoded word segmentation vector corresponding to the business data word segmentation. It should be understood that the way of feature fusion can be the way of vector addition or the way of weighted average, and the embodiments of this application do not limit the way of feature fusion.

[0102] It should be understood that the number of input vectors of N candidate network models is the same, and the number of output vectors of N candidate network models is the same. Here, candidate network model S p is taken as an example for illustration. Candidate network model S p has D1 input vectors, and candidate network model S p has D2 output vectors. Here, D1 can be a positive integer, and here D2 can be a positive integer. For example, D1 can be equal to 512, and D2 can be equal to 64. Among them, the input vectors of candidate network model S p can include encoded word segmentation vectors, and the output vectors of candidate network model S p can include business feature vectors.

[0103] Among them, it can be understood that when the number of encoded word segmentation vectors is less than D1, the encoded word segmentation vectors are supplemented so that the number of input vectors of candidate network model S p is equal to D1; similarly, when the number of business feature vectors is less than D2, the business feature vectors are supplemented so that the number of output vectors of candidate network model S pThe number of output vectors is equal to D2. The portion that supplements the business feature vector or encoded word segmentation vector can be called the pad part (including the input pad part and the output pad part). Therefore, in this embodiment, the encoded word segmentation vector represents the effective input of the candidate network model, not the actual input; the actual input includes the encoded word segmentation vector and the input pad part. Similarly, in this embodiment, the business feature vector represents the effective output of the candidate network model, not the actual output; the actual output includes the business feature vector and the output pad part.

[0104] For ease of understanding, in this application embodiment, the segmented business data can be referred to as a token. A token can consist of a single word (or character) or multiple words (or characters), and this application does not limit this. Therefore, the business feature vector can be used to represent the output of the candidate network model. For each valid token in the output (i.e., the token outside the pad part), there is a corresponding business feature vector, which can be used to represent the probability vector of the valid token. Here, a valid token can represent the segmented business data, and an invalid token can represent the token in the pad part (i.e., the token corresponding to the vector in the pad part).

[0105] Step S103: Based on the business sample data and the business feature vectors output by the N candidate network models, determine the model weights corresponding to the N candidate network models respectively.

[0106] Specifically, terminal devices can base their operations on service sample data and candidate network models S. p The output business feature vector is used to generate candidate network model S. p The corresponding weight adjustment parameters. Furthermore, the terminal device can adjust the weights based on the candidate network model S. p The corresponding weight adjustment parameters and the weight adjustment parameters corresponding to the N candidate network models are used to determine the candidate network model S. p The corresponding model weights. In other words, the terminal device can determine the model weights of the N candidate network models by adjusting the parameters corresponding to the weights of each candidate network model. For example, the weight adjustment parameters can be determined by the cross-entropy loss parameters. The model weights corresponding to the N candidate network models are used to characterize the task similarity between the target task and the N candidate tasks; the candidate network models S... p The corresponding model weights are used to represent the target task and candidate tasks O. p (i.e., candidate network model S) p The task similarity between corresponding candidate tasks.

[0107] It should be understood that the terminal device, based on the candidate network model S pThe corresponding weight adjustment parameters and the weight adjustment parameters corresponding to the N candidate network models are used to determine the candidate network model S. p The specific process for assigning model weights can be described as follows: The terminal device can perform calculations on the weight adjustment parameters corresponding to N candidate network models to obtain the candidate weights corresponding to each of the N candidate network models. Here, "calculation" refers to inverting the weight adjustment parameters and dividing by the difference parameter; this embodiment does not limit the specific process of the calculation. Further, the terminal device can determine the total candidate weights by summing the candidate weights corresponding to the N candidate network models. Further, the terminal device can assign the candidate network models S... p The ratio of the corresponding candidate weights to the total candidate weights is determined as the candidate network model S. p The corresponding model weights. In other words, in this embodiment of the application, the result of processing the candidate weights with softmax (i.e., normalized exponential function) can be used as the model weights.

[0108] Among them, the terminal device is based on the candidate network model S p The corresponding weight adjustment parameters and the weight adjustment parameters corresponding to the N candidate network models are used to determine the candidate network model S. p The specific process for determining the corresponding model weights can be found in formula (1):

[0109]

[0110] in, This can represent the candidate weights corresponding to N candidate network models. It can represent the total candidate weight. Can be used to represent candidate network model S p The corresponding candidate weights, weight p Can be used to represent candidate network model S p The corresponding model weights, Z j Z can represent the weight adjustment parameters corresponding to N candidate network models. p Can be used to represent candidate network model S p The corresponding weight adjustment parameter, j, can be any positive integer from 1 to n. n (i.e., N) is the total number of existing tasks (i.e., candidate tasks), and tem (i.e., the difference parameter) can represent a hyperparameter. tem is used to regulate the absolute difference in similarity between different existing tasks and the target task. This application's embodiments do not... n The specific value of tem is limited.

[0111] When the number of business sample data is one, the terminal device can determine the candidate network model S based on the above process. pThe corresponding weight adjustment parameters. Optionally, when there are at least two business sample data points, the terminal device can determine the candidate network model S based on the weight adjustment parameters corresponding to at least two business sample data points respectively. p The corresponding weight adjustment parameters. For example, the terminal device can determine the candidate network model S by summing the weight adjustment parameters corresponding to at least two service sample data. p The corresponding weight adjustment parameters. For example, a terminal device can determine the candidate network model S by taking the average of the weight adjustment parameters corresponding to at least two service sample data points. p The corresponding weight adjustment parameters.

[0112] Among them, candidate network model S p Includes a parameter tuning subnetwork; candidate network model S p The parameter tuning subnetwork includes adjustable parameters; all N candidate network models include a text processing subnetwork with the same model structure and parameters, and each of the N candidate network models includes a parameter tuning subnetwork with the same specific model structure. The model parameters of the parameter tuning subnetworks in the N candidate network models are different; optionally, the model parameters of the parameter tuning subnetworks in the N candidate network models can also be the same. The parameter tuning subnetwork can be a small-scale inserted neural module or adjustable parameters. Optionally, the candidate network model S... p The parameter-adjustable subnetwork can also include some non-adjustable parameters, candidate network model S p The text processing subnetwork includes non-adjustable parameters.

[0113] Step S104: Generate the initial network model corresponding to the target task based on the N candidate network models and their respective model weights.

[0114] Specifically, the terminal device can use the candidate network model S p The parameter information of the adjustable parameters in the parameter adjustment subnetwork and the candidate network model S p The corresponding model weights are used to generate candidate network models S. p The corresponding model adjustment parameters. Specifically, the terminal device can adjust the candidate network model S. p The parameter information of the adjustable parameters in the parameter adjustment subnetwork and the candidate network model S p The product of the corresponding model weights is used to determine the candidate network model S. p The corresponding model adjustment parameters. Further, the terminal device can generate the total model adjustment parameters based on the model adjustment parameters corresponding to each of the N candidate network models. Specifically, the terminal device can determine the total model adjustment parameters by summing the model adjustment parameters corresponding to each of the N candidate network models. Further, the terminal device can...p The adjustable parameters in the parameter adjustment subnetwork of the target network are adjusted to become the total model adjustment parameters. The adjusted parameter adjustment subnetwork and the text processing subnetwork are then used as the initial network model for the target task. In other words, the terminal device can obtain any one candidate network model from N candidate network models, adjust the adjustable parameters in the parameter adjustment subnetwork of the obtained candidate network model to become the total model adjustment parameters, and then use the adjusted parameter adjustment subnetwork and the text processing subnetwork as the initial network model for the target task.

[0115] It is understood that the model weights corresponding to the N candidate network models can be the same or different. When the weights of the N models are all the same, the embodiments of this application can be understood as averaging the N candidate network models to obtain the initial network model; optionally, when the weights of the N models are different, the embodiments of this application can be understood as weighted summing of the N candidate network models using the model weights to obtain the initial network model.

[0116] Among them, candidate network model S p The number of adjustable parameters in the parameter adjustment subnetwork is at least one, and the terminal device can adjust the parameters according to the candidate network model S. p Parameter information of at least one adjustable parameter in the parameter adjustment subnetwork and candidate network model S p The corresponding model weights are used to generate candidate network models S. p At least one corresponding model adjustment parameter is generated, which in turn generates at least one total model adjustment parameter. The parameter information can represent the value of the adjustable parameter. For example, the adjustable parameter can be 'a', and the parameter information for adjustable parameter 'a' can be 1, meaning 'a' equals 1.

[0117] For easier understanding, please refer to Figure 4 , Figure 4 This is a schematic diagram illustrating a scenario for generating an initial network model, as provided in an embodiment of this application. Figure 4 As shown, the N candidate network models may specifically include candidate network model 40a and candidate network model 40b. Candidate network model 40a may include parameter adjustment subnetwork 41a and text processing subnetwork 41b. Candidate network model 40b may include parameter adjustment subnetwork 42a and text processing subnetwork 42b. The model structure and model parameters of text processing subnetwork 41b and text processing subnetwork 42b are the same.

[0118] For ease of understanding, this application embodiment uses an example where the number of adjustable parameters in parameter adjustment subnetworks 41a and 42a is two, and an example where the number of non-adjustable parameters in text processing subnetworks 41b and 42b is one. It should be understood that this application embodiment does not limit the number of adjustable parameters in the parameter adjustment subnetworks, nor does it limit the number of non-adjustable parameters in the text processing subnetworks.

[0119] like Figure 4 As shown, parameter adjustment subnetworks 41a and 42a may include adjustable parameters X1 and X2, while text processing subnetworks 41b and 42b may include non-adjustable parameter X3. In parameter adjustment subnetwork 41a, the parameter information for adjustable parameter X1 is parameter information 43a, and the parameter information for adjustable parameter X2 is parameter information 43d; in parameter adjustment subnetwork 42a, the parameter information for adjustable parameter X1 is parameter information 44a, and the parameter information for adjustable parameter X2 is parameter information 44d.

[0120] like Figure 4 As shown, the terminal device can generate the model adjustment parameter 43c corresponding to the adjustable parameter X1 based on parameter information 43a and model weights 43b (i.e., the model weights corresponding to candidate network model 40a); the terminal device can also generate the model adjustment parameter 44c corresponding to the adjustable parameter X1 based on parameter information 44a and model weights 44b (i.e., the model weights corresponding to candidate network model 40b). Furthermore, the terminal device can generate the total model adjustment parameter 45a corresponding to the adjustable parameter X1 based on the model adjustment parameter 43c and model adjustment parameter 44c. Similarly, the terminal device can generate the total model adjustment parameter 45b corresponding to the adjustable parameter X2.

[0121] It should be understood that after generating the initial network model corresponding to the target task, the terminal device can input business sample data into the initial network model and output a predicted feature vector. Further, the terminal device can obtain the business sample labels corresponding to the business sample data, obtain the label segmentation vectors corresponding to the business sample labels, and determine the model loss value of the initial network model based on the label segmentation vectors and the predicted feature vectors. Further, the terminal device can adjust the parameters of the initial network model based on the model loss value. When the parameter-adjusted initial network model meets the model convergence condition, the parameter-adjusted initial network model is determined as the target network model. The target network model has the ability to identify business data belonging to the task type of the target task.

[0122] In this context, it can be understood that business data belonging to the task type of the target task can be considered sample business data. Both sample business data and business sample data are business data of the task type of the target task. When sample business data is input into the target network model, the target network model can output the corresponding business sample result. Here, the sample business data can be the answer to the question in the task, and the business sample result can be the answer to the question in the task. For example, the sample business data can be "What is the speed of light in kilometers per second?", and the business sample result can be "299792.458 kilometers per second".

[0123] For example, taking N equal to 3 as an example, the task types of the N candidate tasks can be historical, health, and traffic types, while the task type of the target task can be common sense. In this case, this embodiment can generate an initial network model corresponding to the common sense type based on the business data (i.e., business sample data), the candidate network models corresponding to the historical type, the health type, and the traffic type. Then, based on the common sense type business data (i.e., business sample data), the initial network model is further trained to obtain the target network model. This enables the target network model to recognize common sense type business data, and by using the candidate network models corresponding to the historical type, the health type, and the traffic type, the accuracy of the initial network model is improved, thereby improving the accuracy of the target network model. In other words, this embodiment can obtain other common sense type business data besides business sample data (e.g., the aforementioned sample business data), input the sample business data into the target network model, and accurately identify the business sample results corresponding to the sample business data (i.e., accurately identify the answers to common sense type questions through the target network model).

[0124] Therefore, this application embodiment, by inputting the business sample data corresponding to the target task into N candidate network models to determine the model weights of the N candidate network models, can fully mine data information and more deeply explore the similarity between candidate tasks and target tasks from the perspective of task data. Thus, this application embodiment proposes an evaluation index (i.e., model weight) to measure the similarity between candidate tasks and target tasks. Based on the model weights, the N candidate network models can be weighted. In this way, for different candidate tasks, different candidate network models make different contributions to the initialization of the initial network model, thereby realizing a method of initializing the target task using the model (i.e., candidate network model) corresponding to the existing task (i.e., candidate task), thereby improving the accuracy of the initial network model obtained through initialization.

[0125] Further, please see Figure 5 , Figure 5 This is a flowchart illustrating a data processing method provided in an embodiment of this application. The data processing method may include the following steps S201-S204, and steps S201-S204 are... Figure 3 This is a specific embodiment of step S103 in the corresponding embodiment. The N candidate network models include candidate network model S. p Here, p can be a positive integer less than or equal to N.

[0126] Step S201: Perform word segmentation on the business sample tags corresponding to the business sample data to obtain the business tag word segments of the business sample tags;

[0127] The specific process of the terminal device performing word segmentation on the business sample tags can be found in the above. Figure 3 The description of word segmentation processing of business sample data in the corresponding embodiments will not be repeated here.

[0128] For ease of understanding, in this application embodiment, business data segmentation and business tag segmentation can be collectively referred to as tokens. A token can consist of a single word (or character) or multiple words (or characters), and this application does not limit this. For example, a token can be "how much" or "I like". Another example is that a token can be "more", "few", "I", or "like".

[0129] Step S202: Obtain the tag segmentation vectors corresponding to the business tag segmentation, and generate the tag segmentation vectors and candidate network model S based on the loss function. p The cross-entropy loss parameter between the output business feature vectors;

[0130] Specifically, the terminal device can obtain the tag segmentation vectors corresponding to the business tag segmentation. Among them, the candidate network model S... p The number of output business feature vectors is H, where H can be a positive integer; the number of tag segmentation vectors is A, where A can be a positive integer. The number of business feature vectors output by the N candidate network models can be the same or different. Furthermore, the terminal device can evaluate the candidate network models S... p The output H business feature vectors are each normalized to obtain the sample normalized vectors corresponding to the H business feature vectors. Among them, the candidate network model S... pThis can include a normalization network layer. The terminal device can input H service feature vectors into the normalization network layer, which will then normalize each of the H service feature vectors, outputting sample normalized vectors corresponding to the H service feature vectors. Furthermore, the terminal device can determine the cross-entropy loss parameter between the label segmentation vectors and sample normalized vectors at the same feature position based on the order of A label segmentation vectors, the order of H sample normalized vectors, and a loss function. The loss function (e.g., a multi-class cross-entropy loss function) represents the degree of difference between the predicted value (i.e., the sample normalized vector) and the actual value (i.e., the label segmentation vector). The smaller the loss value corresponding to the loss function (e.g., the cross-entropy loss parameter), the better the model.

[0131] Where H equals A, the terminal device can determine the cross-entropy loss parameter between the sample normalized vectors and the label segmentation vectors at the same feature position based on the order of the H sample normalized vectors, the order of the A label segmentation vectors, and the loss function. Optionally, if H is greater than A, the terminal device can obtain A sample normalized vectors from the H sample normalized vectors, and determine the cross-entropy loss parameter between the sample normalized vectors and the label segmentation vectors at the same feature position based on the order of the A sample normalized vectors, the order of the A label segmentation vectors, and the loss function. Optionally, if H is less than A, the terminal device can obtain H label segmentation vectors from the A label segmentation vectors, and determine the cross-entropy loss parameter between the sample normalized vectors and the label segmentation vectors at the same feature position based on the order of the H sample normalized vectors, the order of the H label segmentation vectors, and the loss function.

[0132] The tag segmentation vector is a vector composed of indicator variables. The vector dimension of the tag segmentation vector is equal to the number of sample classification label information. When the business tag segmentation is the same as the sample classification label information, the indicator variable can be 1; when the business tag segmentation is different from the sample classification label information, the indicator variable can be 0. The sample classification label information can be understood as words in the dictionary. Furthermore, the tag segmentation vector can also be called a one-hot vector. For example, if the sample classification label information is (car, apple, sky), and the business tag segmentation is "sky," then the tag segmentation vector can be represented as (0,0,1), or if the business tag segmentation is "car," then the tag segmentation vector can be represented as (1,0,0).

[0133] The sample normalization vector is a vector composed of predicted values. The vector dimension of the sample normalization vector equals the number of sample classification labels. The sample normalization vector represents the probability that a predicted value belongs to a sample classification label. One sample normalization vector corresponds to one predicted word segmentation information, and the predicted value in the sample normalization vector represents the probability that the predicted word segmentation information belongs to the sample classification label. For example, if the sample classification labels are (car, apple, sky), the corresponding sample normalization vector for the predicted word segmentation information could be (0.7, 0.2, 0.1), which means that the probability of the predicted word segmentation information belonging to "car" is 0.7, the probability of belonging to "apple" is 0.2, and the probability of belonging to "sky" is 0.1.

[0134] Step S203: Generate candidate network model S based on the cross-entropy loss parameter. p The corresponding weight adjustment parameters;

[0135] Specifically, the terminal device can determine the candidate network model S by summing the cross-entropy loss parameters between the normalized vectors of samples at the same feature positions and the tag segmentation vectors. p The corresponding weight adjustment parameters. Where H equals A, the terminal device can determine the candidate network model S by the sum of the cross-entropy loss parameters between the sample normalization vectors and the label segmentation vectors at H (or A) feature positions. p The corresponding weight adjustment parameters. Optionally, if H is greater than A, the terminal device can determine the candidate network model S by the sum of the cross-entropy loss parameters between the sample normalization vectors and the label segmentation vectors at A feature positions. p The corresponding weight adjustment parameters. Optionally, if H is less than A, the terminal device can determine the candidate network model S by the sum of the cross-entropy loss parameters between the sample normalization vectors and the label segmentation vectors at the H feature positions. p The corresponding weight adjustment parameters.

[0136] Step S204, based on the candidate network model S p The corresponding weight adjustment parameters and the weight adjustment parameters corresponding to the N candidate network models are used to determine the candidate network model S. p The corresponding model weights.

[0137] Among them, the terminal device is based on the candidate network model S p The corresponding weight adjustment parameters and the weight adjustment parameters corresponding to the N candidate network models are used to determine the candidate network model S. p For details on the specific process of applying the corresponding model weights, please refer to the above. Figure 3 The description of step S103 in the corresponding embodiments will not be repeated here.

[0138] After determining the model weights corresponding to the N candidate network models, the terminal device can generate the initial network model corresponding to the target task based on the N candidate network models and their corresponding model weights. The specific process of the terminal device generating the initial network model corresponding to the target task based on the N candidate network models and their corresponding model weights can be found in formula (2):

[0139]

[0140] Where L can be Z in the above formula (1), and L can represent the weight adjustment parameter determined according to the cross-entropy loss parameter. Can be used to represent candidate network model S i The corresponding model weights, delta_model i Can be used to represent candidate network model S i delta_model can represent the initial network model, where i can be any positive integer from 1 to n.

[0141] It should be understood that the greater the difference between the candidate task and the target task (i.e., the smaller the task similarity), the larger the cross-entropy loss parameter and the larger the weight adjustment parameter. Therefore, after taking the negative of the weight adjustment parameter, the candidate network model corresponding to the candidate task with the smaller weight adjustment parameter (i.e., the smaller the difference between the candidate task and the target task) will have larger model weights.

[0142] Therefore, this embodiment of the application can determine the cross-entropy loss parameter between the label segmentation vector and the sample normalization vector at the same feature position, generate the weight adjustment parameter corresponding to the candidate network model based on the cross-entropy loss parameter, and then determine the model weight corresponding to the candidate network model based on the weight adjustment parameter. Thus, this embodiment of the application can measure the task similarity between the target task and the candidate task based on the model weight corresponding to the candidate network model, thereby improving the accuracy of the initial network model when weighting the candidate network model based on the model weight.

[0143] Further, please see Figure 6 , Figure 6 This is a flowchart illustrating a data processing method provided in an embodiment of this application. The data processing method may include the following steps S301-S304, and steps S301-S304 are... Figure 3 This is a specific embodiment of step S103 in the corresponding embodiment. The N candidate network models include candidate network model S. p Here, p can be a positive integer less than or equal to N.

[0144] Step S301, based on the business sample labels corresponding to the business sample data and the candidate network model S p The output business feature vector is used to generate candidate network model S. p The candidate model loss during the first backpropagation of parameters;

[0145] Specifically, the terminal device can be based on the above. Figure 5 In the corresponding embodiment, steps S201-S202 generate cross-entropy loss parameters between the sample normalization vector and the label segmentation vector at the same feature position, and then the sum of the cross-entropy loss parameters is determined as the candidate network model S. p The candidate model loss during the initial backpropagation. It should be understood that the embodiments of this application do not limit the specific type of loss function used to determine the candidate model loss.

[0146] Step S302, based on the candidate network model S p The candidate model loss during the first backpropagation is used to determine the candidate network model S. p The parameter gradient of the adjustable parameter in the text;

[0147] The first backpropagation of parameters represents the process for candidate network model S. p The model loss during the first parameter tuning, candidate network model S p The parameter gradient representation of the adjustable parameters in the candidate network model S p The gradient during the first parameter adjustment. In other words, the gradient of the candidate network model S. p The parameter gradient in the adjustable parameter represents the gradient of each part of the model of the efficient few-parameter fine-tuning method with respect to the final loss after the first backpropagation.

[0148] Step S303, select candidate network model S p The sum of the gradients of the adjustable parameters in the model is used to determine the candidate network model S. p The corresponding weight adjustment parameters;

[0149] Understandably, candidate network model S p The number of adjustable parameters in the candidate network model S is at least one. p Includes parameter tuning subnetwork, candidate network model S p The adjustable parameters in the candidate network model S belong to p The parameter adjustment subnetwork. Where, if the candidate network model S... p If the number of adjustable parameters is one, the terminal device can determine the gradient of that adjustable parameter as the candidate network model S. p The corresponding weight adjustment parameters; optional, if the candidate network model S pIf there are at least two adjustable parameters, the terminal device can determine the candidate network model S by the sum of the gradients of the at least two adjustable parameters. p The corresponding weight adjustment parameters.

[0150] Step S304, based on the candidate network model S p The corresponding weight adjustment parameters and the weight adjustment parameters corresponding to the N candidate network models are used to determine the candidate network model S. p The corresponding model weights.

[0151] Among them, the terminal device is based on the candidate network model S p The corresponding weight adjustment parameters and the weight adjustment parameters corresponding to the N candidate network models are used to determine the candidate network model S. p For details on the specific process of applying the corresponding model weights, please refer to the above. Figure 3 The description of step S103 in the corresponding embodiments will not be repeated here.

[0152] After determining the model weights corresponding to the N candidate network models, the terminal device can generate the initial network model corresponding to the target task based on the N candidate network models and their corresponding model weights. The specific process of the terminal device generating the initial network model corresponding to the target task based on the N candidate network models and their corresponding model weights can be found in formula (3):

[0153]

[0154] Where G can be Z in the above formula (1), and G can represent the weight adjustment parameter determined according to the parameter gradient. Can be used to represent candidate network model S i The corresponding model weights, delta_model i Can be used to represent candidate network model S i delta_model can represent the initial network model, where i can be any positive integer from 1 to n.

[0155] It should be understood that a larger parameter gradient indicates that the candidate network model makes greater changes to adapt to the target task, and a larger weight adjustment parameter, meaning a greater difference between the candidate task and the target task (i.e., a smaller task similarity). Therefore, after taking the negative of the weight adjustment parameter, the candidate network model corresponding to the candidate task with a smaller weight adjustment parameter (i.e., a smaller difference between the candidate task and the target task) has a larger model weight.

[0156] Therefore, the embodiments of this application can determine the parameter gradient of the adjustable parameters, generate weight adjustment parameters corresponding to the candidate network model based on the parameter gradient, and then determine the model weights corresponding to the candidate network model based on the weight adjustment parameters corresponding to the candidate network model. Thus, the embodiments of this application can measure the task similarity between the target task and the candidate task based on the model weights corresponding to the candidate network model, thereby improving the accuracy of the initial network model when weighting the candidate network model based on the model weights.

[0157] Further, please see Figure 7 , Figure 7 This is a flowchart illustrating a data processing method provided in an embodiment of this application. The data processing method may include the following steps S401-S404, and steps S401-S404 are... Figure 3 This is a specific embodiment of step S103 in the corresponding embodiment. The N candidate network models include candidate network model S. p Here, p can be a positive integer less than or equal to N.

[0158] Step S401: Obtain the auxiliary network model corresponding to the target task;

[0159] Specifically, the terminal device can input the business sample data corresponding to the target task into the random initialization network model, and the random initialization network model outputs a random predicted feature vector. Further, the terminal device can obtain the business sample labels corresponding to the business sample data, obtain the label segmentation vectors corresponding to the business sample labels, and determine the random model loss value of the random initialization network model based on the label segmentation vectors and the random predicted feature vectors. Further, the terminal device can adjust the parameters of the random initialization network model based on the random model loss value. When the parameter-adjusted random initialization network model meets the model convergence condition, the parameter-adjusted random initialization network model is determined as the auxiliary network model. In other words, the auxiliary network model is obtained by adjusting the parameters of the random initialization network model based on the target task; that is, the auxiliary network model is a model obtained by fine-tuning the target task using the data of the target task on a randomly initialized model (i.e., the random initialization network model), which is an efficient, low-parameter fine-tuning method.

[0160] Step S402: Input the business sample data into the auxiliary network model, and output the auxiliary feature vector through the auxiliary network model;

[0161] The auxiliary network model can include an auxiliary fully connected network layer, and the auxiliary feature vector can be the output of the auxiliary fully connected network layer; similarly, the candidate network model S p It can include candidate fully connected network layers, and the business feature vector can be the output of the candidate fully connected network layer.

[0162] Among them, candidate network model S p The number of output business feature vectors is H, where H can be a positive integer; the number of auxiliary feature vectors is R, where R can be a positive integer. The number of business feature vectors output by the N candidate network models can be the same or different.

[0163] Step S403, based on auxiliary feature vectors and candidate network model S p The output business feature vector is used to generate candidate network model S. p The corresponding weight adjustment parameters;

[0164] Specifically, if H equals R, the terminal device can determine the vector divergence between the service feature vectors and auxiliary feature vectors at the same feature position based on the order of the H service feature vectors and the order of the R auxiliary feature vectors. The sum of the vector divergences at each feature position is then used to determine the candidate network model S. p The corresponding weight adjustment parameters. Optionally, if H is greater than R, the terminal device can obtain R service feature vectors from the H service feature vectors. Based on the order of the R service feature vectors and the order of the R auxiliary feature vectors, the vector divergence between the service feature vectors and auxiliary feature vectors at the same feature position is determined. The sum of the vector divergences at each feature position is used to determine the candidate network model S. p The corresponding weight adjustment parameters. Optionally, if H is less than R, the terminal device can obtain H auxiliary feature vectors from R auxiliary feature vectors. Based on the order of the H service feature vectors and the order of the H auxiliary feature vectors, the vector divergence between the service feature vectors and auxiliary feature vectors at the same feature position is determined. The sum of the vector divergences at each feature position is used to determine the candidate network model S. p The corresponding weight adjustment parameters. Among them, the vector divergence can be KL divergence (Kullback-Leibler divergence). In other words, the terminal device can obtain the KL divergence corresponding to valid word phrases (i.e., tokens excluding the pad part).

[0165] Step S404, based on the candidate network model S p The corresponding weight adjustment parameters and the weight adjustment parameters corresponding to the N candidate network models are used to determine the candidate network model S. p The corresponding model weights.

[0166] Among them, the terminal device is based on the candidate network model S p The corresponding weight adjustment parameters and the weight adjustment parameters corresponding to the N candidate network models are used to determine the candidate network model S. p For details on the specific process of applying the corresponding model weights, please refer to the above. Figure 3The description of step S103 in the corresponding embodiments will not be repeated here.

[0167] After determining the model weights corresponding to the N candidate network models, the terminal device can generate the initial network model corresponding to the target task based on the N candidate network models and their corresponding model weights. The specific process of the terminal device generating the initial network model corresponding to the target task based on the N candidate network models and their corresponding model weights can be found in formula (4):

[0168]

[0169] Where K can be Z in the above formula (1), and K can represent the weight adjustment parameter determined according to the vector divergence. Can be used to represent candidate network model S i The corresponding model weights, delta_model i Can be used to represent candidate network model S i delta_model can represent the initial network model, where i can be any positive integer from 1 to n.

[0170] It should be understood that a larger vector divergence indicates that the two vectors are less similar, and a larger weight adjustment parameter indicates a greater difference between the candidate task and the target task (i.e., a smaller task similarity). Therefore, after taking the negative of the weight adjustment parameter, the candidate network model corresponding to the candidate task with a smaller weight adjustment parameter (i.e., a smaller difference between the candidate task and the target task) has a larger model weight.

[0171] Therefore, the embodiments of this application can determine the vector divergence between business feature vectors and auxiliary feature vectors at the same feature locations, generate weight adjustment parameters corresponding to candidate network models based on the vector divergence, and then determine the model weights corresponding to the candidate network models based on the weight adjustment parameters. Thus, the embodiments of this application can measure the task similarity between the target task and candidate tasks based on the model weights corresponding to the candidate network models, thereby improving the accuracy of the initial network model when weighting the candidate network models based on the model weights.

[0172] Further, please see Figure 8 , Figure 8 This is a flowchart illustrating a data processing method provided in an embodiment of this application. The data processing method may include the following steps S501-S504, and steps S501-S504 are... Figure 3 This is a specific embodiment of step S103 in the corresponding embodiment. The N candidate network models include candidate network model S. pHere, p can be a positive integer less than or equal to N.

[0173] Step S501: Perform word segmentation on the business sample tags corresponding to the business sample data to obtain the business tag word segments of the business sample tags;

[0174] The specific process of the terminal device performing word segmentation on the business sample tags can be found in the above. Figure 3 The description of word segmentation processing of business sample data in the corresponding embodiments will not be repeated here.

[0175] Step S502: Obtain the tag segmentation vectors corresponding to the business tag segmentation, and generate the tag segmentation vectors and candidate network model S. p Similarity parameters between the output business feature vectors;

[0176] Specifically, the terminal device can obtain the tag segmentation vectors corresponding to the business tag segmentation. Among them, the candidate network model S... p The number of output business feature vectors is H, where H can be a positive integer; the number of tag word segmentation vectors is A, where A can be a positive integer. The number of business feature vectors output by the N candidate network models can be the same or different. Furthermore, the terminal device can determine the similarity parameter between tag word segmentation vectors and business feature vectors at the same feature position based on the order of the A tag word segmentation vectors and the order of the H business feature vectors.

[0177] Where H equals A, the terminal device can determine the similarity parameter between the business feature vectors and tag segmentation vectors at the same feature position based on the order of the H business feature vectors and the order of the A tag segmentation vectors. Optionally, if H is greater than A, the terminal device can obtain A business feature vectors from the H business feature vectors, and determine the similarity parameter between the business feature vectors and tag segmentation vectors at the same feature position based on the order of the A business feature vectors and the order of the A tag segmentation vectors. Optionally, if H is less than A, the terminal device can obtain H tag segmentation vectors from the A tag segmentation vectors, and determine the similarity parameter between the business feature vectors and tag segmentation vectors at the same feature position based on the order of the H business feature vectors and the order of the H tag segmentation vectors.

[0178] The specific process by which the terminal device obtains the tag segmentation vector corresponding to the business tag segmentation can be found in the above. Figure 5 The description of step S202 in the corresponding embodiments will not be repeated here.

[0179] The similarity parameter can be determined by Euclidean distance or cosine similarity. Optionally, the similarity parameter can also be determined by Manhattan distance, standardized Euclidean distance, Pearson distance, etc., and this application does not limit this.

[0180] Step S503: Generate candidate network model S based on similarity parameters. p The corresponding weight adjustment parameters;

[0181] Specifically, the terminal device can determine the candidate network model S by summing the similarity parameters between the business feature vectors and the tag segmentation vectors at the same feature locations. p The corresponding weight adjustment parameters. Wherein, if H equals A, the terminal device can determine the candidate network model S by the sum of the similarity parameters between the business feature vectors and the tag segmentation vectors at H (or A) feature positions. p The corresponding weight adjustment parameters. Optionally, if H is greater than A, the terminal device can determine the candidate network model S by the sum of the similarity parameters between the business feature vectors and the tag word segmentation vectors at the A feature positions. p The corresponding weight adjustment parameters. Optionally, if H is less than A, the terminal device can determine the candidate network model S by the sum of the similarity parameters between the business feature vectors and the tag word segmentation vectors at the H feature positions. p The corresponding weight adjustment parameters.

[0182] Step S504, based on the candidate network model S p The corresponding weight adjustment parameters and the weight adjustment parameters corresponding to the N candidate network models are used to determine the candidate network model S. p The corresponding model weights.

[0183] Among them, the terminal device is based on the candidate network model S p The corresponding weight adjustment parameters and the weight adjustment parameters corresponding to the N candidate network models are used to determine the candidate network model S. p For details on the specific process of applying the corresponding model weights, please refer to the above. Figure 3 The description of step S103 in the corresponding embodiments will not be repeated here.

[0184] After determining the model weights corresponding to the N candidate network models, the terminal device can generate the initial network model corresponding to the target task based on the N candidate network models and their corresponding model weights. The specific process of the terminal device generating the initial network model corresponding to the target task based on the N candidate network models and their corresponding model weights can be found in formulas (5) and (6):

[0185]

[0186] Where D can be Z in the above formula (1), and D can represent the weight adjustment parameter determined according to the Euclidean distance. Can be used to represent candidate network model S i The corresponding model weights, delta_model i Can be used to represent candidate network model S i delta_model can represent the initial network model, where i can be any positive integer from 1 to n. Here, D represents the sum of the Euclidean distances corresponding to all valid word groups.

[0187]

[0188] Where C can be Z in the above formula (1), and C can represent the weight adjustment parameter determined based on cosine similarity. Can be used to represent candidate network model S i The corresponding model weights, delta_model i Can be used to represent candidate network model S i delta_model can represent the initial network model, where i can be any positive integer from 1 to n. Here, D represents the sum of the cosine similarities corresponding to all valid word groups.

[0189] It should be understood that a larger cosine similarity (or Euclidean distance) indicates that the two vectors are less similar, and a larger weight adjustment parameter means a greater difference between the candidate task and the target task (i.e., a smaller task similarity). Therefore, after taking the negative of the weight adjustment parameter, the candidate network model corresponding to the candidate task with a smaller weight adjustment parameter (i.e., a smaller difference between the candidate task and the target task) has a larger model weight.

[0190] Therefore, this embodiment of the application can determine the similarity parameter between the tag segmentation vector and the business feature vector at the same feature position, generate the weight adjustment parameter corresponding to the candidate network model based on the similarity parameter, and then determine the model weight corresponding to the candidate network model based on the weight adjustment parameter corresponding to the candidate network model. Thus, this embodiment of the application can measure the task similarity between the target task and the candidate task based on the model weight corresponding to the candidate network model, thereby improving the accuracy of the initial network model when weighting the candidate network model based on the model weight.

[0191] For ease of understanding, please refer to Table 1. Table 1 is a comparison list of the accuracy of different methods provided in the embodiments of this application. This comparison list can store the accuracy of different methods (e.g., random initialization, existing method 1, existing method 2, cross-entropy loss initialization, KL divergence initialization, model parameter gradient initialization, probability vector-label initialization (Euclidean distance version), probability vector-label initialization (cosine similarity version)) under different few-parameter pre-trained model fine-tuning methods (e.g., Adapter Tuning, LoRA, Prefix Tuning). As shown in Table 1:

[0192] Table 1

[0193]

[0194] As shown in Table 1, this application trained five instance models for three efficient, low-parameter fine-tuning methods (i.e., Adapter Tuning, LoRA, and Prefix Tuning). These five instance models (i.e., cross-entropy loss initialization, KL divergence initialization, model parameter gradient initialization, probability vector-label initialization (Euclidean distance version), and probability vector-label initialization (cosine similarity version)) and three instance models from existing technologies (i.e., random initialization, existing method 1, and existing method 2) can be directly applied to the corresponding question-answering tasks. While existing methods 1 and 2 improved the test results of the target task to some extent, the improvement was not significant compared to random initialization. Cross-entropy loss initialization, KL divergence initialization, model parameter gradient initialization, probability vector-label initialization (Euclidean distance version), and probability vector-label initialization (cosine similarity version) almost all outperformed the random initialization method and the two existing methods. Only the model parameter gradient initialization method was slightly lower than existing method 2 in the Adapter Tuning and LoRA cases (but still significantly higher than the random initialization method).

[0195] Existing method 1 calculates the cosine similarity between the auxiliary network model and the N candidate network models (i.e., the cosine similarity between the vector formed by the model parameters of the auxiliary network model and the vector formed by the model parameters of the N candidate network models), and uses the cosine similarity as a weight to perform a weighted average of the N candidate network models. Existing method 2 can directly average the N candidate network models (i.e., the model parameters of the N candidate network models).

[0196] As shown in Table 1, the eight example models selected 32 question-answering tasks for testing. For each task, it was considered the target task, while the remaining 31 tasks were considered candidate tasks, and test results were calculated. The average of the 32 test results (i.e., accuracy) was taken as the final result. In other words, this application can divide the data corresponding to the target task into two categories: one is business sample data and business sample labels, and the other is test sample data and test sample labels. After generating the target network model based on the business sample data and business sample labels, the target network model can be tested based on the test sample data and test sample labels to obtain the accuracy of the target network model.

[0197] It should be understood that, in this application embodiment, the weight adjustment parameter determined based on the cross-entropy loss parameter can be referred to as the first weight adjustment parameter, the weight adjustment parameter determined based on the parameter gradient can be referred to as the second weight adjustment parameter, the weight adjustment parameter determined based on the vector divergence can be referred to as the third weight adjustment parameter, the weight adjustment parameter determined based on the Euclidean distance can be referred to as the fourth weight adjustment parameter, and the weight adjustment parameter determined based on the cosine similarity can be referred to as the fifth weight adjustment parameter. This application embodiment can determine the weight adjustment parameter based on the first, second, third, fourth, or fifth weight adjustment parameter. Optionally, this application embodiment can determine the weight adjustment parameter based on at least two of the first, second, third, fourth, or fifth weight adjustment parameters. For example, this application embodiment can determine the weight adjustment parameter as the sum of the first and second weight adjustment parameters. As another example, this application embodiment can determine the weight adjustment parameter as the sum of the first, second, third, fourth, and fifth weight adjustment parameters.

[0198] Therefore, this application provides a library of methods for model initialization in efficient, low-parameter fine-tuning methods based on task data. This library can be applied to efficient, low-parameter fine-tuning methods and may include cross-entropy loss initialization, KL divergence initialization, model parameter gradient initialization, and probability vector-label initialization. The experimental results shown in Table 1 demonstrate the effectiveness of the library provided in this application.

[0199] Further, please see Figure 9 , Figure 9 This is a schematic diagram of the structure of a data processing device provided in an embodiment of this application. The data processing device 1 may include: a model acquisition module 11, a data input module 12, a weight determination module 13, and a model generation module 14; further, the data processing device 1 may also include: a loss determination module 15 and a parameter adjustment module 16.

[0200] The model acquisition module 11 is used to acquire candidate network models corresponding to N candidate tasks; N is a positive integer greater than 1.

[0201] The data input module 12 is used to input the business sample data corresponding to the target task into N candidate network models respectively, and output the business feature vectors through the N candidate network models respectively.

[0202] Among them, the data input module 12 is specifically used to perform word segmentation processing on the business sample data corresponding to the target task to obtain the business data word segmentation of the business sample data;

[0203] The data input module 12 is specifically used to obtain the encoded word segmentation vector corresponding to the word segmentation of business data, and to extract features from the encoded word segmentation vector through N candidate network models to obtain the business feature vectors output by the N candidate network models respectively.

[0204] The weight determination module 13 is used to determine the model weights corresponding to the N candidate network models based on the business sample data and the business feature vectors output by the N candidate network models respectively; the model weights corresponding to the N candidate network models are used to characterize the task similarity between the target task and the N candidate tasks.

[0205] Among them, N candidate network models include candidate network model S p p is a positive integer less than or equal to N;

[0206] The weight determination module 13 includes: a first parameter determination unit 131 and a weight determination unit 132;

[0207] The first parameter determination unit 131 is used to determine parameters based on business sample data and candidate network model S. p The output business feature vector is used to generate candidate network model S.p The corresponding weight adjustment parameters;

[0208] The first parameter determination unit 131 is specifically used to perform word segmentation on the business sample tags corresponding to the business sample data to obtain business tag word segmentation of the business sample tags.

[0209] The first parameter determination unit 131 is specifically used to obtain the tag segmentation vector corresponding to the business tag segmentation, and to generate the tag segmentation vector and candidate network model S based on the loss function. p The cross-entropy loss parameter between the output business feature vectors;

[0210] The first parameter determination unit 131 is specifically used to generate candidate network models S based on the cross-entropy loss parameters. p The corresponding weight adjustment parameters.

[0211] Among them, candidate network model S p The number of output business feature vectors is H, where H is a positive integer; the number of tag word segmentation vectors is A, where A is a positive integer;

[0212] The first parameter determination unit 131 is specifically used for determining the candidate network model S. p The output H business feature vectors are normalized respectively to obtain the sample normalized vectors corresponding to the H business feature vectors;

[0213] The first parameter determination unit 131 is specifically used to determine the cross-entropy loss parameter between the tag segmentation vectors and the sample normalization vectors at the same feature position based on the order of A tag segmentation vectors, the order of H sample normalization vectors, and the loss function.

[0214] The first parameter determination unit 131 is specifically used to determine the cross-entropy loss parameter between the sample normalization vector and the label segmentation vector at the same feature position if H equals A, based on the order of the H sample normalization vectors, the order of the A label segmentation vectors, and the loss function.

[0215] The first parameter determination unit 131 is specifically used to obtain A sample normalization vectors from H sample normalization vectors if H is greater than A, and determine the cross-entropy loss parameter between sample normalization vectors and label segmentation vectors at the same feature position according to the order of A sample normalization vectors, the order of A label segmentation vectors and the loss function.

[0216] The first parameter determination unit 131 is specifically used to obtain H tag segmentation vectors from A tag segmentation vectors if H is less than A, and determine the cross-entropy loss parameter between the sample normalization vectors and the tag segmentation vectors at the same feature position based on the order of the H sample normalization vectors, the order of the H tag segmentation vectors and the loss function.

[0217] Specifically, the first parameter determination unit 131 is used to determine the business sample labels corresponding to the business sample data and the candidate network model S. p The output business feature vector is used to generate candidate network model S. p The candidate model loss during the first backpropagation of parameters;

[0218] The first parameter determination unit 131 is specifically used to determine the candidate network model S based on the parameter determination unit 131. p The candidate model loss during the first backpropagation is used to determine the candidate network model S. p The parameter gradient of the adjustable parameter in the text;

[0219] The first parameter determination unit 131 is specifically used to determine the candidate network model S. p The sum of the gradients of the adjustable parameters in the model is used to determine the candidate network model S. p The corresponding weight adjustment parameters.

[0220] The first parameter determination unit 131 is specifically used to obtain the auxiliary network model corresponding to the target task; the auxiliary network model is obtained by adjusting the parameters of the random initial network model based on the target task.

[0221] The first parameter determination unit 131 is specifically used to input business sample data into the auxiliary network model and output auxiliary feature vectors through the auxiliary network model.

[0222] The first parameter determination unit 131 is specifically used for determining parameters based on auxiliary feature vectors and candidate network models S. p The output business feature vector is used to generate candidate network model S. p The corresponding weight adjustment parameters.

[0223] Among them, candidate network model S p The number of output business feature vectors is H, where H is a positive integer; the number of auxiliary feature vectors is R, where R is a positive integer.

[0224] The first parameter determination unit 131 is specifically used to determine the vector divergence between the business feature vectors and auxiliary feature vectors at the same feature position based on the order of the H business feature vectors and the order of the R auxiliary feature vectors if H equals R, and to determine the sum of the vector divergences at each feature position as the candidate network model S. p The corresponding weight adjustment parameters;

[0225] The first parameter determination unit 131 is specifically used to: if H is greater than R, obtain R business feature vectors from the H business feature vectors; determine the vector divergence between business feature vectors and auxiliary feature vectors at the same feature position according to the order of the R business feature vectors and the order of the R auxiliary feature vectors; and determine the sum of the vector divergences at each feature position as the candidate network model S. p The corresponding weight adjustment parameters;

[0226] The first parameter determination unit 131 is specifically used to: if H is less than R, obtain H auxiliary feature vectors from R auxiliary feature vectors; determine the vector divergence between the business feature vectors and auxiliary feature vectors at the same feature position according to the order of the H business feature vectors and the order of the H auxiliary feature vectors; and determine the sum of the vector divergences at each feature position as the candidate network model S. p The corresponding weight adjustment parameters.

[0227] The first parameter determination unit 131 is specifically used to perform word segmentation on the business sample tags corresponding to the business sample data to obtain business tag word segmentation of the business sample tags.

[0228] The first parameter determination unit 131 is specifically used to obtain the tag segmentation vector corresponding to the business tag segmentation, and to generate the tag segmentation vector and the candidate network model S. p Similarity parameters between the output business feature vectors;

[0229] The first parameter determination unit 131 is specifically used to generate candidate network models S based on similarity parameters. p The corresponding weight adjustment parameters.

[0230] Weight determination unit 132 is used to determine the weights based on the candidate network model S. p The corresponding weight adjustment parameters and the weight adjustment parameters corresponding to the N candidate network models are used to determine the candidate network model S. p The corresponding model weights.

[0231] Among them, the weight determination unit 132 is specifically used to perform calculations on the weight adjustment parameters corresponding to the N candidate network models respectively, so as to obtain the candidate weights corresponding to the N candidate network models respectively.

[0232] The weight determination unit 132 is specifically used to determine the total candidate weight by summing the candidate weights corresponding to the N candidate network models.

[0233] Weight determination unit 132 is specifically used to determine the weights of candidate network models S. p The ratio of the corresponding candidate weights to the total candidate weights is determined as the candidate network model S.p The corresponding model weights.

[0234] The specific implementation methods of the first parameter determination unit 131 and the weight determination unit 132 can be found in the above description. Figure 3 In the corresponding embodiment, step S103, Figure 5 In the corresponding embodiment, steps S201-S204, Figure 6 In the corresponding embodiment, steps S301-S304, Figure 7 In the corresponding embodiment, steps S401-S404, and Figure 8 The descriptions of steps S501-S504 in the corresponding embodiments will not be repeated here.

[0235] The model generation module 14 is used to generate an initial network model corresponding to the target task based on N candidate network models and their respective model weights.

[0236] Among them, N candidate network models include candidate network model S p p is a positive integer less than or equal to N; candidate network model S p Includes a parameter tuning subnetwork; candidate network model S p The parameter adjustment subnetwork includes adjustable parameters; all N candidate network models include a text processing subnetwork with the same model structure and the same model parameters;

[0237] The model generation module 14 includes: a second parameter determination unit 141 and a model generation unit 142;

[0238] The second parameter determination unit 141 is used to determine the candidate network model S based on the parameter determination unit 141. p The parameter information of the adjustable parameters in the parameter adjustment subnetwork and the candidate network model S p The corresponding model weights are used to generate candidate network models S. p The corresponding model adjustment parameters;

[0239] The second parameter determination unit 141 is used to generate the total model adjustment parameters based on the model adjustment parameters corresponding to the N candidate network models respectively.

[0240] Model generation unit 142 is used to generate candidate network models S p The parameter information of the adjustable parameters in the parameter adjustment subnetwork is adjusted to the total model adjustment parameters. The parameter adjustment subnetwork and the text processing subnetwork after parameter adjustment are determined as the initial network model corresponding to the target task.

[0241] The specific implementation methods of the second parameter determination unit 141 and the model generation unit 142 can be found in the above description. Figure 3 The description of step S104 in the corresponding embodiment will not be repeated here.

[0242] Optionally, the loss determination module 15 is used to input business sample data into the initial network model and output a predicted feature vector through the initial network model;

[0243] The loss determination module 15 is used to obtain the business sample labels corresponding to the business sample data, obtain the label word segmentation vectors corresponding to the business sample labels, and determine the model loss value of the initial network model based on the label word segmentation vectors and the predicted feature vectors.

[0244] The parameter adjustment module 16 is used to adjust the parameters of the initial network model according to the model loss value. When the initial network model after parameter adjustment meets the model convergence condition, the initial network model after parameter adjustment is determined as the target network model. The target network model has the ability to identify business data belonging to the task type of the target task.

[0245] The specific implementation methods of the model acquisition module 11, data input module 12, weight determination module 13, model generation module 14, loss determination module 15, and parameter adjustment module 16 can be found in the above description. Figure 3 In the corresponding embodiment, steps S101-S104, Figure 5 In the corresponding embodiment, steps S201-S204, Figure 6 In the corresponding embodiment, steps S301-S304, Figure 7 In the corresponding embodiment, steps S401-S404, and Figure 8 The descriptions of steps S501-S504 in the corresponding embodiments will not be repeated here. Furthermore, the beneficial effects of using the same method will also not be repeated.

[0246] Further, please see Figure 10 , Figure 10 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. The computer device may be a terminal device or a server. Figure 10As shown, the computer device 1000 may include a processor 1001, a network interface 1004, and a memory 1005. Furthermore, the computer device 1000 may also include a user interface 1003 and at least one communication bus 1002. The communication bus 1002 is used to enable communication between these components. In some embodiments, the user interface 1003 may include a display screen and a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. Optionally, the network interface 1004 may include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be high-speed RAM or non-volatile memory, such as at least one disk storage device. Optionally, the memory 1005 may also be at least one storage device located remotely from the processor 1001. Figure 10 As shown, the memory 1005, which is a computer-readable storage medium, may include an operating system, a network communication module, a user interface module, and a device control application.

[0247] In such Figure 10 In the computer device 1000 shown, the network interface 1004 provides network communication functionality; the user interface 1003 is mainly used to provide an input interface for the user; and the processor 1001 can be used to call the device control application stored in the memory 1005 to achieve:

[0248] Obtain the candidate network models corresponding to N candidate tasks; N is a positive integer greater than 1.

[0249] The business sample data corresponding to the target task is input into N candidate network models respectively, and the business feature vectors are output by the N candidate network models respectively.

[0250] Based on the business sample data and the business feature vectors output by N candidate network models, the model weights corresponding to the N candidate network models are determined. The model weights corresponding to the N candidate network models are used to characterize the task similarity between the target task and the N candidate tasks.

[0251] Based on the N candidate network models and their respective model weights, an initial network model corresponding to the target task is generated.

[0252] It should be understood that the computer device 1000 described in the embodiments of this application can execute the foregoing text. Figure 3 , Figure 5 , Figure 6 , Figure 7 or Figure 8The description of the data processing method in the corresponding embodiments can also be performed as described above. Figure 9 The description of the data processing device 1 in the corresponding embodiments will not be repeated here. Furthermore, the beneficial effects of using the same method will also not be repeated here.

[0253] Furthermore, it should be noted that this application embodiment also provides a computer-readable storage medium, which stores a computer program executed by the aforementioned data processing device 1. When the processor executes the computer program, it can execute the aforementioned... Figure 3 , Figure 5 , Figure 6 , Figure 7 or Figure 8 The description of the data processing method in the corresponding embodiments is already provided and will not be repeated here. Furthermore, the beneficial effects of using the same method will also not be repeated. For technical details not disclosed in the computer-readable storage medium embodiments related to this application, please refer to the description of the method embodiments of this application.

[0254] Furthermore, it should be noted that this application also provides a computer program product, which may include a computer program that can be stored in a computer-readable storage medium. The processor of a computer device reads the computer program from the computer-readable storage medium, and the processor can execute the computer program, causing the computer device to perform the aforementioned... Figure 3 , Figure 5 , Figure 6 , Figure 7 or Figure 8 The description of the data processing method in the corresponding embodiments is already provided and will not be repeated here. Furthermore, the beneficial effects of using the same method will also not be repeated. For technical details not disclosed in the computer program product embodiments related to this application, please refer to the description of the method embodiments of this application.

[0255] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0256] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.

Claims

1. A data processing method, characterized in that, include: Obtain the candidate network models corresponding to N candidate tasks; N is a positive integer greater than 1; The business sample data corresponding to the target task is input into N candidate network models, and each of the N candidate network models outputs a business feature vector. The business feature vector represents the probability vector for word segmentation of the business data. Word segmentation of the business data is obtained by recombinating the continuous character sequences corresponding to the business sample data into word sequences according to a preset standard. The N candidate network models include candidate network model S. p p is a positive integer less than or equal to N; Based on the business sample data and the candidate network model S p The output business feature vector is used to generate the candidate network model S. p The corresponding weight adjustment parameters; According to the candidate network model S p The candidate network model S is generated by using the corresponding weight adjustment parameters and the difference parameters used to regulate the absolute difference in similarity between the N candidate tasks and the target task. p The corresponding candidate weights; The sum of the candidate weights corresponding to the N candidate network models is determined as the total candidate weight. The candidate network models S... p The ratio of the corresponding candidate weights to the total candidate weights is determined as the candidate network model S. p The corresponding model weights; The model weights corresponding to the N candidate network models are used to characterize the task similarity between the target task and the N candidate tasks; An initial network model corresponding to the target task is generated based on the N candidate network models and their respective model weights.

2. The method according to claim 1, characterized in that, The step of inputting the business sample data corresponding to the target task into N candidate network models, and outputting business feature vectors through the N candidate network models respectively, includes: The business sample data corresponding to the target task is segmented into words to obtain the business data words of the business sample data. Obtain the encoded word segmentation vector corresponding to the word segmentation of the business data, and extract features from the encoded word segmentation vector through N candidate network models respectively to obtain the business feature vectors output by the N candidate network models respectively.

3. The method according to claim 1, characterized in that, The data based on the business sample data and the candidate network model S p The output business feature vector is used to generate the candidate network model S. p The corresponding weight adjustment parameters include: The business sample tags corresponding to the business sample data are segmented into words to obtain the business tag words of the business sample tags. Obtain the tag segmentation vector corresponding to the business tag segmentation, and generate the tag segmentation vector and the candidate network model S based on the loss function. p The cross-entropy loss parameter between the output business feature vectors; Based on the cross-entropy loss parameters, the candidate network model S is generated. p The corresponding weight adjustment parameters.

4. The method according to claim 3, characterized in that, The candidate network model S p The number of output business feature vectors is H, where H is a positive integer; the number of tag word segmentation vectors is A, where A is a positive integer; The tag segmentation vector and the candidate network model S are generated based on the loss function. p The cross-entropy loss parameters between the output business feature vectors include: For the candidate network model S p The H output business feature vectors are each normalized to obtain the sample normalized vectors corresponding to the H business feature vectors respectively; Based on the order of the A tag segmentation vectors, the order of the H sample normalization vectors, and the loss function, the cross-entropy loss parameter between the tag segmentation vectors and sample normalization vectors at the same feature position is determined.

5. The method according to claim 4, characterized in that, The method for determining the cross-entropy loss parameter between the tag segmentation vectors and sample normalization vectors at the same feature position based on the order of A tag segmentation vectors, the order of H sample normalization vectors, and a loss function includes: If H equals A, then the cross-entropy loss parameter between the sample normalization vectors and the label segmentation vectors at the same feature position is determined according to the order of the H sample normalization vectors, the order of the A label segmentation vectors, and the loss function. If H is greater than A, then A sample normalized vectors are obtained from the H sample normalized vectors. Based on the order of the A sample normalized vectors, the order of the A tag segmentation vectors, and the loss function, the cross-entropy loss parameter between the sample normalized vectors and the tag segmentation vectors at the same feature position is determined. If H is less than A, then H tag segmentation vectors are obtained from A tag segmentation vectors. Based on the order of the H sample normalization vectors, the order of the H tag segmentation vectors, and the loss function, the cross-entropy loss parameter between the sample normalization vectors and the tag segmentation vectors at the same feature position is determined.

6. The method according to claim 1, characterized in that, The data based on the business sample data and the candidate network model S p The output business feature vector is used to generate the candidate network model S. p The corresponding weight adjustment parameters include: Based on the business sample labels corresponding to the business sample data and the candidate network model S p The output business feature vector is used to generate the candidate network model S. p The candidate model loss during the first backpropagation of parameters; According to the candidate network model S p The candidate model S is determined by the candidate model loss during the first backpropagation. p The parameter gradient of the adjustable parameter in the text; The candidate network model S p The sum of the gradients of the adjustable parameters in the model is used to determine the candidate network model S. p The corresponding weight adjustment parameters.

7. The method according to claim 1, characterized in that, The data based on the business sample data and the candidate network model S p The output business feature vector is used to generate the candidate network model S. p The corresponding weight adjustment parameters include: Obtain the auxiliary network model corresponding to the target task; the auxiliary network model is obtained by adjusting the parameters of a randomly initialized network model based on the target task. The business sample data is input into the auxiliary network model, and the auxiliary network model outputs an auxiliary feature vector. Based on the auxiliary feature vector and the candidate network model S p The output business feature vector is used to generate the candidate network model S. p The corresponding weight adjustment parameters.

8. The method according to claim 7, characterized in that, The candidate network model S p The number of output business feature vectors is H, where H is a positive integer; the number of auxiliary feature vectors is R, where R is a positive integer. The auxiliary feature vector and the candidate network model S p The output business feature vector is used to generate the candidate network model S. p The corresponding weight adjustment parameters include: If H equals R, then based on the order of the H business feature vectors and the order of the R auxiliary feature vectors, the vector divergence between the business feature vectors and auxiliary feature vectors at the same feature position is determined, and the sum of the vector divergences at each feature position is determined as the candidate network model S. p The corresponding weight adjustment parameters; If H is greater than R, then R business feature vectors are obtained from the H business feature vectors. Based on the order of the R business feature vectors and the order of the R auxiliary feature vectors, the vector divergence between the business feature vectors and auxiliary feature vectors at the same feature position is determined. The sum of the vector divergences at each feature position is determined as the candidate network model S. p The corresponding weight adjustment parameters; If H is less than R, then H auxiliary feature vectors are obtained from the R auxiliary feature vectors. Based on the order of the H business feature vectors and the H auxiliary feature vectors, the vector divergence between the business feature vectors and auxiliary feature vectors at the same feature position is determined. The sum of the vector divergences at each feature position is then used to determine the candidate network model S. p The corresponding weight adjustment parameters.

9. The method according to claim 1, characterized in that, The data based on the business sample data and the candidate network model S p The output business feature vector is used to generate the candidate network model S. p The corresponding weight adjustment parameters include: The business sample tags corresponding to the business sample data are segmented into words to obtain the business tag words of the business sample tags. Obtain the tag segmentation vector corresponding to the business tag segmentation, and generate the tag segmentation vector and the candidate network model S. p Similarity parameters between the output business feature vectors; Based on the similarity parameter, the candidate network model S is generated. p The corresponding weight adjustment parameters.

10. The method according to claim 1, characterized in that, The N candidate network models include candidate network model S p The p is a positive integer less than or equal to N; the candidate network model S p Includes a parameter adjustment subnetwork; the candidate network model S p The parameter adjustment subnetwork includes adjustable parameters; all N candidate network models include a text processing subnetwork with the same model structure and the same model parameters; The step of generating an initial network model corresponding to the target task based on the N candidate network models and their respective model weights includes: According to the candidate network model S p The parameter information of the adjustable parameters in the parameter adjustment subnetwork and the candidate network model S p The corresponding model weights are used to generate the candidate network model S. p The corresponding model adjustment parameters; Based on the model adjustment parameters corresponding to the N candidate network models, generate the total model adjustment parameters; The candidate network model S p The parameter information of the adjustable parameters in the parameter adjustment subnetwork is adjusted to the total model adjustment parameters. The parameter adjustment subnetwork and the text processing subnetwork after parameter adjustment are determined as the initial network model corresponding to the target task.

11. The method according to claim 1, characterized in that, The method further includes: The business sample data is input into the initial network model, and the initial network model outputs a predicted feature vector. Obtain the business sample label corresponding to the business sample data, obtain the label segmentation vector corresponding to the business sample label, and determine the model loss value of the initial network model based on the label segmentation vector and the predicted feature vector; The parameters of the initial network model are adjusted based on the model loss value. When the initial network model after parameter adjustment meets the model convergence condition, the initial network model after parameter adjustment is determined as the target network model. The target network model has the ability to identify business data belonging to the task type of the target task.

12. A data processing apparatus, characterized in that, include: The model acquisition module is used to acquire candidate network models corresponding to N candidate tasks. N is a positive integer greater than 1; The data input module is used to input the business sample data corresponding to the target task into N candidate network models respectively, and output business feature vectors through the N candidate network models respectively; the business feature vector refers to the probability vector representing the word segmentation of the business data, and the word segmentation of the business data is obtained by recombinating the continuous character sequence corresponding to the business sample data into a word sequence according to a preset standard; the N candidate network models include candidate network model S p p is a positive integer less than or equal to N; The weight determination module is used to determine the weights based on the business sample data and the candidate network model S. p The output business feature vector is used to generate the candidate network model S. p The corresponding weight adjustment parameters are based on the candidate network model S. p The candidate network model S is generated by using the corresponding weight adjustment parameters and the difference parameters used to regulate the absolute difference in similarity between the N candidate tasks and the target task. p The corresponding candidate weights; The weight determination module is further configured to determine the total candidate weight by summing the candidate weights corresponding to the N candidate network models, and to determine the total candidate weight by setting the candidate network model S. p The ratio of the corresponding candidate weights to the total candidate weights is determined as the candidate network model S. p The corresponding model weights; the model weights corresponding to the N candidate network models are used to characterize the task similarity between the target task and the N candidate tasks; The model generation module is used to generate an initial network model corresponding to the target task based on the N candidate network models and the model weights corresponding to the N candidate network models.

13. A computer device, characterized in that, include: Processor and memory; The processor is connected to the memory, wherein the memory is used to store a computer program, and the processor is used to invoke the computer program to cause the computer device to perform the method according to any one of claims 1-11.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program adapted to be loaded and executed by a processor to cause a computer device having the processor to perform the method according to any one of claims 1-11.

15. A computer program product, characterized in that, The computer program product includes a computer program stored in a computer-readable storage medium and adapted to be read and executed by a processor to cause a computer device having the processor to perform the method of any one of claims 1-11.

Images