Incremental content recognition network training method, content recognition method and device

Abstract

The embodiment of the invention provides an incremental content recognition network training method and device and a content recognition method and device, and relates to the technical field of neural networks. And calculating a causal effect parameter for each model parameter, calculating an influence factor of each historical task on the current task according to the correlation projection and the task affinity, updating the current gradient data according to the influence factor and the causal effect matrix to obtain updated gradient data, and optimizing the model parameters based on the updated gradient data. The influence degree of different model parameters on the loss function of the current task is quantified through the causal effect parameters, so that the fine tuning process does not need to update the full-quantity model parameters, and partial model parameters which have obvious influence on the current task are selectively focused, the calculation amount is reduced, and the fine tuning efficiency is improved. And meanwhile, current gradient data are updated based on the influence factors and the causal effect matrix, knowledge migration is realized, and the gradient calculation amount can be reduced and the iteration convergence speed can be accelerated while the fine tuning accuracy is ensured.

Description

Technical Field

[0001] This application relates to the field of neural network technology, and in particular to incremental content recognition network training methods, content recognition methods and apparatus. Background Technology

[0002] With the development of network information technology, online content exhibits characteristics of continuous growth and dynamic change. Machine learning-based content recognition models are prone to catastrophic forgetting when processing incremental network data. However, pre-trained large models possess rich pre-training knowledge and powerful learning and reasoning capabilities. Therefore, in processing incremental network data, a parameter-efficient fine-tuning technique based on pre-trained models is proposed to reduce the impact of catastrophic forgetting.

[0003] To address the catastrophic forgetting problem in incremental network content recognition, low-rank adaptation (LoRA) is employed for parameter fine-tuning. This method mitigates catastrophic forgetting by limiting the modifications made to the old task model by the new task model update, achieving performance comparable to full-parameter training with only a small number of parameters adjusted. However, this parameter fine-tuning method treats all parameters uniformly. In practical applications, the impact of different parameters varies across different stages of the task, and some parameters may be invalid. Therefore, full-parameter fine-tuning not only fails to improve task performance but also reduces the efficiency of the parameter fine-tuning process. Summary of the Invention

[0004] The main objective of this application is to propose an incremental content recognition network training method, content recognition method, and apparatus to improve the efficiency of parameter fine-tuning of pre-trained models.

[0005] To achieve the above objectives, a first aspect of this application proposes an incremental content recognition network training method, comprising:

[0006] Obtain the data sample and data label corresponding to the current task, input the data sample into the content recognition network for recognition processing, obtain the corresponding prediction output, calculate the current loss function based on the prediction output and data label, and obtain the current gradient data through backpropagation of the current loss function.

[0007] For each model parameter in the content recognition network, a causal effect parameter is calculated based on the current loss function, and a causal effect matrix is ​​obtained based on the causal effect parameter.

[0008] Obtain at least one historical task preceding the current task; for each historical task, obtain corresponding historical gradient data; calculate a correlation projection based on the historical gradient data and the current gradient data; calculate task affinity based on the correlation projection and the historical gradient data; and calculate the influence factor of each historical task on the current task based on the correlation projection and the task affinity.

[0009] The current gradient data is updated based on the influencing factor and the causal effect matrix to obtain updated gradient data. The model parameters are then optimized based on the updated gradient data until the training process of the content recognition network for the current task is completed.

[0010] In some embodiments, calculating the causal effect parameters based on the current loss function includes:

[0011] Based on the current weight matrix corresponding to the current task, obtain the weight parameters corresponding to each model parameter;

[0012] The current loss function is expanded using first-order and second-order Taylor expansions at the weight parameters to obtain first-order and second-order intermediate values.

[0013] The initial causal effect parameter is obtained by first multiplying the first-order intermediate value and the weight parameter, and second multiplying the second-order intermediate value and the square of the weight parameter. The initial causal effect parameter is then normalized to obtain the causal effect parameter.

[0014] In some embodiments, calculating the correlation projection based on the historical gradient data and the current gradient data includes:

[0015] Singular value decomposition is performed on the historical gradient data to obtain the basis information of the input subspace corresponding to the historical task;

[0016] The correlation projection of the current task in the input subspace of the historical task is obtained by multiplying the basis information, the transpose of the basis information, and the current gradient data.

[0017] In some embodiments, calculating task affinity based on the correlation projection and the historical gradient data includes:

[0018] For each column in the historical gradient data, obtain the corresponding historical gradient column vector and the corresponding correlation column vector in the correlation projection;

[0019] Calculate the first quotient of the correlation column vector and the corresponding first norm, and the second quotient of the historical gradient column vector and the corresponding second norm. Obtain the corresponding similarity value based on the product of the first quotient and the second quotient.

[0020] The similarity values ​​are standardized to obtain an affinity column vector, and the task affinity is obtained based on all the affinity column vectors.

[0021] In some embodiments, the standardization of the similarity value to obtain the affinity value includes:

[0022] If the similarity value is greater than zero, the affinity column vector is obtained based on the all-one matrix;

[0023] If the similarity value is equal to zero, the affinity column vector is obtained from the all-zero matrix;

[0024] If the similarity value is less than zero, the affinity column vector is obtained based on the all-negative-one matrix.

[0025] In some embodiments, calculating the influence factor of each historical task on the current task based on the relevance projection and the task affinity includes:

[0026] The task correlation is obtained by calculating the quotient of the third norm corresponding to the correlation projection and the fourth norm corresponding to the current gradient data;

[0027] Calculate the third quotient of the correlation projection and the third norm, and obtain the gradient projection based on the product of the third quotient and the task correlation.

[0028] For each historical task, the Hadamard product of the task affinity and the gradient projection is calculated to obtain the corresponding influence factor.

[0029] In some embodiments, updating the current gradient data based on the influence factor and the causal effect matrix to obtain updated gradient data includes:

[0030] Summing up all the aforementioned impact factors and adding them to the all-one matrix yields the total impact factor;

[0031] The updated gradient data is obtained by sequentially calculating the Hadamard product of the causal effect matrix, the current gradient data, and the total influence factor.

[0032] In some embodiments, optimizing the model parameters based on the updated gradient data includes:

[0033] At each time step, the iterative weight matrix is ​​obtained. Based on the updated gradient data, the first low-rank matrix and the second low-rank matrix corresponding to the previous time step, the low-rank data corresponding to the time step is obtained. Based on the iterative weight matrix and the low-rank data, the time step weight matrix corresponding to the time step is obtained. The time step weight matrix of the previous time step is used as the iterative weight matrix of the next time step. The initial value of the iterative weight matrix is ​​the weight matrix corresponding to the previous historical task.

[0034] Repeat the iteration until the weight matrix of the last time step is obtained, and then optimize the model parameters.

[0035] In some embodiments, obtaining the low-rank data corresponding to the time step based on the updated gradient data, the first low-rank matrix and the second low-rank matrix corresponding to the previous time step includes:

[0036] Obtain the transposes of the first low-rank matrix and the second low-rank matrix corresponding to the previous time step, respectively, to obtain the first transpose matrix and the second transpose matrix;

[0037] Calculate the product of the preset learning rate, the updated gradient data, and the first transpose matrix to obtain the second changed data; calculate the product of the preset learning rate, the second transpose matrix, and the updated gradient data to obtain the first changed data.

[0038] Calculate the difference between the first low-rank matrix of the previous time step and the first changed data to obtain the first low-rank matrix corresponding to the time step;

[0039] The difference between the second low-rank matrix of the previous time step and the second changed data is calculated to obtain the second low-rank matrix corresponding to the time step;

[0040] The low-rank data is obtained by multiplying the first low-rank matrix and the second low-rank matrix corresponding to the time step.

[0041] To achieve the above objectives, a second aspect of this application provides a content recognition method, comprising:

[0042] Obtain the data to be identified;

[0043] The data to be identified is input into a content recognition network for recognition and prediction to obtain the recognition result. The content recognition network is trained by the incremental content recognition network training method described in any one of the first aspects.

[0044] To achieve the above objectives, a third aspect of this application provides an incremental content recognition network training apparatus, comprising:

[0045] Forward propagation module: used to acquire data samples and data labels corresponding to the current task, input the data samples into the content recognition network for recognition processing, obtain the corresponding prediction output, calculate the current loss function based on the prediction output and data labels, and obtain the current gradient data through backpropagation based on the current loss function;

[0046] Causal effect determination module: used to calculate causal effect parameters for each model parameter in the content recognition network based on the current loss function, and obtain a causal effect matrix based on the causal effect parameters;

[0047] Gradient correlation measurement module: used to obtain at least one historical task before the current task, obtain corresponding historical gradient data for each historical task, calculate correlation projection based on the historical gradient data and the current gradient data, calculate task affinity based on the correlation projection and the historical gradient data, and calculate the influence factor of each historical task on the current task based on the correlation projection and the task affinity.

[0048] Gradient update module: used to update the current gradient data according to the influence factor and the causal effect matrix to obtain updated gradient data, and optimize the model parameters based on the updated gradient data until the training process of the content recognition network in the current task is completed.

[0049] To achieve the above objectives, a fourth aspect of the present application provides an electronic device, the electronic device including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method described in the first or second aspect.

[0050] To achieve the above objectives, a fifth aspect of the present application provides a storage medium that stores a computer program, which, when executed by a processor, implements the method described in the first or second aspect.

[0051] The incremental content recognition network training method, content recognition method, and apparatus proposed in this application involve acquiring data samples and data labels corresponding to the current task, inputting the data samples into the content recognition network for recognition processing to obtain the corresponding prediction output, calculating the current loss function based on the prediction output and data labels, obtaining the current gradient data through backpropagation of the current loss function, calculating causal effect parameters based on the current loss function for each model parameter in the content recognition network, obtaining a causal effect matrix based on the causal effect parameters, acquiring at least one historical task prior to the current task, acquiring the corresponding historical gradient data for each historical task, calculating the relevance projection based on the historical gradient data and the current gradient data, calculating the task affinity based on the relevance projection and the historical gradient data, calculating the influence factor of each historical task on the current task based on the relevance projection and the task affinity, updating the current gradient data based on the influence factor and the causal effect matrix to obtain updated gradient data, and optimizing the model parameters based on the updated gradient data until the training process of the content recognition network for the current task is completed. This application's embodiments quantify the impact of different model parameters on the loss function of the current task by calculating the causal effect parameters of each model parameter and constructing a causal effect matrix. This eliminates the need to update all model parameters during fine-tuning, allowing selective focus on model parameters that significantly affect the current task, reducing computational load and improving fine-tuning efficiency. Simultaneously, for historical tasks preceding the current task, the correlation projection between their data sample sets and the current gradient data is calculated. Combined with historical gradient data, task affinity is obtained, and the influence factor of each historical task on the current task is calculated. Based on this influence factor and the causal effect matrix, the current gradient data is updated, achieving knowledge transfer and providing regularization constraints for the optimization process. This avoids overfitting on small samples and ensures the semantic continuity of parameter updates. Through these improvements, the accuracy of fine-tuning can be maintained while reducing gradient computation and accelerating iterative convergence. Attached Figure Description

[0052] Figure 1 This is a flowchart of the incremental content recognition network training method provided in the embodiments of this application.

[0053] Figure 2 This is a flowchart of calculating causal effect parameters based on the current loss function, provided in an embodiment of this application.

[0054] Figure 3 This is a flowchart of calculating the correlation projection based on the data sample set and the current gradient data, provided in an embodiment of this application.

[0055] Figure 4 This is a flowchart of calculating task affinity based on correlation projection and historical gradient data provided in an embodiment of this application.

[0056] Figure 5 The flowchart illustrates the standardization process of similarity values ​​to obtain affinity values, as provided in this embodiment of the application.

[0057] Figure 6 This is a flowchart provided in an embodiment of the present application for calculating the influence factor of each historical task on the current task based on relevance projection and task affinity.

[0058] Figure 7 This is a flowchart provided in an embodiment of the present application for updating the current gradient data based on the influence factor and the causal effect matrix to obtain the updated gradient data.

[0059] Figure 8 This is a flowchart illustrating the optimization of model parameters based on updated gradient data, as provided in this application.

[0060] Figure 9 This is a flowchart provided in an embodiment of the present application for obtaining the low-rank data corresponding to the time step based on the updated gradient data, the first low-rank matrix and the second low-rank matrix corresponding to the previous time step.

[0061] Figure 10 This is an overall schematic diagram of the incremental content recognition network training method provided in the embodiments of this application.

[0062] Figure 11 This is a structural block diagram of an incremental content recognition network training device provided in another embodiment of this application.

[0063] Figure 12 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0064] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0065] It should be noted that although functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart.

[0066] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0067] First, let's analyze some of the terms used in this application:

[0068] Artificial Intelligence (AI) is a new branch of computer science that studies, develops, and applies theories, methods, technologies, and systems to simulate, extend, and expand human intelligence. It aims to understand the essence of intelligence and produce intelligent machines that can react in a way similar to human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing, and expert systems. AI can simulate the information processes of human consciousness and thought. Furthermore, AI utilizes digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceiving the environment, acquiring knowledge, and using that knowledge to achieve optimal results.

[0069] With the development of network information technology, network content exhibits characteristics of continuous increase and dynamic change. Machine learning content recognition models are prone to catastrophic forgetting when processing incremental network data; that is, the model's ability to retain knowledge from previous tasks significantly declines when learning new task knowledge, leading to a degradation in recognition performance. This deficiency is particularly prominent in scenarios such as network text recognition and abnormal network traffic analysis. However, pre-trained large models possess rich pre-training knowledge and powerful learning and reasoning capabilities. Therefore, in processing incremental network data, a parameter-efficient fine-tuning technique based on pre-trained models is proposed to reduce the impact of catastrophic forgetting.

[0070] To address the catastrophic forgetting problem in incremental network content recognition, related technologies employ methods such as prompt-tuning, adapter-tuning, and low-rank adaptation (LoRA) for parameter fine-tuning. These methods mitigate the catastrophic forgetting problem by limiting the modifications made to the old task model by the new task model update, achieving performance comparable to full-parameter training with only a few parameter adjustments. However, this parameter constraint strategy hinders reverse knowledge transfer, negatively impacting the overall performance of the task sequence. In real-world scenarios, the learned parameters for the new task do not necessarily interfere with the old task. When a positive correlation exists between the new and old tasks, it may even promote reverse knowledge transfer, providing useful information for the old task and thus better mitigating catastrophic forgetting. This parameter fine-tuning method treats all parameters uniformly, but in practical applications, the effects of different parameters vary across different stages of the task, and some parameters may be invalid. Furthermore, as the number of tasks increases, the parameter solution space of the new task becomes constrained by the old task, leading to a performance degradation. In this way, full-scale fine-tuning not only fails to improve task performance, but also reduces the efficiency of the parameter fine-tuning process.

[0071] Based on this, embodiments of this application provide an incremental content recognition network training method, content recognition method, and apparatus. By calculating the causal effect parameters of each model parameter and constructing a causal effect matrix, the influence of different model parameters on the loss function of the current task is quantified. This allows the fine-tuning process to selectively focus on the model parameters that significantly affect the current task without updating all model parameters, reducing computational load and improving fine-tuning efficiency. Simultaneously, for historical tasks preceding the current task, the correlation projection between their data sample sets and the current gradient data is calculated, and the task affinity is obtained by combining the historical gradient data. Furthermore, the influence factor of each historical task on the current task is calculated. Based on this influence factor and the causal effect matrix, the current gradient data is updated, achieving knowledge transfer and providing regularization constraints for the optimization process, avoiding small-sample overfitting and ensuring the semantic continuity of parameter updates. Through these improvements, the accuracy of fine-tuning can be maintained while reducing gradient computation and accelerating iterative convergence.

[0072] This application provides an incremental content recognition network training method, a content recognition method, and an apparatus, which are specifically described through the following embodiments. First, the incremental content recognition network training method in this application embodiment is described.

[0073] This application's embodiments can acquire and process relevant data based on artificial intelligence (AI) technology. AI is the theory, methods, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine that can react in a way similar to human intelligence. AI also studies the design principles and implementation methods of various intelligent machines, enabling them to possess perception, reasoning, and decision-making capabilities.

[0074] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0075] The incremental content recognition network training method provided in this application relates to the field of neural network technology. This method can be applied to a terminal, a server, or a computer program running on either the terminal or the server. For example, the computer program can be a native program or software module in an operating system; it can be a native application (APP), i.e., a program that needs to be installed in the operating system to run, such as a client supporting incremental content recognition network training, i.e., a program that only needs to be downloaded to a browser environment to run; or it can be a small program that can be embedded in any APP. In short, the above-mentioned computer program can be any form of application, module, or plugin. The terminal communicates with the server via a network. This incremental content recognition network training method can be executed by the terminal or the server, or by the terminal and the server working together.

[0076] In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, or smartwatch, etc. The server can be a standalone server, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms; it can also be a service node in a blockchain system, where the service nodes form a peer-to-peer (P2P) network. The P2P protocol is an application layer protocol running on top of the Transmission Control Protocol (TCP). The terminal and server can connect via Bluetooth, Universal Serial Bus (USB), or a network, etc., and this embodiment does not impose any limitations.

[0077] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0078] The incremental content recognition network training method in the embodiments of this application is described below.

[0079] Figure 1 This is an optional flowchart of the incremental content recognition network training method provided in the embodiments of this application. Figure 1 The method may include, but is not limited to, steps 110 to 140. It is also understood that this embodiment... Figure 1 The order of steps 110 to 140 is not specifically limited. The order of steps can be adjusted or some steps can be reduced or added according to actual needs.

[0080] Step 110: Obtain the data samples and data labels corresponding to the current task, input the data samples into the content recognition network for recognition processing, obtain the corresponding prediction output, calculate the current loss function based on the prediction output and data labels, and obtain the current gradient data through backpropagation based on the current loss function.

[0081] In one embodiment, an incremental content recognition network refers to a network that sequentially learns multiple tasks at different stages, effectively identifying the category of data for each task. During the learning process, the content recognition network does not receive all task information at once, but rather inputs data step-by-step according to the stage sequence of the tasks. After completing each stage of learning, it adapts to new tasks based on the accumulated knowledge. For example, in a medical image analysis scenario, the content recognition network can first learn basic organ structure recognition tasks, and then sequentially learn advanced tasks such as lesion detection and benign / malignant assessment.

[0082] Assume the task sequence is Each task Corresponding to an independent dataset in, Let j represent the j-th data sample. The data label corresponding to this data sample. Indicates the size of the dataset.

[0083] Based on the current task For example, its All of these are historical tasks corresponding to the current task, and This refers to the previous historical task for the current task. The ultimate goal of incremental learning is to train the content recognition network f. Θ This makes it not only in the current task Performed well in all historical missions. Maintain good performance.

[0084] At this point, the previous t-1 historical tasks have been trained, and the current task... At the start of training, the content recognition network for the current task is represented as f. Θ The parameter matrix formed by its model parameters is represented by Θ, and the corresponding current weight matrix is ​​represented as follows. The current weight matrix here is the weight matrix corresponding to the content recognition network after the previous task was trained. Simultaneously, it is used when learning the current task... Only in the dataset Training will be conducted on [the platform].

[0085] Dataset Corresponding data samples Input content recognition network f Θ Perform recognition processing to obtain the current gradient data. and the corresponding prediction output And based on the predicted output and data tags The current loss function is calculated. Represented as:

[0086]

[0087] in, This represents the current loss function.

[0088] Step 120: For each model parameter in the content recognition network, calculate the causal effect parameter based on the current loss function, and obtain the causal effect matrix based on the causal effect parameter.

[0089] In one embodiment, in order to quantize the model parameter Θ in the i-th row and j-th column of the parameter matrix Θ (i,j) The extent of its impact on the current task is quantified by calculating its corresponding causal effect parameters. (Refer to...) Figure 2 , Figure 2This is a flowchart of calculating causal effect parameters based on the current loss function provided in an embodiment of this application, specifically including the following steps:

[0090] Step 210: Obtain the weight parameters corresponding to each model parameter based on the current weight matrix corresponding to the current task.

[0091] In one embodiment, the model parameter Θ (i,j) For example, in the current weight matrix The corresponding weight parameters are expressed as follows:

[0092] Step 220: Perform first-order and second-order Taylor expansions on the weight parameters of the current loss function to obtain the first-order and second-order intermediate values.

[0093] In one embodiment, the current loss function is expanded using first-order and second-order Taylor expansions at the weight parameters, and the resulting first-order and second-order intermediate values ​​are expressed as follows: and

[0094] Step 230: Obtain the initial causal effect parameters based on the first product of the first-order intermediate value and the weight parameter, and the second product of the squares of the second-order intermediate value and the weight parameter. Normalize the initial causal effect parameters to obtain the causal effect parameters.

[0095] In one embodiment, the first product based on the first-order intermediate value and the weight parameter is expressed as:

[0096]

[0097] The second product of the squares of the second-order intermediate value and the weight parameter is expressed as:

[0098]

[0099] The initial causal effect parameters are expressed as follows:

[0100]

[0101] In essence, the embodiments of this application are for quantizing the given model parameter Θ in the parameter matrix. (i,j) Corresponding weight parameters The causal effect is first determined through the current weight matrix. The weight parameter is set to zero to perform causal intervention, generating a counterfactual weight matrix. Then, the causal effect of the model parameters is defined using the initial causal effect parameters. The causal effect refers to the change in training loss when the weight parameter is included versus when the weight parameter is removed.

[0102] Therefore, the initial causal effect parameter is expressed as:

[0103]

[0104] in, Indicates will Causal intervention operations set to zero, This indicates that there are no weight parameters. In the case of calculating the reverse fact loss function.

[0105] However, directly interfering with each weight parameter to calculate the causal effect incurs a very high computational cost. To address this issue, embodiments of this application modify the current weight matrix... A second-order Taylor expansion is used to approximate the counterfactual loss function, and the approximated initial causal effect parameters are expressed as follows:

[0106]

[0107] in,

[0108] Therefore, we can deduce that:

[0109]

[0110] In one embodiment, considering that directly calculating the second-order intermediate value would result in quadratic complexity, a diagonal Hessian approximation can be used to reduce the computational cost to linear time. This facilitates the approximate extension of the causal effect calculation. When the initial causal effect parameters... A positive value indicates that removing the weights corresponding to the model parameters will lead to an increase in training loss, suggesting that optimizing the model parameters is effective in improving task performance. Conversely, when the initial causal effect parameters... When the value is negative, the optimization of the model parameters is invalid.

[0111] Next, the initial causal effect parameters will be... Normalization is performed, and the Softmax function is used to map the initial causal effect parameters to a value between 0 and 1, thus obtaining the causal effect parameters. Represented as:

[0112]

[0113] Understandable A larger value indicates a greater causal effect of the model parameter on improving task performance. Prioritizing effective parameters with larger causal effects during the training of the current task can reduce the influence of ineffective parameters and enhance the model's ability to learn subsequent tasks.

[0114] Furthermore, we have the causal effect parameters corresponding to each model parameter. Then, the causal effect matrix corresponding to the parameter matrix can be obtained. Where, d I For the input dimension, d O For output dimensions.

[0115] Step 130: Obtain at least one historical task preceding the current task. For each historical task, obtain the corresponding historical gradient data. Calculate the correlation projection based on the historical gradient data and the current gradient data. Calculate the task affinity based on the correlation projection and the historical gradient data. Calculate the influence factor of each historical task on the current task based on the correlation projection and the task affinity.

[0116] In one embodiment, after obtaining the causal effect matrix corresponding to the parameter matrix, it is also necessary to accurately estimate the correlation between the current task and historical tasks to ensure that the causal effect matrix can be effectively combined with task relevance, thereby appropriately improving the historical tasks. To this end, this embodiment performs cross-task gradient adaptation, selectively updating weight parameters beneficial to historical tasks when learning the current task. Specifically, firstly, task relevance is quantified through cross-task gradient projection to determine whether knowledge transfer will occur. Then, task affinity is calculated through gradient-related relevance projection to determine whether knowledge transfer is a beneficial positive transfer or a harmful negative transfer. Based on the causal effect matrix, task relevance, and task affinity, gradient components beneficial to historical tasks in the current task are identified. Finally, by prioritizing the training of gradient components beneficial to historical tasks in the current task, the weight parameters of the current task are selectively updated, thereby achieving reverse knowledge transfer.

[0117] In one embodiment, the current task The corresponding historical mission is For each historical task, it is necessary to calculate its impact factor on the current task. (Taking historical tasks as an example...) Let's take an example. To determine whether the current task satisfies the condition of back-transferring knowledge to historical tasks, we use relevance projection to measure the relevance between the input subspaces of the current task and the historical tasks. (Refer to...) Figure 3 , Figure 3 This is a flowchart illustrating the calculation of correlation projection based on a set of data samples and current gradient data, provided in an embodiment of this application. The flowchart specifically includes the following steps:

[0118] Step 310: Perform singular value decomposition on the historical gradient data to obtain the basis information of the input subspace corresponding to the historical task.

[0119] In one embodiment, for historical tasks Its data sample set is its corresponding dataset. Where m is the number of data samples, n is the feature dimension of the data samples after feature extraction, and singular value decomposition is performed on the historical gradient data to obtain an orthogonal matrix. The column vectors of the diagonal matrix and orthogonal matrix U are left singular vectors, corresponding to the sample space basis of the input subspace, which is the basis information here, expressed as:

[0120]

[0121] Where r << min(d) I ,d O ) indicates rank.

[0122] Step 320: Obtain the correlation projection of the current task in the input subspace of the historical task based on the product of the basis information, the transpose of the basis information, and the current gradient data.

[0123] In one embodiment, based on base information transpose of base information and current gradient data The product yields the relevance projection of the current task onto the input subspace of the historical tasks. Represented as:

[0124]

[0125] Next, task affinity is calculated. Task affinity is used to evaluate whether the knowledge transfer from the current task to historical tasks is beneficial. Since the similarity of task gradients in the shared parameter space can reflect the relationship between tasks, task affinity is quantified by calculating the directional similarity between the projected gradient components of the current task and the gradients of historical tasks. (See reference...) Figure 4 , Figure 4 This is a flowchart of calculating task affinity based on correlation projection and historical gradient data provided in an embodiment of this application, specifically including the following steps:

[0126] Step 410: For each column in the historical gradient data, obtain the corresponding historical gradient column vector and the corresponding correlation column vector in the correlation projection.

[0127] In one embodiment, for historical tasks Its historical gradient data is The historical gradient column vector corresponding to its j-th column is represented as follows: Simultaneously, obtain the correlation column vector corresponding to the j-th column in the correlation projection.

[0128] Step 420: Calculate the first quotient of the correlation column vector and the corresponding first norm, the second quotient of the historical gradient column vector and the corresponding second norm, and obtain the corresponding similarity column vector by multiplying the first quotient and the second quotient.

[0129] In one embodiment, the first quotient of the correlation column vector and the corresponding first norm is represented as:

[0130]

[0131] Where |||2 represents the calculation of the 2-norm. This represents the first norm.

[0132] The second quotient of the historical gradient column vector and its corresponding second norm is expressed as:

[0133]

[0134] in, This represents the second norm.

[0135] Therefore, the similarity value corresponding to column j This represents the similarity between the j-th column of the gradients of the current task and historical tasks in the input subspace. The similarity value is expressed as:

[0136]

[0137] Step 430: Standardize the similarity values ​​to obtain the affinity column vector, and obtain the task affinity based on all affinity column vectors.

[0138] In one embodiment, reference is made to Figure 5 , Figure 5 The flowchart for standardizing similarity values ​​to obtain affinity values ​​provided in this application embodiment specifically includes the following steps:

[0139] Step 510: If the similarity value is greater than zero, obtain the affinity column vector based on the all-one matrix.

[0140] Step 520: If the similarity value is zero, obtain the affinity column vector based on the all-zero matrix.

[0141] Step 530: If the similarity value is less than zero, obtain the affinity column vector based on the all-negative-one matrix.

[0142] In one embodiment, the affinity column vector is represented as:

[0143]

[0144] Among them, set This set represents all possible values ​​for task affinity, where each element is a d-dimensional string. O The column vector has values ​​in the range {+1,0,-1}.

[0145] In one embodiment, once the affinity column vectors are obtained, the current task can be derived based on all affinity column vectors. With historical mission Task affinity Task affinity is used to indicate whether the current task has a positive transfer to historical tasks.

[0146] Next, refer to Figure 6 , Figure 6 This is a flowchart provided in an embodiment of the present application for calculating the influence factor of each historical task on the current task based on relevance projection and task affinity, specifically including the following steps:

[0147] Step 610: Calculate the quotient of the third norm corresponding to the correlation projection and the fourth norm corresponding to the current gradient data to obtain the task correlation.

[0148] In one embodiment, for the current task With historical mission Its correlation projection The corresponding third norm is expressed as:

[0149]

[0150] The fourth norm corresponding to the current gradient data is expressed as:

[0151]

[0152] Therefore, task relevance Represented as:

[0153]

[0154] Task relevance is used to represent any current task. and its gradient Its historical mission Relevance. Task relevance. Larger means In historical mission The larger the projection on the input subspace, the greater the overlap between the two task input subspaces, indicating a greater correlation.

[0155] Step 620: Calculate the correlation projection and the third quotient of the third norm, and obtain the gradient projection based on the product of the third quotient and the task correlation.

[0156] In one embodiment, the third quotient of the correlation projection and the third norm is expressed as:

[0157]

[0158] Therefore, the gradient projection is expressed as:

[0159]

[0160] Where · represents scalar multiplication.

[0161] Step 630: For each historical task, calculate the Hadamard product of task affinity and gradient projection to obtain the corresponding influence factor.

[0162] In one embodiment, for historical tasks The influence factor, derived from the Hadamard product of gradient projection and task affinity, is expressed as follows:

[0163]

[0164] in, This represents the element-wise (Hadamard product) product of matrices.

[0165] It's understandable that a Hadamard product exists for each historical task. Furthermore, if the current task is the first task, the corresponding current gradient data is obtained using the standard training method, and the weights are updated.

[0166] Step 140: Update the current gradient data based on the influence factor and causal effect matrix to obtain updated gradient data, and optimize the model parameters based on the updated gradient data until the training process of the content recognition network for the current task is completed.

[0167] In one embodiment, reference is made to Figure 7 , Figure 7 This application provides a flowchart for updating the current gradient data based on the influence factor and causal effect matrix to obtain updated gradient data, which specifically includes the following steps:

[0168] Step 710: Sum all the impact factors and add them to the all-one matrix to obtain the total impact factor.

[0169] Step 720: Calculate the Hadamard product of the causal effect matrix, the current gradient data, and the total influence factor in sequence to obtain the updated gradient data.

[0170] In one embodiment, for the current task, the total impact factor is obtained by summing all impact factors and adding a one matrix, and is expressed as follows:

[0171]

[0172] Therefore, updating the gradient data is represented as:

[0173]

[0174] in, This represents a matrix of all ones.

[0175] Based on the above updated gradient data, it can be seen that the updated gradient data in this embodiment of the application is... The current task can be adaptively adjusted during incremental learning. The current gradient data is adaptively updated to achieve effective reverse knowledge transfer. The specific update process is as follows.

[0176] The first scenario: If there is no significant correlation between the current task and historical tasks, that is, The value is very small (i.e.) There is no possibility of reverse knowledge transfer, or task affinity. When the value is zero, the total influence factor is zero, therefore the updated gradient data only includes... Since it affects the current gradient data, it will only update the weight parameters corresponding to the effective model parameters with strong causal effects, without considering the impact on historical tasks.

[0177] The second scenario: If there is a strong correlation between the current task and historical tasks, that is... Contains large values, but task affinity When the vector consists of -1s, it indicates that the current task is interfering with the historical task, and the parameters of the historical task will be protected to prevent catastrophic forgetting. The updated gradient data is represented as:

[0178]

[0179] By removing gradient components aligned with historical tasks in this way, residual gradients are preserved. Updating parameters based on these residual gradients helps mitigate catastrophic forgetting.

[0180] The third scenario: If there is a strong correlation between the current task and historical tasks, that is... Contains large values ​​and task affinity When the vector consists of 1s, it indicates that the current task has a positive effect on the historical task, and will selectively update the parameters in the current task that are beneficial to the historical task, thereby promoting reverse knowledge transfer.

[0181] The following describes how to use the updated gradient data for parameter updates in the current task. (Refer to...) Figure 8 , Figure 8 The flowchart provided in this application for optimizing model parameters based on updated gradient data specifically includes the following steps:

[0182] Step 810: At each time step, obtain the iterative weight matrix. Based on the updated gradient data, the first low-rank matrix and the second low-rank matrix corresponding to the previous time step, obtain the low-rank data corresponding to the current task. Based on the iterative weight matrix and the low-rank data, obtain the time step weight matrix corresponding to the time step. Use the time step weight matrix of the previous time step as the iterative weight matrix of the next time step.

[0183] In one embodiment, LoRA parameter fine-tuning constrains the model's training parameters by restricting the updates of the model's weight parameters to a low-rank subspace during the fine-tuning process. Assume the initial weight matrix of the pre-trained network layers of a given content recognition network is... Where d I For the input dimension, d O For the output dimension. Two low-rank matrices are introduced during the training process of each task: the first low-rank matrix and the second low-rank matrix. With the second low-rank matrix The initial weight matrix is ​​reparameterized, with the first low-rank matrix A used for dimensionality reduction and the second low-rank matrix B used to restore the dimensionality.

[0184] Understandably, during LoRA initialization, the second low-rank matrix B is set to an all-zero matrix, and the first low-rank matrix A is randomly initialized with a Gaussian distribution. During training, the initial weight matrix W of the pre-trained model is frozen, and only the first low-rank matrix A and the second low-rank matrix B are fine-tuned.

[0185] However, while LoRA can constrain parameter updates to a low-rank subspace, this low-rank constraint does not guarantee that all parameters will effectively improve task performance. This limitation is particularly pronounced in incremental learning scenarios, as the parameters of the current task are often constrained to protect historical tasks. When invalid parameters from historical tasks are used to constrain the current task, it not only fails to mitigate catastrophic forgetting but may also impair the performance of the current task. Therefore, this application's embodiments utilize updated gradient data for improvement.

[0186] With the current task For example, the forward propagation process of a content recognition network is modified as follows:

[0187]

[0188] Where h represents the input data sample, This indicates the predicted output.

[0189] Therefore, the current task The previous historical mission Corresponding weight matrix The update is performed based on the previous one, and the weight matrix is ​​frozen during the update process. Only update the first low-rank matrix A and the second low-rank matrix B.

[0190] Since the training process for the current task is divided into multiple time steps, the same iterative action is performed at each time step. Assume time step s+1 represents the (s+1)th update step, and the previous time step is s, corresponding to the update at time step s. Taking the (s+1)th time step as an example, the weight matrix output from the previous time step is obtained as the iterative weight matrix, with the initial value being the weight matrix corresponding to the previous historical task. Therefore, the iterative weight matrix for the (s+1)th time step is expressed as:

[0191]

[0192] Next, refer to Figure 9 , Figure 9 This is a flowchart provided in this application embodiment for obtaining the low-rank data corresponding to a time step based on the updated gradient data, the first low-rank matrix corresponding to the previous time step, and the second low-rank matrix, specifically including the following steps:

[0193] Step 910: Obtain the transposes of the first low-rank matrix and the second low-rank matrix corresponding to the previous time step, respectively, to obtain the first transpose matrix and the second transpose matrix.

[0194] In one embodiment, the first low-rank matrix and the second low-rank matrix corresponding to the previous time step are respectively represented as: and After calculating the transpose matrix, we obtain the first transpose matrix. Second transpose matrix

[0195] Step 920: Calculate the product of the preset learning rate, the updated gradient data, and the first transpose matrix to obtain the second changed data; calculate the product of the preset learning rate, the second transpose matrix, and the updated gradient data to obtain the first changed data.

[0196] In one embodiment, the first change data is obtained by calculating the product of the preset learning rate, the second transpose matrix, and the updated gradient data:

[0197]

[0198] The second change data is obtained by multiplying the preset learning rate, the updated gradient data, and the first transpose matrix:

[0199]

[0200] Step 930: Calculate the difference between the first low-rank matrix of the previous time step and the first changed data to obtain the first low-rank matrix corresponding to the time step.

[0201] In one embodiment, the first low-rank matrix corresponding to time step s+1 is:

[0202]

[0203] Step 940: Calculate the difference between the second low-rank matrix of the previous time step and the second changed data to obtain the second low-rank matrix corresponding to the time step.

[0204] In one embodiment, the second low-rank matrix corresponding to time step s+1 is:

[0205]

[0206] Step 950: Obtain the low-rank data by multiplying the first low-rank matrix and the second low-rank matrix corresponding to the time step.

[0207] In one embodiment, the low-rank data corresponding to time step s+1 is:

[0208]

[0209] Therefore, the time step weight matrix corresponding to time step s+1 obtained from the iterative weight matrix and low-rank data is expressed as:

[0210]

[0211] Step 820: Repeat the iteration until the time step weight matrix corresponding to the last time step is obtained for model parameter optimization.

[0212] In one embodiment, the weight matrix of the previous time step is used as the iterative weight matrix for the next time step. This iterative process is repeated until the weight matrix corresponding to the last time step is obtained. Based on this, the weight parameters of the model are optimized and adjusted. Thus, the content recognition network performs its current task... The training process has ended.

[0213] In one embodiment, reference is made to Figure 10 , Figure 10 This is an overall schematic diagram of the incremental content recognition network training method provided in the embodiments of this application.

[0214] As shown in the figure, based on LoRA parameter fine-tuning, during the training process of each task, the weight matrix W completed by the previous task is frozen as the pre-trained weights, and then the first low-rank matrix corresponding to the previous task is... With the second low-rank matrix Based on this, parameter fine-tuning is performed. At this point, counterfactual parameters and factual parameters are determined based on the causal effect matrix. Factual parameters are considered valid parameters, while counterfactual parameters are considered invalid parameters.

[0215] Next, for each historical task, the corresponding data sample set and historical gradient data are obtained. Based on the data sample set and current gradient data, a correlation projection is calculated to determine the correlation projection and task affinity between the historical and current tasks. The task affinity value ranges from {+1, 0, -1}, where 1 represents positive affinity, 0 represents no affinity, and -1 represents negative affinity. Furthermore, if the correlation projection indicates a weak correlation between tasks, it is treated similarly to no affinity.

[0216] Then, an adaptive gradient data update process is performed. If there is no significant correlation between the current task and historical tasks, or if the task affinity is zero, only the weight parameters corresponding to the effective model parameters with strong causal effects are updated, without considering the impact on historical tasks—that is, no transfer occurs. If there is a strong correlation between the current task and historical tasks, but the task affinity is a vector consisting of -1 values, the parameters of the historical tasks are protected to prevent catastrophic forgetting. If there is a strong correlation between the current task and historical tasks, and the task affinity is a vector consisting of 1 values, the parameters in the current task that are beneficial to the historical tasks are selectively updated, thereby achieving reverse knowledge transfer.

[0217] Therefore, in order to achieve reverse knowledge transfer of incremental content recognition networks based on efficient parameter fine-tuning, thereby alleviating the catastrophic forgetting problem and improving the effectiveness of incremental network content recognition, this application proposes an incremental network content recognition framework based on causal low-rank fine-tuning. Based on the efficient parameter fine-tuning framework based on low-rank adaptation, a parameter-level counterfactual attribution method and a cross-task gradient update method are designed.

[0218] Specifically, the parameter-level counterfactual attribution method estimates the causal effect of LoRA parameters through causal intervention and counterfactual inference, enabling the content recognition network to identify effective parameters with strong causal effects on the current task while reducing the influence of ineffective parameters, thereby enhancing its ability to learn the current task. The cross-task gradient adaptation method measures the correlation between tasks through gradient projection and calculates task affinity based on gradient similarity in the task space. By jointly utilizing causal effects, task correlation, and task affinity, the cross-task gradient adaptation method adaptively adjusts task gradients, achieving effective reverse knowledge transfer without data replay. Unlike methods that mitigate catastrophic forgetting by restricting modifications to historical tasks by the current task, the embodiments of this application enable reverse knowledge transfer in incremental network content recognition with efficient parameter fine-tuning, thereby mitigating the catastrophic forgetting problem and improving the effectiveness of incremental network content recognition.

[0219] The technical solution provided in this application involves acquiring data samples and data labels corresponding to the current task, inputting the data samples into a content recognition network for recognition processing, obtaining the corresponding prediction output, calculating the current loss function based on the prediction output and data labels, obtaining the current gradient data through backpropagation of the current loss function, calculating causal effect parameters based on the current loss function for each model parameter in the content recognition network, obtaining a causal effect matrix based on the causal effect parameters, acquiring at least one historical task prior to the current task, acquiring the corresponding historical gradient data for each historical task, calculating the relevance projection based on the historical gradient data and the current gradient data, calculating the task affinity based on the relevance projection and the historical gradient data, calculating the influence factor of each historical task on the current task based on the relevance projection and the task affinity, updating the current gradient data based on the influence factor and the causal effect matrix to obtain updated gradient data, and optimizing the model parameters based on the updated gradient data until the training process of the content recognition network for the current task is completed. This application's embodiments quantify the impact of different model parameters on the loss function of the current task by calculating the causal effect parameters of each model parameter and constructing a causal effect matrix. This eliminates the need to update all model parameters during fine-tuning, allowing selective focus on model parameters that significantly affect the current task, reducing computational load and improving fine-tuning efficiency. Simultaneously, for historical tasks preceding the current task, the correlation projection between their data sample sets and the current gradient data is calculated. Combined with historical gradient data, task affinity is obtained, and the influence factor of each historical task on the current task is calculated. Based on this influence factor and the causal effect matrix, the current gradient data is updated, achieving knowledge transfer and providing regularization constraints for the optimization process. This avoids overfitting on small samples and ensures the semantic continuity of parameter updates. Through these improvements, the accuracy of fine-tuning can be maintained while reducing gradient computation and accelerating iterative convergence.

[0220] In one embodiment, a content recognition method is also provided, specifically described as follows: acquiring data to be recognized; inputting the data to be recognized into a content recognition network for recognition prediction, and obtaining a recognition result. The content recognition network here can perform recognition of various tasks, and is trained using the incremental content recognition network training method described in any of the above embodiments. It is understood that the content recognition here can be different types of data such as image recognition, text recognition, and audio recognition.

[0221] This application also provides an incremental content recognition network training device, which can implement the above-described incremental content recognition network training method, referring to... Figure 11 The device includes:

[0222] Forward propagation module 1110: It is used to obtain the data samples and data labels corresponding to the current task, input the data samples into the content recognition network for recognition processing, obtain the corresponding prediction output, calculate the current loss function based on the prediction output and data labels, and obtain the current gradient data through backpropagation based on the current loss function.

[0223] Causal effect determination module 1120: for each model parameter in the content recognition network, it calculates the causal effect parameters based on the current loss function and obtains the causal effect matrix based on the causal effect parameters.

[0224] The gradient correlation measurement module 1130 is used to obtain at least one historical task before the current task, obtain the corresponding historical gradient data for each historical task, calculate the correlation projection based on the historical gradient data and the current gradient data, calculate the task affinity based on the correlation projection and the historical gradient data, and calculate the influence factor of each historical task on the current task based on the correlation projection and the task affinity.

[0225] Gradient update module 1140: It is used to update the current gradient data according to the influence factor and causal effect matrix to obtain updated gradient data, and optimize the model parameters based on the updated gradient data until the training process of the content recognition network in the current task is completed.

[0226] The specific implementation of the incremental content recognition network training device in this embodiment is basically the same as the specific implementation of the incremental content recognition network training method described above, and will not be repeated here.

[0227] This application also provides an electronic device, including:

[0228] At least one memory;

[0229] At least one processor;

[0230] At least one program;

[0231] The program is stored in a memory, and the processor executes the at least one program to implement the incremental content recognition network training method described above. The electronic device can be any smart terminal, including mobile phones, tablets, personal digital assistants (PDAs), and in-vehicle computers.

[0232] Please see Figure 12 , Figure 12 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes:

[0233] The processor 1201 can be implemented using a general-purpose central processing unit (CPU), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0234] The memory 1202 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 1202 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1202 and is called and executed by the processor 1201 to execute the incremental content recognition network training method, or the content recognition method, of the embodiments of this application.

[0235] The input / output interface 1203 is used to implement information input and output;

[0236] The communication interface 1204 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0237] Bus 1205 transmits information between various components of the device (e.g., processor 1201, memory 1202, input / output interface 1203, and communication interface 1204);

[0238] The processor 1201, memory 1202, input / output interface 1203 and communication interface 1204 are connected to each other within the device via bus 1205.

[0239] This application embodiment also provides a storage medium that stores a computer program. When the computer program is executed by a processor, it implements the above-described incremental content recognition network training method, s or content recognition method.

[0240] Memory, as a non-transitory storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0241] The incremental content recognition network training method, content recognition method, and apparatus proposed in this application involve acquiring data samples and data labels corresponding to the current task, inputting the data samples into the content recognition network for recognition processing, obtaining the corresponding prediction output, calculating the current loss function based on the prediction output and data labels, obtaining the current gradient data through backpropagation of the current loss function, calculating causal effect parameters based on the current loss function for each model parameter in the content recognition network, obtaining a causal effect matrix based on the causal effect parameters, acquiring at least one historical task prior to the current task, acquiring the corresponding historical gradient data for each historical task, calculating the relevance projection based on the historical gradient data and the current gradient data, calculating the task affinity based on the relevance projection and the historical gradient data, calculating the influence factor of each historical task on the current task based on the relevance projection and the task affinity, updating the current gradient data based on the influence factor and the causal effect matrix to obtain updated gradient data, and optimizing the model parameters based on the updated gradient data until the training process of the content recognition network for the current task is completed. This application's embodiments quantify the impact of different model parameters on the loss function of the current task by calculating the causal effect parameters of each model parameter and constructing a causal effect matrix. This eliminates the need to update all model parameters during fine-tuning, allowing selective focus on model parameters that significantly affect the current task, reducing computational load and improving fine-tuning efficiency. Simultaneously, for historical tasks preceding the current task, the correlation projection between their data sample sets and the current gradient data is calculated. Combined with historical gradient data, task affinity is obtained, and the influence factor of each historical task on the current task is calculated. Based on this influence factor and the causal effect matrix, the current gradient data is updated, achieving knowledge transfer and providing regularization constraints for the optimization process. This avoids overfitting on small samples and ensures the semantic continuity of parameter updates. Through these improvements, the accuracy of fine-tuning can be maintained while reducing gradient computation and accelerating iterative convergence.

[0242] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0243] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0244] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0245] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0246] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0247] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0248] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0249] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0250] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0251] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0252] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A method for training an incremental content recognition network, characterized in that, include: Obtain the data sample and data label corresponding to the current task, input the data sample into the content recognition network for recognition processing, obtain the corresponding prediction output, calculate the current loss function based on the prediction output and data label, and obtain the current gradient data through backpropagation of the current loss function. For each model parameter in the content recognition network, a causal effect parameter is calculated based on the current loss function, and a causal effect matrix is ​​obtained based on the causal effect parameter. Obtain at least one historical task preceding the current task; for each historical task, obtain corresponding historical gradient data; calculate a correlation projection based on the historical gradient data and the current gradient data; calculate task affinity based on the correlation projection and the historical gradient data; and calculate the influence factor of each historical task on the current task based on the correlation projection and the task affinity. The current gradient data is updated based on the influencing factor and the causal effect matrix to obtain updated gradient data. The model parameters are then optimized based on the updated gradient data until the training process of the content recognition network for the current task is completed.

2. The incremental content recognition network training method according to claim 1, characterized in that, The calculation of causal effect parameters based on the current loss function includes: Based on the current weight matrix corresponding to the current task, obtain the weight parameters corresponding to each model parameter; The current loss function is expanded using first-order and second-order Taylor expansions at the weight parameters to obtain first-order and second-order intermediate values. The initial causal effect parameter is obtained by first multiplying the first-order intermediate value and the weight parameter, and second multiplying the second-order intermediate value and the square of the weight parameter. The initial causal effect parameter is then normalized to obtain the causal effect parameter.

3. The incremental content recognition network training method according to claim 1, characterized in that, The step of calculating the correlation projection based on the historical gradient data and the current gradient data includes: Singular value decomposition is performed on the historical gradient data to obtain the basis information of the input subspace corresponding to the historical task; The correlation projection of the current task in the input subspace of the historical task is obtained by multiplying the basis information, the transpose of the basis information, and the current gradient data.

4. The incremental content recognition network training method according to claim 3, characterized in that, The step of calculating task affinity based on the correlation projection and the historical gradient data includes: For each column in the historical gradient data, obtain the corresponding historical gradient column vector and the corresponding correlation column vector in the correlation projection; Calculate the first quotient of the correlation column vector and the corresponding first norm, and the second quotient of the historical gradient column vector and the corresponding second norm. Obtain the corresponding similarity value based on the product of the first quotient and the second quotient. The similarity values ​​are standardized to obtain an affinity column vector, and the task affinity is obtained based on all the affinity column vectors.

5. The incremental content recognition network training method according to claim 4, characterized in that, The standardization process for the similarity values, resulting in an affinity column vector, includes: If the similarity value is greater than zero, the affinity column vector is obtained based on the all-one matrix; If the similarity value is equal to zero, the affinity column vector is obtained from the all-zero matrix; If the similarity value is less than zero, the affinity column vector is obtained based on the all-negative-one matrix.

6. The incremental content recognition network training method according to claim 1, characterized in that, The step of calculating the influence factor of each historical task on the current task based on the correlation projection and the task affinity includes: The task correlation is obtained by calculating the quotient of the third norm corresponding to the correlation projection and the fourth norm corresponding to the current gradient data; Calculate the third quotient of the correlation projection and the third norm, and obtain the gradient projection based on the product of the third quotient and the task correlation. For each historical task, the Hadamard product of the task affinity and the gradient projection is calculated to obtain the corresponding influence factor.

7. The incremental content recognition network training method according to claim 1, characterized in that, The step of updating the current gradient data based on the influencing factor and the causal effect matrix to obtain updated gradient data includes: Summing up all the aforementioned impact factors and adding them to the all-one matrix yields the total impact factor; The updated gradient data is obtained by sequentially calculating the Hadamard product of the causal effect matrix, the current gradient data, and the total influence factor.

8. The incremental content recognition network training method according to claim 1, characterized in that, The optimization of the model parameters based on the updated gradient data includes: At each time step, the iterative weight matrix is ​​obtained. Based on the updated gradient data, the first low-rank matrix and the second low-rank matrix corresponding to the previous time step, the low-rank data corresponding to the time step is obtained. Based on the iterative weight matrix and the low-rank data, the time step weight matrix corresponding to the time step is obtained. The time step weight matrix of the previous time step is used as the iterative weight matrix of the next time step. The initial value of the iterative weight matrix is ​​the weight matrix corresponding to the previous historical task. Repeat the iteration until the weight matrix of the last time step is obtained, and then optimize the model parameters.

9. The incremental content recognition network training method according to claim 8, characterized in that, The step of obtaining the low-rank data corresponding to the time step based on the updated gradient data and the first low-rank matrix and the second low-rank matrix corresponding to the previous time step includes: Obtain the transposes of the first low-rank matrix and the second low-rank matrix corresponding to the previous time step, respectively, to obtain the first transpose matrix and the second transpose matrix; Calculate the product of the preset learning rate, the updated gradient data, and the first transpose matrix to obtain the second changed data; calculate the product of the preset learning rate, the second transpose matrix, and the updated gradient data to obtain the first changed data. Calculate the difference between the first low-rank matrix of the previous time step and the first changed data to obtain the first low-rank matrix corresponding to the time step; The difference between the second low-rank matrix of the previous time step and the second changed data is calculated to obtain the second low-rank matrix corresponding to the time step; The low-rank data is obtained by multiplying the first low-rank matrix and the second low-rank matrix corresponding to the time step.

10. A content recognition method, characterized in that, include: Obtain the data to be identified; The data to be identified is input into the content recognition network for recognition and prediction to obtain the recognition result. The content recognition network is trained by the incremental content recognition network training method according to any one of claims 1 to 9.

11. An incremental content recognition network training device, characterized in that, include: Forward propagation module: used to acquire data samples and data labels corresponding to the current task, input the data samples into the content recognition network for recognition processing, obtain the corresponding prediction output, calculate the current loss function based on the prediction output and data labels, and obtain the current gradient data through backpropagation based on the current loss function; Causal effect determination module: used to calculate causal effect parameters for each model parameter in the content recognition network based on the current loss function, and obtain a causal effect matrix based on the causal effect parameters; Gradient correlation measurement module: used to obtain at least one historical task before the current task, obtain corresponding historical gradient data for each historical task, calculate correlation projection based on the historical gradient data and the current gradient data, calculate task affinity based on the correlation projection and the historical gradient data, and calculate the influence factor of each historical task on the current task based on the correlation projection and the task affinity. Gradient update module: used to update the current gradient data according to the influence factor and the causal effect matrix to obtain updated gradient data, and optimize the model parameters based on the updated gradient data until the training process of the content recognition network in the current task is completed.

12. An electronic device, characterized in that, The electronic device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the incremental content recognition network training method according to any one of claims 1 to 9, or the content recognition method according to claim 10.

13. A storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the incremental content recognition network training method according to any one of claims 1 to 9, or the content recognition method according to claim 10.

Images