A capacity incremental expansion method and system of a neural network model and related products

Abstract

The application discloses a capacity incremental expansion method and system of a neural network model and related products, and the method comprises the following steps: acquiring a sample set and first parameters of each layer structure in a neural network model; inputting the sample set into the neural network model to obtain output features of the first parameters; assigning second parameters to each layer structure in the neural network model; connecting the second parameters with the first parameters based on the output features of the first parameters to obtain a neural network model after parameter connection; and training the neural network model after parameter connection based on the sample set to obtain a neural network model after capacity incremental expansion. The application guarantees the effectiveness of each parameter, avoids the occurrence of multiple repeated parameters in the same layer structure, reduces the size of the neural network model, and reduces the memory requirement of a computing device deploying the neural network model.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method, system, and related products for incremental capacity expansion of neural network models. Background Technology

[0002] With the continuous advancement of artificial intelligence technology, the development of neural networks is also progressing rapidly. Deploying neural network models on terminals such as mobile and embedded devices has become an important trend in AI applications. However, in this era of information overload, to ensure that AI can continuously meet user needs, neural network models must be updated regularly.

[0003] There are two existing methods for incrementally expanding the capacity of neural networks: The first is to add a subnetwork to the model when a new task appears, enabling it to learn the new task. However, this method requires a large amount of memory from the computing device, as each subnetwork needs independent storage space. The second method allows the model to dynamically expand the network according to the task size, and provides task indices during training and inference to protect the relevant parameters of previous tasks and reduce interference between tasks. However, this method violates the principle of generality of large models, because the boundaries between tasks processed by large models are usually fuzzy, making it impossible to provide specific knowledge sources for batches. Furthermore, this method significantly limits the feasibility of incremental learning in practical applications. Summary of the Invention

[0004] This application provides a method, system, and related products for incremental capacity expansion of neural network models, aiming to reduce the memory required for incremental capacity expansion of models while adhering to the principle of generality of large models, thereby reducing the memory requirements of computing devices that deploy neural network models.

[0005] The first aspect of this application provides a method for incremental capacity expansion of a neural network model, the method being executed by a system for incremental capacity expansion of the neural network model, the system being deployed on or including at least one computing device, the method comprising:

[0006] Obtain a sample set and the first parameters of each layer in the neural network model, the neural network model including a multi-layer structure, the neural network model being used to process input data by running the multi-layer structure through the hardware in the computing device;

[0007] The sample set is input into the neural network model to obtain the output features of the first parameter of each layer in the neural network model;

[0008] Assign a second parameter to each layer in the neural network model;

[0009] Based on the output features of the first parameter of each layer in the neural network model, a connection is established between the second parameter of each layer in the neural network model and the first parameter of each layer in the neural network model to obtain a neural network model with parameter connections.

[0010] The neural network model with the parameters connected is trained based on the sample set to obtain a neural network model with incremental capacity expansion; wherein, during the training process, the first parameter of each layer in the neural network model is frozen.

[0011] A second aspect of this application provides a capacity incremental expansion system for a neural network model, wherein the capacity incremental expansion system for the neural network model is deployed on or includes at least one computing device, and the system comprises:

[0012] An acquisition module is used to acquire a sample set and the first parameters of each layer in the neural network model, wherein the neural network model includes a multi-layer structure, and the neural network model is used to process input data by running the multi-layer structure through the hardware in the computing device;

[0013] The output feature determination module is used to input the sample set into the neural network model to obtain the output features of the first parameter of each layer structure in the neural network model;

[0014] An allocation module is used to allocate a second parameter to each layer in the neural network model;

[0015] The connection module is used to establish a connection between the second parameter of each layer in the neural network model and the first parameter of each layer in the neural network model based on the output features of the first parameter of each layer in the neural network model, so as to obtain a neural network model with parameter connection.

[0016] The capacity incremental expansion module is used to train the neural network model after parameter concatenation based on the sample set to obtain the neural network model after capacity incremental expansion; wherein, during the training process, the first parameter of each layer in the neural network model is frozen.

[0017] A third aspect of this application provides a computer device, the device comprising a processor and a memory:

[0018] The memory is used to store computer programs and to transfer the computer programs to the processor;

[0019] The processor is configured to execute the steps of the capacity incremental expansion method for the neural network model provided in the first aspect according to the instructions in the computer program.

[0020] A fourth aspect of this application provides a computer-readable storage medium for storing a computer program that, when executed, implements the steps of the capacity incremental expansion method for the neural network model provided in the first aspect.

[0021] The fifth aspect of this application provides a computer program product, including a computer program that, when executed, implements the steps of the capacity incremental expansion method for the neural network model provided in the first aspect.

[0022] As can be seen from the above technical solutions, the embodiments of this application have the following advantages:

[0023] This application is executed by a capacity incremental expansion system for a neural network model, which is deployed on or includes at least one computing device. The process involves: obtaining a sample set and the first parameters of each layer in the neural network model, which includes multiple layers and is used to process input data by running these multiple layers through hardware on the computing device; inputting the sample set into the neural network model to obtain the output features of the first parameters of each layer; assigning second parameters to each layer in the neural network model; establishing connections between the second parameters and the first parameters of each layer based on the output features of the first parameters, resulting in a parameter-connected neural network model; and training the parameter-connected neural network model based on the sample set to obtain a capacity-increased neural network model. During training, the first parameters of each layer in the neural network model are frozen.

[0024] This application assigns a second parameter to each layer of the neural network model, replacing the existing method of establishing subnetworks for the neural network model. This effectively reduces the overall size of the neural network model and lowers the memory requirements of the computing devices that deploy the neural network model. Furthermore, this application establishes a connection between the second parameter and the first parameter of each layer in the neural network model, replacing the existing method of providing task numbers during training and inference, thus ensuring the versatility of large models.

[0025] In summary, this application assigns a second parameter to each layer of the neural network model and establishes a connection between the second parameter and the first parameter. The neural network model with these parameter connections is then trained to obtain a capacity-incremented neural network model. This allows the capacity-incremented neural network model to learn features of new tasks while retaining features of the original tasks. Furthermore, when executing a new task, both the first and second parameters in the capacity-incremented neural network model are simultaneously invoked, ensuring the effectiveness of each parameter, avoiding multiple duplicate parameters in the same layer, reducing the size of the neural network model, and consequently reducing the memory requirements of the computing devices deploying the neural network model. Attached Figure Description

[0026] Figure 1 This is a schematic diagram of a method for incrementally expanding the capacity of a neural network.

[0027] Figure 2 A flowchart illustrating a method for incremental capacity expansion of a neural network model provided in this application embodiment;

[0028] Figure 3 This is a neural network model with parameters connected but without filtering the first parameter, provided in an embodiment of this application.

[0029] Figure 4 This application provides a neural network model for filtering the first parameter after parameter concatenation.

[0030] Figure 5 A schematic diagram of the structure of a capacity incremental expansion system for a neural network model provided in an embodiment of this application;

[0031] Figure 6 A schematic diagram of a server structure provided in an embodiment of this application;

[0032] Figure 7 This is a schematic diagram of a terminal device structure provided in an embodiment of this application. Detailed Implementation

[0033] Both existing methods for incrementally expanding the capacity of neural networks have drawbacks. The first method requires the establishment of multiple sub-networks, which places high demands on the memory of computing devices. The second method requires a sequence of tasks, which violates the principle of generality of large models.

[0034] The first method for incremental expansion of neural network capacity adopts the mainstream strategy for model expansion. It mainly involves allocating an independent sub-neural network for each downstream task and integrating the output features of all neural networks in the last layer. During training, only the newly initialized network parameters can be updated. This update strategy ensures that the parameters of the newly learned task will not overwrite the knowledge of the previous task, which greatly alleviates the problem of catastrophic forgetting. Figure 1 This is a schematic diagram of a method for incrementally expanding the capacity of a neural network, as shown below. Figure 1 The three sub-neural networks shown are independent, and the task features they learn are also different. This approach initializes a separate sub-neural network for each sub-task, which ensures that each sub-task can obtain sufficient parameter resources, while reducing signal interference between different tasks and achieving good performance in a fixed scenario.

[0035] However, this method limits the flexibility of incremental model capacity expansion. In industrial applications, continuous learning is usually based on dynamic updates of large-scale pre-trained models to adapt to new business scenarios and update outdated knowledge-based data. However, this method requires initializing a new network with the same architecture for each new task, which is often costly and unacceptable in terms of resource consumption and computational cost.

[0036] At the same time, such as Figure 1 As shown, to reduce the catastrophic forgetting problem caused by crosstalk between tasks, the newly initialized sub-neural network does not share features or data with the previous sub-neural network. This design approach violates the general idea of ​​pre-training and fine-tuning. Since the parameters of different tasks are independent of each other, the downstream tasks that are subsequently learned cannot utilize the knowledge learned during pre-training, which greatly limits the generality of the model.

[0037] The second approach to incrementally expanding neural network capacity employs a finer-grained model expansion strategy. This strategy focuses on learning features for a sub-task using a set of parameters. When training for a new task, a masking mechanism is used to conceal irrelevant parameters, training only those relevant to the new task. During model training and inference, the task number is provided. The model determines the relevant parameter set based on the task number and uses the masking mechanism to hide irrelevant parameters. This approach significantly improves feature reuse across tasks, reduces interference between tasks, and allows the model to scale the network according to task size.

[0038] However, to address the interference between different tasks, this approach requires displaying the task number for each task during model training and inference to protect relevant parameters from previous tasks and reduce inter-task interference. However, providing task numbers during model training and inference violates the principle of generality for large models. Large models often lack clear boundaries between tasks, such as real-time updated world knowledge, and users cannot be required to specify the exact source of the knowledge needed by the model. This approach significantly limits the application of incremental methods in real-world scenarios.

[0039] In summary, to ensure that incremental model capacity expansion methods adhere to the principle of generality of large models while reducing the memory required for incremental model capacity expansion, thereby reducing the memory requirements of computing devices deploying neural network models, it is necessary to simultaneously meet the requirements that the model can dynamically adjust the scale of incremental model capacity based on the complexity of new tasks (data), that new model parameters can utilize knowledge learned from existing models (i.e., backward knowledge transfer), and that interference between different tasks should be minimized. During training, the similarity between knowledge from different tasks should be used to simplify the training process, while existing parameters should not be modified to avoid catastrophic forgetting of previous tasks.

[0040] In view of the above problems, this application provides a method, system, and related products for incremental capacity expansion of neural network models. In the technical solution provided in this application, a second parameter is assigned to each layer of the neural network model, and a connection between the first and second parameters is established based on the output features of the first parameter of each layer, resulting in a parameter-connected neural network model. Finally, the parameter-connected neural network model is trained based on a sample set to obtain a capacity-increased neural network model. This allows the capacity-increased neural network model to simultaneously utilize both the first and second parameters when performing new tasks, ensuring the effectiveness of each parameter, avoiding multiple duplicate parameters in the same layer, reducing the size of the neural network model, and thus reducing the memory requirements of the computing devices deploying the neural network model. Furthermore, this application establishes a connection between the first and second parameters of each layer in the neural network model, replacing the prior art's provision of task numbers during training and inference, thus ensuring the versatility of large models.

[0041] First, we will explain several terms that may be involved in the embodiments of this application below.

[0042] Continuous learning: A machine learning strategy aimed at enabling models to adapt to new data and knowledge, perform knowledge transfer, learn online, learn multi-task, learn incrementally, selectively forget, and learn across modalities, while maintaining stable performance over long periods. This strategy is crucial for building intelligent systems that can self-update and improve in constantly changing environments.

[0043] Incremental learning with model capacity: One of the mainstream methods of continuous learning, the core idea of ​​this type of method is to treat the model as a scalable graph structure, adjusting the model's complexity by adding or removing nodes and edges. When a new task arrives, the model can dynamically adjust its structure and parameters according to the characteristics of the task to improve performance on the new task while maintaining its generalization ability to previously learned tasks.

[0044] The execution subject of the incremental capacity expansion method for neural network models provided in this application embodiment can be a terminal device. As an example, the terminal device may include, but is not limited to, mobile phones, desktop computers, tablet computers, laptops, PDAs, smart voice interaction devices, smart home appliances, vehicle terminals, and aircraft. The execution subject of the incremental capacity expansion method for neural network models provided in this application embodiment can also be a server, meaning the incremental capacity expansion system for neural network models can be deployed on a server. The method provided in this application embodiment can also be executed collaboratively by a terminal device and a server. Therefore, this application embodiment does not limit the implementation subject of the technical solution of this application.

[0045] A server can be a standalone physical server, a server cluster consisting of multiple physical servers, or a distributed system. Furthermore, a server can also be a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0046] Figure 2 A flowchart illustrating a method for incremental capacity expansion of a neural network model provided in this application embodiment. Now, in conjunction with... Figure 2 This paper describes a method for incremental capacity expansion of a neural network model.

[0047] The incremental capacity expansion method for neural network models is executed by a neural network model incremental capacity expansion system, which is deployed on or includes at least one computing device. The method includes:

[0048] S101: Obtain the first parameter of each layer in the sample set and the neural network model. The neural network model includes multiple layers and is used to process input data by running the multiple layers through the hardware in the computing device.

[0049] The neural network model in S101 has been trained using one or more training sets. This application does not limit the number of training sets; the neural network model can be trained using either a training set or a sample set. The difference between a training set and a sample set is that they are datasets for different tasks, and the neural network model has already learned the features of the task corresponding to the training set. This application does not limit the task type of the training set and sample set. As an optional embodiment, if the neural network model has been trained using a training set that is a dataset for a road condition recognition task, then the first parameter of each layer in the neural network model learns the features of the road condition recognition task; the sample set is a dataset for an optimal route recommendation task.

[0050] S102: Input the sample set into the neural network model to obtain the output features of the first parameter of each layer in the neural network model.

[0051] By inputting the sample set into the neural network model, the output features of the first parameter of each layer in the neural network model under the new task can be obtained, preparing for the establishment of connections between the first and second parameters in subsequent iterations. As an optional embodiment, the sample set is a sample set for the optimal route recommendation task, and the first parameter of each layer in the neural network model learns features from the road condition recognition task. In this case, inputting the sample set into the neural network model yields the output features of each first parameter under the optimal route recommendation task.

[0052] S103: Assign a second parameter to each layer in the neural network model.

[0053] This application replaces the existing method of constructing subnetworks by assigning a second parameter to each layer of the neural network model. The first parameter has already learned the features of the task corresponding to the training set, while the second parameter in S103 is the initialized parameter. In other words, the first parameter is the original parameter of each layer of the neural network model, while the second parameter is the new parameter assigned to each layer of the neural network model. This application does not limit the number of new parameters assigned to each layer.

[0054] As an optional embodiment, the neural network model includes an input layer, a convolutional layer, and an output layer. For any training set, the input layer has one parameter that learns the features of the task corresponding to that training set, the convolutional layer has two parameters that learn the features of the task corresponding to that training set, and the output layer has three parameters that learn the features of the task corresponding to that training set. When assigning a second parameter to each layer, one second parameter can be assigned to the input layer, two second parameters to the convolutional layer, and three second parameters to the output layer, based on this principle.

[0055] As an optional embodiment, the neural network model includes an input layer, a convolutional layer, and an output layer. Analyzing the correlation between the tasks corresponding to the sample set and the training set, if the correlation between the tasks corresponding to the sample set and the training set is large, it proves that the output features of the first parameter of each layer in the neural network model are highly correlated with the label data of the sample set. This further proves that many first parameters can be applied to the learning of the sample set. Therefore, when assigning second parameters to each layer in the neural network model, the number of second parameters can be appropriately reduced. For example, one second parameter can be assigned to each of the input layer, convolutional layer, and output layer.

[0056] The above only demonstrates two relatively detailed methods for allocating the second parameter, but the methods are not limited to these two. Regardless of the method used, allocating the second parameter to each layer significantly reduces the computing memory required to deploy the neural network model compared to building a new subnetwork. Directly adjusting the parameters on the existing model not only simplifies the model structure but also improves resource utilization efficiency, making the model more flexible in handling new tasks and consuming fewer system resources. This strategy ensures model performance while optimizing memory usage, making it suitable for resource-constrained environments. Compared to building entirely new subnetworks, directly extending the parameters on the existing model saves more computing memory and processing power. This is because it eliminates the need to allocate independent storage space and computing resources for additional subnetworks, resulting in higher overall resource utilization efficiency.

[0057] S104: Based on the output features of the first parameter of each layer in the neural network model, establish connections between the second parameter of each layer and the first parameter of each layer in the neural network model to obtain the neural network model with parameter connections.

[0058] Establishing a connection between the second and first parameters based on the output features of the first parameter allows for more precise tuning and optimization of the neural network model. This connection method ensures greater compatibility of the second parameter with the existing structure, thereby improving the overall performance of the model. Furthermore, the connection between the first and second parameters allows the model to simultaneously invoke both parameters when performing new tasks, improving parameter effectiveness, reducing model complexity and redundancy, and making the model easier to maintain and update. Especially in large-scale deep learning models, this method of reducing model complexity through the allocation of the second parameter helps lower debugging difficulty and maintenance costs.

[0059] S105: Train the neural network model with connected parameters based on the sample set to obtain the neural network model with capacity incremental expansion.

[0060] During the training process, the first parameter of each layer in the neural network model is frozen.

[0061] Training the neural network model with the parameters connected using a sample set is primarily to adjust the second parameter, enabling it to learn the features of the task corresponding to the sample set. Freezing the first parameter ensures that the neural network model after capacity incremental expansion does not forget the features of the task corresponding to the old sample set, allowing it to learn features of both the new and old tasks.

[0062] The above describes the main technical solution of this application. Further implementations of the main technical solution are now introduced. Details are as follows:

[0063] S104 primarily establishes connections between the second and first parameters, thereby achieving fine-grained (feature-level) parameter expansion and knowledge sharing. However, to establish more effective connections, it is necessary to analyze which first parameters of the neural network model are helpful in solving the corresponding task for the sample set, thus establishing effective connections between the second and first parameters.

[0064] The output of each layer in a neural network model can be viewed as a high-dimensional embedding of the input data in the representation space. Evaluating the effect of the features learned by the first parameter of each layer on the sample set is equivalent to evaluating the property of the first parameter's high-dimensional embedding in this representation space. Evaluation metrics are used to assess the discriminative power of the first parameter (feature dimension) on the sample set, thus achieving the effect of filtering noise while saving parameter space.

[0065] Based on the above analysis, this application provides an optional embodiment:

[0066] Calculate the correlation value of the output features of the first parameter of each layer in the neural network model.

[0067] This embodiment uses the correlation value as an evaluation index to assess the discriminative power of the first parameter on the sample set. This application does not limit the calculation method of the correlation value. As an optional embodiment, the correlation value of the output feature of the first parameter of each layer in the neural network model is calculated using the linear discriminant analysis method.

[0068] As an optional embodiment, the output features of the first parameter of each layer in the neural network model are classified according to a preset classification rule to obtain a multi-class output feature set.

[0069] This application does not limit the preset classification rules. The main purpose of classifying the output features is to calculate the generalized Ruili entropy set, thereby determining the correlation value of the output features of each first parameter. For example, the preset classification rules include determining the samples in the sample set corresponding to the output features of the first parameter of each layer in the neural network model, and classifying the output features of the first parameter of each layer in the neural network model according to the sample semantics. In short, the output features of the first parameter of each layer in the neural network model corresponding to samples with the same semantics are grouped into one category.

[0070] The overall mean vector of the sample set is obtained based on the output features of the first parameter of each layer in the neural network model.

[0071] The class mean vector corresponding to each class of output feature set is obtained based on each class of output feature set.

[0072] The generalized Ruili entropy set is obtained based on the overall mean vector of the sample set and the category mean vector corresponding to each class of output feature set.

[0073] The correlation values ​​of the output features of the first parameter of each layer in the neural network model are determined based on the generalized Ruili entropy set.

[0074] This application provides a specific embodiment of a linear discriminant analysis method:

[0075] Given a sample set D = {(x i y i )} n i＝1 , where: x i ∈R d Given a d-dimensional eigenvector, x in this application i Let y be the output feature obtained by running the neural network model on the i-th sample data in the sample set. i ∈{1, 2, ..., c} represents the label data of the i-th sample data in the sample set, and n represents the number of samples in the sample set. In this embodiment, before inputting the sample set into the neural network model, the sample set is first classified according to semantics, and then each type of sample set is input into the neural network model. This is not substantially different from the above embodiment, which first inputs the sample set into the neural network model and then classifies the output features. Both are for the purpose of obtaining the generalized Ruili entropy set.

[0076] The sample set D is classified according to its semantic relevance values ​​to obtain multiple sample sets; D k ={(x i y i )|y i =k} ​​is the sample set of the kth class; n k =|D k | represents the number of samples in the k-th class sample set.

[0077] Each sample set is input into a neural network model to obtain the output features of each sample in each set running on the model. Based on the output features of each sample in the set running on the neural network model, the overall mean vector μ of the sample set is calculated.

[0078]

[0079] The class mean vector μ of the k-th class sample set is calculated based on the output features obtained by running the neural network model on each sample data in the k-th class sample set. k :

[0080]

[0081] According to μ k Calculate the within-class scatter matrix S of the k-th class sample set. w :

[0082]

[0083] According to μ k The inter-class scatter matrix S is calculated using μ. b :

[0084]

[0085] Based on S w and S b Solve for the generalized Ruili entropy set W, where W is the criterion for judging the correlation of each parameter:

[0086] S b w=λS w w is equivalent to:

[0087] Wherein, the corresponding eigenvalue λ represents the degree of inter-class separation in that direction. W={(w ii )},w ij The generalized Ruili entropy represents the output feature of the j-th parameter in the i-th layer of a neural network model.

[0088] W is used to determine the importance of the output features of the first parameter of each layer in the neural network model.

[0089] This application provides an embodiment for determining the importance of the output features of the first parameter of each layer in a W-discriminative neural network model:

[0090] The generalized Ruili entropy w of the output feature of the j-th parameter in the i-th layer of the neural network model is calculated using the method described above. ij .

[0091] Based on w ij The correlation s of the output feature of the j-th parameter in the i-th layer of the neural network model is obtained. ij :

[0092] s ij =|w ij |

[0093] Based on s ij The average correlation score of the output feature of the j-th parameter in the neural network model is calculated. j :

[0094]

[0095] Where c is the number of categories.

[0096] Based on score j The normalized correlation value of the output feature of the j-th parameter in the neural network model is calculated as:

[0097]

[0098] Arrange the d first parameters of the neural network model in descending order of normalized correlation values:

[0099]

[0100] By calculating the generalized Rayleigh entropy, the correlation value of the output feature of each first parameter in the neural network model was quantified, objectively evaluating the correlation between the first parameter and the sample set. Furthermore, by calculating the average correlation of the output features of the first parameter, the transparency and interpretability of the neural network model were enhanced. Arranging the first parameters in the neural network model in descending order according to the normalized correlation values ​​more clearly demonstrates the relationship between the output features of the first parameter and the sample set, indicating the direction for the connection between the first and second parameters. For example, a connection can be established between the second parameter and the first parameter corresponding to a high correlation value, improving the effectiveness of the connection between the first and second parameters.

[0101] This embodiment is mainly to demonstrate the specific method of using linear discriminant analysis to calculate the correlation value of the output feature of the first parameter of each layer of a neural network model, in order to prepare for the subsequent selection of the first parameter.

[0102] After calculating the correlation value of each first parameter, the computing device establishes connections between the second parameters of each layer in the neural network model and the first parameters of each layer based on the correlation values, thus obtaining a parameter-connected neural network model. As an optional embodiment, the first parameters of each layer in the neural network model are filtered based on the correlation values ​​to determine the first parameters whose correlation values ​​are greater than a preset threshold, and these first parameters with correlation values ​​greater than the preset threshold are defined as target parameters. Connections are then established between the target parameters and the second parameters of each layer in the neural network model to obtain the parameter-connected neural network model.

[0103] Filtering the first parameter of each layer in a neural network model using the magnitude of correlation values ​​ensures a strong data correlation between the selected target parameter and the second parameter. This makes the connection between the target parameter and the second parameter more effective and simplifies the structure of the neural network model. Without filtering the first parameter, the structure of the neural network model after parameter connections would be quite messy and contain many invalid connections, such as... Figure 3 As shown; conversely, if the first parameter is filtered to obtain the target parameter, and a connection is established between the second parameter and the target parameter, the structure of the neural network model after parameter connection is clearer and more efficient, as shown. Figure 4 As shown. By comparison Figure 4 and Figure 3 It is also clear that filtering the first parameter and then establishing connections between the parameters can make the structure of the neural network model clearer.

[0104] This application does not limit the specific connection method between the first parameter and the second parameter. As an optional embodiment, a connection can be established between the first parameter and the second parameter that can cross layers, or a connection can be established between the first parameter and the second parameter of adjacent layers.

[0105] This application provides an optional embodiment for establishing a connection between the first and second parameters of adjacent layers, where the value of k is set to 1. Setting k to 1 is mainly to avoid missing connections.

[0106] A connection is established between the target parameter of the k-th layer in the neural network model and the second parameter of the (k+1)-th layer. For example, a one-way arrow connection is used between the target parameter of the first layer and the second parameter of the second layer. A one-way arrow connection means that the information flow is unidirectional, passing from lower to higher layers, which mainly helps to maintain the clarity of the data flow and reduce the risk of confusion or circular dependencies.

[0107] If k+1 equals the preset iteration value, the iteration ends and the neural network model with connected parameters is obtained; if k+1 does not equal the preset iteration value, the value of k is incremented by 1, and the step of establishing a connection between the target parameter of the k-th layer structure in the neural network model and the second parameter of the (k+1)-th layer structure in the neural network model is returned.

[0108] This embodiment helps improve the overall performance of the model by systematically establishing connections between the first and second parameters, especially in application environments that require dynamic updates to adapt to new business scenarios or changes in knowledge-based data. Furthermore, since each iteration only involves establishing connections between two adjacent layers, it effectively manages computing resources, reduces unnecessary computational overhead, and improves resource utilization efficiency.

[0109] In short, this embodiment can not only effectively establish the connection between the first parameter and the second parameter, but also ensure the efficiency and scalability of the model, making it suitable for more complex industrial application scenarios.

[0110] S104 primarily establishes the connection between the second parameter and the first parameter. This embodiment only provides a more specific example to illustrate the method of establishing the second parameter and the first parameter based on the output features of the first parameter of each layer in the neural network model. After establishing the connection between the first parameter and the second parameter, a neural network model with parameter connections is obtained. However, the neural network model with parameter connections only has the connection relationship between the second parameter and the first parameter; the second parameter has not yet learned the features of the task corresponding to the sample set. Therefore, after obtaining the neural network model with parameter connections, it is necessary to train it. For example, in S105, the neural network model with parameter connections is trained based on the sample set to obtain a neural network model with incremental capacity expansion.

[0111] Regarding S105, this application provides an optional embodiment:

[0112] The first parameter of each layer in the neural network model after parameter concatenation is frozen to obtain the frozen neural network model.

[0113] Freezing the first parameter is mainly to preserve the task features learned by the first parameter from the old sample set, to avoid the destruction of this important information during the fine-tuning process for the new task, and to enable the second parameter to understand the task features corresponding to the old sample set.

[0114] Based on the sample set, the second parameter of each layer in the frozen neural network model is adjusted to obtain the neural network model with capacity increment expansion.

[0115] This application does not limit the specific training process. As an optional embodiment, this application uses a loss function to train the frozen neural network model, specifically including:

[0116] The input data of the sample set is fed into the frozen neural network model to obtain the output of the frozen neural network model.

[0117] The purpose of feeding the input data of the sample set into the frozen neural network model is to evaluate the performance of the frozen neural network model on the sample set.

[0118] The loss value is calculated based on the output of the frozen neural network model and the labeled data of the sample set. The loss value reflects the difference between the output of the frozen neural network model and the expected output.

[0119] If the loss value is less than the preset loss value, the training ends and the neural network model with capacity increment expansion is obtained; if the loss value is greater than the preset loss value, the second parameter of each layer in the frozen neural network model is adjusted, and the process returns to the step of inputting the input data of the sample set into the frozen neural network model to obtain the output of the frozen neural network model.

[0120] In practical industrial applications, models often need to quickly adapt to constantly changing business needs and data distributions. By freezing existing knowledge and fine-tuning the model accordingly, efficient knowledge transfer and improved model adaptability are achieved. Compared to completely retraining the model, this significantly reduces training time and computational resource requirements, making it particularly suitable for large-scale datasets and complex model architectures.

[0121] In summary, this embodiment not only effectively protects the original knowledge of the model, but also improves the model's adaptability and performance through targeted fine-tuning, while reducing the consumption of computing resources. It is a very practical model optimization strategy.

[0122] This application does not limit the application scenarios. As an optional embodiment, the application scenarios include knowledge base updates for models that require world knowledge, such as code generation models and chat models; or, the application scenarios include continuous fine-tuning of general large models in different business scenarios; or, the application scenarios include continuous model updates customized for individual users or streaming data.

[0123] This application provides a relatively general method for incremental capacity expansion of neural network models. The correlation numerical evaluation and parameter expansion involved in this method are defined at the matrix operation level, without specifying the model's architecture. The loss function or training dataset used to train the model can be widely applied to capacity expansion of various deep neural networks. This application ensures that new knowledge is learned without forgetting existing knowledge, and it can also utilize the model's existing knowledge to learn new knowledge, achieving a positive knowledge transfer effect.

[0124] Based on the embodiments described above, this application also provides a system for incremental capacity expansion of a neural network model, wherein the system is deployed on at least one computing device or includes at least one computing device. The following is in conjunction with... Figure 5 Please provide an explanation. Figure 5 This is a schematic diagram of the structure of a capacity incremental expansion system for a neural network model provided in an embodiment of this application. Figure 5 The system shown includes:

[0125] The acquisition module is used to acquire the sample set and the first parameters of each layer in the neural network model. The neural network model includes multiple layers and is used to process the input data by running the multiple layers through the hardware in the computing device.

[0126] The output feature determination module is used to input the sample set into the neural network model and obtain the output features of the first parameter of each layer in the neural network model.

[0127] The allocation module is used to assign a second parameter to each layer in the neural network model.

[0128] The connection module is used to establish connections between the second parameter of each layer in the neural network model and the first parameter of each layer in the neural network model based on the output features of the first parameter of each layer, so as to obtain the neural network model with parameter connections.

[0129] The capacity incremental expansion module is used to train the neural network model with connected parameters based on the sample set to obtain the neural network model with increased capacity; during the training process, the first parameter of each layer in the neural network model is frozen.

[0130] As an optional embodiment, the connection module includes: a correlation numerical calculation unit and a connection unit. The function of each unit in the connection module is now described:

[0131] The correlation value calculation unit is used to calculate the correlation value of the output features of the first parameter of each layer in the neural network model.

[0132] The connection unit is used to establish connections between the second parameter of each layer in the neural network model and the first parameter of each layer in the neural network model based on the correlation value, so as to obtain the neural network model with parameter connections.

[0133] As an optional embodiment, the correlation numerical calculation unit is specifically used for:

[0134] The correlation values ​​of the output features of the first parameter of each layer in the neural network model are calculated using the linear discriminant analysis method.

[0135] As an optional embodiment, the connection unit includes a filtering subunit and a connection subunit. The function of each subunit in the connection unit is now described:

[0136] The filtering sub-unit is used to filter the first parameter of each layer in the neural network model based on the correlation value, determine the first parameter corresponding to the correlation value being greater than a preset threshold, and define the first parameter corresponding to the correlation value being greater than the preset threshold as the target parameter.

[0137] The connecting subunit is used to establish connections between the target parameters and the second parameters of each layer in the neural network model, resulting in a neural network model with connected parameters.

[0138] As an optional embodiment, the connecting subunit is specifically used for:

[0139] Set the value of k to 1; establish a connection between the target parameter of the k-th layer structure in the neural network model and the second parameter of the (k+1)-th layer structure in the neural network model; determine whether k+1 is equal to the preset iteration value and obtain the first judgment result; if the first judgment result is yes, end the iteration and obtain the neural network model after parameter connection; if the first judgment result is no, increment the value of k by 1 and return to establish a connection between the target parameter of the k-th layer structure in the neural network model and the second parameter of the (k+1)-th layer structure in the neural network model.

[0140] As an optional embodiment, the capacity incremental expansion module includes a freeze unit and a parameter adjustment unit. The function of each unit in the capacity incremental expansion module is now described:

[0141] The freeze unit is used to freeze the first parameter of each layer in the neural network model after the parameters are connected, so as to obtain the frozen neural network model.

[0142] The parameter adjustment unit is used to adjust the second parameter of each layer in the frozen neural network model based on the sample set, so as to obtain the neural network model after capacity increment expansion.

[0143] As an optional embodiment, the parameter adjustment unit includes:

[0144] The output determination subunit of the frozen neural network model is used to input the input data of the sample set into the frozen neural network model to obtain the output of the frozen neural network model.

[0145] The loss value calculation subunit is used to calculate the loss value based on the output of the frozen neural network model and the labeled data of the sample set.

[0146] The judgment sub-unit is used to determine whether the loss value is less than the preset loss value and obtain the second judgment result. If the second judgment result is yes, the training ends and the neural network model after capacity increment expansion is obtained. If the second judgment result is no, the second parameter of each layer structure in the frozen neural network model is adjusted and the output determination sub-unit of the frozen neural network model is returned.

[0147] This application provides a device, which can be a server. Figure 6 This is a schematic diagram of a server structure provided in an embodiment of this application. The server 900 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 922 (e.g., one or more processors) and memory 932, and one or more storage media 930 (e.g., one or more mass storage devices) for storing application programs 942 or data 944. The memory 932 and storage media 930 can be temporary or persistent storage. The program stored in the storage media 930 may include one or more modules (not shown in the diagram), each module may include a series of instruction operations on the server. Furthermore, the CPU 922 may be configured to communicate with the storage media 930 and execute the series of instruction operations in the storage media 930 on the server 900.

[0148] Server 900 may also include one or more power supplies 926, one or more wired or wireless network interfaces 950, one or more input / output interfaces 958, and / or one or more operating systems 941.

[0149] CPU 922 is used to perform the following steps:

[0150] Obtain the first parameters of each layer in the sample set and the neural network model, which includes a multi-layer structure. The neural network model is used to process input data by running the multi-layer structure through the hardware in the computing device.

[0151] The sample set is input into the neural network model to obtain the output features of the first parameter of each layer in the neural network model.

[0152] Assign a second parameter to each layer in the neural network model.

[0153] Based on the output features of the first parameter of each layer in the neural network model, the second parameter of each layer in the neural network model is connected to the first parameter of each layer in the neural network model to obtain the neural network model with parameter connections.

[0154] The neural network model with parameters connected is trained based on the sample set to obtain a neural network model with incremental capacity expansion; during the training process, the first parameter of each layer in the neural network model is frozen.

[0155] This application also provides another device, which can be a terminal device. For example... Figure 7 As shown, for ease of explanation, only the parts related to the embodiments of this application are shown. For specific technical details not disclosed, please refer to the method section of the embodiments of this application. Taking a mobile phone as an example:

[0156] Figure 7 The diagram shown is a block diagram of a portion of the structure of a mobile phone provided in an embodiment of this application. (Reference) Figure 7 The mobile phone includes: a radio frequency (RF) circuit 1010, a memory 1020, an input unit 1030, a display unit 1040, a sensor 1050, an audio circuit 1060, a wireless fidelity (WiFi) module 1070, a processor 1080, and a power supply 1090, etc. Those skilled in the art will understand that... Figure 7 The mobile phone structure shown does not constitute a limitation on the mobile phone and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0157] The following is combined Figure 7 A detailed introduction to each component of a mobile phone:

[0158] The RF circuit 1010 can be used for receiving and transmitting signals during information transmission or calls. Specifically, it receives downlink information from the base station and processes it with the processor 1080; additionally, it transmits uplink data to the base station. Typically, the RF circuit 1010 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier (LNA), a duplexer, etc. Furthermore, the RF circuit 1010 can also communicate wirelessly with networks and other devices. The aforementioned wireless communications may use any communication standard or protocol, including but not limited to Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, and Short Messaging Service (SMS).

[0159] The memory 1020 can be used to store software programs and modules. The processor 1080 executes various mobile phone functions and data processing by running the software programs and modules stored in the memory 1020. The memory 1020 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, applications required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory 1020 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0160] The input unit 1030 can be used to receive input numerical or character information, and to generate key signal inputs related to user settings and function control of the mobile phone. Specifically, the input unit 1030 may include a touch panel 1031 and other input devices 1032. The touch panel 1031, also known as a touch screen, can collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch panel 1031), and drive the corresponding connection system according to a pre-set program. Optionally, the touch panel 1031 may include two parts: a touch detection system and a touch controller. The touch detection system detects the user's touch position and the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection system, converts it into touch point coordinates, and sends it to the processor 1080, and can also receive and execute commands sent by the processor 1080. In addition, the touch panel 1031 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 1031, the input unit 1030 may also include other input devices 1032. Specifically, other input devices 1032 may include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc.

[0161] The display unit 1040 can be used to display information input by the user or information provided to the user, as well as various menus of the mobile phone. The display unit 1040 may include a display panel 1041, which may optionally be configured as a Liquid Crystal Display (LCD), Organic Light-Emitting Diode (OLED), or similar display panel 1041. Further, a touch panel 1031 may cover the display panel 1041. When the touch panel 1031 detects a touch operation on or near it, it transmits the information to the processor 1080 to determine the type of touch event. Subsequently, the processor 1080 provides corresponding visual output on the display panel 1041 according to the type of touch event. Although in Figure 7 In this embodiment, the touch panel 1031 and the display panel 1041 are two separate components to realize the input and output functions of the mobile phone. However, in some embodiments, the touch panel 1031 and the display panel 1041 can be integrated to realize the input and output functions of the mobile phone.

[0162] The mobile phone may also include at least one sensor 1050, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor. The ambient light sensor can adjust the brightness of the display panel 1041 according to the ambient light level, and the proximity sensor can turn off the display panel 1041 and / or the backlight when the phone is moved to the ear. As a type of motion sensor, an accelerometer sensor can detect the magnitude of acceleration in various directions (generally three axes). When stationary, it can detect the magnitude and direction of gravity and can be used for applications that recognize the phone's posture (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration recognition-related functions (such as pedometer, taps), etc. Other sensors that may be configured in the mobile phone, such as gyroscopes, barometers, hygrometers, thermometers, and infrared sensors, will not be described in detail here.

[0163] The audio circuit 1060, speaker 1061, and microphone 1062 provide an audio interface between the user and the mobile phone. The audio circuit 1060 converts the received audio data into electrical signals and transmits them to the speaker 1061, where the speaker 1061 converts them into sound signals for output. On the other hand, the microphone 1062 converts the collected sound signals into electrical signals, which are then received by the audio circuit 1060, converted into audio data, and then processed by the processor 1080 before being transmitted via the RF circuit 1010 to, for example, another mobile phone, or the audio data can be output to the memory 1020 for further processing.

[0164] WiFi is a short-range wireless transmission technology. Through the WiFi module 1070, mobile phones can help users send and receive emails, browse web pages, and access streaming media, providing users with wireless broadband internet access. Although Figure 7 The WiFi module 1070 is shown, but it is understood that it is not an essential component of a mobile phone and can be omitted as needed without changing the essence of the invention.

[0165] The processor 1080 is the control center of the mobile phone, connecting various parts of the phone through various interfaces and lines. It executes software programs and / or modules stored in the memory 1020 and calls data stored in the memory 1020 to perform various functions and process data, thereby collecting overall data and information from the phone. Optionally, the processor 1080 may include one or more processing units; preferably, the processor 1080 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 1080.

[0166] The mobile phone also includes a power supply 1090 (such as a battery) that supplies power to various components. Preferably, the power supply can be logically connected to the processor 1080 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system.

[0167] Although not shown, mobile phones may also include a camera, Bluetooth module, etc., which will not be described in detail here.

[0168] In this embodiment of the application, the processor 1080 included in the mobile phone also has the following functions:

[0169] Obtain the first parameters of each layer in the sample set and the neural network model, which includes a multi-layer structure. The neural network model is used to process input data by running the multi-layer structure through the hardware in the computing device.

[0170] The sample set is input into the neural network model to obtain the output features of the first parameter of each layer in the neural network model.

[0171] Assign a second parameter to each layer in the neural network model.

[0172] Based on the output features of the first parameter of each layer in the neural network model, the second parameter of each layer in the neural network model is connected to the first parameter of each layer in the neural network model to obtain the neural network model with parameter connections.

[0173] The neural network model with parameters connected is trained based on the sample set to obtain a neural network model with incremental capacity expansion; during the training process, the first parameter of each layer in the neural network model is frozen.

[0174] This application also provides a computer-readable storage medium for storing a computer program that, when run, enables any one of the methods for incrementally expanding the capacity of a neural network model according to the foregoing embodiments.

[0175] This application also provides a computer program product including a computer program, which, when run on a computer, enables the execution of any one of the capacity incremental expansion methods for a neural network model of the foregoing embodiments.

[0176] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the system and equipment described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0177] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0178] In the several embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the system division is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple systems may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between systems or units, and may be electrical, mechanical, or other forms.

[0179] The system described as separate components may or may not be physically separate. Similarly, the components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0180] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0181] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media capable of storing computer programs.

[0182] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for incremental capacity expansion of a neural network model, characterized in that, The method is executed by a capacity incremental scaling system for a neural network model, the capacity incremental scaling system for the neural network model being deployed on or including at least one computing device, and the method comprising: Obtain a sample set and the first parameters of each layer in the neural network model, the neural network model including a multi-layer structure, the neural network model being used to process input data by running the multi-layer structure through the hardware in the computing device; The sample set is input into the neural network model to obtain the output features of the first parameter of each layer in the neural network model; Assign a second parameter to each layer in the neural network model; Based on the output features of the first parameter of each layer in the neural network model, a connection is established between the second parameter of each layer in the neural network model and the first parameter of each layer in the neural network model to obtain a neural network model with parameter connections. The neural network model with the parameters connected is trained based on the sample set to obtain a neural network model with incremental capacity expansion; wherein, during the training process, the first parameter of each layer in the neural network model is frozen.

2. The method according to claim 1, characterized in that, Based on the output features of the first parameter of each layer in the neural network model, a connection is established between the second parameter of each layer and the first parameter of each layer in the neural network model to obtain a parameter-connected neural network model, specifically including: Calculate the correlation value of the output features of the first parameter of each layer in the neural network model; Based on the correlation values, the second parameter of each layer in the neural network model is connected to the first parameter of each layer in the neural network model to obtain the neural network model with the parameters connected.

3. The method according to claim 2, characterized in that, Based on the correlation values, a connection is established between the second parameter of each layer in the neural network model and the first parameter of each layer in the neural network model to obtain the neural network model after parameter connection, including: Based on the correlation value, the first parameter of each layer in the neural network model is filtered to determine the first parameter corresponding to the correlation value being greater than a preset threshold, and the first parameter corresponding to the correlation value being greater than the preset threshold is defined as the target parameter. A connection is established between the target parameter and the second parameter of each layer in the neural network model to obtain the neural network model after parameter connection.

4. The method according to claim 3, characterized in that, Establishing connections between the target parameters and the second parameters of each layer in the neural network model to obtain the neural network model with the parameters connected specifically includes: Let the value of k be 1; Establish a connection between the target parameter of the k-th layer structure in the neural network model and the second parameter of the (k+1)-th layer structure in the neural network model; If k+1 equals the preset iteration value, then the iteration ends, and the neural network model with the parameters connected is obtained; If k+1 is not equal to the preset iteration value, then increment the value of k by 1 and return to the step of establishing a connection between the target parameter of the k-th layer structure in the neural network model and the second parameter of the (k+1)-th layer structure in the neural network model.

5. The method according to claim 1, characterized in that, The neural network model with the parameters connected is trained based on the sample set to obtain a neural network model with incremental capacity expansion, specifically including: The first parameter of each layer in the neural network model after the parameters are connected is frozen to obtain the frozen neural network model. Based on the sample set, the second parameter of each layer in the frozen neural network model is adjusted to obtain the neural network model after capacity increment expansion.

6. The method according to claim 5, characterized in that, Based on the sample set, the second parameter of each layer in the frozen neural network model is adjusted to obtain the neural network model with the capacity increment expansion, specifically including: The input data of the sample set is input into the frozen neural network model to obtain the output of the frozen neural network model; The loss value is calculated based on the output of the frozen neural network model and the label data of the sample set; If the loss value is less than the preset loss value, then the training ends and the neural network model after the capacity increment is obtained; If the loss value is greater than the preset loss value, then the second parameter of each layer in the frozen neural network model is adjusted, and the process returns to the step of inputting the input data of the sample set into the frozen neural network model to obtain the output of the frozen neural network model.

7. The method according to claim 2, characterized in that, Calculating the correlation value of the output features of the first parameter of each layer in the neural network model specifically includes: The correlation values ​​of the output features of the first parameter of each layer in the neural network model are calculated using the linear discriminant analysis method.

8. The method according to claim 7, characterized in that, The correlation values ​​of the output features of the first parameter of each layer in the neural network model are calculated using linear discriminant analysis, specifically including: The output features of the first parameter of each layer in the neural network model are classified according to a preset classification rule to obtain a multi-class output feature set; The overall mean vector of the sample set is obtained based on the output features of the first parameter of each layer in the neural network model. Based on each class of output feature set, obtain the category mean vector corresponding to each class of output feature set; The generalized Ruili entropy set is obtained based on the overall mean vector of the sample set and the category mean vector corresponding to each type of output feature set; The correlation values ​​of the output features of the first parameter of each layer in the neural network model are determined based on the generalized Ruili entropy set.

9. A capacity incremental expansion system for a neural network model, characterized in that, The capacity incremental scaling system for the neural network model is deployed on or includes at least one computing device, and the system comprises: An acquisition module is used to acquire a sample set and the first parameters of each layer in the neural network model, wherein the neural network model includes a multi-layer structure, and the neural network model is used to process input data by running the multi-layer structure through the hardware in the computing device; The output feature determination module is used to input the sample set into the neural network model to obtain the output features of the first parameter of each layer structure in the neural network model; An allocation module is used to allocate a second parameter to each layer in the neural network model; The connection module is used to establish a connection between the second parameter of each layer in the neural network model and the first parameter of each layer in the neural network model based on the output features of the first parameter of each layer in the neural network model, so as to obtain a neural network model with parameter connection. The capacity incremental expansion module is used to train the neural network model after parameter concatenation based on the sample set to obtain the neural network model after capacity incremental expansion; wherein, during the training process, the first parameter of each layer in the neural network model is frozen.

10. A computer device, characterized in that, The device includes a processor and a memory: The memory is used to store computer programs and to transfer the computer programs to the processor; The processor is configured to execute the capacity incremental expansion method of the neural network model according to any one of claims 1 to 8, based on instructions in the computer program.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a computer program that, when executed, implements the capacity incremental expansion method of the neural network model according to any one of claims 1 to 8.

12. A computer program product, characterized in that, Includes a computer program that, when executed, implements the capacity incremental expansion method for the neural network model according to any one of claims 1 to 8.

Images