Model training method and related apparatus

Abstract

Disclosed in embodiments of the present application are a model training method and a related apparatus. Multiple incremental branches are added and combined with an original main branch of a first model to constitute multiple processing branches. When sample data corresponding to a new data processing objective is processed, processing operations of the same category can be performed in parallel by means of the multiple processing branches, and a processing result determined by a model can be outputted on the basis of processing results of the multiple processing branches. Because the multiple processing branches are independent of each other during data processing, processing dimensions for executing the processing operations by means of the incremental branches can be enriched. Thus, when model training is completed by adjusting model parameters of the multiple incremental branches, the model can fully learn data processing manners for new data processing objectives, thereby improving the ability of the model to achieve the new data processing objectives without excessively increasing the number of model parameters.

Description

A model training method and related apparatus

[0001] This application claims priority to Chinese Patent Application No. 2024109347641, filed on July 12, 2024, entitled “A Model Training Method and Related Apparatus”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of data processing technology, and in particular to a model training method and related apparatus. Background Technology

[0003] Models are one of the most popular tools in the field of data processing. They can efficiently achieve various complex data processing goals, such as object detection, image processing, and probability prediction. To enable a model to achieve a specific data processing goal, training samples can be constructed for that goal. The model is then trained using these samples, allowing it to learn how to achieve that goal.

[0004] However, the data processing goals that the model needs to achieve may differ in different data processing scenarios. For example, in object recognition scenarios, recognizing people and recognizing objects require different object recognition capabilities from the model, necessitating parameter tuning. Related technologies suggest adding parameters to the original model parameters and adjusting only these newly added parameters during the tuning process. This allows the adjusted parameters to provide the model with specific data processing capabilities to achieve the new data processing goals. These newly added parameters constitute only a small portion of the overall model parameters, thus reducing the amount of parameter tuning and improving tuning efficiency.

[0005] Because the role of newly added parameters in the model data processing process is relatively singular in related technologies, this parameter adjustment method limits the model knowledge learned and makes it difficult to bring high-quality model processing performance. Summary of the Invention

[0006] To address the aforementioned technical issues, this application provides a model training method and related apparatus that can significantly improve model training accuracy in incremental training scenarios.

[0007] The embodiments of this application disclose the following technical solutions:

[0008] In a first aspect, embodiments of this application disclose a model training method, executed by a computer device, the method comprising:

[0009] Acquire a first model and sample data, wherein the first model is trained based on initial sample data, and the sample data is different from the initial sample data;

[0010] By adding multiple incremental branches to the first model, a second model is formed. The multiple incremental branches and the main branch in the first model are used to perform the same type of processing operations on the same data. The total number of model parameters of the multiple incremental branches is less than the number of model parameters of the main branch.

[0011] The second model is used to obtain a pending processing result based on the sample data. The pending processing result is determined by combining the processing results corresponding to the multiple incremental branches and the main branch respectively.

[0012] Based on the difference between the pending processing result and the labeled processing result of the sample data, the model parameters of the multiple incremental branches are adjusted to obtain the third model.

[0013] Secondly, embodiments of this application disclose a model training apparatus, the apparatus comprising an acquisition unit, a composition unit, a determination unit, and an adjustment unit:

[0014] The acquisition unit is used to acquire a first model and sample data, wherein the first model is trained based on initial sample data, and the sample data is different from the initial sample data.

[0015] The constituent unit is used to form a second model by adding multiple incremental branches to the first model. The multiple incremental branches and the main branch in the first model are used to perform the same type of processing operation on the same data. The total number of model parameters of the multiple incremental branches is less than the number of model parameters of the main branch.

[0016] The determining unit is used to obtain a pending processing result based on the sample data through the second model. The pending processing result is determined by combining the processing results corresponding to the multiple incremental branches and the main branch respectively.

[0017] The adjustment unit is used to adjust the model parameters of the multiple incremental branches according to the difference between the pending processing result and the labeled processing result of the sample data, so as to obtain a third model.

[0018] Thirdly, embodiments of this application disclose a computer device, which includes a processor and a memory:

[0019] The memory is used to store computer programs and to transfer the computer programs to the processor;

[0020] The processor is configured to execute the model training method described in any one of the first aspects according to the instructions in the computer program;

[0021] Fourthly, embodiments of this application disclose a computer-readable storage medium for storing a computer program for executing the model training method described in any one of the first aspects;

[0022] Fifthly, embodiments of this application disclose a computer program product including a computer program, which, when run on a computer device, causes the computer device to execute the model training method described in any one of the first aspects.

[0023] As can be seen from the above technical solution, when a third model needs to be trained based on the first model to achieve a new data processing goal, a second model can be constructed by adding multiple incremental branches to the first model. This results in a second model with multiple processing branches, including the original main branch of the first model and multiple incremental branches. The total number of model parameters for the multiple incremental branches is less than that of the main branch. By adjusting the model parameters of the multiple incremental branches during training, the second model can learn how to achieve the new data processing goal. Both the multiple incremental branches and the main branch in the first model are used to perform the same type of processing operations on the same data. During the data processing of the sample data, the multiple processing branches can independently process the input data, obtaining processing results corresponding to each branch. The second model can combine these multiple processing results to output the final pending processing result. The labeled processing result corresponding to the sample data is the processing result obtained by accurately processing the sample data when achieving the new data processing goal. Therefore, by adjusting the model parameters of the multiple incremental branches based on the difference between the pending processing result and the labeled processing result, the second model can learn how to achieve the new data processing goal. Firstly, because this application adds multiple incremental branches during incremental training, it enriches the number of parameters in the incremental model, thus improving the model's knowledge learning ability through parameter tuning. Secondly, the multiple incremental branches in this application can independently constitute multiple processing branches, which can independently process data and complete the same processing operations. Therefore, even with a relatively low overall number of parameters in the incremental model, it can enrich the data analysis process and improve the processing accuracy. Thirdly, because this application only adjusts the incremental model parameters while keeping the original model parameters of the first model unchanged, it can learn new model knowledge through the incremental model parameters while retaining the model knowledge of the first model, enriching the model's knowledge base and further improving data processing accuracy. Furthermore, since the overall number of model parameters in the multiple incremental branches is less than that in the main branch, this parameter tuning method requires less parameter tuning and has higher model training efficiency. Fourthly, since the training effect brought by the multi-processing branch model architecture of this application is better, the data processing flow targeted by the multi-processing branch in this application can be a partial processing flow, that is, it only plays a role in a part of the model architecture, which can also ensure the accuracy of the overall data processing of the model. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 is a schematic diagram of a model training method in a practical application scenario provided by an embodiment of this application;

[0026] Figure 2 is a flowchart of a model training method provided in an embodiment of this application;

[0027] Figure 3 is a schematic diagram of a model architecture provided in an embodiment of this application;

[0028] Figure 4 is a schematic diagram of a model architecture provided in an embodiment of this application;

[0029] Figure 5 is a schematic diagram of model parameter adjustment provided in an embodiment of this application;

[0030] Figure 6 is a schematic diagram of a model architecture provided in an embodiment of this application;

[0031] Figure 7 is a schematic diagram of a learning rate reduction strategy provided in an embodiment of this application;

[0032] Figure 8 is a schematic diagram of a model architecture provided in an embodiment of this application;

[0033] Figure 9 is a flowchart of a model training method in a practical application scenario provided by an embodiment of this application;

[0034] Figure 10 is a schematic diagram of a model architecture in a practical application scenario provided by an embodiment of this application;

[0035] Figure 11 is a structural block diagram of a model training device provided in an embodiment of this application;

[0036] Figure 12 is a structural diagram of a terminal provided in an embodiment of this application;

[0037] Figure 13 is a structural diagram of a server provided in an embodiment of this application. Detailed Implementation

[0038] The embodiments of this application will now be described with reference to the accompanying drawings.

[0039] In the field of data processing, using models for data processing is a common approach. The data processing goals that a model can achieve typically depend on how it is trained. For example, training a model using sample data and reference processing results obtained by performing a specific data processing task on the sample data allows the model to learn how to accurately perform that task.

[0040] However, the data processing goals a model needs to achieve in real-world applications may differ from those achieved during training. For example, during training, the model might only achieve first-level precision data processing, while in real-world applications, second-level precision might be required. Alternatively, the data processing tasks performed in real-world applications might differ from those during training. For instance, when the model is an object recognition model, the training samples primarily consist of images containing objects, thus the model mainly learns how to recognize objects within images. However, in real-world applications, the model might need to identify people within images. Using the same model for object recognition could lead to inaccuracies. Therefore, when a new data processing goal is required, the model needs to learn the necessary knowledge to achieve that new goal.

[0041] In related technologies, to improve model training efficiency, one approach is to avoid adjusting existing model parameters and instead add a small number of new parameters. These new parameters are then adjusted using sample data not used in the original training process, allowing the model to learn how to achieve new data processing goals. However, because these new parameters are small and can only be processed once in the data processing flow, the model's learning capacity is limited, making it difficult to achieve the desired new data processing objectives.

[0042] To address the aforementioned technical issues, this application provides a model training method. By adding multiple incremental branches, which, together with the original main branch in the first model, constitute multiple processing branches in the second model, when processing sample data used to teach the second model how to achieve new data processing goals, the multiple incremental branches and the main branch in the first model perform the same type of processing operations on the same data, obtaining corresponding processing results. Combining the processing results from each of the multiple processing branches, the processing result determined by the model can be output. Since the multiple processing branches are independent of each other during data processing, the processing dimensions of data processing can be enriched through multiple incremental branches. Therefore, when adjusting the model parameters of the multiple incremental branches to complete model training, the model can fully learn the data processing methods of new data processing tasks, improving the model's processing accuracy for new data processing tasks without excessively increasing the number of model parameters.

[0043] Understandably, this method can be applied to computer devices capable of model training, such as terminal devices or servers. This method can be executed independently by a terminal device or server, or it can be applied to network scenarios where the terminal device and server communicate, executing in cooperation. The terminal device can be a mobile phone, tablet, laptop, desktop computer, etc. The terminal device can also include various virtual reality devices, such as augmented reality (AR) devices like AR glasses and AR screens, and virtual reality (VR) devices like VR headsets. The server can be understood as an application server or a web server. In actual deployment, the server can be a standalone server, a cluster server, or a cloud server, etc.

[0044] To facilitate understanding of the technical solution provided in this application, the model training method provided in this application will be introduced next in conjunction with a practical application scenario.

[0045] Referring to Figure 1, which is a schematic diagram of a model training method in a practical application scenario provided by an embodiment of this application, in this practical application scenario, the computer device can be a server 101 with model training function, the first model is a model capable of performing a first data processing task, and the purpose of model training is to enable the first model to perform a second data processing task. The server 101 can first obtain the first model and sample data. The first model is used to perform the first data processing task, which can be, for example, an object recognition task for objects. The sample data is the data to be processed in the second data processing task, which can be, for example, an object recognition task for people. Accordingly, the sample data can be an image including people.

[0046] A second model can be constructed based on the first model and N (N is an integer greater than 1) incremental branches. This second model is used to learn how to execute the second data processing task. The incremental model parameters of each of the N incremental branches can be set in various ways, which are not limited here. The model parameters of the second model include the target model parameters of the first model itself and the incremental model parameters of each of the N incremental branches. Among them, the main branch composed of the target model parameters of the first model and the N incremental branches together constitute multiple processing branches. Multiple processing branches are used to independently execute the target processing flow in the second data processing task through processing operations of the same category. After the sample data is input into the second model, the second model will determine the input data corresponding to the target processing flow. This input data will be input into multiple processing branches respectively, and each processing branch will determine the corresponding processing result. The second model can combine the processing results determined by the multiple processing branches to determine the final output pending processing result. This pending processing result is the processing result obtained by the second model after executing the second data processing task on the sample data.

[0047] Since this application can construct multiple processing branches through the main branch and N incremental branches in the first model, and independently and in parallel complete the determination of the desired processing result, the model architecture of this second model can perform more detailed data processing on the target processing flow, improve the complexity of data analysis, and thus improve the accuracy of data processing. Server 101 can obtain the labeled processing results corresponding to the sample data, which are the processing results obtained after performing the accurate second data processing task on the sample data. Therefore, based on the difference between the labeled processing results and the desired processing result, the processing accuracy of the second model when performing the second data processing task can be characterized. This allows for the adjustment of the model parameters of multiple incremental branches, enabling the second model to learn how to accurately perform the second data processing task, resulting in a third model for performing the second data processing task.

[0048] Because this application increases the complexity of the model parameters corresponding to the incremental branches participating in the data processing in the target processing flow by adding multiple incremental branches, even though the total number of model parameters of the N incremental branches is still less than the number of parameters of the target model, it also enriches the number of incremental model parameters compared with related technologies, which can bring richer data processing performance to the target processing flow. Furthermore, by adjusting the model parameters only for the incremental branches, the effectiveness of the model parameter adjustment can be ensured while reducing the amount of model parameter adjustment. This allows the second model to effectively learn how to accurately execute the second data processing task, while also retaining the model knowledge used in the first model to execute the first data processing task, so that this model knowledge can assist the third model in executing the second data processing task.

[0049] Next, the technical solution provided in this application will be described in detail with reference to the accompanying drawings.

[0050] Referring to Figure 2, which is a flowchart of a model training method provided in an embodiment of this application, in this embodiment, the computer device used to execute the model training method can be any type of computer device capable of model training, and the method includes:

[0051] S201: Obtain the first model and sample data.

[0052] This application primarily addresses incremental training scenarios, specifically the retraining of a pre-trained model. The first model is used to achieve the old data processing objective, while incremental training aims to obtain a model capable of achieving a new data processing objective, which can be different from the old one. Since the data processing objective achievable by the model largely depends on the data used during model training, to ensure that the third model obtained through incremental training can achieve a different data processing objective than the first model, when the first model is trained based on initial sample data, the sample data used during incremental training differs from the initial sample data.

[0053] It is understood that data processing objectives can include multi-dimensional objectives. For example, two different data processing objectives can refer to achieving different levels of accuracy for the same data processing task, such as an old data processing objective of achieving 80% accuracy in object recognition and a new data processing objective of achieving 90% accuracy in object recognition. Two different data processing objectives can also refer to achieving two different data processing tasks, such as an old data processing objective of recognizing human objects in an image and a new data processing objective of recognizing animal objects in an image. The model training method in this application is applicable to the above-mentioned scenarios. For ease of distinction, in the embodiments of this application, the aforementioned old data processing objective is referred to as the first data processing objective, and the aforementioned new data processing objective is referred to as the second data processing objective.

[0054] The first model can be any model. During the training process of the model corresponding to the first model, since the first model is trained based on the initial sample data, the first model has model knowledge learned from the initial sample data to achieve the old data processing goals.

[0055] To enable the first model to learn how to achieve a new data processing goal, the computer device can acquire new sample data with corresponding labeled processing results. These labeled processing results represent the processing outcome obtained by performing data processing on the sample data when achieving the new data processing goal. For example, when the second data processing goal is the same data processing task with higher accuracy than the first data processing goal, the new sample data can be the sample data corresponding to that data processing task, and the labeled processing results represent the processing outcome obtained by accurately performing the data processing task on the sample data. Since this sample data and labeled processing results are not involved in the training process of the first model, the first model can learn model knowledge for accurately performing the new data processing task, thereby improving the accuracy of performing the data processing task. Alternatively, when the second data processing goal is to achieve a data processing task different from the first data processing goal, the new sample data can be the sample data in the data processing task corresponding to the second data processing goal; that is, the labeled processing results corresponding to the sample data represent the processing outcome obtained by accurately performing that data processing task on the sample data.

[0056] S202: The second model is constructed by adding multiple incremental branches to the first model.

[0057] As can be seen from the above analysis, the inaccuracies in model training methods in related technologies are mainly due to the following two aspects:

[0058] 1. The incremental model has too few parameters.

[0059] Because of the insufficient number of parameters, even if these incremental model parameters can be fully adjusted, it is difficult for the model to learn model knowledge related to the new data processing objectives.

[0060] Second, the data processing flow involved in the incremental model parameters is too simple.

[0061] In related technologies, all incremental model parameters are processed only once in a single data processing task. This results in a relatively simple data processing flow based on incremental model parameters, making it difficult to perform complex and detailed data analysis. Consequently, models trained in this way are less likely to accurately execute new data processing tasks.

[0062] In this application, to address the two aforementioned technical deficiencies, a second model can be constructed based on multiple incremental branches and a first model. The model parameters of the second model include the model parameters corresponding to the first model and the model parameters of each of the multiple incremental branches. The total number of model parameters for the multiple incremental branches is less than the number of model parameters for the main branch (i.e., the branch corresponding to the first model). The second model includes multiple processing branches (i.e., the aforementioned main branch and multiple incremental branches), as shown in Figure 1. Both the multiple incremental branches and the main branch in the first model are used to perform the same type of processing operations on the same data, thereby enabling multiple data processing operations to be performed on the same data processing flow. The new data processing target is the data processing objective achieved by performing this processing operation. This application increases the number of incremental model parameters (i.e., the model parameters of the multiple incremental branches) while keeping the number of parameters smaller than that of the second model, and enriches the model architecture involved in the incremental model parameters in the second model, providing architectural support for the more complex processing flow described below.

[0063] It is important to emphasize that the processing operations performed by these multiple processing branches can be any process involved in achieving the new data processing goal, that is, all or part of the processing process. The model parameters of the main branch are the model parameters used to perform this processing operation in the first model parameters.

[0064] Understandably, the purpose of setting up multiple incremental branches is to replace the original main branch of the first model in learning model knowledge. Therefore, this application allows multiple incremental branches to perform the same data processing operations as the main branch; that is, multiple processing branches perform the same data processing operations. The target processing branch is used to perform this data processing operation on the input data corresponding to the target processing flow using the model parameters corresponding to the target processing branch, to obtain the processing result corresponding to the target processing branch. For example, this data processing operation can be a feature extraction operation; that is, multiple processing branches are all used to perform feature extraction operations on the input data corresponding to the target processing flow to obtain the processing result corresponding to each processing branch. The target processing branch can be any one of the multiple processing branches.

[0065] Since the incremental branch performs the same data processing operations as the main branch, adjusting the model parameters in the incremental branch during model training can effectively simulate the adjustment method of the model parameters in the main branch. This allows the second model to learn accurate model knowledge by adjusting the model parameters in the incremental branch, thereby accurately achieving the second data processing goal.

[0066] It should be emphasized that the same data processing operation does not necessarily mean the same data processing method. For example, both the main branch and the incremental branch can perform feature extraction operations (such as convolution operations). However, the main branch directly extracts features through its model parameters, while the incremental branch can extract features using the first and second parameters through the aforementioned parameter reduction method. This also falls under the same data processing operation described in this application.

[0067] S203: Using the second model, the pending processing results are obtained based on the sample data.

[0068] The pending processing result is determined by combining the processing results corresponding to multiple processing branches. Since multiple processing branches process data independently during the execution of the target processing flow, even if the number of model parameters corresponding to each incremental branch is small, that is, the data processing method of each incremental branch is relatively simple, the combination of multiple incremental branches can still process the input data in a more detailed and complex manner. Thus, the data processing result obtained by combining multiple processing branches can support high accuracy and rationality. In other words, this data processing method can support the second model to effectively learn sufficient model knowledge through the model parameters of the incremental branches, thereby effectively learning how to achieve new data processing goals.

[0069] Meanwhile, since the data processing of multiple processing branches is independent of each other, multiple processing branches can be executed in parallel when performing the same processing operation, which ensures the execution efficiency of the second model in performing data processing tasks, and in turn ensures the training efficiency of the model, avoiding the loss of data processing efficiency due to the increase in the number of model parameters in the incremental branch.

[0070] S204: Based on the difference between the pending processing results and the labeled processing results of the sample data, adjust the model parameters of multiple incremental branches to obtain the third model.

[0071] The model parameters of multiple incremental branches can also be called incremental model parameters. Since the labeled processing result is the result obtained by processing sample data to achieve a new data processing goal, the difference between the labeled processing result and the pending processing result can characterize the gap between the current data processing goal achieved by the second model and the new data processing goal; the larger the difference, the greater the gap. Therefore, the computer device can adjust the parameters of multiple incremental models based on this difference, enabling the second model to learn how to determine a pending processing result close to the labeled processing result through the incremental model parameters. This allows the second model to learn model knowledge for achieving a new third data processing goal by adjusting the incremental model parameters, resulting in a third model that can be used to achieve the new third data processing goal. For example, training with new sample data for the same data processing task can enable the third model to achieve higher accuracy in data processing for that task; using sample data corresponding to a new data processing task, the third model can be used to execute the new data processing task.

[0072] Since the incremental model parameters of this application can be used for more complex and detailed analysis of sample data, sufficient model knowledge can be learned by adjusting only the incremental model parameters during model training, without the need to adjust the original model parameters of the first model as a whole. Therefore, the amount of parameter adjustment required for model training can be reduced to a certain extent, thereby improving the model training efficiency.

[0073] In summary, this application can bring about the following technical effects:

[0074] Firstly, since this application adds multiple incremental branches during incremental training, it enriches the number of parameters in the incremental model, thereby improving the model's knowledge learning ability brought about by adjusting the parameters of the incremental model.

[0075] Secondly, the multiple incremental branches in this application and the original main branch of the first model can independently constitute multiple processing branches. When executing the data processing flow, they can perform data processing independently. Therefore, under the premise that the overall number of parameters in the incremental model is low, it can also enrich the data analysis process during data processing and improve the processing accuracy of data processing.

[0076] Thirdly, multiple independent processing branches can perform data processing in parallel. Therefore, even if multiple processing branches are added, it will not have too much impact on the data processing efficiency of the model, thus ensuring the data processing efficiency of the model.

[0077] Fourthly, since this application only adjusts the incremental model parameters when adjusting the model parameters, while keeping the original model parameters of the first model unchanged, it can learn new model knowledge through the incremental model parameters while retaining the model knowledge of the first model, thus enriching the model knowledge and further improving the accuracy of data processing.

[0078] Fifthly, since the total number of parameters in multiple incremental branches is less than the number of parameters in the first model, this parameter tuning method requires less parameter tuning and has higher model training efficiency.

[0079] Sixthly, since the training effect brought about by the multi-processing branch model architecture in this application is better, the data processing flow corresponding to the data processing operation executed by the multi-processing branch in this application can be a part of the complete data processing flow, that is, it only plays a role in a part of the model architecture, and can also ensure the accuracy of the overall data processing of the model.

[0080] Since the incremental model parameters in this application can support relatively complex data processing functions, even if the incremental model parameters only participate in a portion of the data processing task, the data processing task can still achieve high processing accuracy. Based on this, in one possible implementation, the model parameters of the main branch are a subset of the model parameters in the first model, and the processing operations performed by multiple processing branches are used to complete the target processing flow in the third model. The target processing flow can be a portion of the data processing task. Because it does not need to participate in the entire data processing task, the number of incremental model parameters required is reduced to a certain extent. This further reduces the number of model parameters that need to be adjusted during model training while ensuring data processing accuracy, thereby further improving model training efficiency.

[0081] The third model in this application can be applied to various data processing tasks. Next, the model training method of this application will be introduced in conjunction with a specific data processing task.

[0082] In one possible application scenario, the data processing task can be used to identify target objects included in the image information based on the image information. The data processing task can include a feature extraction process, a location recognition process, and a category recognition process. The target processing process can include the location recognition process and the category recognition process.

[0083] The feature extraction process extracts image feature data corresponding to the image information. This image feature data serves as the input data for the target processing process. The location recognition process determines the location data of the target object in the image information based on the image feature data, such as determining the bounding box corresponding to the target object. The category recognition process determines the object category data corresponding to the target object based on the image feature data, such as identifying the object's identity, the object's corresponding type, or the species of organism. With the participation of the above multiple incremental branches, both the location recognition and category recognition processes in the second model can perform accurate data processing. Therefore, even without adjusting the feature extraction process, it can still provide relatively accurate object recognition capabilities. While ensuring object recognition accuracy, it improves the training efficiency of the model used for object recognition. Thus, when the data processing objective changes, such as changes in object recognition accuracy or the application scenario of object recognition, the model training method of this application can be used to quickly train an object recognition model that can accurately achieve the new data processing objective.

[0084] The initial model architecture is shown in Figure 3. After the image information is input, it first passes through the feature extraction module of the corresponding feature extraction process to extract complete image features. This feature extraction module can be constructed using a backbone network, which can be of various types. For example, in this application, a parallel high-resolution network (HRNetV2P) can be used as the backbone network. HRNet has a stronger ability to extract panoramic information compared to a residual network (ResNet).

[0085] The local feature extraction module extracts local features from the image features and inputs them into various network heads. Local feature extraction can be achieved through a local feature extractor, pooling. The network heads execute the aforementioned target processing flow and include a classification head and a regression head. The classification head acquires category information and determines the classification result corresponding to the local features, while the regression head determines the location information and outputs the bounding boxes corresponding to the objects. The overall bounding boxes are obtained by the second model through overall location recognition of the image information, corresponding to multiple objects, and are used to guide the extraction method of local features.

[0086] In one possible implementation, when the data processing task of the third model is image recognition, both the sample data used to train the second model and the initial sample data used to train the first model can be images. The first model is used to identify the category of a first object in the initial sample data; that is, when training the first model with the initial sample data, it is able to acquire the ability to identify the category of the first object in the initial sample data. In this implementation, the purpose of incremental training is to enable the second model to have a stronger category recognition ability. For example, the third model obtained by training the second model can be used to identify the category of the first object in the initial sample data and the category of the second object in the sample data, i.e., it simultaneously possesses the object recognition ability of the first model and a new object recognition ability for the second object.

[0087] In this model, the first and second objects can be of the same type or different types. For example, the purpose of incremental training can be to enable the model to recognize more categories for the same type of object. For example, for the category of "cat," the first model may only be able to recognize categories such as orange cats and tabby cats, while the sample data and annotation results provide the second model with the ability to recognize the category of Siamese cats. The purpose of incremental training can also be to enable the trained third model to recognize new types of objects. For example, the first model is used to classify objects of the category "cat," and after model training with sample data and annotation results, the third model can be equipped with the ability to classify objects of the category "dog."

[0088] Next, we will provide a detailed introduction to the technical aspects involved in the model training process.

[0089] In one possible implementation, in order to further reduce the amount of model parameter adjustment during training and improve model training efficiency, the computer device can simulate large incremental branches through multiple sub-branches by splitting parameters, thereby reducing the number of parameters in the incremental model.

[0090] Taking the target increment branch as an example, it can be any one of multiple increment branches. The target increment branch includes a first sub-branch and a second sub-branch. The model parameters of the target increment branch include the first parameter of the first sub-branch and the second parameter of the second sub-branch. To ensure that the processing result corresponding to the target increment branch can be combined with the processing result corresponding to the main branch, the processing results of the two branches need to have the same data dimension, such as the same feature dimension. Since the inputs of multiple processing branches are the same input data, the model parameters corresponding to the target increment branch need to have the same dimension as the model parameters of the main branch. That is, the first and second parameters are used to simulate equivalent model parameters with the same dimension as the model parameters of the main branch.

[0091] When executing step S203, the computer device may execute steps S2031-S2033 (not shown in the figure). Steps S2031-S2033 are one possible implementation of step S203, including:

[0092] S2031: Based on the sample data, determine the input data for multiple processing branches.

[0093] When the target processing flow corresponding to the processing operations executed by multiple processing branches is the entire processing flow in the data processing task, the input data can be the input data itself; when the target processing flow is a part of the processing flow in the data processing task, the input data can be the input data itself, or it can be the data after the sample data has been processed by other processing flows in the data processing task, such as the image features extracted as mentioned above.

[0094] S2032: Based on the input data, determine the processing results corresponding to each of the multiple processing branches through multiple processing branches.

[0095] The multiple processing branches include the main branch and multiple incremental branches. The main branch is used to determine the processing result corresponding to the main branch based on its own model parameters and input data. The target incremental branch is used to determine the first intermediate data based on the input data and the first parameter, and to determine the processing result corresponding to the target incremental branch based on the first intermediate data and the second parameter. The processing results corresponding to the multiple processing branches have the same data dimension.

[0096] In other words, for the target incremental branch, processing the input data sequentially through a first sub-branch containing the first parameter and a second sub-branch containing the second parameter simulates the effect of processing the input data using equivalent model parameters with the same model parameter format dimension as the main branch. This results in a processing result with the same data dimension, allowing the processing result of the target incremental branch to be combined with the processing result of the main branch to obtain the final processing result. The sum of the first and second parameters can be less than the number of equivalent model parameters. Therefore, using the first and second parameters to replace the equivalent model parameters further reduces the number of model parameters in each incremental branch. Simultaneously, the same data dimension ensures the effectiveness of data processing, thereby improving model training efficiency.

[0097] S2033: Combine the processing results corresponding to multiple processing branches to determine the pending processing result.

[0098] There are several ways to reduce the number of model parameters by using multiple sub-branches to simulate a single branch. The following section will introduce a specific simulation method.

[0099] In one possible implementation, the input data is n-dimensional feature data, and the processing results corresponding to multiple processing branches are m-dimensional feature data. Therefore, processing the input data into the processing results requires model parameters capable of n-dimensional to m-dimensional transformation. Since the dimensions of the input data and the processing results are usually high, if the n-dimensional to m-dimensional transformation is directly completed through a single branch, the model parameters of that branch need to have a large number of parameters, for example, the model parameters need to be an n*m matrix parameter.

[0100] To reduce the number of model parameters in the incremental branch while still achieving the aforementioned dimensionality transformation effect, the computer device can split the model parameters of the incremental branch used for n-to-m dimensional transformation into two lower-dimensional model parameters: a first parameter and a second parameter. The first intermediate data consists of r-dimensional feature data, where r is less than n and r is less than m. The first parameter is used to reduce the dimensionality of the input data to determine the first intermediate data, and the second parameter is used to increase the dimensionality of the first intermediate data to determine the processing result corresponding to the target incremental branch. Since the data dimensionality is reduced during data processing, the first and second parameters can have a lower number of parameters. This allows for the simulation of the parameter effects of the original n*m-dimensional model parameters while reducing the number of model parameters in the incremental branch, thus maintaining the data processing performance of the incremental branch. Furthermore, since this application has a large number of incremental branches, this parameter reduction method can achieve a more significant reduction in parameters while ensuring model performance, preventing an excessive increase in the number of model parameters and ensuring model training efficiency.

[0101] For example, as shown in Figure 4, the input data is din-dimensional data, and the model parameters of the main branch are d. in *d out The parameters of dimension d can be directly determined through these model parameters. out Dimension processing result 1; the first parameter in the target incremental branch is d. in The parameters are r-dimensional. By using the first parameter to reduce the dimensionality of the input data, we can obtain the first intermediate data in r dimensions. This can be achieved through r*d... out The second parameter of the dimension is used to increase the dimension of the first intermediate data, which yields d. out Result 2 of the dimension processing. Since r is less than d. out Therefore, the sum of the parameters of the first and second parameters is much smaller than d. in *d outThis reduces the number of parameters in the incremental model, thus maintaining the effectiveness of the data processing workflow while further reducing the number of parameters. Both the first and second parameters can be matrix parameters. The first parameter is initialized using a Kaiming uniform distribution initialization method (e.g., a zero-mean Gaussian distribution with a bias of zero mean and zero variance). The number of output channels of the convolution kernel is specified as r. The second parameter can be a 1x1 convolution, increasing the dimension of the r-dimensional features to d. out This allows for combination with the processing results of the main branch.

[0102] Next, we will introduce the specific methods for adjusting model parameters based on the above data processing method. When executing step S204, the computer device can execute steps S2041-S2044 (not shown in the figure). Steps S2041-S2042 are one possible implementation of step S204, including:

[0103] S2041: Adjust the second parameter according to the difference between the pending processing result and the labeled processing result of the sample data, and adjust the first intermediate data according to the difference between the pending processing result and the labeled processing result of the sample data to obtain the second intermediate data.

[0104] As shown in Figure 5, the pending processing result is determined by two types of data: firstly, the first intermediate data determined by the first parameter, and secondly, the second parameter. Therefore, based on this difference, the computer can first determine how to adjust the pending processing result to make it closer to the labeled processing result. Then, through gradient propagation, it can determine how to adjust the second parameter and the first intermediate data to meet the adjustment requirements for the pending processing result. Thus, the second parameter and the first intermediate data can be adjusted according to the difference between the pending processing result and the labeled processing result. Based on the adjustment of the first intermediate data, second intermediate data that meets the adjustment requirements can be obtained.

[0105] S2042: Adjust the first parameter based on the difference between the second intermediate data and the first intermediate data.

[0106] The first intermediate data is determined by two factors: the first parameter and the input data. Since the input data is independent of the model parameters of the incremental branch, this application primarily adjusts the first parameter to ensure that the determined first intermediate data meets the adjustment method required for the desired processing result; that is, the second intermediate data is obtained by adjusting the first parameter. Therefore, by analyzing the difference between the first and second intermediate parameters, an adjustment method for the first parameter can be determined. This adjustment method ensures that the determined first intermediate parameter gradually approaches the second intermediate parameter, thereby making the desired processing result gradually approach the reference processing result.

[0107] In this way, the difference between the pending processing result and the labeled processing result can be transmitted to the first parameter and the second parameter through layer-by-layer gradient propagation. This allows the computer equipment to learn how to adjust the first parameter and the second parameter to make the target processing flow more accurate. Furthermore, by adjusting the model parameters of the incremental branch, it can learn more effectively how to achieve the second data processing goal.

[0108] As mentioned above, this application can improve the complexity and accuracy of data processing through multiple incremental branches. Therefore, even if the data processing accuracy is reduced to some extent for each incremental branch, the combination of multiple incremental branches can compensate for this accuracy loss and ensure the data processing effect of the model. Based on this, in one possible implementation, in order to further improve the model training efficiency, the computer equipment can reduce the amount of data processing in the incremental branches.

[0109] In this implementation, the main branch is specifically used for:

[0110] Feature extraction is performed on the input data to generate the processing result corresponding to the main branch, wherein the processing result corresponding to the main branch is feature data with the first feature size.

[0111] Taking the target increment branch as an example, the target increment branch is specifically used for:

[0112] The first sub-branch extracts features from the input data based on the first parameter, generating first intermediate data, which is feature data with a second feature size. The computational cost of feature extraction is positively correlated with the feature size (i.e., the second feature size) of the generated feature data; that is, the larger the extracted feature size, the more computation is required. Since the second feature size is smaller than the first feature size, the computational cost of feature extraction based on the first parameter can be reduced. Although this reduction in computational cost may lead to a decrease in model data processing accuracy to some extent, the application has multiple incremental branches. Therefore, by combining the independent data processing functions of these multiple incremental branches, the accuracy loss in a single incremental branch can be compensated for to some extent, thereby ensuring the overall data processing accuracy.

[0113] Then, the target incremental branch can use the second sub-branch to process the first intermediate data according to the second parameter to determine the processing result corresponding to the target incremental branch. Data processing includes size enlargement processing to expand the feature size. That is, in the process of determining the processing result, the size of the first intermediate data will be restored, and the processing result will be feature data with the first feature size. It can be understood that the role of the second parameter is to process the first intermediate data to determine a more accurate model output based on the processed data, not to restore the size. That is, the role of the second sub-branch includes two aspects: one is to perform data processing, and the other is to restore the size. Size restoration only requires simple linear data calculation, so the amount of computation required for the restoration process is low. Overall, it can still achieve a significant reduction in the amount of computation for each incremental branch. In addition, since the feature size of the intermediate data involved in the data processing in the incremental branch is reduced, and as seen from the above process, the difference in the intermediate data is one of the bases for adjusting the incremental model parameters, the reduction in feature size will lead to a reduction in the difference in the intermediate data, thereby reducing the amount of adjustment required for model parameter adjustment to a certain extent, improving the model fitting speed, and further improving the model training efficiency.

[0114] Since the correlation between the data processing and dimensional restoration of the second parameter is low, the order of data processing and dimensional restoration is not limited in this application. The technical solutions and effects of different orders will be described in detail below.

[0115] The first method: restore first, then process.

[0116] In one possible implementation, when processing the first intermediate data according to the second parameter to determine the processing result corresponding to the target incremental branch, the computer device can first enlarge the size of the first intermediate data to generate third intermediate data, which is feature data with the first feature size. Then, based on the third intermediate data and the second parameter, the processing result corresponding to the target incremental branch is determined; that is, by processing the third intermediate data using the second parameter, the processing result corresponding to the target incremental branch is obtained.

[0117] The advantage of this approach is that, since the second parameter is used for data processing on the third intermediate data with a larger first feature size, the data processing on the third intermediate data is more complex and detailed. This can improve the processing accuracy of data processing using the second parameter to a certain extent, thereby improving the data processing accuracy of the target incremental branch. The processing procedure is shown in Figure 6.

[0118] The specific execution methods for feature extraction and size restoration can include various methods, and the following will mainly focus on the convolution processing method.

[0119] In one possible implementation, when extracting features from the input data in the main branch and generating the processing result corresponding to the main branch, the computer device can extract features from the input data through p convolution processes and generate the processing result corresponding to the main branch. The number of convolution processes determines the amount of computation required for the convolution process. The more times the convolution process is performed, the greater the amount of computation and the larger the feature size obtained.

[0120] In the incremental branch, when extracting features from the input data based on the first parameter to generate the first intermediate data, the computer can perform feature extraction on the input data through q convolution processes based on the first parameter, generating the first intermediate data. Here, q is less than p, meaning the number of convolutions in the incremental branch is less than the number of convolutions in the main branch, thus requiring less computation. The number of convolutions can be adjusted by adjusting the stride parameter during convolution processing. A larger stride parameter results in a larger span in the data for each convolution process, requiring fewer convolution processes and yielding a smaller feature size. For example, setting the stride parameter to 4 times that of the main branch allows for upsampling by 4 times during convolution processing, resulting in first intermediate data with a feature size 1 / 16th of the feature size corresponding to the main branch processing result.

[0121] Size enlargement processing can include bilinear interpolation upsampling, a simple linear processing method that requires no parameters other than the first intermediate parameter, resulting in low computational cost and high processing efficiency. For example, a computer device can use bilinear interpolation upsampling to enlarge the feature size to 16 times the original size to obtain third intermediate data.

[0122] The second method: process first, then restore.

[0123] In another possible implementation, as shown in Figure 8, when processing the first intermediate data according to the second parameter to determine the processing result corresponding to the target incremental branch, the computer device can first determine the fourth intermediate data based on the first intermediate data and the second parameter. This fourth intermediate data is feature data with a second feature size. That is, when processing the first intermediate data according to the second parameter, the feature size of the first intermediate data is not increased; only data processing is performed. After processing by the second parameter, the size of the fourth intermediate data is increased to generate the processing result corresponding to the target incremental branch, which is the processing result with the first feature size.

[0124] The advantage of this approach is that the feature size corresponding to the data that needs to be processed by the second parameter is relatively small, so the amount of computation required for data processing is small. The amount of computation required for feature restoration is not directly related to whether data processing is performed by the second parameter. Therefore, this approach can further reduce the amount of computation required for data processing by the target incremental branch, thereby further improving the model training efficiency.

[0125] Next, we will introduce the specific methods for adjusting the model parameters.

[0126] As mentioned above, this application improves the complexity and accuracy of data processing by adding multiple incremental branches, enabling more detailed data processing. However, the model parameters of the multiple incremental branches are independent of each other, and the number of model parameters in each incremental branch is relatively low. Therefore, when adjusting the model parameters, the amount of parameter adjustment required for each incremental branch is small, thereby reducing the difficulty of model parameter fitting. Based on this, in one possible implementation, to further improve the model training accuracy and efficiency, the computer device can gradually reduce the learning rate of the model parameters during the parameter adjustment process, thereby obtaining effective model parameters more quickly.

[0127] When performing step S204, the computer device may perform step S2045 (not shown in the figure). Step S2045 is a possible implementation of step S204, including:

[0128] S2045: Based on the difference between the pending processing results and the labeled processing results of the sample data, perform multiple rounds of parameter adjustment on the model parameters of multiple incremental branches.

[0129] Each round of parameter tuning has a corresponding learning rate, which decreases as the number of rounds increases. Taking the target round parameter tuning as an example, this round is any one of the multiple rounds. The learning rate corresponding to the target round parameter tuning controls the adjustment magnitude of the model parameters for multiple incremental branches, and is positively correlated with the adjustment magnitude. Since there are fewer model parameters on each incremental branch, the required parameter tuning is lower. A smaller adjustment magnitude helps to find the incremental model parameters suitable for performing the data processing task more efficiently. Therefore, by gradually decreasing the learning rate during the tuning process, the model fitting speed can be further improved, and the target model used to achieve the second data processing objective can be trained more efficiently.

[0130] Furthermore, because the required parameter adjustment amount is small, the computer equipment can also limit the number of parameter adjustment rounds to a small range. For example, in related technologies, since the parameter adjustment amount is not reduced, it may be necessary to perform 36 rounds of parameter adjustment to achieve effective model training. In this application, because the parameter adjustment amount is reduced, the number of parameter adjustment rounds can be reduced from 36 to 12, the initial value of the learning rate is adjusted to 1 / 10 of the original learning rate, and gradually decreased over 12 rounds of parameter adjustment. Various strategies can be used to decrease the learning rate, such as the cosine annealing (Cosine AnnealingLR) learning rate adjustment strategy. The change of the learning rate controlled by this strategy over multiple rounds of parameter adjustment is shown in Figure 7, where the horizontal axis represents the number of parameter adjustment rounds and the vertical axis represents the learning rate.

[0131] Furthermore, since the model parameters in the multiple incremental branches are independent of each other, not only can data processing efficiency be improved by having multiple incremental branches execute data processing in parallel during data processing, but the parameter adjustment process can also be executed in parallel. In one possible implementation, when executing step S204, the computer device can execute step S2046 (not shown in the figure), where step S2046 is a possible implementation of step S204, including:

[0132] S2046: Based on the difference between the pending processing results and the labeled processing results corresponding to the sample data, adjust the model parameters of multiple incremental branches in parallel to obtain the third model.

[0133] Computer equipment can reduce the time required for parameter tuning by adjusting model parameters in multiple incremental branches in parallel, thereby ensuring model training efficiency while increasing the number of incremental model parameters.

[0134] To facilitate understanding of the technical solution provided in this application, the model training method provided in this application will be introduced next in conjunction with a practical application scenario.

[0135] Referring to Figure 9, which is a flowchart of a model training method in a practical application scenario provided by an embodiment of this application, in this practical application scenario, the computer device can be any computer device capable of model training. The purpose of incremental training is to enable the trained third model to perform new data processing tasks. The method includes:

[0136] S901: Acquire the first object recognition model and sample image information for object recognition.

[0137] In this practical application scenario, the data processing task performed by the first object recognition model (i.e., the first model) is to identify objects in the image information, while the new data processing task is to identify people in the image information. The first object recognition model is used to identify objects in the image information, such as vehicles and houses. The sample image information includes images of people, and the sample image information has corresponding labeled object recognition results, which are the recognition results obtained by accurately identifying people in the sample image information.

[0138] S902: The second model is formed by combining multiple incremental branches with the first object recognition model.

[0139] The second model architecture in this practical application scenario can be shown in Figure 10. Multiple incremental branches have corresponding first and second parameters to form the first and second sub-branches in each incremental branch. Together with the model parameters in the first object recognition model, they form multiple processing branches. These multiple processing branches are used to determine the object recognition result based on image features. The object recognition result includes location data and object category data. For example, the model parameters in the incremental branches and the model parameters in the main branches are the model parameters in the regression head and the classification head, respectively.

[0140] S903: Using the second model, sample image information is used to generate a target object recognition result for a person.

[0141] The data processing flow in step S903 can be shown in steps S9031-S9036.

[0142] S9031: The feature extraction section extracts the image features corresponding to the sample image information.

[0143] The feature extraction part is used to extract image features corresponding to image information. This feature extraction part can use a variety of feature extraction networks, such as ResNet network, visual geometry group network, etc., which are not limited here.

[0144] S9032: Determine location data and object type data through model parameters in the main branch.

[0145] The location data and object type data are the object recognition results corresponding to the main branch, which are the data corresponding to the first feature size. The main branch is used to perform multiple convolution processes on the image features.

[0146] S9033: Feature extraction is performed using the first parameter in the incremental branch to obtain the first intermediate data.

[0147] Taking the incremental branch as an example, the data processing flow of the incremental branch is introduced. The first parameter is used to perform multiple convolution processes on the image features to complete feature extraction. The number of convolution processes is less than the number of convolution processes in the main branch, thus obtaining the first intermediate data of the second feature size. This size scaling can use various magnifications, such as 1 / 2, 1 / 4, 16 / 1, etc., which are not limited here.

[0148] The first parameter is d in The model parameters are r-dimensional, with the second parameter being r*d. out The model parameters are d-dimensional, and the image features are d-dimensional. in The characteristics of the processing branch are used to output the object recognition result d. out Dimensional data, r is less than d out Therefore, the sum of the parameters of the first parameter and the second parameter is much smaller than the number of model parameters of the main branch.

[0149] S9034: The first intermediate data is processed using the second parameter to obtain the second intermediate data.

[0150] This data processing can also be done through convolution, used for further analysis of the first intermediate data, while simultaneously increasing the data dimension to d. out dimension.

[0151] S9035: Increase the size of the second intermediate data to obtain location data and object type data.

[0152] This size processing is used to adjust the feature size of the second intermediate data to the first feature size, so as to obtain the object recognition result output by the target incremental branch, namely the location data and object type data.

[0153] S9036: Combining the location data and object type data corresponding to multiple processing branches, output the identification result of the object to be identified.

[0154] S904: Based on the difference between the identification results of the undetermined object and the identification results of the labeled object, the model parameters of multiple incremental branches are adjusted in parallel to obtain the second object identification model (i.e., the third model mentioned above).

[0155] When adjusting model parameters, the computer device can do so with fewer rounds, for example, 12 rounds. The learning rate gradually decreases as the number of rounds increases, and various learning rate reduction strategies can be used, such as the CosineAnnealingLR strategy and the distributed strategy (MultStep), etc., which are not limited here. The incremental model parameters adjusted in this application can be used not only for the network head, but also for parts of the feature extraction process, such as the fourth stage of the four stages of the backbone network in the feature extraction part. The original model parameters of the first object recognition model are not adjusted.

[0156] S905: Identify people in image information using a second object recognition model.

[0157] As can be seen from the above process, this application has the following outstanding technical effects compared with related technologies:

[0158] 1. Because this application adds multiple incremental branches during incremental training, it enriches the number of parameters in the incremental model, thus improving the model's knowledge learning ability brought about by adjusting the parameters of the incremental branches. Even if only the incremental model parameters are added to the network head, it can still bring accurate object recognition results overall.

[0159] 2. The multiple incremental branches in this application can independently constitute multiple processing branches, and can perform data processing independently when executing the target processing flow. Therefore, under the premise that the overall parameter quantity of the incremental model is low, it can also enrich the data analysis process during data processing and improve the processing accuracy of data processing.

[0160] 3. Multiple independent processing branches can perform data processing and model parameter tuning in parallel. Therefore, even if multiple processing branches are added, it will not have too much impact on the model's data processing efficiency and parameter tuning efficiency, thus ensuring the model's data processing efficiency and model training efficiency.

[0161] 4. Since this application only adjusts the incremental model parameters when adjusting the model parameters, while keeping the model parameters corresponding to the first model unchanged, it can learn new model knowledge through the incremental model parameters while retaining the model knowledge of the first model, thus enriching the model knowledge and further improving the accuracy of data processing.

[0162] 5. Since the total number of parameters in the incremental model is smaller than that in the main branch model, and by splitting the parameters, multiple small incremental model parameters are used to simulate large incremental model parameters, further reducing the number of parameters in the incremental model, this parameter adjustment method requires less parameter adjustment and has higher model training efficiency.

[0163] 6. This application can further reduce the amount of data processing required for each incremental branch and reduce the difficulty of parameter adjustment by reducing the data feature size in the incremental branch. At the same time, multiple incremental branches can ensure the accuracy of overall data processing and further improve the model training efficiency.

[0164] Based on the model training method provided in the above embodiments, this application also provides a model training device. Referring to FIG11, FIG11 is a structural block diagram of a model training device provided in this application. The device 1100 includes an acquisition unit 1101, a configuration unit 1102, a determination unit 1103, and an adjustment unit 1104.

[0165] The acquisition unit 1101 is used to acquire a first model and sample data, wherein the first model is trained based on initial sample data and the sample data is different from the initial sample data.

[0166] The constitutive unit 1102 is used to form a second model by adding multiple incremental branches to the first model. The multiple incremental branches and the main branch in the first model are used to perform the same type of processing operation on the same data. The total number of model parameters of the multiple incremental branches is less than the number of model parameters of the main branch.

[0167] The determining unit 1103 is used to obtain a pending processing result based on the sample data through the second model. The pending processing result is determined by combining the processing results corresponding to the multiple incremental branches and the main branch respectively.

[0168] The adjustment unit 1104 is used to adjust the model parameters of the multiple incremental branches according to the difference between the pending processing result and the labeled processing result of the sample data, so as to obtain a third model.

[0169] In one possible implementation, the target incremental branch is any one of the plurality of incremental branches, the target incremental branch includes a first sub-branch and a second sub-branch, and the incremental model parameters of the target incremental branch include a first parameter of the first sub-branch and a second parameter of the second sub-branch. The determining unit 1103 is specifically used for:

[0170] Based on the sample data, input data for multiple processing branches are determined, wherein the multiple processing branches include the main branch and the multiple incremental branches;

[0171] Through the multiple processing branches, based on the input data, the processing results corresponding to the multiple processing branches are determined respectively. The target incremental branch is used to determine the first intermediate data based on the input data and the first parameter, and to determine the processing result corresponding to the target incremental branch based on the first intermediate data and the second parameter. The processing results corresponding to the multiple processing branches have the same data dimension.

[0172] The pending processing result is determined by combining the processing results corresponding to the multiple processing branches.

[0173] In one possible implementation, the input data is n-dimensional feature data, the processing results corresponding to the multiple processing branches are m-dimensional feature data, the first intermediate data is r-dimensional feature data, where r is less than n and r is less than m, the first sub-branch is used to perform dimensionality reduction processing on the input data to determine the first intermediate data, and the second sub-branch is used to perform dimensionality increase processing on the first intermediate data to determine the processing result corresponding to the target incremental branch.

[0174] In one possible implementation, the adjustment unit 1104 is specifically configured to include:

[0175] The second parameter is adjusted based on the difference between the pending processing result and the labeled processing result, and the first intermediate data is adjusted based on the difference between the pending processing result and the labeled processing result to obtain the second intermediate data;

[0176] The first parameter is adjusted based on the difference between the second intermediate data and the first intermediate data.

[0177] In one possible implementation, the main branch is specifically used for:

[0178] Feature extraction is performed on the input data to generate the processing result corresponding to the main branch. The processing result corresponding to the main branch is feature data with a first feature size.

[0179] The target incremental branch is specifically used for:

[0180] Through the first sub-branch, feature extraction is performed on the input data according to the first parameter to generate the first intermediate data. The first intermediate data is feature data with a second feature size. The computational amount required for feature extraction is positively correlated with the second feature size, and the second feature size is smaller than the first feature size.

[0181] Through the second sub-branch, the first intermediate data is processed according to the second parameter to determine the processing result corresponding to the target incremental branch. The data processing includes size enlargement processing for expanding the feature size, and the processing result is feature data with the first feature size.

[0182] In one possible implementation, data processing is performed on the first intermediate data based on the second parameter to determine the processing result corresponding to the target incremental branch, including:

[0183] The size of the first intermediate data is increased to generate third intermediate data, which is feature data with the first feature size.

[0184] Based on the third intermediate data and the second parameter, the processing result corresponding to the target incremental branch is determined.

[0185] In one possible implementation, the step of extracting features from the input data to generate the processing result corresponding to the main branch includes:

[0186] The input data is processed by p convolutions to extract features and generate the processing result corresponding to the main branch.

[0187] The step of extracting features from the input data based on the first parameter to generate the first intermediate data includes:

[0188] Based on the first parameter, the input data is processed by q convolutions to extract features and generate the first intermediate data, where q is less than p.

[0189] The size enlargement process includes bilinear interpolation upsampling.

[0190] In one possible implementation, the step of processing the first intermediate data according to the second parameter to determine the processing result corresponding to the target incremental branch includes:

[0191] A fourth intermediate data is determined based on the first intermediate data and the second parameter, wherein the fourth intermediate data is feature data having the second feature size;

[0192] The size of the fourth intermediate data is increased to generate the processing result corresponding to the target incremental branch.

[0193] In one possible implementation, the adjustment unit 1104 is specifically used for:

[0194] Based on the difference between the pending processing result and the labeled processing result, the model parameters of the multiple incremental branches are adjusted in multiple rounds. Each round of parameter adjustment has a corresponding learning rate, which decreases as the number of rounds increases. The target round of parameter adjustment is any one of the multiple rounds of parameter adjustment. The learning rate corresponding to the target round of parameter adjustment is used to control the adjustment range of the model parameters of the multiple incremental branches. The learning rate corresponding to the target round of parameter adjustment is positively correlated with the adjustment range.

[0195] In one possible implementation, the adjustment unit 1104 is specifically used for:

[0196] Based on the difference between the pending processing result and the labeled processing result, the model parameters of the multiple incremental branches are adjusted in parallel to obtain a third model.

[0197] In one possible implementation, both the sample data and the initial sample data are images;

[0198] The first model is used to identify the category of the first object in the initial sample data;

[0199] The third model is used to identify the category of the first object in the initial sample data and the category of the second object in the sample data.

[0200] This application also provides a computer device, as shown in Figure 12. This computer device can be a terminal device, taking a mobile phone as an example:

[0201] Figure 12 shows a block diagram of a portion of the structure of a mobile phone related to the terminal device provided in the embodiments of this application. Referring to Figure 12, the mobile phone includes components such as a radio frequency (RF) circuit 710, a memory 720, an input unit 730, a display unit 740, a sensor 750, an audio circuit 760, a wireless Fidelity (WiFi) module 770, a processor 790, and a power supply 790. Those skilled in the art will understand that the mobile phone structure shown in Figure 12 does not constitute a limitation on the mobile phone and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0202] The following section, with reference to Figure 12, provides a detailed introduction to the various components of the mobile phone:

[0203] The RF circuit 710 can be used for receiving and transmitting signals during information transmission or calls. Specifically, it receives downlink information from the base station and processes it with the processor 790; additionally, it transmits uplink data to the base station. Typically, the RF circuit 710 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier (LNA), and a duplexer. Furthermore, the RF circuit 710 can also communicate wirelessly with networks and other devices. The aforementioned wireless communication can use any communication standard or protocol, including but not limited to Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, and Short Messaging Service (SMS).

[0204] The memory 720 can be used to store software programs and modules. The processor 790 executes various mobile phone functions and data processing by running the software programs and modules stored in the memory 720. The memory 720 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory 720 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0205] The input unit 730 can be used to receive input numerical or character information, and to generate key signal inputs related to user settings and function control of the mobile phone. Specifically, the input unit 730 may include a touch panel 731 and other input devices 732. The touch panel 731, also known as a touch screen, can collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch panel 731), and drive the corresponding connected devices according to a pre-set program. Optionally, the touch panel 731 may include two parts: a touch detection device and a touch controller. The touch detection device detects the user's touch position and the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, and sends it to the processor 790, and can also receive and execute commands sent by the processor 790. In addition, the touch panel 731 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 731, the input unit 730 may also include other input devices 732. Specifically, other input devices 732 may include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc.

[0206] The display unit 740 can be used to display information input by the user or information provided to the user, as well as various menus of the mobile phone. The display unit 740 may include a display panel 741, which may optionally be configured as a Liquid Crystal Display (LCD), Organic Light-Emitting Diode (OLED), or similar display panel. Further, a touch panel 731 may cover the display panel 741. When the touch panel 731 detects a touch operation on or near it, it transmits the information to the processor 790 to determine the type of touch event. Subsequently, the processor 790 provides corresponding visual output on the display panel 741 based on the type of touch event. Although in Figure 12, the touch panel 731 and the display panel 741 are shown as two separate components for implementing the input and output functions of the mobile phone, in some embodiments, the touch panel 731 and the display panel 741 can be integrated to achieve the input and output functions of the mobile phone.

[0207] The mobile phone may also include at least one sensor 750, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor. The ambient light sensor can adjust the brightness of the display panel 741 according to the ambient light level, and the proximity sensor can turn off the display panel 741 and / or backlight when the phone is moved to the ear. As a type of motion sensor, an accelerometer sensor can detect the magnitude of acceleration in various directions (generally three axes). When stationary, it can detect the magnitude and direction of gravity and can be used for applications that recognize the phone's posture (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration recognition-related functions (such as pedometer, taps), etc. Other sensors that may be configured in the mobile phone, such as gyroscopes, barometers, hygrometers, thermometers, and infrared sensors, will not be described in detail here.

[0208] Audio circuit 760, speaker 761, and microphone 762 provide an audio interface between the user and the mobile phone. Audio circuit 760 converts received audio data into electrical signals and transmits them to speaker 761, where speaker 761 converts them into sound signals for output. On the other hand, microphone 762 converts collected sound signals into electrical signals, which are received by audio circuit 760, converted into audio data, and then processed by processor 790 before being transmitted via RF circuit 710 to, for example, another mobile phone, or the audio data can be output to memory 720 for further processing.

[0209] WiFi is a short-range wireless transmission technology. A mobile phone using the WiFi module 770 can help users send and receive emails, browse web pages, and access streaming media, providing wireless broadband internet access. Although Figure 12 shows the WiFi module 770, it is understood that it is not an essential component of a mobile phone and can be omitted as needed without altering the essence of the invention.

[0210] The processor 790 is the control center of the mobile phone, connecting various parts of the phone through various interfaces and lines. It executes software programs and / or modules stored in the memory 720, and calls data stored in the memory 720 to perform various functions and process data, thereby performing overall detection of the phone. Optionally, the processor 790 may include one or more processing units; preferably, the processor 790 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may also not be integrated into the processor 790.

[0211] The mobile phone also includes a power supply 790 (such as a battery) that supplies power to various components. Preferably, the power supply can be logically connected to the processor 790 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system.

[0212] Although not shown, mobile phones may also include a camera, Bluetooth module, etc., which will not be described in detail here.

[0213] In this embodiment, the processor 790 included in the terminal device is also used to execute any one of the implementation methods of the model training method provided in the embodiments of this application.

[0214] This application embodiment also provides a server. Please refer to Figure 13, which is a structural diagram of the server 900 provided in this application embodiment. The server 900 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 922 (e.g., one or more processors) and a memory 932, and one or more storage media 930 (e.g., one or more mass storage devices) for storing application programs 942 or data 944. The memory 932 and storage media 930 can be temporary or persistent storage. The program stored in the storage media 930 may include one or more modules (not shown in the figure), each module may include a series of instruction operations on the server. Furthermore, the central processing unit 922 may be configured to communicate with the storage media 930 and execute the series of instruction operations in the storage media 930 on the server 900.

[0215] Server 900 may also include one or more power supplies 926, one or more wired or wireless network interfaces 950, one or more input / output interfaces 959, and / or one or more operating systems 941, such as Windows Server. TM Mac OS X TM Unix TM Linux TM FreeBSD TM etc.

[0216] The steps performed by the server in the above embodiments can be based on the server structure shown in Figure 13.

[0217] This application also provides a computer-readable storage medium for storing a computer program that executes any one of the model training methods described in the foregoing embodiments.

[0218] This application also provides a computer program product including a computer program, which, when run on a computer device, causes the computer device to execute any of the model training methods described in the above embodiments.

[0219] It is understood that in the specific embodiments of this application, data such as user information are involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0220] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium can be at least one of the following media: read-only memory (ROM), RAM, magnetic disk, or optical disk, etc., and other media capable of storing program code.

[0221] It should be noted that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for the device and system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiments. The device and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the solution in this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0222] The above description is merely one specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A model training method, executed by a computer device, the method comprising: Acquire a first model and sample data, wherein the first model is trained based on initial sample data, and the sample data is different from the initial sample data; By adding multiple incremental branches to the first model, a second model is formed. The multiple incremental branches and the main branch in the first model are used to perform the same type of processing operations on the same data. The total number of model parameters of the multiple incremental branches is less than the number of model parameters of the main branch. The second model is used to obtain a pending processing result based on the sample data. The pending processing result is determined by combining the processing results corresponding to the multiple incremental branches and the main branch respectively. Based on the difference between the pending processing result and the labeled processing result of the sample data, the model parameters of the multiple incremental branches are adjusted to obtain the third model.

2. The method according to claim 1, wherein, The target incremental branch is any one of the plurality of incremental branches. The target incremental branch includes a first sub-branch and a second sub-branch. The model parameters of the target incremental branch include a first parameter of the first sub-branch and a second parameter of the second sub-branch. Obtaining the undetermined processing result based on the sample data using the second model includes: Based on the sample data, input data for multiple processing branches are determined, wherein the multiple processing branches include the main branch and the multiple incremental branches; Through the multiple processing branches, based on the input data, the processing results corresponding to the multiple processing branches are determined respectively. The target incremental branch is used to determine the first intermediate data based on the input data and the first parameter, and to determine the processing result corresponding to the target incremental branch based on the first intermediate data and the second parameter. The processing results corresponding to the multiple processing branches have the same data dimension. The pending processing result is determined by combining the processing results corresponding to the multiple processing branches.

3. The method according to claim 2, wherein, The input data is n-dimensional feature data, and the processing results corresponding to the multiple processing branches are m-dimensional feature data. The first intermediate data is r-dimensional feature data, where r is less than n and r is less than m. The first sub-branch is used to perform dimensionality reduction processing on the input data to determine the first intermediate data, and the second sub-branch is used to perform dimensionality increase processing on the first intermediate data to determine the processing result corresponding to the target incremental branch.

4. The method according to claim 2 or 3, wherein, The step of adjusting the model parameters of the multiple incremental branches to obtain a third model based on the difference between the pending processing result and the labeled processing result of the sample data includes: The second parameter is adjusted based on the difference between the pending processing result and the labeled processing result, and the first intermediate data is adjusted based on the difference between the pending processing result and the labeled processing result to obtain the second intermediate data; The first parameter is adjusted based on the difference between the second intermediate data and the first intermediate data.

5. The method according to any one of claims 2 to 4, wherein, The main branch is specifically used for: Feature extraction is performed on the input data to generate the processing result corresponding to the main branch. The processing result corresponding to the main branch is feature data with a first feature size. The target incremental branch is specifically used for: Through the first sub-branch, feature extraction is performed on the input data according to the first parameter to generate the first intermediate data. The first intermediate data is feature data with a second feature size. The computational amount required for feature extraction is positively correlated with the second feature size, and the second feature size is smaller than the first feature size. Through the second sub-branch, the first intermediate data is processed according to the second parameter to determine the processing result corresponding to the target incremental branch. The data processing includes size enlargement processing for expanding the feature size, and the processing result is feature data with the first feature size.

6. The method according to claim 5, wherein, Based on the second parameter, the first intermediate data is processed to determine the processing result corresponding to the target incremental branch, including: The first intermediate data is subjected to the size enlargement process to generate third intermediate data, wherein the third intermediate data is feature data having the first feature size; Based on the third intermediate data and the second parameter, the processing result corresponding to the target incremental branch is determined.

7. The method according to claim 5 or 6, wherein, The step of extracting features from the input data to generate the processing result corresponding to the main branch includes: The input data is processed by p convolutions to extract features and generate the processing result corresponding to the main branch. The step of extracting features from the input data based on the first parameter to generate the first intermediate data includes: Based on the first parameter, the input data is processed by q convolutions to extract features and generate the first intermediate data, where q is less than p. The size enlargement process includes bilinear interpolation upsampling.

8. The method according to any one of claims 5 to 7, wherein, The step of processing the first intermediate data according to the second parameter to determine the processing result corresponding to the target incremental branch includes: A fourth intermediate data is determined based on the first intermediate data and the second parameter, wherein the fourth intermediate data is feature data having the second feature size; The fourth intermediate data is subjected to the size enlargement process to generate the processing result corresponding to the target incremental branch.

9. The method according to any one of claims 1 to 8, wherein, The step of adjusting the model parameters of the multiple incremental branches to obtain a third model based on the difference between the pending processing result and the labeled processing result of the sample data includes: Based on the difference between the pending processing result and the labeled processing result, the model parameters of the multiple incremental branches are adjusted in multiple rounds. Each round of parameter adjustment has a corresponding learning rate, which decreases as the number of rounds increases. The target round of parameter adjustment is any one of the multiple rounds of parameter adjustment. The learning rate corresponding to the target round of parameter adjustment is used to control the adjustment range of the model parameters of the multiple incremental branches. The learning rate corresponding to the target round of parameter adjustment is positively correlated with the adjustment range.

10. The method according to any one of claims 1 to 9, wherein, The step of adjusting the model parameters of the multiple incremental branches to obtain a third model based on the difference between the pending processing result and the labeled processing result of the sample data includes: Based on the difference between the pending processing result and the labeled processing result, the model parameters of the multiple incremental branches are adjusted in parallel to obtain the third model.

11. The method according to any one of claims 1 to 10, characterized in that, Both the sample data and the initial sample data are images; The first model is used to identify the category of the first object in the initial sample data; The third model is used to identify the category of the first object in the initial sample data and the category of the second object in the sample data.

12. A model training device, wherein, The device includes an acquisition unit, a configuration unit, a determination unit, and an adjustment unit: The acquisition unit is used to acquire a first model and sample data, wherein the first model is trained based on initial sample data, and the sample data is different from the initial sample data. The constituent unit is used to form a second model by adding multiple incremental branches to the first model. The multiple incremental branches and the main branch in the first model are used to perform the same type of processing operation on the same data. The total number of model parameters of the multiple incremental branches is less than the number of model parameters of the main branch. The determining unit is used to obtain a pending processing result based on the sample data through the second model. The pending processing result is determined by combining the processing results corresponding to the multiple incremental branches and the main branch respectively. The adjustment unit is used to adjust the model parameters of the multiple incremental branches according to the difference between the pending processing result and the labeled processing result of the sample data, so as to obtain a third model.

13. A computer device, wherein, The computer device includes a processor and memory: The memory is used to store computer programs and to transfer the computer programs to the processor; The processor is configured to execute the model training method according to any one of claims 1-11 according to instructions in the computer program.

14. A computer-readable storage medium, wherein, The computer-readable storage medium is used to store a computer program for executing the model training method according to any one of claims 1-11.

15. A computer program product comprising a computer program, wherein, When the computer program is executed by the processor, it implements the model training method according to any one of claims 1-11.

Images