A model training method and related device
By adding multiple incremental branches to the model and combining the processing results for parameter adjustment, the problems of insufficient incremental parameters and simple data processing flow are solved, the accuracy and efficiency of the model for new data processing tasks are improved, and the original Model knowledge.
Patent Information
- Application Number
- CN202410934764.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-12
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2044-07-12
AI Technical Summary
In the prior art, in model training, the incremental parameters are too small and the data processing flow is too simple, making it difficult for the model to learn new data processing goals and the parameter adjustment efficiency is low.
By adding multiple incremental branches to the model to be trained, the initial model is formed, and multiple incremental branches and the main branch process data together, and parameter adjustment is performed based on the processing results. The overall parameter amount of the incremental model parameters is smaller than the target model parameters, and it is independent Constitute multiple processing branches to enrich the data analysis process.
The model's processing accuracy and efficiency of new data processing tasks is improved, the parameter adjustment amount is reduced, the original model knowledge is maintained, and the model knowledge learning ability is enhanced.
Smart Images

Figure CN118627643B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of data processing technology, and in particular to a model training method and related devices. Background Art
[0002] Models are one of the most popular tools used in the field of data processing. Models can be used to efficiently achieve a variety of complex data processing goals, such as object detection, image processing, probability prediction, and other tasks. In order to enable the model to have the data processing capabilities to achieve the specified data processing goals, training samples can be constructed for the specified data processing goals, and the model can be trained through the training samples so that the model can learn how to achieve the data processing goals.
[0003] However, in different data processing scenarios, the data processing goals that the model needs to achieve may also be different. For example, in an object recognition scenario, recognizing human objects and recognizing physical objects require the model to have different object recognition capabilities, so it is necessary to adjust the model parameters. In related technologies, some parameters can be added on the basis of the original model parameters, and only these newly added parameters are adjusted during the adjustment process, so that the adjusted newly added parameters can bring certain targeted data processing capabilities to the model to achieve new data processing goals. This part of the newly added parameters only accounts for a very small part of the overall model parameters, which can reduce the amount of parameter adjustment and improve the efficiency of parameter adjustment.
[0004] Because in related technologies, the role of newly added parameters in the model data processing process is relatively simple, this parameter adjustment method makes the model knowledge learned by the model limited, and it is difficult to bring high-quality model processing performance. Summary of the invention
[0005] In order to solve the above technical problems, the present application provides a model training method that can significantly improve the model training accuracy in an incremental training scenario.
[0006] The embodiments of the present application disclose the following technical solutions:
[0007] In a first aspect, an embodiment of the present application discloses a model training method, the method comprising:
[0008] Obtaining a model to be trained and sample data, wherein the model to be trained is obtained based on initial sample data, and the sample data is different from the initial sample data;
[0009] An initial model is formed by adding multiple incremental branches to the model to be trained, wherein the multiple incremental branches and the main branch in the model to be trained are used to perform the same type of processing operations on the same data, and the overall parameter amount of the incremental model parameters of the multiple incremental branches is less than the parameter amount of the target model parameters of the main branch;
[0010] Obtaining a pending processing result according to the sample data through the initial model, wherein the pending processing result is determined by combining the processing results corresponding to the plurality of incremental branches and the main branch respectively;
[0011] According to the difference between the pending processing result and the reference processing result of the sample data, a plurality of incremental model parameters are adjusted to obtain a target model, wherein the plurality of incremental model parameters are incremental model parameters of the plurality of incremental branches.
[0012] In a second aspect, an embodiment of the present application discloses a model training device, the device comprising an acquisition unit, a composition unit, a determination unit and an adjustment unit:
[0013] The acquisition unit is used to acquire a model to be trained and sample data, wherein the model to be trained is obtained based on the initial sample data, and the sample data is different from the initial sample data;
[0014] The constituting unit is used to construct an initial model by adding a plurality of incremental branches to the model to be trained, wherein the plurality of incremental branches and the main branch in the model to be trained are used to perform the same type of processing operations on the same data, and the overall parameter amount of the incremental model parameters of the plurality of incremental branches is less than the parameter amount of the target model parameters of the main branch;
[0015] The determining unit is used to obtain a pending processing result according to the sample data through the initial model, wherein the pending processing result is determined by combining the processing results corresponding to the multiple incremental branches and the main branch respectively;
[0016] The adjustment unit is used to adjust multiple incremental model parameters according to the difference between the pending processing result and the reference processing result of the sample data to obtain a target model, and the multiple incremental model parameters are the incremental model parameters of the multiple incremental branches.
[0017] In a possible implementation, the target incremental branch is any one of the multiple incremental branches, the target incremental branch includes a first sub-branch and a second sub-branch, the incremental model parameter of the target incremental branch includes a first parameter of the first sub-branch and a second parameter of the second sub-branch, and the determining unit is specifically used to:
[0018] Determine input data of the plurality of processing branches according to the sample data, the plurality of processing branches including the main branch and the plurality of incremental branches;
[0019] Through the multiple processing branches, the processing results corresponding to the multiple processing branches are determined according to the input data, the main branch is used to determine the processing result corresponding to the main branch according to the target model parameter and the input data, the target incremental branch is used to determine the first intermediate data according to the input data and the first parameter, and determine the processing result corresponding to the target incremental branch according to the first intermediate data and the second parameter, and the processing results corresponding to the multiple processing branches respectively correspond to the same data dimension;
[0020] The pending processing result is determined by combining the processing results respectively corresponding to the multiple processing branches.
[0021] In one possible implementation, the input data is n-dimensional feature data, the processing results corresponding to the multiple processing branches are all m-dimensional feature data, the first intermediate data is r-dimensional feature data, r is less than n, and r is less than m, the first sub-branch is used to perform dimensionality reduction processing on the input data to determine the first intermediate data, and the second sub-branch is used to perform dimensionality increase processing on the first intermediate data to determine the processing result corresponding to the target incremental branch.
[0022] In a possible implementation, the adjusting unit is specifically configured to include:
[0023] adjusting the second parameter according to the difference between the pending processing result and the reference processing result of the sample data, and adjusting the first intermediate data according to the difference between the pending processing result and the reference processing result of the sample data, to obtain second intermediate data;
[0024] The first parameter is adjusted according to a difference between the second intermediate data and the first intermediate data.
[0025] In a possible implementation, the main branch is specifically used for:
[0026] Perform feature extraction on the input data according to the target model parameters to generate a processing result corresponding to the main branch, wherein the processing result corresponding to the main branch is feature data corresponding to a first feature size;
[0027] The target incremental branch is specifically used for:
[0028] By means of the first sub-branch, feature extraction is performed on the input data according to the first parameter to generate the first intermediate data, the first intermediate data is feature data corresponding to a second feature size, the amount of calculation required for feature extraction is positively correlated with the feature size corresponding to the generated feature data, and the second feature size is smaller than the first feature size;
[0029] Through the second sub-branch, the first intermediate data is processed according to the second parameter to determine the processing result corresponding to the target incremental branch, the data processing includes size processing for expanding the feature size, and the processing result is feature data corresponding to the first feature size.
[0030] In a possible implementation, performing data processing on the first intermediate data according to the second parameter to determine a processing result corresponding to the target incremental branch includes:
[0031] Performing size processing on the first intermediate data to generate third intermediate data, wherein the third intermediate data is feature data corresponding to the first feature size;
[0032] A processing result corresponding to the target incremental branch is determined according to the third intermediate data and the second parameter.
[0033] In a possible implementation, extracting features from the input data according to the target model parameters to generate a processing result corresponding to the main branch includes:
[0034] According to the target model parameters, feature extraction is performed on the input data through p convolution processes to generate a processing result corresponding to the main branch;
[0035] The step of extracting features from the input data according to the first parameter to generate the first intermediate data includes:
[0036] According to the first parameter, extracting features from the input data by performing q convolution processes to generate the first intermediate data, where q is less than p;
[0037] The resizing process includes a bilinear interpolation upsampling process.
[0038] In a possible implementation, the performing data processing on the first intermediate data according to the second parameter to determine the processing result corresponding to the target incremental branch includes:
[0039] Determine fourth intermediate data according to the first intermediate data and the second parameter, wherein the fourth intermediate data is feature data corresponding to the second feature size;
[0040] The fourth intermediate data is resized to generate a processing result corresponding to the target incremental branch.
[0041] In a possible implementation manner, the adjusting unit is specifically configured to:
[0042] According to the difference between the pending processing result and the reference processing result of the sample data, multiple rounds of parameter adjustment are performed on the multiple incremental model parameters, and the multiple rounds of parameter adjustment respectively have corresponding learning rates. The learning rate decreases as the number of rounds increases in the multiple rounds of parameter adjustment. The target round parameter adjustment is any round in the multiple rounds of parameter adjustment. The learning rate corresponding to the target round parameter adjustment is used to control the adjustment amplitude of the multiple incremental model parameters, and the learning rate corresponding to the target round parameter adjustment is positively correlated with the adjustment amplitude.
[0043] In a possible implementation manner, the adjusting unit is specifically configured to:
[0044] According to the difference between the pending processing result and the reference processing result corresponding to the sample data, the multiple incremental model parameters are adjusted in parallel to obtain a target model, and the multiple incremental model parameters are incremental model parameters of the multiple incremental branches.
[0045] In a possible implementation manner, the sample data and the initial sample data are both images;
[0046] The model to be trained is used to identify the category of the first object in the initial sample data;
[0047] The target model is used to identify the category of the first object in the initial sample data and the category of the second object in the sample data.
[0048] In a third aspect, an embodiment of the present application discloses a computer device, wherein the computer device includes a processor and a memory:
[0049] The memory is used to store a computer program and transmit the computer program to the processor;
[0050] The processor is used to execute the model training method described in any one of the first aspects according to the instructions in the computer program;
[0051] In a fourth aspect, an embodiment of the present application discloses a computer-readable storage medium, wherein the computer-readable storage medium is used to store a computer program, and the computer program is used to execute the model training method described in any one of the first aspects;
[0052] In a fifth aspect, an embodiment of the present application discloses a computer program product comprising a computer program, which, when executed on a computer device, enables the computer device to execute the model training method described in any one of the first aspects.
[0053] It can be seen from the above technical solution that when it is necessary to obtain a target model for achieving a new data processing target based on the model to be trained, the initial model can be first formed by adding multiple incremental branches to the model to be trained, so that the initial model has multiple processing branches, and the multiple processing branches include the main branch corresponding to the original target model parameters of the model to be trained and the incremental branches corresponding to the multiple incremental model parameters respectively. The overall parameter amount of the incremental model parameters of the multiple incremental branches is less than the parameter amount of the target model parameters. By adjusting the multiple incremental model parameters during the training process, the initial model can learn how to achieve the new data processing target. Among them, the multiple incremental branches and the main branch in the model to be trained are used to perform the same type of processing operations on the same data. In the process of data processing of sample data, multiple processing branches can independently process the input data to obtain the processing results corresponding to the multiple processing branches respectively, and the initial model can combine the multiple processing results to output the final pending processing result. The reference processing result corresponding to the sample data is the processing result obtained by accurately processing the sample data when the new data processing target is achieved. Therefore, the parameters of the multiple incremental model parameters are adjusted based on the difference between the pending processing result and the reference processing result, so that the initial model can learn how to achieve the new data processing target. On the one hand, since the present application adds multiple incremental model parameters during incremental training and enriches the parameter amount of the incremental model parameters, it is possible to improve the model knowledge learning ability brought about by adjusting the parameters of the incremental model parameters; on the other hand, the multiple incremental models in the present application can independently constitute multiple processing branches, and can independently perform data processing and complete the same processing operation. Therefore, under the premise that the overall parameter amount of the incremental model parameters is low, it is also possible to enrich the data analysis process during data processing and improve the processing accuracy of data processing; on the third hand, since the present application only adjusts the incremental model parameters during model parameter adjustment and keeps the original model parameters of the model to be trained unchanged, it is possible to learn new model knowledge through the incremental model parameters while retaining the model knowledge of the model to be trained, enriching the model knowledge amount, thereby further improving the data processing accuracy. In addition, since the overall parameter amount of multiple incremental model parameters is less than the parameter amount of the target model parameters, the parameter adjustment amount required for this parameter adjustment method is low, and the model training efficiency is high. Fourthly, since the training effect brought by the model architecture of multiple processing branches in this application is better and higher quality, the data processing process targeted by constructing multiple processing branches in this application can be a partial processing process, that is, it only plays a role in part of the model architecture, and can also ensure the accuracy of the overall data processing of the model. BRIEF DESCRIPTION OF THE DRAWINGS
[0054] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0055] Figure 1 A schematic diagram of a model training method in an actual application scenario provided in an embodiment of the present application;
[0056] Figure 2 A flowchart of a model training method provided in an embodiment of the present application;
[0057] Figure 3 A schematic diagram of a model architecture provided in an embodiment of the present application;
[0058] Figure 4 A schematic diagram of a model architecture provided in an embodiment of the present application;
[0059] Figure 5 A schematic diagram of model parameter adjustment provided in an embodiment of the present application;
[0060] Figure 6 A schematic diagram of a model architecture provided in an embodiment of the present application;
[0061] Figure 7 A schematic diagram of a learning rate reduction strategy provided in an embodiment of the present application;
[0062] Figure 8 A schematic diagram of a model architecture provided in an embodiment of the present application;
[0063] Fig. 9 A flowchart of a model training method in an actual application scenario provided in an embodiment of the present application;
[0064] Fig.10 A schematic diagram of a model architecture in an actual application scenario provided in an embodiment of the present application;
[0065] Fig.11 A structural block diagram of a model training device provided in an embodiment of the present application;
[0066] Fig.12 A structural diagram of a terminal provided in an embodiment of the present application;
[0067] Fig.13 A structural diagram of a server provided in an embodiment of the present application. DETAILED DESCRIPTION
[0068] The embodiments of the present application are described below in conjunction with the accompanying drawings.
[0069] In the field of data processing, data processing through models is a common data processing method. The data processing goals that a model can achieve usually depend on the training method for the model. For example, model training is performed through sample data and reference processing results obtained by performing a specified data processing task on the sample data, so that the model can learn how to accurately perform the data processing task.
[0070] However, in actual applications, the data processing goals that the model needs to achieve in actual applications and the data processing goals achieved during the training process may be different. For example, during the training process, the model can only achieve data processing with the first precision, while in the actual application process, it needs to achieve data processing with the second precision; or, the data processing tasks performed by the model in actual application and in the training process are different. For example, when the model is an object recognition model, during the training process, the training samples used are mainly images including objects, so the model mainly learns how to perform object recognition for objects in the image. When applied, it may be necessary to use the model to identify people in the image. If the original model is still used for object recognition, inaccurate recognition may occur. Therefore, when the model is required to achieve new data processing goals, the model needs to learn model knowledge used to achieve new data processing goals.
[0071] In related technologies, in order to improve the efficiency of model training, one may choose not to adjust the existing model parameters of the model, but to add a small number of model parameters to the model, and adjust the parameters of these newly added model parameters during the training process using sample data that has not been used in the original training process, so that the model can learn how to achieve new data processing goals. However, since the number of model parameters in this part is small and data processing can only be performed once in the data processing flow, the model knowledge that can be learned by adjusting these model parameters is relatively limited, making it difficult for the model to achieve new data processing goals.
[0072] In order to solve the above technical problems, the present application provides a model training method, which constitutes an initial model by adding incremental branches corresponding to multiple incremental model parameters, and together with the original main branches in the model to be trained, constitute multiple processing branches in the initial model. When processing sample data for the initial model to learn how to achieve new data processing goals, multiple incremental branches and the main branches in the model to be trained are used to perform processing operations of the same category on the same data to obtain corresponding processing results. The processing results corresponding to the multiple processing branches can be combined to output the processing results determined by the model. Since the multiple processing branches are independent of each other during the data processing process, the processing dimensions of data processing through incremental model parameters can be enriched, so that when the model training is completed by adjusting the multiple incremental model parameters, the model can fully learn the data processing method of the new data processing task, and improve the processing accuracy of the model for the new data processing task without increasing the number of model parameters too much.
[0073] It is understandable that the method can be applied to a computer device, which is a computer device capable of model training, such as a terminal device or a server. The method can be executed independently by a terminal device or a server, or it can be applied to a network scenario in which a terminal device and a server communicate, and is executed in cooperation with a terminal device and a server. Among them, the terminal device can be a mobile phone, a tablet computer, a laptop computer, a desktop computer and other devices. The terminal device can also include a variety of virtual reality devices, such as augmented reality (AR) devices, such as AR glasses, AR screens and other devices, and virtual reality technology (VR) devices, such as head-mounted VR glasses and other devices. The server can be understood as an application server, or a Web server. In actual deployment, the server can be an independent server, a cluster server, or a cloud server.
[0074] This application also relates to artificial intelligence (AI) technology, which is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.
[0075] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. Basic artificial intelligence technologies generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, pre-trained model technology, operation / interaction systems, mechatronics, etc. Among them, pre-trained models are also called large models and basic models. After fine-tuning, they can be widely used in downstream tasks in various major directions of artificial intelligence. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.
[0076] This application mainly relates to the machine learning (ML) technology, which is a multi-disciplinary cross-disciplinary subject involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all fields of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and learning by teaching. The pre-trained model is the latest development in deep learning, which integrates the above technologies.
[0077] The present application can use machine learning technology to automatically complete efficient and accurate model training based on the model training method of the present application to obtain a model for performing new data processing tasks.
[0078] In order to facilitate the understanding of the technical solution provided by this application, the model training method provided by this application will be introduced in combination with an actual application scenario.
[0079] See also Figure 1 , Figure 1 A schematic diagram of a model training method in an actual application scenario provided for an embodiment of the present application, in which the computer device may be a server 101 having a model training function, the model to be trained is a model capable of performing a first data processing task, and the purpose of model training is to enable the model to be trained to perform a second data processing task. The server 101 may first obtain the model to be trained and N incremental model parameters, the model to be trained is used to perform the first data processing task, and the first data processing task may be an object recognition task for an object. The parameter values of the initial model parameters of the N incremental model parameters may be set in a variety of ways, which are not limited here.
[0080] An initial model can be formed based on the model to be trained and N incremental model parameters, and the initial model is used to learn how to perform the second data processing task, and the second data processing task can be to identify the person for the object of the person. The model parameters of the initial model include the initial model parameters included in the model to be trained itself and N incremental model parameters, wherein the N incremental model parameters and the target model parameters in the initial model parameters can form multiple processing branches, including a main branch corresponding to the target model parameters and incremental branches corresponding to the N incremental model parameters. Multiple processing branches are used to independently execute the target processing flow in the second data processing task through processing operations of the same category. After the sample data is input into the initial model, the initial model will determine the input data corresponding to the target processing flow, and the input data will be respectively input into multiple processing branches, and the corresponding processing results are determined by the model parameters corresponding to each processing branch. The initial model can combine the processing results determined by the multiple processing branches to determine the pending processing result of the final output, and the pending processing result is the processing result obtained after the initial model performs the second data processing task on the sample data.
[0081] Since the present application can form multiple processing branches through N incremental model parameters to independently and in parallel complete the determination of the pending processing results, the model architecture of the initial model can perform more detailed data processing for the target processing flow, improve the complexity of data analysis, and thus the accuracy of data processing. Server 101 can obtain the reference processing result corresponding to the sample data, and the reference processing result is the processing result obtained after the accurate second data processing task is performed on the sample data. Thus, according to the difference between the reference processing result and the pending processing result, the processing accuracy of the initial model when executing the second data processing task can be characterized, and then the multiple incremental model parameters can be adjusted so that the initial model learns how to accurately execute the second data processing task and obtains the target model for executing the second data processing task.
[0082] Since the present application increases the complexity of data processing in which the incremental model parameters participate in the target processing flow by adding multiple processing branches, even if the overall parameter amount of the N incremental model parameters is still smaller than the parameter amount of the target model parameters, the parameter amount of the incremental model parameters is enriched compared to the related art, and can bring richer data processing performance to the target processing flow. Furthermore, by adjusting the parameters only for the incremental model parameters, the effectiveness of the model parameter adjustment can be guaranteed while reducing the model parameter adjustment amount, so that the initial model can effectively learn how to accurately perform the second data processing task. At the same time, the model knowledge used to perform the first data processing task in the model to be trained can be retained, so that this part of the model knowledge can assist the target model in performing the second data processing task.
[0083] Next, the technical solution provided by this application will be introduced in detail in conjunction with the accompanying drawings.
[0084] See also Figure 2 , Figure 2 A flowchart of a model training method provided in an embodiment of the present application. In this embodiment, the computer device may be any computer device capable of performing model training. The method includes:
[0085] S201: Obtain the model to be trained and sample data.
[0086] This application is mainly aimed at incremental training scenarios, that is, scenarios in which a pre-trained model is retrained. The model to be trained is used to achieve the old data processing goal, and the purpose of incremental training is to obtain a model that can achieve a new data processing goal, which is a different data processing goal from the first data processing goal. Since the data processing goals that the model can achieve mainly depend on the data used in model training, in order to enable the target model obtained by incremental training to achieve different data processing goals from the model to be trained, when the model to be trained is obtained based on the initial sample data, the sample data used in incremental training is different from the initial sample data.
[0087] It is understandable that the data processing target may include targets of multiple dimensions. For example, two different data processing targets may refer to the accuracy achieved for different tasks of the same data processing task, such as the old data processing target is to be able to achieve 80% accuracy in object recognition, and the new data processing target is to be able to achieve 90% accuracy in object recognition; two different data processing targets may also refer to achieving two different data processing tasks, such as the old data processing target may be able to recognize human objects in images, and the new data processing target may be able to recognize animal objects in images. The model training method in this application is applicable to the above-mentioned multiple scenarios.
[0088] Among them, the model to be trained can be any model. In the model training process corresponding to the model to be trained, since the model to be trained is obtained based on the initial sample data, the model parameters included in the model to be trained have the model knowledge learned from the initial sample data for achieving the old data processing goals.
[0089] In order to enable the model to be trained to learn how to achieve a new data processing target, the computer device can obtain new sample data, which has a corresponding reference processing result, and the reference processing result is the processing result obtained by performing data processing on the sample data when the new data processing target is achieved. For example, when the second data processing target is the same data processing task with higher accuracy than the first data processing target, the new sample data can be the sample data corresponding to the data processing task, and the reference processing result is the reference processing result obtained by accurately performing the data processing task on the sample data. Since the sample data and the reference processing result do not participate in the training process of the model to be trained, the model can learn new model knowledge for accurately performing the data processing task, thereby improving the accuracy of performing the data processing task; or, when the second data processing target is to achieve a data processing task different from the first data processing target, the new sample data can be the sample data corresponding to the data processing task corresponding to the second data processing target, that is, the reference processing result corresponding to the sample data is the processing result obtained by accurately performing the data processing task on the sample data.
[0090] S202: An initial model is constructed by adding multiple incremental branches to the model to be trained.
[0091] From the above analysis, it can be seen that the reasons for the inaccurate model training methods in related technologies mainly include the following two aspects:
[0092] 1. The number of parameters of the incremental model parameters is too small.
[0093] Due to the small number of parameters, even if the incremental model parameters can be fully adjusted, it is difficult for the model to learn model knowledge related to the new data processing goals.
[0094] Second, the data processing process involving the incremental model parameters is too simple.
[0095] In the related art, all incremental model parameters only execute the data processing flow once in a data processing task. This results in the data processing flow based on the incremental model parameters being relatively simple, making it difficult to perform more complex and detailed analysis of the data. As a result, the model trained in this way is difficult to accurately execute new data processing tasks.
[0096] In this application, in order to solve the above two technical defects, an initial model can be constructed based on multiple incremental model parameter models and the model to be trained. The model parameters of the initial model include the initial model parameters and multiple incremental model parameters corresponding to the model to be trained. The overall parameter amount of the incremental model parameters of the multiple incremental branches is less than the parameter amount of the target model parameters of the main branch. The initial model includes multiple processing branches, such as Figure 1As shown in . Multiple incremental branches and the main branch in the model to be trained are used to perform the same type of processing operations on the same data, so that multiple data processing operations can be performed for the same data processing flow, and the new data processing target is the data processing target achieved by executing the processing operation. The multiple processing branches include a main branch corresponding to the target model parameters in the initial model parameters and multiple incremental branches, and the multiple incremental branches are incremental branches corresponding to multiple incremental model parameters. Therefore, the present application first increases the parameter amount of the incremental model parameters while keeping the parameter amount of the incremental model parameters smaller than that of the initial model parameters, and at the same time enriches the model architecture involved in the incremental model parameters in the initial model, bringing model architecture support for the more complex processing flow below.
[0097] It should be emphasized that the processing operations performed by the multiple processing branches can be any process involved in achieving the new data processing target, that is, can be all or part of the processing process. The target model parameters are the model parameters in the initial model parameters used to perform the processing operations.
[0098] It is understandable that the purpose of multiple incremental branches is to replace the main branch corresponding to the original target model parameters of the model to be trained to learn model knowledge. Therefore, the present application can enable multiple incremental branches to perform the same data processing operation as the main branch, that is, the data processing operations performed by multiple processing branches are the same, and the target processing branch is used to perform the data processing operation on the input data corresponding to the target processing flow through the model parameters corresponding to the target processing branch, and obtain the processing result corresponding to the target processing branch. For example, the data processing operation can be a feature extraction operation, that is, multiple processing branches are used to perform feature extraction operations on the input data corresponding to the target processing flow to obtain the processing result corresponding to each processing branch. The target processing branch can be any one of the multiple processing branches.
[0099] Since the incremental branch and the main branch have the same data processing operations, adjusting the incremental model parameters in the incremental branch during the model training process can effectively simulate the adjustment method of the target model parameters in the main branch, so that the initial model can learn accurate model knowledge through parameter adjustment of the incremental model parameters, and then accurately achieve the second data processing goal.
[0100] It should be emphasized that the same data processing operation does not necessarily represent the same data processing method. For example, both the main branch and the incremental branch can perform feature extraction operations (such as convolution operations), but the main branch directly extracts features through the target model parameters, while the incremental branch can perform feature extraction using the first parameter and the second parameter through the above-mentioned parameter reduction method, which also belongs to the same data processing operation described in this application.
[0101] S203: Obtain pending processing results based on sample data through the initial model.
[0102] Among them, the pending processing results are determined by combining the processing results corresponding to multiple processing branches. Since multiple processing branches perform data processing independently when executing the target processing flow, even if the parameter amount of the incremental model parameters corresponding to each incremental branch is small, that is, the data processing method of each incremental branch is relatively simple, but by combining multiple incremental branches, the input data can be processed more carefully and complexly, so that the data processing results obtained by combining multiple processing branches can support higher accuracy and rationality, that is, this data processing method can support the initial model to effectively learn sufficient model knowledge through incremental model parameters, thereby effectively learning how to achieve new data processing goals.
[0103] At the same time, since the data processing of multiple processing branches is independent of each other, when performing the same processing operation, multiple processing branches can be executed in parallel, ensuring the execution efficiency of the initial model in performing data processing tasks, and thus also ensuring the training efficiency of the model training, avoiding the loss of data processing efficiency due to increasing the parameter amount of the incremental model parameters.
[0104] S204: According to the difference between the pending processing result and the reference processing result of the sample data, multiple incremental model parameters are adjusted to obtain a target model.
[0105] Among them, multiple incremental model parameters are incremental model parameters of the multiple incremental branches. Since the reference processing result is the processing result obtained by processing the sample data when the new data processing target is achieved, the difference between the reference processing result and the pending processing result can characterize the gap between the data processing target currently achieved by the initial model and the new data processing target. The larger the difference, the larger the gap. Therefore, the computer device can adjust the parameters of multiple incremental model parameters according to the difference, so that the initial model learns how to determine the pending processing result close to the reference processing result through the incremental model parameters, so that the initial model can learn the model knowledge used to achieve the new data processing target by adjusting the incremental model parameters, and obtain the target model, which can be used to achieve the new data processing target. For example, by training with new sample data of the same data processing task, the target model can achieve higher-precision data processing for the data processing task; by using sample data corresponding to the new data processing task, the target model can be used to execute the new data processing task.
[0106] Since the incremental model parameters of the present application can be used for more complex and detailed analysis of sample data, only the incremental model parameters need to be adjusted during model training to learn sufficient model knowledge without the need to make overall adjustments to the original model parameters of the training model. Therefore, the amount of parameter adjustment required for model training can be reduced to a certain extent, thereby improving the efficiency of model training.
[0107] In summary, this application can bring the following technical effects:
[0108] First, since the present application adds multiple incremental model parameters during incremental training, and enriches the parameter quantity of the incremental model parameters, it is possible to improve the model knowledge learning capability brought about by adjusting the parameters of the incremental model parameters;
[0109] Secondly, the multiple incremental models in the present application can independently constitute multiple processing branches, and can independently perform data processing when executing the target processing flow. Therefore, under the premise that the overall parameter amount of the incremental model parameters is low, it is also possible to enrich the data analysis process during data processing and improve the processing accuracy of data processing;
[0110] Thirdly, multiple independent processing branches can perform data processing in parallel. Therefore, even if multiple processing branches are added, the data processing efficiency of the model will not be greatly affected, thus ensuring the data processing efficiency of the model.
[0111] Fourthly, since the present application only adjusts the incremental model parameters when adjusting the model parameters, and keeps the original target model parameters of the model to be trained unchanged, it is possible to learn new model knowledge through the incremental model parameters while retaining the model knowledge of the model to be trained, thereby enriching the model knowledge amount and further improving the data processing accuracy;
[0112] Fifthly, since the overall parameter amount of multiple incremental model parameters is smaller than the parameter amount of the target model parameters, the parameter adjustment amount required by this parameter adjustment method is relatively low, and the model training efficiency is relatively high;
[0113] Sixthly, since the training effect brought by the model architecture of multiple processing branches in the present application is better and higher quality, the data processing flow corresponding to the data processing operations executed by multiple processing branches constructed in the present application can be a partial data processing flow in the complete data processing flow, that is, it only plays a role in part of the architecture of the model, and can also ensure the accuracy of the overall data processing of the model.
[0114] Since the incremental model parameters in this application can support more complex data processing functions, even if the incremental model parameters in this application only participate in part of the processing flow in the data processing task, the data processing task can have a higher processing accuracy. Based on this, in a possible implementation method, the target model parameters are part of the model parameters in the initial model parameters, and the processing operations performed by multiple processing branches are used to complete the target processing flow in the target model, and the target processing flow can be part of the processing flow in the data processing task. Since there is no need to participate in the entire processing flow of the data processing task, the parameter amount of the required incremental model parameters is reduced to a certain extent, thereby ensuring the accuracy of data processing while further reducing the amount of model parameters required to adjust the model training, thereby further improving the efficiency of model training.
[0115] Among them, the target model in this application can be applied to a variety of data processing tasks. Next, the model training method of this application will be introduced in combination with a specific data processing task.
[0116] In a possible application scenario, a data processing task can be used to identify a target object included in image information based on the image information. The data processing task may include a feature extraction process, a position identification process, and a category identification process, wherein the target processing process may include a position identification process and a category identification process.
[0117] The feature extraction process is used to extract image feature data corresponding to the image information, and the image feature data is the input data corresponding to the target processing process. The position recognition process is used to determine the position data corresponding to the target object in the image information based on the image feature data, for example, the bounding box and other data corresponding to the target object in the image information can be determined. The category recognition process is used to determine the object category data corresponding to the target object based on the image feature data, for example, the identity of the object, the object category, the biological category, etc. can be identified. With the participation of the above-mentioned multiple incremental model parameters, the position recognition process and the category recognition process in the initial model can both perform accurate data processing. Therefore, even if the feature extraction process is not adjusted, it can also bring more accurate object recognition capabilities. While ensuring the accuracy of object recognition, the training efficiency of the model used for object recognition is improved, so when the data processing target changes, such as the object recognition accuracy or the application scenario of object recognition changes, the model training method of this application can be used to quickly train to obtain an object recognition model that can accurately achieve the new data processing target.
[0118] The model architecture of the initial model can be as follows Figure 3As shown, after the image information is input, the complete image features are first extracted through the feature extraction module of the corresponding feature extraction process. The feature extraction module can be composed of a backbone network, and the backbone network can adopt a variety of networks. For example, in this application, a parallel high-resolution network (High-Resolution Network Parallel, referred to as HRNetV2P) can be used as the backbone network. Compared with the residual neural network (Residual Neural Network, referred to as ResNet), the high-resolution network (HRNet) has a stronger ability to extract panoramic information.
[0119] The local feature extraction module is used to extract local features from image features and input them into each network head. Local feature extraction can be achieved through the local feature extractor pool. The network head is used to execute the above target processing flow. The network head includes a classification head and a regression head. The classification head is used to obtain category information and determine the classification result corresponding to the local feature. The regression head is used to determine the position information and output the bounding box corresponding to the object. The overall bounding box is the bounding box corresponding to multiple objects in the image information obtained by the initial model through overall position recognition of the image information, which is used to guide the extraction method of local feature extraction.
[0120] In a possible implementation, when the data processing task of the target model is image recognition, the sample data used to train the target model and the initial sample data used to train the model to be trained can both be images. The model to be trained is used to identify the category of the first object in the initial sample data, that is, when the model to be trained is trained by the initial sample data, the model to be trained can have the ability to identify the category of the first object in the initial sample data. In this implementation, the purpose of incremental training is to enable the target model to have a stronger category recognition ability. For example, the target model can be used to identify the category of the first object in the initial sample data and the category of the second object in the sample data, that is, it has both the object recognition ability of the model to be trained and the new object recognition ability for the second object.
[0121] The first object and the second object may be objects of the same type or different types. For example, the purpose of incremental training may be to enable the model to recognize more categories for objects of the same type. For example, for objects of the type "cat", the model to be trained may only be able to recognize categories such as orange cats and tabby cats, while the sample data and reference processing results provide the target model with the ability to recognize the category of Siamese cats. The purpose of incremental training may also be to enable the model to recognize new types of objects. For example, the model to be trained is used to perform category recognition for objects of the type "cat". After model training is performed using sample data and reference processing results, the target model can have the ability to recognize the category of objects of the type "dog".
[0122] Next, the technical details involved in the model training process will be introduced in detail.
[0123] In one possible implementation, in order to further reduce the amount of model parameter adjustment during the training process and improve the model training efficiency, the computer device can simulate large model parameters through multiple small model parameters through parameter splitting to reduce the parameter amount of incremental model parameters.
[0124] Taking the target incremental branch as an example, the target incremental branch can be any one of the multiple incremental branches, the target incremental branch includes a first sub-branch and a second sub-branch, and the incremental model parameters of the target incremental branch include a first parameter of the first sub-branch and a second parameter of the second sub-branch. In order to enable the processing result corresponding to the target incremental branch to be combined with the processing result corresponding to the main branch, the processing results corresponding to the two branches need to correspond to the same data dimension, for example, they need to correspond to the same feature dimension, etc. Since the inputs of multiple processing branches are the same input data, the incremental model parameters corresponding to the target incremental branch need to be the same as the target model parameter dimensions, that is, the first parameter and the second parameter are used to simulate equivalent model parameters with the same dimension as the target model parameter.
[0125] When executing step S203, the computer device may execute steps S2031-S2033 (not shown in the figure). Steps S2031-S2033 are a possible implementation of step S203, including:
[0126] S2031: Determine input data of multiple processing branches based on sample data.
[0127] When the target processing flow corresponding to the processing operations performed by multiple processing branches is the entire processing flow in the data processing task, the input data can be the input data itself; when the target processing flow is a partial processing flow in the data processing task, the input data can be the input data itself, or it can be data after sample data is processed by other processing flows in the data processing task, such as the image features extracted through the above content.
[0128] S2032: Determine the processing results corresponding to the multiple processing branches respectively according to the input data through the multiple processing branches.
[0129] Among them, the main branch is used to determine the processing result corresponding to the main branch according to the target model parameters and the input data. The target incremental branch is used to determine the first intermediate data according to the input data and the first parameter, and to determine the processing result corresponding to the target incremental branch according to the first intermediate data and the second parameter. The processing results corresponding to the multiple processing branches correspond to the same data dimension. That is, by processing the input data in turn through the first sub-branch containing the first parameter and the second sub-branch containing the second parameter, it is possible to simulate the effect of processing the input data through the equivalent model parameters of the target model parameter format dimension, thereby obtaining the processing result with the same data dimension, and then the processing result corresponding to the target incremental branch can be combined with the processing result corresponding to the main branch to obtain the final processing result. Among them, the sum of the parameter amounts of the first parameter and the second parameter can be less than the parameter amount of the equivalent model parameter. Therefore, replacing the equivalent parameter with the first parameter and the second parameter can further reduce the model parameter amount of each incremental branch. At the same time, the same data dimension also ensures the effectiveness of data processing, thereby improving the efficiency of model training.
[0130] S2033: Determine pending processing results by combining the processing results corresponding to the multiple processing branches.
[0131] There are many ways to use multiple model parameters to simulate one model parameter to reduce the amount of model parameters. A specific simulation method will be introduced below.
[0132] In a possible implementation, the input data is n-dimensional feature data, and the processing results corresponding to the multiple processing branches are all m-dimensional feature data, so processing the input data into processing results requires model parameters that can perform n-dimensional to m-dimensional conversion. Since the dimensions of the input data and the processing results are usually high, if the conversion from n-dimensional to m-dimensional is completed directly through a single model parameter, the model parameter needs to have a large parameter amount, for example, the model parameter needs to be an n*m matrix parameter.
[0133] In order to reduce the parameter amount of the incremental model parameters and enable the incremental model parameters to have the parameter effect of the above-mentioned dimensional conversion, the computer device can split the incremental model parameters used for n-to-m-dimensional conversion into two incremental model parameters with lower dimensions, namely, the first parameter and the second parameter, and the first intermediate data is r-dimensional feature data, r is less than n, and r is less than m. The first parameter is used to perform dimensionality reduction processing on the input data to determine the first intermediate data, and the second parameter is used to perform dimensionality increase processing on the first intermediate data to determine the processing result corresponding to the target incremental branch. Since the data dimension involved is reduced during the data processing process, the first parameter and the second parameter can have a lower parameter amount, so that the parameter effect of the original n*m-dimensional incremental model parameters can be simulated under the premise of reducing the parameter amount of the incremental model parameters, thereby maintaining the data processing performance of the incremental branch. In addition, since the present application has more incremental branches, the use of this parameter reduction method in the present application can bring more significant parameter reduction on the basis of ensuring model performance, so that the model training method of the present application will not bring too large an increase in the model parameter amount, thereby ensuring the model training efficiency.
[0134] For example Figure 4 As shown, the input data is din-dimensional data, and the target model parameter is d in *d out The target model parameters can be used to directly determine the d out The processing result of dimension 1; the first parameter in the target increment branch is d in *r-dimensional parameter, the first intermediate data of r dimension can be obtained by reducing the dimension of the input data through the first parameter, and then the first intermediate data of r dimension can be obtained through r*d out The second parameter of dimension can be used to increase the dimension of the first intermediate data to obtain d out The processing result of dimension 2. Since r is less than d out , so the sum of the first and second parameters is much smaller than d in *d out The number of parameters of the model parameters of the dimensional model can be further reduced while maintaining the effectiveness of the data processing flow. The first parameter and the second parameter can both be matrix parameters. The first parameter will be initialized using the Kaiming uniform distribution initialization method (for example, a 0-mean Gaussian distribution can be used, and the bias is a uniform distribution with 0 mean and 0 variance). The number of output channels of the convolution kernel is specified as r, and the second parameter can be a 1*1 convolution, which upgrades the r-dimensional feature to d-dimensional. out dimension so that it can be combined with the processing results of the main branch.
[0135] Next, the specific method of adjusting the model parameters will be introduced for the above data processing method. When executing step S204, the computer device can execute steps S2041-S2044 (not shown in the figure), and steps S2041-S2042 are a possible implementation method of step S204, including:
[0136] S2041: adjusting the second parameter according to the difference between the pending processing result and the reference processing result of the sample data, and adjusting the first intermediate data according to the difference between the pending processing result and the reference processing result of the sample data, to obtain second intermediate data.
[0137] like Figure 5 As shown, the pending processing result is determined by two aspects of data, one is the first intermediate data determined by the first parameter, and the other is the second parameter. Therefore, based on the difference, the computer device can first know how to adjust the pending processing result to make it closer to the reference processing result, and then determine how to adjust the second parameter and the first intermediate data through gradient conduction to meet the adjustment method for the pending processing result, so that the second parameter and the first intermediate data can be adjusted according to the difference between the pending processing result and the reference processing result, and the second intermediate data that meets the adjustment requirements can be obtained based on the adjustment of the first intermediate data.
[0138] S2042: Adjust the first parameter according to the difference between the second intermediate data and the first intermediate data.
[0139] The first intermediate data is determined by two aspects of data, one is the first parameter, and the other is the input data. Since the input data is independent of the incremental model parameters, the present application mainly adjusts the first parameter to make the determined first intermediate data meet the adjustment method required by the pending processing result, that is, the second intermediate data is obtained by adjusting the first parameter. Therefore, through the difference between the first intermediate parameter and the second intermediate parameter, the adjustment method for the first parameter can be determined, and the adjustment method satisfies the determined first intermediate parameter to gradually approach the second intermediate parameter, thereby making the pending processing result gradually approach the reference processing result.
[0140] Through the above method, the difference between the pending processing result and the reference processing result can be transmitted through layers of gradients to the first parameter and the second parameter, so that the computer device can know how to adjust the first parameter and the second parameter to make the target processing flow more accurate, and then more effective learning on how to achieve the second data processing goal can be achieved by adjusting the incremental model parameters.
[0141] As mentioned above, the present application can improve the complexity and accuracy of data processing by processing data in multiple incremental branches. Therefore, in the present application, even if the data processing accuracy is reduced to a certain extent for each incremental branch, the combination of multiple incremental branches can make up for this part of the accuracy loss and ensure the data processing effect of the model. Based on this, in a possible implementation, in order to further improve the efficiency of model training, the computer device can reduce the amount of data processing in the incremental branch.
[0142] In this implementation, the master branch is specifically used for:
[0143] Feature extraction is performed on the input data according to the target model parameters to generate a processing result corresponding to the main branch, wherein the processing result corresponding to the main branch is feature data corresponding to the first feature size.
[0144] Taking the target incremental branch as an example, the target incremental branch is specifically used for:
[0145] Through the first sub-branch, feature extraction is performed on the input data according to the first parameter to generate first intermediate data, and the first intermediate data is feature data corresponding to the second feature size. Among them, the amount of calculation required for feature extraction is positively correlated with the feature size corresponding to the generated feature data, that is, the larger the extracted feature size, the more calculation is required. Since the second feature size is smaller than the first feature size, the amount of calculation required for feature extraction can be reduced when feature extraction is performed based on the first parameter. Although the reduction in the amount of calculation will lead to a reduction in the processing accuracy of the model data to a certain extent, since the present application has multiple incremental branches, the independent data processing functions of the multiple incremental branches can compensate for the loss of accuracy in a single incremental branch to a certain extent, thereby ensuring the processing accuracy of the overall data processing.
[0146] Then, the target incremental branch can process the first intermediate data according to the second parameter through the second sub-branch to determine the processing result corresponding to the target incremental branch. The data processing includes size processing for expanding the feature size, that is, the first intermediate data will be restored in size during the process of determining the processing result, that is, the processing result is the feature data corresponding to the first feature size. It can be understood that the role of the second parameter is to process the first intermediate data so that a more accurate model output can be determined based on the data obtained after processing, not size restoration, that is, the role of the second sub-branch includes two aspects, on the one hand, data processing can be performed, and on the other hand, size restoration is required. Size restoration only requires simple linear data calculation, so the amount of calculation required for the restoration process is low, and overall, it can still play a more obvious effect of reducing the amount of calculation for each incremental branch. In addition, since the feature size of the intermediate data involved in the data processing process in the incremental branch is reduced, and it can be seen from the above process that the difference in the intermediate data is one of the bases for adjusting the incremental model parameters, and the reduction of the feature size will cause the difference in the intermediate data to be reduced, thereby reducing the amount of adjustment required for model parameter adjustment to a certain extent, improving the model fitting speed, and further improving the model training efficiency.
[0147] Since the correlation between data processing and size restoration of the second parameter is relatively low, the order of data processing and size restoration is not limited in this application. The technical solutions and technical effects of different orders will be introduced in detail below.
[0148] The first method: restore first, then process
[0149] In a possible implementation, when the first intermediate data is processed according to the second parameter to determine the processing result corresponding to the target incremental branch, the computer device may first perform size processing on the first intermediate data to generate third intermediate data, where the third intermediate data is feature data corresponding to the first feature size. Then, the processing result corresponding to the target incremental branch is determined according to the third intermediate data and the second parameter, that is, the third intermediate data is processed according to the second parameter to obtain the processing result corresponding to the target incremental branch.
[0150] The advantage of this method is that, since the second parameter is used to process the third intermediate data with a larger first characteristic size, the data processing of the third intermediate data is more complex and detailed, which can improve the processing accuracy of the data processing through the second parameter to a certain extent, thereby improving the data processing accuracy of the target incremental branch. Figure 6 shown.
[0151] There are many specific ways to perform feature extraction and size restoration. The following will mainly introduce the convolution processing method.
[0152] In one possible implementation, when features are extracted from input data in the main branch according to the target model parameters to generate processing results corresponding to the main branch, the computer device can extract features from the input data through p convolution processes according to the target model parameters to generate processing results corresponding to the main branch. The number of convolution processes determines the amount of calculation required for the convolution process. The more times the convolution process is performed, the greater the amount of calculation and the larger the feature size obtained.
[0153] In the target incremental branch, when extracting features from the input data according to the first parameter and generating the first intermediate data, the computer device can extract features from the input data through q convolution processes according to the first parameter to generate the first intermediate data, where q is less than p, i.e., the number of convolutions in the incremental branch is less than the number of convolutions in the main branch, and therefore the amount of computation required in the incremental branch is less. The number of convolutions can be adjusted by adjusting the stride parameter (stride) during the convolution process. The larger the stride parameter, the larger the cross-server of each convolution process in the data, the smaller the number of convolution processes required, and the smaller the feature size obtained. For example, adjusting the stride parameter to 4 times that of the main branch can achieve 4 times upsampling during the convolution process, and obtain the first intermediate data with a feature size that is 1 / 16 of the feature size corresponding to the main branch processing result.
[0154] The size processing may include bilinear interpolation upsampling processing, which is a simple linear processing method that does not require the participation of other parameters other than the first intermediate parameter, requires less calculation and has higher processing efficiency. For example, the computer device may enlarge the feature size to 16 times the original size through bilinear difference upsampling processing to obtain the third intermediate data.
[0155] The second method: process first, then restore
[0156] In another possible implementation, Figure 8 As shown, when the first intermediate data is processed according to the second parameter to determine the processing result corresponding to the target incremental branch, the computer device can first determine the fourth intermediate data according to the first intermediate data and the second parameter, and the fourth intermediate data is the feature data corresponding to the second feature size, that is, when the first intermediate data is processed according to the second parameter, the feature size of the first intermediate data is not enlarged, and only data processing is performed. After the processing is completed by the second parameter, the fourth intermediate data is size processed to generate the processing result corresponding to the target incremental branch, that is, the processing result of the first feature size is obtained.
[0157] The advantage of this method is that the feature size corresponding to the data that needs to be processed by the second parameter is smaller, so the amount of computation required for data processing is smaller, and the amount of computation for feature restoration has no direct correlation with whether the data is processed by the second parameter. Therefore, this method can further reduce the amount of computation required for data processing in the target incremental branch, thereby further improving the efficiency of model training.
[0158] Next, the specific method of adjusting model parameters will be introduced.
[0159] As mentioned above, this application can improve the complexity and accuracy of data processing by adding multiple incremental branches, and perform more detailed data processing. However, multiple incremental model parameters are independent of each other, and the amount of model parameters in each incremental branch is relatively low. Therefore, when adjusting the model parameters, the amount of parameter adjustment required for the model parameters on each incremental branch is small, thereby reducing the difficulty of model parameter fitting. Based on this, in a possible implementation method, in order to further improve the model training accuracy and model training efficiency, the computer device can gradually reduce the learning rate of the model parameters during the parameter adjustment process to obtain effective model parameters more quickly.
[0160] When executing step S204, the computer device may execute step S2045 (not shown in the figure). Step S2045 is a possible implementation of step S204, including:
[0161] S2045: Perform multiple rounds of parameter adjustments on multiple incremental model parameters according to the difference between the pending processing result and the reference processing result of the sample data.
[0162] Among them, multiple rounds of parameter adjustment have corresponding learning rates respectively, and the learning rate decreases as the number of rounds increases in multiple rounds of parameter adjustment. Taking the target round parameter adjustment as an example, the target round parameter adjustment is any round in the multiple rounds of parameter adjustment. The learning rate corresponding to the target round parameter adjustment is used to control the adjustment amplitude of adjusting multiple incremental model parameters, and the learning rate corresponding to the target round parameter adjustment is positively correlated with the adjustment amplitude. Since there are fewer incremental model parameters on each incremental branch, the required parameter adjustment amount is lower. A smaller adjustment amplitude helps to more efficiently find incremental model parameters suitable for performing data processing tasks. Therefore, by gradually reducing the learning rate during the adjustment process, the model fitting speed can be further improved, and more efficient training can be obtained to obtain the target model for achieving the second data processing purpose.
[0163] In addition, since the required parameter adjustment amount is small, the computer device can also limit the number of rounds of parameter adjustment to a smaller range. For example, in the related art, since the parameter adjustment amount is not reduced, 36 rounds of parameter adjustment may be required to achieve effective model training; in the present application, since the parameter adjustment amount is reduced, the number of rounds of parameter adjustment can be reduced from 36 to 12, and the initial value of the learning rate is adjusted to 1 / 10 of the original learning rate, and gradually decreases in 12 rounds of parameter adjustment. The learning rate reduction strategy can adopt a variety of strategies, such as the cosine annealing learning rate adjustment strategy (CosineAnnealingLR). The learning rate controlled by this strategy changes in multiple rounds of parameter adjustment as shown in the following figure. Figure 7 As shown in the figure, the horizontal axis is the number of parameter adjustment rounds, and the vertical axis is the learning rate.
[0164] In addition, since the incremental model parameters in the multiple incremental branches are independent of each other, not only can the multiple incremental branches perform data processing in parallel to improve data processing efficiency during the data processing process, but the parameter adjustment process can also be performed in parallel. In a possible implementation, when executing step S204, the computer device can execute step S2046 (not shown in the figure), which is a possible implementation of step S204, including:
[0165] S2046: According to the difference between the pending processing result and the reference processing result corresponding to the sample data, multiple incremental model parameters are adjusted in parallel to obtain a target model.
[0166] Computer equipment can reduce the time required for parameter adjustment by adjusting the incremental model parameters in multiple incremental branches in parallel, thereby ensuring model training efficiency while doubling the parameter amount of the incremental model parameters.
[0167] In order to facilitate the understanding of the technical solution provided by this application, the model training method provided by this application will be introduced in combination with an actual application scenario.
[0168] See also Fig. 9 , Fig. 9 A flowchart of a model training method in an actual application scenario provided by an embodiment of the present application. In this actual application scenario, the computer device can be any computer device capable of model training. The purpose of incremental training is to enable the target model to perform new data processing tasks. The method includes:
[0169] S901: Acquire a first object recognition model and sample image information for identifying an object.
[0170] In this actual application scenario, the data processing task performed by the first object recognition model (i.e., the model to be trained) is to recognize objects in image information, and the new data processing task is to recognize people in image information. The first object recognition model is used to recognize objects in image information, such as vehicles, houses, etc. The sample image information is image information including people, and the sample image information has a corresponding reference object recognition result, which is a recognition result obtained by accurately recognizing the people in the sample image information.
[0171] S902: constructing an initial model through multiple incremental branches and the first object recognition model.
[0172] The initial model architecture in this practical application scenario can be Fig.10 As shown, multiple incremental model parameters respectively have corresponding first parameters and second parameters to constitute the first sub-branch and the second sub-branch in each incremental branch. Multiple incremental branches can be constituted by multiple incremental model parameters, and together with the target model parameters in the first object recognition model, multiple processing branches are constituted. The multiple processing branches are used to determine the object recognition results based on image features. The object recognition results include location data and object type data. For example, the incremental model parameters and the target model parameters are model parameters in the regression head and the classification head.
[0173] S903: Generate a pending object recognition result for a person through the initial model and sample image information.
[0174] The data processing flow of step S903 may be as shown in steps S9031-S9036.
[0175] S9031: Extract image features corresponding to the sample image information through the feature extraction part.
[0176] The feature extraction part is used to extract image features corresponding to image information. The feature extraction part can use a variety of feature extraction networks, such as ResNet network, visual geometry group network, etc., which are not limited here.
[0177] S9032: Determine the position data and object type data through the target model parameters in the main branch.
[0178] The position data and object type data are the object recognition results corresponding to the main branch, which are data corresponding to the first feature size, and the target model parameters are used to perform multiple convolution processes on the image features.
[0179] S9033: Perform feature extraction using the first parameter in the incremental branch to obtain first intermediate data.
[0180] Taking the target incremental branch as an example, the data processing flow of the incremental branch is introduced. The first parameter is used to perform multiple convolution processes on the image features to complete feature extraction. The number of convolution processes is less than the number of convolution processes in the main branch, so that the first intermediate data of the second feature size can be obtained. The size scaling can adopt a variety of magnifications, such as 1 / 2, 1 / 4, 16 / 1, etc., which are not limited here.
[0181] The first parameter is d in *r-dimensional model parameter, the second parameter is r*d out The model parameters are d-dimensional, and the image features are d in The object recognition result output by the processing branch is d out dimensional data, r is less than d out , so that the sum of the first parameter and the second parameter is much smaller than the parameter of the target model parameter.
[0182] S9034: Process the first intermediate data using the second parameter to obtain second intermediate data.
[0183] The data processing can also be convolution processing, which is used to further analyze the first intermediate data and increase the data dimension to d out dimension.
[0184] S9035: Perform size processing on the second intermediate data to obtain position data and object type data.
[0185] The size processing is used to adjust the characteristic size of the second intermediate data to the first characteristic size, and obtain the object recognition result output by the target incremental branch, that is, the position data and the object type data.
[0186] S9036: Combine the position data and object type data corresponding to the multiple processing branches, and output the pending object recognition result.
[0187] S904: According to the difference between the pending object recognition result and the reference object recognition result, multiple incremental model parameters are adjusted in parallel to obtain a second object recognition model.
[0188] When adjusting the model parameters, the computer device can be adjusted through a smaller number of adjustment rounds, for example, it can be completed through 12 rounds of parameter adjustment. The learning rate of parameter adjustment gradually decreases with the increase in the number of rounds, and a variety of learning rate reduction strategies can be adopted, such as the CosineAnnealingLR strategy, the distribution strategy (MultStep), etc., which are not limited here. The incremental model parameters adjusted in this application can be used not only for network casting, but also for some processes in the feature extraction part, for example, it can be used for the fourth stage of the four stages of the backbone network (Backbone) of the feature extraction part. The original model parameters of the first object recognition model are not adjusted.
[0189] S905: Identify people in the image information using a second object recognition model.
[0190] Through the above process, it can be seen that compared with the related art, this application has the following outstanding technical effects:
[0191] 1. Since the present application adds multiple incremental model parameters during incremental training and enriches the parameter quantity of the incremental model parameters, it is possible to improve the model knowledge learning ability brought about by parameter adjustment of the incremental model parameters. Even if only the incremental model parameters are added to the network head part, it can bring an accurate object recognition effect as a whole;
[0192] 2. The multiple incremental models in this application can independently constitute multiple processing branches, and can independently perform data processing when executing the target processing flow. Therefore, under the premise that the overall parameter amount of the incremental model parameters is low, it can also enrich the data analysis process during data processing and improve the processing accuracy of data processing;
[0193] 3. Multiple independent processing branches can perform data processing and model parameter adjustment in parallel. Therefore, even if multiple processing branches are added, the data processing efficiency and parameter adjustment efficiency of the model will not be affected too much, thus ensuring the data processing efficiency and model training efficiency of the model.
[0194] 4. Since the present application only adjusts the incremental model parameters when adjusting the model parameters, and keeps the initial model parameters corresponding to the model to be trained unchanged, it is possible to learn new model knowledge through the incremental model parameters while retaining the model knowledge of the model to be trained, thereby enriching the model knowledge amount and further improving the data processing accuracy;
[0195] 5. Since the overall parameter amount of multiple incremental model parameters is smaller than the parameter amount of the target model parameters, and through parameter splitting, multiple small incremental model parameters are used to simulate large incremental model parameters, which further reduces the parameter amount of the incremental model parameters. Therefore, this parameter adjustment method requires a lower parameter adjustment amount and has a higher model training efficiency.
[0196] 6. The present application can further reduce the amount of data processing required for each incremental branch and reduce the difficulty of parameter adjustment by reducing the data feature size in the incremental branch. At the same time, it can ensure the accuracy of overall data processing through multiple incremental branches, thereby further improving the efficiency of model training.
[0197] Based on the model training method provided in the above embodiment, the present application also provides a model training device, see Fig.11 , Fig.11 This is a structural block diagram of a model training device provided in the present application. The device 1100 includes an acquisition unit 1101, a formation unit 1102, a determination unit 1103 and an adjustment unit 1104:
[0198] The acquisition unit 1101 is used to acquire a model to be trained and sample data, wherein the model to be trained is obtained based on initial sample data, and the sample data is different from the initial sample data;
[0199] The forming unit 1102 is used to form an initial model by adding multiple incremental branches to the model to be trained, wherein the multiple incremental branches and the main branch in the model to be trained are used to perform the same type of processing operations on the same data, and the overall parameter amount of the incremental model parameters of the multiple incremental branches is less than the parameter amount of the target model parameters of the main branch;
[0200] The determining unit 1103 is used to obtain a pending processing result according to the sample data through the initial model, wherein the pending processing result is determined by combining the processing results corresponding to the multiple incremental branches and the main branch respectively;
[0201] The adjusting unit 1104 is used to adjust a plurality of incremental model parameters according to the difference between the pending processing result and the reference processing result of the sample data to obtain a target model.
[0202] In a possible implementation, the target incremental branch is any one of the multiple incremental branches, the target incremental branch includes a first sub-branch and a second sub-branch, the incremental model parameter of the target incremental branch includes a first parameter of the first sub-branch and a second parameter of the second sub-branch, and the determining unit 1103 is specifically used to:
[0203] Determine input data of the plurality of processing branches according to the sample data, the plurality of processing branches including the main branch and the plurality of incremental branches;
[0204] Through the multiple processing branches, the processing results corresponding to the multiple processing branches are determined according to the input data, the main branch is used to determine the processing result corresponding to the main branch according to the target model parameter and the input data, the target incremental branch is used to determine the first intermediate data according to the input data and the first parameter, and determine the processing result corresponding to the target incremental branch according to the first intermediate data and the second parameter, and the processing results corresponding to the multiple processing branches respectively correspond to the same data dimension;
[0205] The pending processing result is determined by combining the processing results respectively corresponding to the multiple processing branches.
[0206] In one possible implementation, the input data is n-dimensional feature data, the processing results corresponding to the multiple processing branches are all m-dimensional feature data, the first intermediate data is r-dimensional feature data, r is less than n, and r is less than m, the first sub-branch is used to perform dimensionality reduction processing on the input data to determine the first intermediate data, and the second sub-branch is used to perform dimensionality increase processing on the first intermediate data to determine the processing result corresponding to the target incremental branch.
[0207] In a possible implementation, the adjusting unit 1104 is specifically configured to include:
[0208] adjusting the second parameter according to the difference between the pending processing result and the reference processing result of the sample data, and adjusting the first intermediate data according to the difference between the pending processing result and the reference processing result of the sample data, to obtain second intermediate data;
[0209] The first parameter is adjusted according to a difference between the second intermediate data and the first intermediate data.
[0210] In a possible implementation, the main branch is specifically used for:
[0211] Perform feature extraction on the input data according to the target model parameters to generate a processing result corresponding to the main branch, wherein the processing result corresponding to the main branch is feature data corresponding to a first feature size;
[0212] The target incremental branch is specifically used for:
[0213] By means of the first sub-branch, feature extraction is performed on the input data according to the first parameter to generate the first intermediate data, the first intermediate data is feature data corresponding to a second feature size, the amount of calculation required for feature extraction is positively correlated with the feature size corresponding to the generated feature data, and the second feature size is smaller than the first feature size;
[0214] Through the second sub-branch, the first intermediate data is processed according to the second parameter to determine the processing result corresponding to the target incremental branch, the data processing includes size processing for expanding the feature size, and the processing result is feature data corresponding to the first feature size.
[0215] In a possible implementation, performing data processing on the first intermediate data according to the second parameter to determine a processing result corresponding to the target incremental branch includes:
[0216] Performing size processing on the first intermediate data to generate third intermediate data, wherein the third intermediate data is feature data corresponding to the first feature size;
[0217] A processing result corresponding to the target incremental branch is determined according to the third intermediate data and the second parameter.
[0218] In a possible implementation, extracting features from the input data according to the target model parameters to generate a processing result corresponding to the main branch includes:
[0219] According to the target model parameters, feature extraction is performed on the input data through p convolution processes to generate a processing result corresponding to the main branch;
[0220] The step of extracting features from the input data according to the first parameter to generate the first intermediate data includes:
[0221] According to the first parameter, extracting features from the input data by performing q convolution processes to generate the first intermediate data, where q is less than p;
[0222] The resizing process includes a bilinear interpolation upsampling process.
[0223] In a possible implementation, the performing data processing on the first intermediate data according to the second parameter to determine the processing result corresponding to the target incremental branch includes:
[0224] Determine fourth intermediate data according to the first intermediate data and the second parameter, wherein the fourth intermediate data is feature data corresponding to the second feature size;
[0225] The fourth intermediate data is resized to generate a processing result corresponding to the target incremental branch.
[0226] In a possible implementation, the adjusting unit 1104 is specifically configured to:
[0227] According to the difference between the pending processing result and the reference processing result of the sample data, multiple rounds of parameter adjustment are performed on the multiple incremental model parameters, and the multiple rounds of parameter adjustment respectively have corresponding learning rates. The learning rate decreases as the number of rounds increases in the multiple rounds of parameter adjustment. The target round parameter adjustment is any round in the multiple rounds of parameter adjustment. The learning rate corresponding to the target round parameter adjustment is used to control the adjustment amplitude of the multiple incremental model parameters, and the learning rate corresponding to the target round parameter adjustment is positively correlated with the adjustment amplitude.
[0228] In a possible implementation, the adjusting unit 1104 is specifically configured to:
[0229] According to the difference between the pending processing result and the reference processing result corresponding to the sample data, the plurality of incremental model parameters are adjusted in parallel to obtain a target model.
[0230] In a possible implementation manner, the sample data and the initial sample data are both images;
[0231] The model to be trained is used to identify the category of the first object in the initial sample data;
[0232] The target model is used to identify the category of the first object in the initial sample data and the category of the second object in the sample data.
[0233] The present application also provides a computer device, see Fig.12 As shown, the computer device may be a terminal device, and a mobile phone is taken as an example:
[0234] Fig.12 FIG. 1 is a block diagram showing a partial structure of a mobile phone related to a terminal device provided in an embodiment of the present application. Fig.12 The mobile phone includes: a radio frequency (RF) circuit 710, a memory 720, an input unit 730, a display unit 740, a sensor 750, an audio circuit 760, a wireless fidelity (WiFi) module 770, a processor 790, and a power supply 790. Those skilled in the art can understand that Fig.12 The mobile phone structure shown in the figure does not constitute a limitation on the mobile phone, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.
[0235] Combine the following Fig.12 A detailed introduction to the various components of the mobile phone:
[0236] The RF circuit 710 can be used for receiving and sending signals during the process of sending and receiving information or making calls. In particular, after receiving the downlink information of the base station, it is sent to the processor 790 for processing; in addition, the designed uplink data is sent to the base station. Usually, the RF circuit 710 includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (Low Noise Amplifier, referred to as LNA), a duplexer, etc. In addition, the RF circuit 710 can also communicate with the network and other devices through wireless communication. The above-mentioned wireless communication can use any communication standard or protocol, including but not limited to the Global System of Mobile communication (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, Short Messaging Service (SMS), etc.
[0237] The memory 720 can be used to store software programs and modules. The processor 790 executes various functional applications and data processing of the mobile phone by running the software programs and modules stored in the memory 720. The memory 720 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory 720 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.
[0238] The input unit 730 can be used to receive input digital or character information, and generate key signal input related to the user settings and function control of the mobile phone. Specifically, the input unit 730 may include a touch panel 731 and other input devices 732. The touch panel 731, also known as a touch screen, can collect the user's touch operation on or near it (such as the user's operation on the touch panel 731 or near the touch panel 731 using any suitable object or accessory such as a finger, stylus, etc.), and drive the corresponding connection device according to a pre-set program. Optionally, the touch panel 731 may include a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into the touch point coordinates, and then sends it to the processor 790, and can receive and execute the command sent by the processor 790. In addition, the touch panel 731 can be implemented in various types such as resistive, capacitive, infrared and surface acoustic wave. In addition to the touch panel 731, the input unit 730 may further include other input devices 732. Specifically, the other input devices 732 may include but are not limited to one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, and the like.
[0239] The display unit 740 can be used to display information input by the user or information provided to the user and various menus of the mobile phone. The display unit 740 may include a display panel 741. Optionally, the display panel 741 may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc. Further, the touch panel 731 may cover the display panel 741. When the touch panel 731 detects a touch operation on or near it, it is transmitted to the processor 790 to determine the type of touch event. Subsequently, the processor 790 provides a corresponding visual output on the display panel 741 according to the type of touch event. Although in Fig.12 In the embodiment, the touch panel 731 and the display panel 741 are used as two independent components to realize the input and output functions of the mobile phone, but in some embodiments, the touch panel 731 and the display panel 741 can be integrated to realize the input and output functions of the mobile phone.
[0240] The mobile phone may also include at least one sensor 750, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display panel 741 according to the brightness of the ambient light, and the proximity sensor may turn off the display panel 741 and / or the backlight when the mobile phone is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in all directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that identify the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for other sensors that can be configured in the mobile phone, such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., they will not be repeated here.
[0241] The audio circuit 760, the speaker 761, and the microphone 762 can provide an audio interface between the user and the mobile phone. The audio circuit 760 can transmit the received audio data to the speaker 761 after converting the received audio data into an electrical signal, which is converted into a sound signal for output; on the other hand, the microphone 762 converts the collected sound signal into an electrical signal, which is received by the audio circuit 760 and converted into audio data, and then the audio data is output to the processor 790 for processing, and then sent to another mobile phone through the RF circuit 710, or the audio data is output to the memory 720 for further processing.
[0242] WiFi is a short-range wireless transmission technology. The mobile phone can help users send and receive emails, browse web pages and access streaming media through the WiFi module 770. It provides users with wireless broadband Internet access. Fig.12 A WiFi module 770 is shown, but it is understandable that it is not an essential component of the mobile phone and can be omitted as needed without changing the essence of the invention.
[0243] The processor 790 is the control center of the mobile phone. It uses various interfaces and lines to connect various parts of the entire mobile phone. By running or executing software programs and / or modules stored in the memory 720, and calling data stored in the memory 720, it executes various functions of the mobile phone and processes data, thereby performing overall detection of the mobile phone. Optionally, the processor 790 may include one or more processing units; preferably, the processor 790 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 790.
[0244] The mobile phone also includes a power supply 790 (such as a battery) for supplying power to various components. Preferably, the power supply can be logically connected to the processor 790 through a power management system, so that the power management system can manage charging, discharging, power consumption and other functions.
[0245] Although not shown, the mobile phone may also include a camera, a Bluetooth module, etc., which will not be described in detail here.
[0246] In this embodiment, the processor 790 included in the terminal device also has the following functions:
[0247] Obtaining a model to be trained and sample data, wherein the model to be trained is obtained based on initial sample data, and the sample data is different from the initial sample data;
[0248] An initial model is formed by adding multiple incremental branches to the model to be trained, wherein the multiple incremental branches and the main branch in the model to be trained are used to perform the same type of processing operations on the same data, and the overall parameter amount of the incremental model parameters of the multiple incremental branches is less than the parameter amount of the target model parameters of the main branch;
[0249] Obtaining a pending processing result according to the sample data through the initial model, wherein the pending processing result is determined by combining the processing results corresponding to the plurality of incremental branches and the main branch respectively;
[0250] According to the difference between the pending processing result and the reference processing result of the sample data, the plurality of incremental model parameters are adjusted to obtain a target model.
[0251] The present application also provides a server. Fig.13 As shown, Fig.13 The structural diagram of the server 900 provided in the embodiment of the present application, the server 900 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 922 (for example, one or more processors) and memory 932, one or more storage media 930 (for example, one or more mass storage devices) storing application programs 942 or data 944. Among them, the memory 932 and the storage medium 930 can be short-term storage or permanent storage. The program stored in the storage medium 930 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the server. Furthermore, the central processing unit 922 can be configured to communicate with the storage medium 930 and execute a series of instruction operations in the storage medium 930 on the server 900.
[0252] The server 900 may also include one or more power supplies 926, one or more wired or wireless network interfaces 950, one or more input and output interfaces 959, and / or one or more operating systems 941, such as Windows Server 2000 or Windows Server 2003. TM , Mac OS X TM , Unix TM , Linux TM , FreeBSD TM etc.
[0253] The steps performed by the server in the above embodiment can be based on Fig.13 The server structure shown.
[0254] An embodiment of the present application also provides a computer-readable storage medium for storing a computer program, which is used to execute any one of the implementation methods of the model training methods described in the aforementioned embodiments.
[0255] An embodiment of the present application also provides a computer program product including a computer program, which, when executed on a computer device, enables the computer device to execute the model training method described in any one of the above embodiments.
[0256] It is understandable that in the specific implementation of this application, related data such as user information is involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.
[0257] A person skilled in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the above program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the above storage medium can be at least one of the following media: read-only memory (English: read-only memory, abbreviated: ROM), RAM, magnetic disk or optical disk, etc. Various media that can store program codes.
[0258] It should be noted that each embodiment in this specification is described in a progressive manner, and the same and similar parts between the embodiments can refer to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiments. The device and system embodiments described above are merely schematic, in which the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative work.
[0259] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. A model training method, characterized in that: The method comprises: Obtaining a model to be trained and sample data, wherein the model to be trained is obtained based on initial sample data, and the sample data is different from the initial sample data; An initial model is formed by adding multiple incremental branches to the model to be trained, wherein the multiple incremental branches and the main branch in the model to be trained are used to perform the same type of processing operations on the same data, and the overall parameter amount of the incremental model parameters of the multiple incremental branches is smaller than the parameter amount of the target model parameters of the main branch; the processing operations of the main branch and the multiple incremental branches are independent of each other; Obtaining a pending processing result according to the sample data through the initial model, wherein the pending processing result is determined by combining the processing results corresponding to the plurality of incremental branches and the main branch respectively; According to the difference between the pending processing result and the reference processing result of the sample data, a plurality of incremental model parameters are adjusted to obtain a target model, wherein the plurality of incremental model parameters are incremental model parameters of the plurality of incremental branches.
2. The method according to claim 1, characterized in that The target incremental branch is any one of the multiple incremental branches, the target incremental branch includes a first sub-branch and a second sub-branch, the incremental model parameter of the target incremental branch includes a first parameter of the first sub-branch and a second parameter of the second sub-branch, and obtaining a pending processing result according to the sample data includes: Determine input data of a plurality of processing branches according to the sample data, the plurality of processing branches including the main branch and the plurality of incremental branches; Through the multiple processing branches, the processing results corresponding to the multiple processing branches are determined according to the input data, the main branch is used to determine the processing result corresponding to the main branch according to the target model parameter and the input data, the target incremental branch is used to determine the first intermediate data according to the input data and the first parameter, and determine the processing result corresponding to the target incremental branch according to the first intermediate data and the second parameter, and the processing results corresponding to the multiple processing branches respectively correspond to the same data dimension; The pending processing result is determined by combining the processing results respectively corresponding to the multiple processing branches.
3. The method according to claim 2, characterized in that The input data is n-dimensional feature data, the processing results corresponding to the multiple processing branches are all m-dimensional feature data, the first intermediate data is r-dimensional feature data, r is less than n, and r is less than m, the first sub-branch is used to perform dimensionality reduction processing on the input data to determine the first intermediate data, and the second sub-branch is used to perform dimensionality increase processing on the first intermediate data to determine the processing result corresponding to the target incremental branch.
4. The method according to claim 2, characterized in that: The step of adjusting a plurality of incremental model parameters according to the difference between the pending processing result and the reference processing result of the sample data to obtain a target model comprises: adjusting the second parameter according to the difference between the pending processing result and the reference processing result of the sample data, and adjusting the first intermediate data according to the difference between the pending processing result and the reference processing result of the sample data, to obtain second intermediate data; The first parameter is adjusted according to a difference between the second intermediate data and the first intermediate data.
5. The method according to claim 2, characterized in that: The main branch is specifically used for: Perform feature extraction on the input data according to the target model parameters to generate a processing result corresponding to the main branch, wherein the processing result corresponding to the main branch is feature data corresponding to a first feature size; The target incremental branch is specifically used for: By means of the first sub-branch, feature extraction is performed on the input data according to the first parameter to generate the first intermediate data, the first intermediate data is feature data corresponding to a second feature size, the amount of calculation required for feature extraction is positively correlated with the feature size corresponding to the generated feature data, and the second feature size is smaller than the first feature size; Through the second sub-branch, the first intermediate data is processed according to the second parameter to determine the processing result corresponding to the target incremental branch, the data processing includes size processing for expanding the feature size, and the processing result is feature data corresponding to the first feature size.
6. The method according to claim 5, characterized in that Performing data processing on the first intermediate data according to the second parameter to determine a processing result corresponding to the target incremental branch includes: Performing size processing on the first intermediate data to generate third intermediate data, wherein the third intermediate data is feature data corresponding to the first feature size; A processing result corresponding to the target incremental branch is determined according to the third intermediate data and the second parameter.
7. The method according to claim 6, characterized in that The extracting features of the input data according to the target model parameters to generate a processing result corresponding to the main branch includes: According to the target model parameters, feature extraction is performed on the input data through p convolution processes to generate a processing result corresponding to the main branch; The step of extracting features from the input data according to the first parameter to generate the first intermediate data includes: According to the first parameter, extracting features from the input data by performing q convolution processes to generate the first intermediate data, where q is less than p; The resizing process includes a bilinear interpolation upsampling process.
8. The method according to claim 5, characterized in that The performing data processing on the first intermediate data according to the second parameter to determine the processing result corresponding to the target incremental branch includes: Determine fourth intermediate data according to the first intermediate data and the second parameter, wherein the fourth intermediate data is feature data corresponding to the second feature size; The fourth intermediate data is resized to generate a processing result corresponding to the target incremental branch.
9. The method according to claim 1, characterized in that: The step of adjusting a plurality of incremental model parameters according to the difference between the pending processing result and the reference processing result of the sample data to obtain a target model comprises: According to the difference between the pending processing result and the reference processing result of the sample data, multiple rounds of parameter adjustment are performed on the multiple incremental model parameters, and the multiple rounds of parameter adjustment respectively have corresponding learning rates. The learning rate decreases as the number of rounds increases in the multiple rounds of parameter adjustment. The target round parameter adjustment is any round in the multiple rounds of parameter adjustment. The learning rate corresponding to the target round parameter adjustment is used to control the adjustment amplitude of the multiple incremental model parameters, and the learning rate corresponding to the target round parameter adjustment is positively correlated with the adjustment amplitude.
10. The method according to claim 1, characterized in that The step of adjusting a plurality of incremental model parameters according to the difference between the pending processing result and the reference processing result of the sample data to obtain a target model comprises: According to the difference between the pending processing result and the reference processing result corresponding to the sample data, the plurality of incremental model parameters are adjusted in parallel to obtain a target model.
11. The method according to claim 1, characterized in that: The sample data and the initial sample data are both images; The model to be trained is used to identify the category of the first object in the initial sample data; The target model is used to identify the category of the first object in the initial sample data and the category of the second object in the sample data.
12. A model training device, characterized in that: The device comprises an acquisition unit, a formation unit, a determination unit and an adjustment unit: The acquisition unit is used to acquire a model to be trained and sample data, wherein the model to be trained is obtained based on the initial sample data, and the sample data is different from the initial sample data; The constituting unit is used to constitute an initial model by adding a plurality of incremental branches to the model to be trained, wherein the plurality of incremental branches and the main branch in the model to be trained are used to perform the same type of processing operations on the same data, and the overall parameter amount of the incremental model parameters of the plurality of incremental branches is less than the parameter amount of the target model parameters of the main branch; and the processing operations of the main branch and the plurality of incremental branches are independent of each other; The determining unit is used to obtain a pending processing result according to the sample data through the initial model, wherein the pending processing result is determined by combining the processing results corresponding to the multiple incremental branches and the main branch respectively; The adjustment unit is used to adjust multiple incremental model parameters according to the difference between the pending processing result and the reference processing result of the sample data to obtain a target model, and the multiple incremental model parameters are the incremental model parameters of the multiple incremental branches.
13. A computer device, characterized in that: The computer device comprises a processor and a memory: The memory is used to store a computer program and transmit the computer program to the processor; The processor is used to execute the model training method described in any one of claims 1-11 according to the instructions in the computer program.
14. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store a computer program, and the computer program is used to execute the model training method described in any one of claims 1-11.
15. A computer program product comprising a computer program, characterized in that When it runs on a computer device, the computer device executes the model training method described in any one of claims 1-11.
Citation Information
Patent Citations
Model training method and related device
CN110956265A
Target detection method and device based on incremental learning
CN113205142A
Multi-attribute identification and model training method and device, equipment and storage medium
CN116129214A
Natural language processing method and device, model training method and device and storage medium
CN116775807A
Compliance verification in large deployments
WO2022269526A1
