Model adjustment and business processing method, device, equipment and storage medium

Abstract

The application discloses a model adjustment and business processing method, device, equipment and storage medium, wherein the method comprises the following steps: determining a target adjustment type of a target model to be adjusted; determining existing ability data of the target model according to the target adjustment type; determining a target weight to be adjusted in the target model according to the target adjustment type; acquiring target training data corresponding to the target weight; and adjusting the target model based on the existing ability data and the target training data, wherein the adjustment comprises adjustment of the target weight, so that the target model learns new ability while keeping old pattern recognition ability, thereby improving the ability of the system to flexibly and quickly adjust according to business changes.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to a model adjustment method, a business processing method, a model adjustment device, a business processing device, a business processing equipment, a computer readable storage medium, and a computer program product. BACKGROUND

[0002] Considering the real-time and scale problems of short video / picture review and recommendation, most of the current review and recommendation processes tend to use a combination of machines and manual methods, with the machine processing part occupying a dominant position. For example, machine review can quickly discover and handle more than 90% of violations while ensuring a certain accuracy.

[0003] Current mainstream deep learning machine review / machine push systems tend to use a planar method to solve a wide range of complex review / recommendation application scenarios, that is, multiple functionally different models are used in parallel to meet different business needs. When the business environment changes, due to the low reuse capability of the business system, it is difficult to adapt to rapidly changing business needs and data distribution, and it takes a long time to update and adjust the system, especially in the case of multiple model collaboration, which takes even longer. SUMMARY

[0004] The present application provides a model adjustment and business processing method, device, equipment and storage medium to solve the problem that when the business environment changes, the system is difficult to adapt to rapidly changing business needs and data distribution, and it takes a long time to update and adjust the system in the prior art.

[0005] In a first aspect, the embodiments of the present application provide a model adjustment method, which comprises:

[0006] determining a target adjustment type of a target model to be adjusted;

[0007] determining existing capability data of the target model according to the target adjustment type;

[0008] determining a target weight in the target model that needs to be adjusted according to the target adjustment type;

[0009] obtaining target training data corresponding to the target weight;

[0010] adjusting the target model based on the existing capability data and the target training data, and the adjustment includes adjusting the target weight.

[0011] In a second aspect, the embodiments of the present application also provide a business processing method, which comprises:

[0012] a business data obtaining module, configured to obtain business data to be processed;

[0013] a target model loading module, configured to load the target model obtained by the method in the first aspect;

[0014] a business processing module, configured to process the business data by using the target model.

[0015] In a third aspect, an embodiment of the present application further provides a model adjusting apparatus, and the apparatus comprises:

[0016] a type of adjustment determining module, configured to determine a target adjustment type of a target model to be adjusted;

[0017] an existing capability data determining module, configured to determine, according to the target adjustment type, existing capability data of the target model;

[0018] a target weight determining module, configured to determine, according to the target adjustment type, a target weight in the target model that needs to be adjusted;

[0019] a target training data obtaining module, configured to obtain target training data corresponding to the target weight;

[0020] a model adjusting module, configured to adjust the target model based on the existing capability data and the target training data, and the adjustment comprises adjustment of the target weight.

[0021] In a fourth aspect, an embodiment of the present application further provides a business processing apparatus, and the apparatus comprises:

[0022] obtaining business data to be processed;

[0023] loading the target model obtained by the method in the first aspect;

[0024] processing the business data by using the target model.

[0025] In a fifth aspect, an embodiment of the present application further provides a business processing device, and the business processing device comprises:

[0026] one or more processors;

[0027] a storage apparatus, configured to store one or more programs,

[0028] when the one or more programs are executed by the one or more processors, the one or more processors implement the method in the first aspect or the second aspect.

[0029] In a sixth aspect, the embodiments of the present application further provide a computer readable storage medium, having stored thereon a computer program, which, when executed by a processor, implements the method of the first aspect or the second aspect.

[0030] In a seventh aspect, the embodiments of the present application further provide a computer program product, comprising computer executable instructions for implementing the method of the first aspect or the second aspect when executed by a processor.

[0031] The technical solutions provided by the present application have the following beneficial effects:

[0032] In the embodiment, when the target model needs to be adjusted, the adjustment direction is determined by determining the target adjustment type of the target model, and then the existing ability data of the target model is determined according to the target adjustment type, and the target weight that needs to be adjusted in the target model is determined. The target model is adjusted according to the existing ability data and the target training data, so as to adjust the target weight, so that the target model retains the old pattern recognition ability while learning new ability; so that the business system can extend new functions under the premise of retaining the original system ability as much as possible according to the changing business requirements and data distribution, so as to avoid the performance of the system under the new data distribution and business conditions. Greatly reduced, or even make the system have certain automatic optimization and intelligent improvement ability, improve the ability of the system to flexibly adjust according to the business changes. BRIEF DESCRIPTION OF DRAWINGS

[0033] Figure 1 is a flowchart of a model adjustment method embodiment provided by the first embodiment of the present application;

[0034] Figure 2 is a flowchart of a model adjustment method embodiment provided by the second embodiment of the present application;

[0035] Figure 3 is a flowchart of a model adjustment method embodiment provided by the third embodiment of the present application;

[0036] Figure 4 is a schematic diagram of range expansion branch division provided by the third embodiment of the present application;

[0037] Figure 5 is a flowchart of a model adjustment method embodiment provided by the fourth embodiment of the present application;

[0038] Figure 6 is an architecture schematic diagram of a mapping connection layer when expanding new task functions provided by the fourth embodiment of the present application;

[0039] Figure 7is a structural block diagram of a model adjustment device embodiment provided by Embodiment Five of the present application.

[0040] Figure 8 is a flowchart of a service processing method embodiment provided by Embodiment Six of the present application.

[0041] Figure 9 is a flowchart of a service processing device embodiment provided by Embodiment Seven of the present application.

[0042] Figure 10 is a structural schematic diagram of a service processing device provided by Embodiment Six of the present application. DETAILED DESCRIPTION

[0043] The present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application. In addition, it should be noted that, for the purpose of description, only the parts related to the present application are shown in the drawings, but not all the structures.

[0044] Embodiment One

[0045] Figure 1 is a flowchart of a model adjustment method embodiment provided by Embodiment One of the present application. The present embodiment can be applied to the scenario of adaptive adjustment of the model or model set used by the business system, such as Figure 1 As shown, the present embodiment can include the following steps:

[0046] Step 110, determining the target adjustment type of the target model to be adjusted.

[0047] The target model can include a model that needs to be updated or adjusted, and can include various models required by the business system to cope with different business needs and different functions, such as behavior recognition model, scene classification model, age regression model, gender recognition model, speech recognition model, user portrait model, multi-modal integrated model, sensitive word detection model, gun detection model, etc.

[0048] The target model can be a model specified by the user to be updated, or a model determined by the system to be updated according to a preset model updating strategy, and the present embodiment does not limit this.

[0049] The target adjustment type is used to guide the direction of the target model adjustment, and different target models can have different adjustment types.

[0050] In one embodiment, step 110 can further include the following steps:

[0051] At step 110-1, model requirement information input by the user for the target model is acquired.

[0052] The model requirement information can include a business function that the user expects the target model to implement, or a type of task that the user expects the target model to perform. For example, assuming that the user expects a model to implement a business function of task classification and task segmentation, the model requirement information can include identification information indicating task classification, and identification information indicating task segmentation.

[0053] In an implementation, an interactive interface can be presented to the user, the interactive interface can include a plurality of model function elements and a plurality of base models for the user to select from, the user can select one or more base models from the plurality of base models as the target model according to actual requirements, and select one or more model function elements from the model function elements as a function that the target model is expected to implement, and generate model requirement information according to the model function elements selected by the user.

[0054] In other embodiments, the user can also input model requirement information in the form of a command line or other instructions, and the model requirement information can also carry an identifier of the target model, and the present embodiment is not limited in this regard.

[0055] The model requirement information input by the user facilitates more flexible configuration of artificial conditions, and enhances flexibility in processing business requirements.

[0056] At step 110-2, existing model information of the target model is acquired.

[0057] The existing model information refers to description information indicating existing capabilities of the target model, for example, the existing model information can include a model architecture of the target model, model parameters and corresponding parameter values, implemented functions of the model, and the like.

[0058] In implementation, when the target model is determined, data of the target model can be loaded, and the existing model information of the target model can be extracted from the data of the target model.

[0059] At step 110-3, the model requirement information is compared with the existing model information to determine model extension information.

[0060] When the model requirement information of the target model and the existing model information are determined, the two can be compared to determine the capability information that the current target model does not have from the model requirement information as the model expansion information. In implementation, one or more requirements contained in the model requirement information are determined by analyzing the model requirement information, and then it is judged whether each requirement is a requirement that the target model has already implemented (i.e., the existing capability). If one or more requirements are requirements that the target model has not implemented, the requirements are taken as the model expansion information.

[0061] In step 110-4, one or more preset adjustment types of adjustment logic are obtained.

[0062] In an implementation, the adjustment type is adjustment information that is abstracted in advance by the developer according to experience and is used to guide the adjustment direction of the model. Different adjustment types can have different adjustment emphases, and the adjustment emphases of different adjustment types can be described by using adjustment logic.

[0063] For example, because the model to be adjusted in the business system involves a variety of task types and structural designs, in order to reduce the complexity of the system as much as possible, various model task types can be sorted into the following adjustment types: mode adjustment type, range adjustment type, and function adjustment type.

[0064] The mode adjustment type involves adjustment of the diversity of data mode coverage, that is, adjustment of the breadth of recognition of data within a category. For example, assuming that the model is a gun recognition model, the original gun recognition model can only recognize a certain type of gun in a certain region or work. If the user wants the model to be able to recognize most types of guns in the world or cover gun recognition in multiple works, this is adjustment of the diversity of mode coverage.

[0065] The range adjustment type involves adjustment of the range of data that can be processed within the same task function, for example, increasing the category of a classification task, increasing the number of key point detection, increasing the anchor box type of a detection task, and the like. For example, an example of "increasing the category of a classification task" can be as follows: Assuming that the original model can recognize two animals, cats and dogs, if the user wants the model to be able to recognize lions, this adjustment does not change the task type (both are classification tasks), but only increases the number of recognized categories, and therefore this adjustment belongs to the adjustment of the range of the model.

[0066] Unlike the mode adjustment type, the range adjustment type can change the type range that can be recognized by the model, while the mode adjustment type does not increase or decrease the type range to be recognized, but increases the breadth of recognition of the same category (from a certain region to the whole country or the world, etc.).

[0067] The function adjustment type is related to the adjustment of the number or combination of task functions that the model can handle, such as changing a single-task function model into a multi-task function model, or adding a task function that the original model does not have, such as expanding a model that only has classification ability into a dual-energy model that has both classification and detection abilities, or upgrading a simple key point recognition ability to a relational recognition ability.

[0068] Unlike the above range adjustment type, the function adjustment type can increase or decrease the task function. For example, assuming that the original model can implement a classification task, if the user wants the model to also be able to implement a segmentation task, this adjustment belongs to the adjustment of the task function.

[0069] Of course, in addition to the above three adjustment types, those skilled in the art can also abstract other adjustment types according to actual needs and define the adjustment logic of each adjustment type, and the present embodiment does not limit this.

[0070] Step 110-5, match the model expansion information with the adjustment logic of the one or more preset adjustment types, and take the matched adjustment type as the target adjustment type.

[0071] Among them, by analyzing the model expansion information, it can be determined that the expansion of the target model belongs to which aspect of expansion, so as to determine the target adjustment type. In one implementation, the target adjustment type can be determined in the following way:

[0072] According to the model expansion information, it is determined whether it involves the increase or decrease of the task function, if yes, the target adjustment type is determined as the function adjustment type; if not, it is further determined whether the model expansion information involves the increase or decrease of the task category of the same task function, if yes, the target adjustment type is determined as the range adjustment type; if not, it is further determined whether the model expansion information involves the increase or decrease of the mode (or breadth) of the same task category of the same task function, if yes, the target adjustment type is determined as the mode adjustment type; if not, the model expansion information can be provided to the developer for manual judgment.

[0073] Step 120, according to the target adjustment type, determine the existing ability data of the target model.

[0074] Among them, the existing ability data refers to the data that the target model already has and is associated with the target adjustment type. The existing model information in the above step 110-2 refers to all the ability data that the target model already has.

[0075] In implementation, the existing capability data can be extracted from the existing model information of the target model according to the adjustment logic of the target adjustment type. Since the adjustment logic of different adjustment types is different, the existing capability data corresponding to different adjustment types can also be different.

[0076] In step 130, the target weight to be adjusted in the target model is determined according to the target adjustment type.

[0077] In practice, when adjusting the model, the adjustment of each weight of the model is included. When adjusting the weight, all weights of the model can be adjusted, or part of the weights can also be adjusted. In implementation, the target weight to be adjusted can be determined according to the adjustment logic of the target adjustment type. Since the adjustment logic of different adjustment types is different, the target weight corresponding to different adjustment types can also be different.

[0078] In step 140, the target training data corresponding to the target weight is obtained.

[0079] The target training data can be existing training data or newly generated training data, and the embodiment is not limited thereto.

[0080] In implementation, the target training data corresponding to the target weight can be found from a database for storing training data. For example, if the target weight is the weight of the processing layer for identifying lions, the training data labeled with lions can be used as the target training data.

[0081] In step 150, the target model is adjusted based on the existing capability data and the target training data.

[0082] In this step, the target model can be adjusted according to the adjustment logic of the target adjustment type, using the existing capability data and the target training data, which can include the adjustment of the target weight. The above adjustment can include iterative updating of the target model, or expanding the target model (such as expanding the target model used in the old field to the new field), or reducing the target model (such as reducing some redundant functions), etc.

[0083] In an implementation, when adjusting the target model, at least one of a transfer learning method and a continual learning method can be used to adjust the target model according to the existing capability data of the target model. The transfer learning is a model learning method that migrates the capability of a processing model in a current field to a new model or extends to a new field, mainly hoping to use the learned pattern recognition capability. The continual learning is a learning method that avoids catastrophic forgetting, aiming to preserve the old pattern recognition capability while learning new capabilities. Catastrophic forgetting usually refers to a phenomenon that a model newly trained based on old weight parameters loses the pattern recognition capability for past data in subsequent training, and mainly occurs when the data distribution changes greatly.

[0084] The dual learning mechanism of the transfer learning method and the continual learning method used in this embodiment can reduce the development cost when adjusting the capability of the business system. Further, from the time axis of system development, the dual learning mechanism also shoulders an important mission in different development stages. In the expansion stage in the early stage of system development, there is a lack of labeled data, and at this time, the self-supervised idea of transfer learning plays a great role. Many different business requirements can be transferred from a single task field according to the commonality of the data distribution. In the optimization / extension stage in the middle and later stages, the idea of continual learning is used to avoid catastrophic forgetting of the existing capability, maintain the original capability, and steadily expand and gradually strengthen the response capability to variable human conditions, so that the system can also maintain high accuracy in a changing environment.

[0085] It should be noted that the above-mentioned adjustment of the target model is based on the adjustment generated by expansion, and in practice, the adjustment can also include adjustment generated by reduction of the target model. In this case, the redundant branches or the entire model that are not needed can be shielded accordingly without being deleted or processed in structure. This is mainly to prevent the increased adjustment overhead when the reduced function is needed again in the future. Some special reduction requirements can be combined by a preset post-processing strategy.

[0086] In the embodiment, when the target model needs to be adjusted, the adjustment direction is determined by determining the target adjustment type of the target model, and then the existing ability data of the target model is determined according to the target adjustment type, and the target weight that needs to be adjusted in the target model is determined. The target model is adjusted according to the existing ability data and the target training data, so as to adjust the target weight, so that the target model learns new ability while preserving the old pattern recognition ability; so that the business system can extend new functions under the premise of preserving the original system ability as much as possible according to the changing business requirements and data distribution, so as to avoid the performance of the system under the new data distribution and business conditions. Even let the system have certain automatic optimization and intelligent improvement ability, improve the ability of the system to adjust quickly according to the business changes.

[0087] Since the adjustment of the target model in the embodiment is based on the existing ability data of the reused model, supplemented by a specific learning mechanism, and the data labels fed back by the current system can be periodically used for ordinary iterative optimization, the system adjustment can be realized under stable conditions with low cost, so as to match the rapid changes of the attention points of short video / picture and other businesses. At the same time, the system can quickly adapt to changes without causing a large decline in effect.

[0088] Embodiment two

[0089] Figure 2 A flowchart of a model adjustment method embodiment provided by embodiment two of the application is provided. Based on embodiment one, the model adjustment process based on the pattern adjustment type is described, as shown in Figure 2 The embodiment can include the following steps:

[0090] Step 210, determining the target adjustment type of the target model to be adjusted.

[0091] Step 220, obtaining the pattern library of the target model.

[0092] In the embodiment, the idea of "replay" in continuous learning is used to create a pattern library. Replay is a special continuous learning method, which mainly operates according to the learned data or data prototype to first deduce the recognition ability of the old data distribution in the new model, and then learn the pattern of the new data. The pattern library can include multiple pattern entries, and the pattern entries can be information generated based on the specified feature information extracted from the target model. The pattern entries can be used to quickly restore the recognition ability of the model to the old data pattern when the target model is adjusted in the future, so as to avoid the catastrophic forgetting of the model to the data that can be recognized before.

[0093] In an implementation, the schema library can be stored in a server where the business system is located, or can be managed by a dedicated storage server. The corresponding schema library can be found through the identification of the target model.

[0094] In step 230, weights of each processing layer of the target model are obtained, the processing layers including fixed processing layers and non-fixed processing layers.

[0095] In an implementation, the fixed processing layers and the non-fixed processing layers in the target model can be specified by a developer. For example, the developer can specify which processing layers are fixed processing layers according to prior knowledge, and the remaining processing layers are non-fixed processing layers. Alternatively, the developer can also specify the characteristic information possessed by the fixed processing layers, and then the system can determine the fixed processing layers from the multiple processing layers of the target model according to the characteristic information. For example, the feature extraction layer or the attention layer can be taken as the fixed processing layer.

[0096] In step 240, the schema library and the weights of each processing layer are taken as the existing capability data.

[0097] In the embodiment, the schema library of the target model and the weights of each processing layer of the target model are taken as the existing capability data of the target model. The existing capability data refers to the model data related to the target adjustment type.

[0098] In step 250, the weights of the non-fixed processing layers are initialized, and the initialized weights are taken as the target weights in the target model that need to be adjusted.

[0099] When the target model is updated and adjusted, the weights of the fixed processing layers can be fixed, and then the weights of the non-fixed processing layers are initialized, and the initialized weights of the non-fixed processing layers are taken as the target weights in the target model that need to be adjusted.

[0100] In step 260, target training data corresponding to the target weights is obtained.

[0101] The target training data is new training data, that is, the training data used for updating the target model.

[0102] In step 270, old schema characteristic information of the target model is determined based on the schema entries in the schema library.

[0103] In this step, all schema entries can be taken out from the schema library of the target model, and the old schema characteristic information of each schema entry is generated.

[0104] In an embodiment, step 270 can further include the following steps:

[0105] Step 270-1, generate one or more perturbation vectors corresponding to each mode entry by using a random vector generator, wherein the random vector generator is a random vector generator simulating a normal distribution of multi-modal.

[0106] Step 270-2, respectively perturb the mode entries by using the one or more perturbation vectors, to obtain the old mode feature information corresponding to the mode entries.

[0107] The perturbation vector is used to perturb the mode entry to generate the feature information (i.e., the old mode feature information) of the mode entry. For one mode entry, one perturbation vector can be generated, or multiple perturbation vectors can be generated. If multiple perturbation vectors are generated, the mode entry is perturbed multiple times, and each perturbation vector corresponds to an old mode embedding (i.e., old mode feature information) generated.

[0108] The random vector generator simulating a normal distribution of multi-modal can be used to generate one or more perturbation vectors corresponding to each mode entry. Then, the generated perturbation vectors are used to perturb the current mode entry to obtain the old mode feature information corresponding to the mode entry.

[0109] For example, assuming that the feature vector of a mode entry is (x1, x2, x3,..., xn), and the perturbation vector is (d1, d2,..., dn), then the perturbation result of one time can be (x1+d1, x2+d2,..., xn+dn). When multiple perturbations are needed, the feature vector of the mode entry is taken as the center, and each perturbation vector acts on this "center" (such as each perturbation vector is added to the feature vector of the center), to obtain multiple corresponding old mode embeddings.

[0110] Step 280, input the target training data into the target model, and use the fixed processing layer to extract features of the target training data in the target model to obtain new feature information.

[0111] In this embodiment, in addition to calculating the old mode embedding, the embedding (i.e., new feature information) of the new data based on the new training data (i.e., target training data) also needs to be calculated. In implementation, the target training data can be input into the target model, and the new feature information can be calculated by the fixed processing layer in the target model.

[0112] Step 290, adjust the target model by using the old mode feature information and the new feature information.

[0113] In this step, the old mode feature information and the new feature information are used together to adjust the target model. In one embodiment, the old mode feature information is used to update the target weights, and the new feature information is used to update the weights of all processing layers. That is, the old mode feature information is only used to update the initialized part of the weights (i.e., the target weights), and the new feature information is used to update all weight parameters, so that the model quickly recovers the recognition ability for the old data mode and learns the new data mode.

[0114] In one embodiment, in order to enable the target model to quickly recover the memory of the old mode, when adjusting the target model, an L2 regularization term is used to expand the loss function until the model converges. The L2 regularization term is a regularization term combining Laplace estimation and Fisher encoding. Laplace estimation is a posterior probability estimation method for evaluating the proximity of a new distribution to an old distribution using a Laplace matrix and maximum likelihood estimation; Fisher encoding is a curvature-based encoding method that can force the model to learn a specific data distribution by controlling the size of the curvature. This can enable the model to quickly learn the old data distribution in the early stage of model training, and then promote learning of the new data distribution in the later stage of training.

[0115] In one embodiment, after the adjustment of the target model is completed, the embodiment can further include the following steps:

[0116] When the adjustment of the target model is completed, the adjusted target model is used to extract feature information from newly generated training data; all obtained feature information is clustered to obtain a cluster center; and the cluster center is added to the mode library as a mode entry.

[0117] In this embodiment, each time the adjustment of the model is completed, the newly converged model is used to extract embedding (feature information) from newly generated training data. The embedding can be a plurality of feature vectors or feature maps. Then, a density clustering method can be used to obtain a cluster center from the embedding of all newly generated training data, and the cluster center can be stored in the mode library as a mode entry.

[0118] In other implementations, in addition to using a density clustering method to obtain a cluster center, a hierarchical clustering method can also be used to determine a cluster center, and hierarchical information can be used to filter out part information with negative effects.

[0119] The embodiment can greatly reduce the space required to store the "replay prototype" by storing the cluster center and the disturbance recovery.

[0120] In this embodiment, in order to improve the data coverage diversity, the old mode feature information is reused by creating a mode library through the idea of replay, and the target model is adjusted by combining the new feature information obtained through new training data, so as to quickly restore the original ability while learning the mode of new data. On the premise of not expanding the weight parameter amount of the model, the purpose of improving data coverage is achieved, the required storage space is reduced, and the reuse ability of the developed module is improved.

[0121] Embodiment three

[0122] Figure 3 A flowchart of a model adjustment method embodiment provided for embodiment three of the present application, which is based on embodiment one and describes the model adjustment process based on the task range adjustment type, as shown in Figure 3 The embodiment can include the following steps:

[0123] Step 310, determining the target adjustment type of the target model to be adjusted.

[0124] In an embodiment, step 310 can further include the following steps:

[0125] Obtaining model requirement information input by a user for the target model; obtaining existing model information of the target model; comparing the model requirement information with the existing model information to determine model expansion information; obtaining adjustment logic of one or more preset adjustment types; matching the model expansion information with the adjustment logic of the one or more preset adjustment types, and taking the matched adjustment type as the target adjustment type.

[0126] Step 320, obtaining the weights of each processing layer of the target model, including low-level processing layer, middle-level processing layer and high-level processing layer.

[0127] In this embodiment, the processing layers of the target model can be divided into low-level processing layer, middle-level processing layer and high-level processing layer, and the division strategy can be specified artificially.

[0128] Step 330, obtaining the weights of each existing output branch of the target model.

[0129] For example, for a classification model processing a classification task, the model can have multiple output branches, and each output branch corresponds to a class.

[0130] Step 340, taking the weights of each processing layer and the weights of each existing output branch as existing ability data.

[0131] Step 350, generating a corresponding range expansion branch for the model expansion information.

[0132] In this embodiment, when the target model needs to expand the task range, model expansion information can be obtained, which is information for indicating that the target model needs to expand. After the model expansion information is obtained, a small processing branch, referred to as a range expansion branch, can be added in the target model for the model expansion information.

[0133] In practice, for the scenario of expanding the task range of the target model, the following two cases can be considered: one is to expand the task type not contained in the current model output, that is, to expand the unknown task range; the other is to expand or disassemble the task type of the existing output into a more detailed type, that is, to expand the known task range. The above two cases are handled by using the idea of transfer learning, which migrates the current learned pattern recognition ability to a larger range of tasks.

[0134] In order to maintain the generalization ability of the model and avoid the influence of negative transfer (negative transfer refers to the transfer learning operation that has a negative effect on learning new domain tasks) on the existing recognition ability, as shown in FIG. 3B, when the unknown task range of the model needs to be expanded, a specified output option can be set in the newly created range expansion branch (i.e., the new disassembly branch in FIG. 3B), which exemplarily can include an “other” type. The specified output option can correspond to unlabeled training data. When the known task range needs to be expanded, the specified output option does not need to be set in the newly created range expansion branch, because the output of the range expansion branch actually comes from a certain output of the original branch, and the unlabeled data only needs to be concentrated in the specified output option of the original branch, so that the model can inherit the recognition ability of the original type pattern as much as possible. Figure 4 Figure 4

[0135] Step 360, initializing the weight of the range expansion branch.

[0136] In this step, the weight of the newly created range expansion branch can be an initial weight.

[0137] Step 370, taking the initialized weight of the range expansion branch and the weight of the high-level processing layer as the target weight.

[0138] Step 380, respectively fixing the weight of the low-level processing layer and the weight of the middle-level processing layer.

[0139] ​​In one implementation, when training the target model for the first time, the low-level processing layer and the middle-level processing layer of the target model can be trained by using a large-scale general data set, and the weights of the low-level processing layer and the weights of the middle-level processing layer are fixed. Each subsequent adjustment only updates the weights of the newly added range expansion branch and the weights of the high-level processing layer, so that the main part of feature extraction has strong generalization feature extraction capability, and the training overhead of each adjustment is reduced.

[0140] In step 390, target training data corresponding to the target weight is obtained.

[0141] For example, in this embodiment, the target training data can include training data before model expansion and training data after model expansion, and the training data after model expansion mainly refers to training data corresponding to the range expansion branch.

[0142] In step 3110, the target weight is adjusted by using the target training data.

[0143] In one implementation, when the target weight is adjusted by using the target training data, the loss function used can be a domain matching-based loss function, where domain matching refers to a self-learning scheme for unlabeled data, which generally infers unknown data by using part of existing data. The domain matching-based loss function can include, for example, a Laplace estimation-based loss function.

[0144] In actual training, the gradient calculated by the existing output branch and the gradient calculated by the range expansion branch jointly act on the weight parameters that need to be updated. In this way, the generalization capability of the feature extraction part can be continuously enhanced during adjustment. At the same time, the expansion in the form of a small branch also maximizes the increase in the number of parameters, avoiding an excessively large model in the later stage; and provides better expansion capability for businesses with growth characteristics.

[0145] In one embodiment, the existing capability data further includes a specified output option of the existing output branch; and step 3110 can further include the following steps:

[0146] Obtaining first training data corresponding to the specified output option of the existing output branch;

[0147] Determining whether the range expansion branch contains the specified output option;

[0148] If yes, obtaining second training data of the specified output option of the range expansion branch, and adjusting the weights of the range expansion branch and the weights of the existing output branch in a self-supervised manner based on the first training data and the second training data;

[0149] If not, the weights of the range expansion branch and the weights of the existing output branch are adjusted in a self-supervised manner based on the first training data.

[0150] Specifically, when adjusting the model, if it is determined that the range expansion branch contains the specified output option, it indicates that the current range expansion is an expansion of an unknown range. Second training data of the specified output option of the range expansion branch can be obtained, and the first training data corresponding to the specified output option of the existing output branch is combined as a training data set. Then, the training data set is learned in a self-supervised manner to adjust the weights of the current range expansion branch and the weights of the existing output branch.

[0151] It should be noted that the specified output option (such as the "other" type) of the existing output branch will only perform back propagation on the last layer of the decision layer to fine-tune the parameters. The specified output option is set to maintain the generalization ability of the model and avoid the influence of "negative transfer" on the existing recognition ability.

[0152] On the other hand, if it is determined that the range expansion branch does not contain the specified output option, it indicates that the current range expansion is an expansion of a known range. When adjusting the weights of the current range expansion branch, the original task type corresponding to the range expansion branch can be shielded, and the weights of the current range expansion branch are mainly trained during training. During training, the weights of the range expansion branch and the weights of the existing output branch can be adjusted in a self-supervised manner based on the first training data.

[0153] In the embodiment, the idea of transfer learning is used to transfer the learned pattern recognition ability to a larger range of tasks in response to the need to expand the task range of the same task. After generating the corresponding range expansion branch based on the model expansion information, the weights of the range expansion branch and the weights of the high-level processing layer are adjusted during the adjustment of the target model. Expanding in the form of a small branch can maximize the control of the increase of the number of parameters, avoid the model becoming too large in the later stage, and provide better expansion capability for the business with growth characteristics.

[0154] In addition, the embodiment makes corresponding design for expanding unknown task range and known task range, respectively, and leaves room for further expansion of the task range by self-supervised learning of uncertain label data.

[0155] Embodiment Four

[0156] Figure 5 A flowchart of a model adjustment method embodiment provided by Embodiment Four of the present application is based on Embodiment One, and the model adjustment process based on the task function adjustment type is described as shown in Figure 5 The embodiment can include the following steps:

[0157] In step 510, a target adjustment type of the target model to be adjusted is determined.

[0158] In an embodiment, step 510 can further include the following steps:

[0159] obtaining model requirement information input by a user for the target model; obtaining existing model information of the target model; comparing the model requirement information with the existing model information to determine model expansion information; obtaining one or more preset adjustment logics of adjustment types; matching the model expansion information with the one or more preset adjustment logics of adjustment types, and taking a matched adjustment type as the target adjustment type.

[0160] In step 520, weights of each existing function branch of the target model are obtained.

[0161] In step 530, the weights of each existing function branch are taken as the existing capability data.

[0162] In step 540, a corresponding function expansion branch is generated for the model expansion information.

[0163] In step 550, a weight of the function expansion branch is initialized.

[0164] In step 560, a mapping and bridging layer is constructed between the function expansion branch and each existing function branch, the mapping and bridging layer is used to map feature information of the existing function branch, and determine a shared weight from the feature information of the existing function branch, the shared weight is used as the existing capability data.

[0165] In step 570, the initialized weight of the expansion branch and the weight of the mapping and bridging layer are taken as the target weight.

[0166] In step 580, the target model is adjusted based on the existing capability data and the target training data, the adjustment includes adjustment of the target weight.

[0167] The model adjustment method of the task function adjustment type, and Figure 3 The task range adjustment type in the embodiment is similar to the task function adjustment type, and also generates a corresponding expansion branch for model expansion information, which is called a function expansion branch in this embodiment, and is used to carry new task functions. The newly constructed small function expansion branch is connected to the output corresponding to the expanded task function, and the function expansion branch is trained using the loss function related to the corresponding task function.

[0168] In practice, the output of the new task function can be quite different from the original task function, such as the segmentation task function taking a complete heat map as the output, and the classification function only including a confidence sequence of multiple categories. Therefore, the function expansion branch of the embodiment is more complex than the range expansion branch Figure 3 .

[0169] When adjusting the target model, the weights of the function expansion branch are initialized and trained. At the same time, the weights of a part of the feature extraction layer are shared with the old task function, mainly because the learning mechanism is mainly for the change of the task function, and the data distribution is basically consistent or fluctuates slightly, and the feature extraction ability of the original task function will also help the performance of the new task function.

[0170] Specifically, as shown in Figure 6 , when expanding the new task function, a mapping connection layer (i.e. the connection layer in Figure 6 ) is constructed between the function expansion branch and each existing function branch, mainly because the discrimination of the function expansion branch also needs to combine the feature extraction ability of the existing function branch, but the output of the decision will be greatly affected by the output of the new task function, so the feature information of the existing function branch also needs to be mapped to be reasonably used by the new function. At the same time, because it is not clear which feature information of the existing function branch is more conducive to the decision of the new function, the feature extraction layers of multiple existing function branches will be connected with the mapping connection layer, and the mapping connection layer will judge and select the appropriate shared feature information for use. During training, the mapping connection layer will also be trained together with the new function expansion branch, and the gradient of the loss function will also be back-propagated to the mapping connection layer and the new function expansion branch.

[0171] In the embodiment, by combining the connection mapping layer and the new function branch, the expansion of the task function is realized, which not only utilizes the feature extraction ability of the old task function, but also provides suitable mapping for the decision of the new task function.

[0172] In addition, the embodiment combines the ideas of continuous learning and transfer learning, which does not cause catastrophic forgetting of the past learned ability, and also completes the diffusion of the single function field ability to the multi-function field, so that the system can keep up with the changes of business needs and data distribution.

[0173] Embodiment five

[0174] Figure 7 The structure block diagram of a model adjustment device embodiment provided by the embodiment five of the application can include the following modules:

[0175] The adjustment type determination module 710 is configured to determine the target adjustment type of the target model to be adjusted.

[0176] The existing capability data determination module 720 is configured to determine the existing capability data of the target model according to the target adjustment type.

[0177] The target weight determination module 730 is configured to determine the target weight to be adjusted in the target model according to the target adjustment type.

[0178] The target training data acquisition module 740 is configured to acquire target training data corresponding to the target weight.

[0179] The model adjustment module 750 is configured to adjust the target model based on the existing capability data and the target training data, and the adjustment includes adjustment of the target weight.

[0180] In an embodiment, the adjustment type determination module 710 is specifically configured to:

[0181] acquire model requirement information input by a user for the target model;

[0182] acquire existing model information of the target model;

[0183] compare the model requirement information with the existing model information to determine model expansion information;

[0184] acquire one or more preset adjustment logic of adjustment types;

[0185] match the model expansion information with the one or more preset adjustment logic of adjustment types, and match the adjustment type to the target adjustment type.

[0186] In an embodiment, the existing capability data determination module 720 is specifically configured to:

[0187] acquire a pattern library of the target model, the pattern library including a plurality of pattern entries;

[0188] acquire weights of each processing layer of the target model;

[0189] use the pattern library and the weights of each processing layer as the existing capability data.

[0190] In an embodiment, the processing layer includes a fixed processing layer and a non-fixed processing layer, and the target weight determination module 730 is specifically configured to:

[0191] initialize the weights of the non-fixed processing layer;

[0192] use the initialized weights as the target weight to be adjusted in the target model.

[0193] In an embodiment, the model adjustment module 750 can further include the following sub-modules:

[0194] an old pattern feature information determination sub-module configured to determine old pattern feature information of the target model based on the pattern entries in the pattern library;

[0195] a new feature information acquisition sub-module configured to input the target training data into the target model, extract features of the target training data by using the fixed processing layers in the target model, and obtain new feature information;

[0196] a model adjustment sub-module configured to adjust the target model by using the old pattern feature information and the new feature information.

[0197] In an embodiment, the old pattern feature information determination sub-module is specifically configured to:

[0198] generate one or more perturbation vectors corresponding to each pattern entry by using a random vector generator, wherein the random vector generator is a random vector generator simulating a normal distribution of multi-modal;

[0199] perturb the pattern entries by using the one or more perturbation vectors respectively, and obtain the old pattern feature information corresponding to the pattern entries.

[0200] In an embodiment, the model adjustment sub-module is specifically configured to:

[0201] update the target weights by using the old pattern feature information;

[0202] update the weights of all processing layers by using the new feature information.

[0203] In an embodiment, the apparatus can further include the following modules:

[0204] a loss function expansion module configured to expand a loss function by using an L2 regular term when adjusting the target model, wherein the L2 regular term is a regular term combining Laplace estimation and Fisher encoding.

[0205] In an embodiment, the apparatus can further include the following modules:

[0206] a pattern addition module configured to extract feature information of newly generated training data by using the adjusted target model after the adjustment of the target model is completed, cluster all obtained feature information to obtain a cluster center, and add the cluster center as a pattern entry to the pattern library.

[0207] In another embodiment, the target adjustment type includes a task scope adjustment type, and the capability data determination module 720 is specifically used for:

[0208] Obtain the weights of each processing layer of the target model;

[0209] Obtain the weights of each existing output branch of the target model;

[0210] The weights of each processing layer and the weights of each existing output branch are used as the existing capability data.

[0211] In one embodiment, the processing layer includes a higher-level processing layer; the target weight determination module 730 is specifically used for:

[0212] Generate corresponding range extension branches for the model extension information;

[0213] Initialize the weights of the range extension branch;

[0214] The weights initialized by the range expansion branch and the weights of the higher-level processing layer are used as the target weights.

[0215] In one embodiment, the processing layer further includes a low-level processing layer and a mid-level processing layer; the model adjustment module 750 is specifically used for:

[0216] The weights of the lower-level processing layer and the weights of the middle-level processing layer are fixed respectively.

[0217] The target weights are adjusted using the target training data.

[0218] In one embodiment, the existing capability data further includes specified output options for existing output branches;

[0219] The model adjustment module 750 is specifically used for:

[0220] Obtain the first training data corresponding to the specified output option of the existing output branch;

[0221] Determine whether the range expansion branch contains the specified output option;

[0222] If so, then obtain the second training data for the specified output option of the range expansion branch, and adjust the weights of the range expansion branch and the weights of the existing output branch in a self-supervised manner based on the first training data and the second training data.

[0223] If not, then based on the first training data, a self-supervised approach is used to adjust the weights of the range expansion branch and the weights of the existing output branch.

[0224] In yet another embodiment, the existing capability data determination module 720 is specifically configured to:

[0225] obtain the weight of each existing function branch of the target model;

[0226] use the weight of each existing function branch as the existing capability data.

[0227] In one embodiment, the target weight determination module 730 is specifically configured to:

[0228] generate a corresponding function extension branch according to the model extension information;

[0229] initialize the weight of the function extension branch;

[0230] construct a mapping adaptation layer between the function extension branch and each existing function branch, the mapping adaptation layer being configured to map the feature information of the existing function branch and determine a shared weight from the feature information of the existing function branch, the shared weight being used as the existing capability data;

[0231] use the initialized weight of the extension branch and the weight of the mapping adaptation layer as the target weight.

[0232] The model adjustment device provided in the embodiments of the present application can perform the model adjustment method of any one of the embodiments one to four, and has the corresponding function modules and beneficial effects of the execution method.

[0233] Embodiment six

[0234] Figure 8 The flowchart of the service processing method embodiment provided in the embodiment six of the present application can include the following steps:

[0235] Step 810, obtaining service data to be processed;

[0236] Step 820, loading the target model obtained according to any one of the embodiments one to four;

[0237] Step 830, processing the service data by using the target model.

[0238] In the embodiment, the target model used in the service processing is a real-time adjusted model. The new function is extended for the changed service requirement and data distribution while the original model capability is preserved as much as possible. Since the adjustment is based on the reuse of the existing model and supplemented by a specific learning mechanism, and the data labeling period is periodically optimized by the model real-time feedback, the model adjustment can be realized in stable conditions with low cost to match the rapid changes of the service focus, and the effect is not greatly reduced while adapting to the changes.

[0239] Embodiment seven

[0240] Figure 9 The flowchart of a service processing device embodiment provided for the seventh embodiment of the present application can include the following modules:

[0241] The service data acquisition module 910 is configured to acquire service data to be processed.

[0242] The target model loading module 920 is configured to load the target model obtained according to any one of the first to fourth embodiments.

[0243] The service processing module 930 is configured to process the service data by using the target model.

[0244] The service processing device provided in the embodiments of the present application can execute the service processing method of the seventh embodiment of the present application, and has the corresponding function modules and beneficial effects of the execution method.

[0245] Embodiment eight

[0246] Figure 10 As shown in the structural schematic diagram of a service processing device provided for the eighth embodiment of the present application, the service processing device includes a processor 1010, a memory 1020, an input device 1030 and an output device 1040. Figure 10 The number of processors 1010 in the service processing device can be one or more, and one processor 1010 is taken as an example in the embodiment. Figure 10 The processor 1010, the memory 1020, the input device 1030 and the output device 1040 in the service processing device can be connected through a bus or other means, and the connection through the bus is taken as an example in the embodiment. Figure 10

[0247] ​The memory 1020, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as program instructions / modules corresponding to any of the embodiments one to four or embodiment six in this application. The processor 1010 executes various functional applications and data processing of the business processing device by running the software programs, instructions, and modules stored in the memory 1020, thereby implementing the methods mentioned in any of the embodiments one to four or embodiment six.

[0248] The memory 1020 may primarily include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on terminal usage. Furthermore, the memory 1020 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 1020 may further include memory remotely located relative to the processor 1010, which can be connected to the device / terminal / server via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0249] Input device 1030 can be used to receive input digital or character information, and to generate key signal inputs related to user settings and function control of the business processing equipment. Output device 1040 may include display devices such as a display screen.

[0250] Example 9

[0251] Embodiment 7 of this application also provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform any of the embodiments of the above-described method embodiments 1 to 4 or the method of embodiment 6.

[0252] Of course, the computer-executable instructions provided in the embodiments of this application are not limited to the method operations described above, but can also perform related operations in the methods provided in any embodiment of this application.

[0253] Example 10

[0254] Embodiment 10 of this application also provides a computer program product, which includes computer-executable instructions. When executed by a computer processor, the computer-executable instructions are used to perform any of the embodiments of the above-described method embodiments 1 to 4 or the method of embodiment 6.

[0255] Of course, the computer executable instructions of the computer program product provided by the embodiments of the present application are not limited to the method operations described above, and can also perform related operations in the method provided by any of the embodiments of the present application.

[0256] Through the above description of the embodiments, those skilled in the art can clearly understand that the present application can be realized by means of software and necessary universal hardware, and of course can also be realized by hardware, but in many cases the former is a better implementation. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a floppy disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a FLASH, a hard disk, or an optical disc, etc., and includes a number of instructions for causing a business processing device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments of the present application.

[0257] It is worth noting that in the embodiments of the above device, each unit and module included is only divided according to the function logic, but is not limited to the above division, as long as the corresponding function can be realized; in addition, the specific name of each functional unit is only for convenient mutual distinction, and does not limit the protection scope of the present application.

[0258] Note that the above are only preferred embodiments of the present application and the technical principles applied. Those skilled in the art will understand that the present application is not limited to the specific embodiments described herein, and those skilled in the art can make various obvious changes, readjustments and substitutions without departing from the scope of the present application. Therefore, although the present application has been described in more detail through the above embodiments, the present application is not limited to the above embodiments, and can include more other equivalent embodiments without departing from the concept of the present application, and the scope of the present application is determined by the scope of the appended claims.

Claims

1. A method of video and / or picture model adjustment, characterized in that, The method comprises: determining a target adjustment type of a target video and / or picture model to be adjusted; wherein the adjustment type comprises a mode adjustment type, a task range adjustment type, and a task function adjustment type; determining existing capability data of the target video and / or picture model according to the target adjustment type; wherein the existing capability data refers to data that the target video and / or picture model already has and is associated with the target adjustment type; determining a target weight of a video and / or picture processing layer that needs to be adjusted in the target video and / or picture model according to the target adjustment type; obtaining target video and / or picture training data corresponding to the target weight; adjusting the target video and / or picture model based on the existing capability data and the target video and / or picture training data, wherein the adjustment comprises adjustment of the target weight. The method comprises: obtaining model requirement information input by a user for the target video and / or picture model; the model requirement information comprises a business function that the user expects the target video and / or picture model to be able to implement, or a task type that the user expects the target video and / or picture model to be able to perform; obtaining existing model information of the target video and / or picture model; comparing the model requirement information and the existing model information to determine model expansion information; obtaining one or more preset adjustment logic of adjustment types; matching the model expansion information and the one or more preset adjustment logic of adjustment types, and taking a matched adjustment type as the target adjustment type.

2. The method of claim 1, wherein, The target adjustment type comprises a mode adjustment type, and the method comprises: obtaining a mode library of the target video and / or picture model, wherein the mode library comprises a plurality of mode entries; obtaining weights of each processing layer of the target video and / or picture model; taking the mode library and the weights of each processing layer as the existing capability data.

3. The method of claim 2, wherein, The processing layer comprises a fixed processing layer and a non-fixed processing layer. The method comprises: initializing the weights of the non-fixed processing layer; taking the initialized weights as the target weight of the video and / or picture processing layer that needs to be adjusted in the target video and / or picture model.

4. The method of claim 3, wherein, The method comprises: determining old mode feature information of the target video and / or picture model based on the mode entries in the mode library; inputting the target video and / or picture training data into the target video and / or picture model, and using the fixed processing layer to extract features of the target video and / or picture training data in the target video and / or picture model to obtain new feature information. Adopting the old mode feature information and the new feature information to adjust the target video and / or picture model.

5. The method of claim 4, wherein, The old mode feature information of the target video and / or picture model is determined based on the mode entries in the mode library, including: Adopting a random vector generator to generate one or more perturbation vectors corresponding to each mode entry, wherein the random vector generator is a random vector generator simulating a normal distribution of multi-modal; Taking the mode entry as the center, respectively adopting the one or more perturbation vectors to perturb the mode entry to obtain the old mode feature information corresponding to the mode entry.

6. The method of claim 4, wherein, The adjustment of the target video and / or picture model includes: Adopting the old mode feature information to update the target weight; Adopting the new feature information to update the weight of all processing layers.

7. The method of claim 4, wherein, The method further includes: When adjusting the target video and / or picture model, adopting an L2 regularization term to expand the loss function, wherein the L2 regularization term is a regularization term combining Laplace estimation and Fisher encoding.

8. The method of claim 2, wherein, The method further includes: After the adjustment of the target video and / or picture model is completed, adopting the adjusted target video and / or picture model to extract feature information from newly generated training data; Clustering all obtained feature information to obtain a cluster center; Adding the cluster center to the mode library as a mode entry.

9. The method of claim 1, wherein, The target adjustment type includes a task range adjustment type, and the existing capability data of the target video and / or picture model is determined according to the target adjustment type, including: Obtaining the weight of each processing layer of the target video and / or picture model; Obtaining the weight of each existing output branch of the target video and / or picture model; Taking the weight of each processing layer and the weight of each existing output branch as the existing capability data.

10. The method of claim 9, wherein, The processing layer includes a high-level processing layer; The target weight of the video and / or picture processing layer that needs to be adjusted in the target video and / or picture model is determined according to the target adjustment type, including: Generating a corresponding range expansion branch for the model expansion information; Initializing the weight of the range expansion branch; Taking the initialized weight of the range expansion branch and the weight of the high-level processing layer as the target weight.

11. The method of claim 10, wherein, The processing layer further includes a low-level processing layer and a middle-level processing layer; The adjustment of the target video and / or picture model based on the existing capability data and the target video and / or picture training data includes: Respectively fixing the weight of the low-level processing layer and the weight of the middle-level processing layer; Adopting the target video and / or picture training data to adjust the target weight.

12. The method of claim 10, wherein, The existing capability data further includes a specified output option of an existing output branch; The adjustment of the target weight using the target video and / or picture training data includes: Obtaining first training data corresponding to the specified output option of the existing output branch; Judging whether the range expansion branch contains the specified output option; If yes, second training data of a specified output option of the range expansion branch is acquired, and weights of the range expansion branch and the existing output branch are adjusted in a self-supervised manner based on the first training data and the second training data; If no, weights of the range expansion branch and the existing output branch are adjusted in a self-supervised manner based on the first training data.

13. The method of claim 1, wherein, The target adjustment type includes a task function adjustment type, and the existing capability data of the target video and / or picture model is determined according to the target adjustment type, including: Weights of each existing function branch of the target video and / or picture model are acquired; The weights of each existing function branch are taken as the existing capability data.

14. The method of claim 13, wherein, The target weight of the video and / or picture processing layer that needs to be adjusted in the target video and / or picture model is determined according to the target adjustment type, including: A corresponding function expansion branch is generated for the model expansion information; Weights of the function expansion branch are initialized; A mapping connection layer is constructed between the function expansion branch and each existing function branch, the mapping connection layer is used to map the feature information of the existing function branch, and a shared weight is determined from the feature information of the existing function branch, the shared weight is used as the existing capability data; The initialized weights of the expansion branch and the weights of the mapping connection layer are taken as the target weight.

15. A service processing method characterized by comprising: The method includes: Acquiring business data to be processed; Loading the target video and / or picture model obtained by the method according to any one of claims 1-14; Processing the business data by using the target video and / or picture model.

16. An apparatus for video and / or picture model adjustment, the apparatus comprising: The device includes: An adjustment type determination module is configured to determine a target adjustment type of a target video and / or picture model to be adjusted; wherein the adjustment type includes a mode adjustment type, a task range adjustment type and a task function adjustment type; An existing capability data determination module is configured to determine existing capability data of the target video and / or picture model according to the target adjustment type; wherein the existing capability data refers to data of the target video and / or picture model that is already available and associated with the target adjustment type; A target weight determination module is configured to determine a target weight of a video and / or picture processing layer that needs to be adjusted in the target video and / or picture model according to the target adjustment type; A target training data acquisition module is configured to acquire target video and / or picture training data corresponding to the target weight; A model adjustment module is configured to adjust the target video and / or picture model based on the existing capability data and the target video and / or picture training data, and the adjustment includes adjustment of the target weight; The adjustment type determination module is specifically configured to: Acquire model requirement information input by a user for the target video and / or picture model; the model requirement information includes a business function that the user expects the target video and / or picture model to be able to implement, or a task type that the user expects the target video and / or picture model to be able to perform; Acquire existing model information of the target video and / or picture model; comparing the model requirement information with the existing model information to determine model extension information; obtaining one or more preset adjustment types of adjustment logic; matching the model extension information with the one or more preset adjustment types of adjustment logic, and taking the matched adjustment type as the target adjustment type.

17. A service processing apparatus, characterized by comprising: The apparatus comprises: a business data obtaining module, configured to obtain business data to be processed; a target model loading module, configured to load a target video and / or picture model obtained according to the method of any one of claims 1-14; a business processing module, configured to process the business data by using the target video and / or picture model.

18. A service processing device, characterized by The business processing device comprises: one or more processors; a storage device, configured to store one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the method of any one of claims 1-15.

19. A computer readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the method of any one of claims 1-15.

20. A computer program product comprising computer executable instructions for implementing the method of any one of claims 1-15 when executed.

Images