A data processing method, device and equipment
By combining features from the original and target datasets to train the model, the problem of model retraining caused by business changes was solved, reducing the workload of calibration and improving model performance and security.
Patent Information
- Application Number
- CN202110587053.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-05-27
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2041-05-27
AI Technical Summary
When machine learning models are applied to real-world business applications, changes in business operations can lead to changes in the target category, requiring the model to be retrained. This results in a large workload and high resource consumption for relabeling a large number of training images.
By inputting the training data from the original dataset into the first sub-network of the original trained model and the training data from the target dataset into the first sub-network of the target model to be trained, and combining the features of both, the target trained model is obtained, thus avoiding the need to recalibrate all training images in the original dataset.
It reduces the workload of recalibration, saves calibration resources, improves the performance and security of the target trained model, and avoids the leakage of sensitive data.
Smart Images

Figure CN113240035B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of artificial intelligence technology, and in particular to a data processing method, device and equipment. Background Art
[0002] Machine learning is a way to achieve artificial intelligence. It is a multi-disciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. Machine learning is used to study how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning pays more attention to algorithm design, enabling computers to automatically learn patterns from data and use patterns to predict unknown data. Machine learning has been widely used, such as deep learning, data mining, computer vision, natural language processing, biometric recognition, search engines, medical diagnosis, speech recognition and handwriting recognition.
[0003] In order to implement artificial intelligence processing by machine learning, a data set can be constructed, a machine learning model can be trained based on the data set, and artificial intelligence processing can be implemented by the machine learning model. The data set includes a large amount of training data, each of which includes a training image and a label category, and the label category is used to indicate the category of the training image. For example, assuming that the machine learning model is used to detect the target category, and the target category is category A or category B, then the label category is used to indicate that the category of the training image is category A or category B.
[0004] When applying machine learning models to actual businesses, changes in the business may cause changes in the target categories detected by the machine learning model. For example, at the beginning of the business, the target category detected by the machine learning model is category A or category B. After the business has been deployed for a period of time, the business needs change, and the target category detected by the machine learning model is category A, category B, or category C.
[0005] Obviously, when the target category detected by the machine learning model changes, the above machine learning model can no longer achieve good performance and needs to be retrained. In order to retrain the machine learning model, it is necessary to recalibrate all the training images in the dataset, that is, recalibrate the label category of each training image, that is, the label category is category A, category B or category C.
[0006] Since there are a large number of training images in the data set, when recalibrating a large number of training images in the data set, the workload of recalibration is very large, which will lead to a large consumption of calibration resources. Summary of the invention
[0007] The present application provides a data processing method, the method comprising:
[0008] Input the training data in the original data set to the first original sub-network of the original trained model to obtain the first original feature; wherein the original trained model also includes a second original sub-network, the first original sub-network includes the first M network layers in the original trained model, and the second original sub-network includes the remaining network layers in the original trained model except the first M network layers;
[0009] Input the training data in the target data set to the first target sub-network of the target model to be trained to obtain a first target feature; wherein the target model to be trained also includes a second target sub-network, the first target sub-network includes the first M network layers in the target model to be trained, and the second target sub-network includes the remaining network layers in the target model to be trained except the first M network layers;
[0010] Inputting the first original feature and the first target feature into a second target sub-network of the target to-be-trained model, and having the second target sub-network output a second target feature;
[0011] The first target sub-network is trained based on the first target feature, the second target sub-network is trained based on the second target feature, and a target trained model is obtained based on the trained first target sub-network and the trained second target sub-network; wherein the target trained model is used to implement data processing.
[0012] In a possible implementation, the number of original trained models is N, the N original trained models correspond one-to-one to the N original data sets, and the first original features input to the second target subnetwork are the N first original features corresponding to the N original trained models;
[0013] For each original trained model, the training process of the original trained model includes:
[0014] The training data in the original data set corresponding to the original trained model is input into the original model to be trained to obtain a second original feature; the original model to be trained is trained based on the second original feature to obtain an original trained model; wherein the original trained model is used to realize data processing.
[0015] In a possible implementation, the training data in the original data set includes original training images and original label categories corresponding to the original training images; the training data in the target data set includes target training images and target label categories corresponding to the target training images;
[0016] The target category set corresponding to the target data set is the union of N original category sets corresponding to N original data sets, and the N original data sets correspond to the N original category sets one by one;
[0017] The target category set includes all target label categories in the target data set;
[0018] The original category set includes all original label categories in the original data set.
[0019] In a possible implementation, the training of the first target sub-network based on the first target feature, the training of the second target sub-network based on the second target feature, and the acquisition of the target trained model based on the trained first target sub-network and the trained second target sub-network include:
[0020] Determine whether the first target subnetwork has converged; if not, adjust the parameters of the first target subnetwork based on the first target feature to obtain an adjusted subnetwork, determine the adjusted subnetwork as the first target subnetwork, and return to execute the operation of inputting the training data in the target data set to the first target subnetwork; if yes, determine the first target subnetwork as the first converged target subnetwork;
[0021] Determine whether the second target subnetwork has converged; if not, adjust the parameters of the second target subnetwork based on the second target feature to obtain an adjusted subnetwork, determine the adjusted subnetwork as the second target subnetwork, and return to perform the operation of inputting the first original feature and the first target feature into the second target subnetwork; if yes, determine the second target subnetwork as the second converged target subnetwork;
[0022] A target trained model is obtained based on the first converged target sub-network and the second converged target sub-network.
[0023] In a possible implementation, determining whether the first target subnetwork has converged and determining whether the second target subnetwork has converged includes: if the first target subnetwork satisfies the convergence condition and the second target subnetwork satisfies the convergence condition, determining that the first target subnetwork has converged and determining that the second target subnetwork has converged; or, if the first target subnetwork satisfies the convergence condition and the second target subnetwork does not satisfy the convergence condition, determining that the first target subnetwork has converged and determining that the second target subnetwork has converged; or, if the first target subnetwork does not satisfy the convergence condition and the second target subnetwork satisfies the convergence condition, determining that the first target subnetwork has converged and determining that the second target subnetwork has converged.
[0024] In a possible implementation manner, after acquiring the target trained model based on the trained first target sub-network and the trained second target sub-network, the method further includes:
[0025] If the target trained model is used to realize target detection, after obtaining the data to be processed, the data to be processed is input into the target trained model, and the target trained model performs detection processing on the data to be processed to obtain a detection result, and performs target detection based on the detection result;
[0026] If the target trained model is used to achieve target classification, then after obtaining the data to be processed, the data to be processed is input into the target trained model, and the target trained model classifies the data to be processed to obtain a classification result, and the target classification is performed based on the classification result.
[0027] In a possible implementation, the network structure of the first M network layers in the original trained model is the same as the network structure of the first M network layers in the target trained model;
[0028] If there are N original trained models, where N is a positive integer greater than 1, the network structures of the first M network layers in the N original trained models are the same.
[0029] The present application proposes a data processing device, the device comprising:
[0030] An acquisition module is used to input the training data in the original data set to the first original sub-network of the original trained model to obtain the first original feature; wherein the original trained model also includes a second original sub-network, the first original sub-network includes the first M network layers in the original trained model, and the second original sub-network includes the remaining network layers in the original trained model except the first M network layers;
[0031] And, inputting the training data in the target data set into the first target sub-network of the target model to be trained to obtain the first target feature; wherein the target model to be trained also includes a second target sub-network, the first target sub-network includes the first M network layers in the target model to be trained, and the second target sub-network includes the remaining network layers in the target model to be trained except the first M network layers;
[0032] And, inputting the first original feature and the first target feature into a second target sub-network of the target model to be trained, and the second target sub-network outputs a second target feature;
[0033] A training module, used to train the first target sub-network based on the first target feature, train the second target sub-network based on the second target feature, and obtain a target trained model based on the trained first target sub-network and the trained second target sub-network;
[0034] The target trained model is used to implement data processing.
[0035] In a possible implementation, the training module is specifically used for:
[0036] Determine whether the first target subnetwork has converged; if not, adjust the parameters of the first target subnetwork based on the first target feature to obtain an adjusted subnetwork, determine the adjusted subnetwork as the first target subnetwork, and return to execute the operation of inputting the training data in the target data set to the first target subnetwork; if yes, determine the first target subnetwork as the first converged target subnetwork;
[0037] Determine whether the second target subnetwork has converged; if not, adjust the parameters of the second target subnetwork based on the second target feature to obtain an adjusted subnetwork, determine the adjusted subnetwork as the second target subnetwork, and return to perform the operation of inputting the first original feature and the first target feature into the second target subnetwork; if yes, determine the second target subnetwork as the second converged target subnetwork;
[0038] A target trained model is obtained based on the first converged target sub-network and the second converged target sub-network.
[0039] The present application proposes a data processing device, comprising: a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions that can be executed by the processor;
[0040] The processor is used to execute machine executable instructions to implement the following steps:
[0041] Input the training data in the original data set to the first original sub-network of the original trained model to obtain the first original feature; wherein the original trained model also includes a second original sub-network, the first original sub-network includes the first M network layers in the original trained model, and the second original sub-network includes the remaining network layers in the original trained model except the first M network layers;
[0042] Input the training data in the target data set to the first target sub-network of the target model to be trained to obtain a first target feature; wherein the target model to be trained also includes a second target sub-network, the first target sub-network includes the first M network layers in the target model to be trained, and the second target sub-network includes the remaining network layers in the target model to be trained except the first M network layers;
[0043] Inputting the first original feature and the first target feature into a second target sub-network of the target to-be-trained model, and having the second target sub-network output a second target feature;
[0044] The first target sub-network is trained based on the first target feature, the second target sub-network is trained based on the second target feature, and a target trained model is obtained based on the trained first target sub-network and the trained second target sub-network; wherein the target trained model is used to implement data processing.
[0045] It can be seen from the above technical solutions that in the embodiment of the present application, the original trained model can be obtained based on the original data set training, and artificial intelligence processing is implemented through the original trained model to detect the target category. When the original trained model is applied to the actual business, if the business change causes the target category detected by the original trained model to change, the target trained model is obtained based on the target data set training, and artificial intelligence processing is implemented through the target trained model to detect the changed target category. Since the target data set is different from the original data set, it is not necessary to recalibrate all the training images in the original data set, that is, it is not necessary to recalibrate the label category of each training image, which reduces the workload of recalibration, saves calibration resources, reduces the resource consumption investment of recalibration, and integrates the target data set for joint training without supplementing the label category of the original data set. The training data in the target data set can be less than the training data in the original data set, that is, only a small amount of training data is needed to train the target trained model. The original features corresponding to the training data in the original data set can be involved in the training process of the target trained model to improve the performance of the target trained model, and the accuracy of the intelligent analysis results of the target trained model is high. Since it is the original features that participate in the training process of the target trained model, rather than the training data in the original data set, sensitive data is avoided from being involved in the training process of the target trained model, the confidential information of the training data is hidden, and security is improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] Figure 1 It is a flowchart of a data processing method in one embodiment of the present application;
[0047] Figure 2It is a schematic diagram of the training process in one embodiment of the present application;
[0048] Figure 3 is a schematic diagram of the training process of the original trained model in one embodiment of the present application;
[0049] Figure 4 is a schematic diagram of the training process of a target trained model in one embodiment of the present application;
[0050] Figure 5 It is a structural schematic diagram of a data processing device in one embodiment of the present application. DETAILED DESCRIPTION
[0051] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, rather than limiting the present application. The singular forms of "a", "said" and "the" used in the present application and claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to any or all possible combinations of one or more associated listed items.
[0052] It should be understood that, although the terms first, second, third, etc. may be used to describe various information in the embodiments of the present application, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, in addition, the word "if" used may be interpreted as "at..." or "when..." or "in response to determination".
[0053] Before introducing the technical solution of the present application, concepts related to the embodiments of the present application are first introduced.
[0054] Machine learning: Machine learning is a way to achieve artificial intelligence. It is used to study how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Deep learning is a subcategory of machine learning. It is a process of using mathematical models to model specific problems in the real world to solve similar problems in the field.
[0055] The models in the embodiments of the present application all refer to machine learning models, such as deep learning models in machine learning models. Of course, the deep learning model is just an example and there is no limitation on this.
[0056] Taking the model of the embodiment of the present application as a deep learning model, an example of a deep learning model is a neural network model. In subsequent embodiments, the structure and function of the neural network model are introduced by taking the neural network model as an example. Other subclasses of the deep learning model are similar to the structure and function of the neural network model.
[0057] Exemplarily, the model in the embodiment of the present application can be a detection model, that is, used to achieve target detection, or a classification model, that is, used to achieve target classification, and there is no restriction on the type of this model. For the detection model, it can be a deep learning model for achieving target detection, such as a detection model for detecting the location of a specified category of target in an image, such as yolov2, FRCNN, etc. For the classification model, it can be a deep learning model for achieving target classification, such as a detection model for detecting the type of a specified target in an image.
[0058] Neural network model: Neural network models include but are not limited to convolutional neural network models, recurrent neural network models, fully connected network models, etc. The structural units of the neural network model may include but are not limited to convolutional layers (Conv), pooling layers (Pool), excitation layers, fully connected layers (FC), etc., and there is no restriction on this. In practical applications, one or more convolutional layers, one or more pooling layers, one or more excitation layers, and one or more fully connected layers can be combined to construct a neural network model according to different requirements.
[0059] In the convolution layer, the input data features are enhanced by using a convolution kernel to perform a convolution operation on the input data features. The convolution kernel can be a matrix of size m*n. The input data features of the convolution layer are convolved with the convolution kernel to obtain the output data features of the convolution layer. The convolution operation is actually a filtering process.
[0060] In the pooling layer, by taking the maximum value, minimum value, and average value of the input data features (such as the output of the convolutional layer), the principle of local correlation is used to sub-sample the input data features, reduce the processing amount, and maintain feature invariance. The pooling layer operation is actually a downsampling process.
[0061] In the excitation layer, an activation function (such as a nonlinear function) can be used to map the input data features, thereby introducing nonlinear factors so that the neural network can enhance its expression ability through nonlinear combinations. The activation function may include but is not limited to a ReLU (Rectified Linear Units) function, which is used to set features less than 0 to 0, while features greater than 0 remain unchanged.
[0062] In the fully connected layer, all data features input to this fully connected layer are fully connected to obtain a feature vector, which may include multiple data features.
[0063] In order to realize artificial intelligence processing, an original data set can be constructed, an original trained model can be trained based on the original data set, and artificial intelligence processing can be realized by using the original trained model. The original data set includes a large amount of training data, each of which includes a training image and a label category, and the label category is used to indicate the category of the training image. For example, if the original trained model is used to detect the target category, and the target category is category A or category B, then the label category is used to indicate that the training image is category A or category B.
[0064] When the original trained model is applied to actual business, the target category detected by the original trained model may change due to business changes. For example, after the business requirements change, the target category detected by the original trained model is category A, category B, or category C. Obviously, the above original trained model cannot detect category C, that is, it is impossible to obtain accurate detection results, and the model needs to be retrained. In order to retrain the model, it is necessary to recalibrate all training images in the original data set, that is, recalibrate the label category of each training image, that is, the label category is category A, category B, or category C. Obviously, when recalibrating a large number of training images in the original data set, the workload of recalibration is very large.
[0065] In view of the above findings, an embodiment of the present application proposes a data processing method, in which an original trained model is obtained based on the original data set training, and a target trained model is obtained based on the target data set training. Since the target data set is different from the original data set, there is no need to recalibrate all the training images in the original data set, that is, there is no need to recalibrate the label category of each training image, thereby reducing the recalibration workload. Without performing label category supplementary calibration on the original data set, the target data set is fused for joint training.
[0066] The technical solution of the embodiments of the present application is described below in conjunction with specific embodiments.
[0067] See also Figure 1 FIG. 1 is a flow chart of a data processing method, which may include:
[0068] Step 101, input the training data in the original data set to the first original sub-network of the original trained model to obtain the first original feature. Exemplarily, the original trained model may also include a second original sub-network, the first original sub-network includes the first M network layers in the original trained model, and the second original sub-network includes the remaining network layers in the original trained model except the first M network layers.
[0069] Exemplarily, the number of original trained models may be N, where N is a positive integer, and the N original trained models correspond one to one with the N original data sets. Before step 101, the original model to be trained may be trained based on the training data in the original data set to obtain the original trained model. For example, original model to be trained 1 is trained based on the training data in original data set 1 to obtain original trained model 1 corresponding to original model to be trained 1; original model to be trained 2 is trained based on the training data in original data set 2 to obtain original trained model 2 corresponding to original model to be trained 2, and so on.
[0070] Exemplarily, for each original trained model, the training process of the original trained model may include: inputting the training data in the original data set corresponding to the original trained model into the original model to be trained to obtain a second original feature; training the original model to be trained based on the second original feature to obtain an original trained model, and the original trained model is used to implement data processing.
[0071] In the process of training the original model to be trained based on the second original feature to obtain the original trained model, it can be determined whether the original model to be trained has converged; if not, the parameters of the original model to be trained are adjusted based on the second original feature to obtain the adjusted model, and the adjusted model is determined as the original model to be trained, and the operation of inputting the training data in the original data set to the original model to be trained is returned; if so, the original model to be trained is determined as the original trained model.
[0072] Step 102, input the training data in the target data set to the first target sub-network of the target model to be trained to obtain the first target feature. Exemplarily, the target model to be trained may also include a second target sub-network, the first target sub-network includes the first M network layers in the target model to be trained, and the second target sub-network includes the remaining network layers in the target model to be trained except the first M network layers.
[0073] Exemplarily, the training data in the original data set may include the original training image and the original label category corresponding to the original training image, and the training data in the target data set may include the target training image and the target label category corresponding to the target training image. The target category set corresponding to the target data set may be the union of N original category sets corresponding to N original data sets, and the N original data sets correspond to the N original category sets one-to-one. The target category set may include all target label categories in the target data set, and the original category set may include all original label categories in the original data set.
[0074] For example, suppose that the N original data sets are original data set 1 and original data set 2, original data set 1 corresponds to original category set 1, and original category set 1 may include all original label categories of original data set 1, such as category A and category B, and original data set 2 corresponds to original category set 2, and original category set 2 may include all original label categories of original data set 2, such as category A and category C.
[0075] Based on this, the target data set corresponds to a target category set, which is the union of the original category set 1 and the original category set 2, that is, the target category set includes category A, category B and category C. In other words, all target label categories of the target data set can be category A, category B and category C.
[0076] Step 103, the first original feature and the first target feature are input into the second target sub-network of the target model to be trained, and the second target sub-network outputs the second target feature. That is to say, the first original feature output by the first original sub-network of the original trained model and the first target feature output by the first target sub-network of the target model to be trained are input into the second target sub-network of the target model to be trained, and the second target sub-network processes the first original feature and the first target feature to obtain the second target feature.
[0077] Exemplarily, since the number of original trained models may be N, the first original features input to the second target sub-network are N first original features corresponding to the N original trained models.
[0078] Step 104: train the first target sub-network based on the first target feature, train the second target sub-network based on the second target feature, and obtain a target trained model based on the trained first target sub-network and the trained second target sub-network, and the target trained model is used to implement data processing.
[0079] Exemplarily, it is possible to determine whether the first target subnetwork has converged; if not, the parameters of the first target subnetwork are adjusted based on the first target feature to obtain an adjusted subnetwork, the adjusted subnetwork is determined as the first target subnetwork, and the operation of inputting the training data in the target data set to the first target subnetwork is returned to be executed; if so, the first target subnetwork is determined as the first converged target subnetwork. Determine whether the second target subnetwork has converged; if not, the parameters of the second target subnetwork are adjusted based on the second target feature to obtain an adjusted subnetwork, the adjusted subnetwork is determined as the second target subnetwork, and the operation of inputting the first original feature and the first target feature to the second target subnetwork is returned to be executed; if so, the second target subnetwork is determined as the second converged target subnetwork. On this basis, the target trained model can be obtained based on the first converged target subnetwork and the second converged target subnetwork, that is, the target trained model includes the first converged target subnetwork and the second converged target subnetwork, the first converged target subnetwork includes the first M network layers in the target trained model, and the second converged target subnetwork includes the remaining network layers in the target trained model except the first M network layers.
[0080] In the above embodiment, if the first target subnetwork meets the convergence condition and the second target subnetwork meets the convergence condition, it can be determined that the first target subnetwork has converged and the second target subnetwork has converged. Alternatively, if the first target subnetwork meets the convergence condition and the second target subnetwork does not meet the convergence condition, it can be determined that the first target subnetwork has converged and the second target subnetwork has converged; or, if the first target subnetwork does not meet the convergence condition and the second target subnetwork meets the convergence condition, it can be determined that the first target subnetwork has converged and the second target subnetwork has converged.
[0081] In a possible implementation, after obtaining a target trained model based on the trained first target sub-network and the trained second target sub-network, artificial intelligence processing can also be implemented based on the target trained model. For example, if the target trained model is used to implement target detection, then after obtaining the data to be processed, the data to be processed can be input into the target trained model, and the target trained model performs detection processing on the data to be processed to obtain a detection result, and target detection is performed based on the detection result. Alternatively, if the target trained model is used to implement target classification, then after obtaining the data to be processed, the data to be processed can be input into the target trained model, and the target trained model performs classification processing on the data to be processed to obtain a classification result, and target classification is performed based on the classification result.
[0082] In a possible implementation, the network structure of the first M network layers in the original trained model is the same as the network structure of the first M network layers in the target trained model. If there are N original trained models, N can be a positive integer greater than 1, then the network structures of the first M network layers in the N original trained models are the same, that is, the network structures of the first M network layers in the N original trained models are the same as the network structures of the first M network layers in the target trained model, and M can be a positive integer.
[0083] It can be seen from the above technical solutions that in the embodiment of the present application, the original trained model can be obtained based on the original data set training, and artificial intelligence processing is implemented through the original trained model to detect the target category. When the original trained model is applied to the actual business, if the business change causes the target category detected by the original trained model to change, the target trained model is obtained based on the target data set training, and artificial intelligence processing is implemented through the target trained model to detect the changed target category. Since the target data set is different from the original data set, it is not necessary to recalibrate all the training images in the original data set, that is, it is not necessary to recalibrate the label category of each training image, which reduces the workload of recalibration, saves calibration resources, reduces the resource consumption investment of recalibration, and integrates the target data set for joint training without supplementing the label category of the original data set. The training data in the target data set can be less than the training data in the original data set, that is, only a small amount of training data is needed to train the target trained model. The original features corresponding to the training data in the original data set can be involved in the training process of the target trained model to improve the performance of the target trained model, and the accuracy of the intelligent analysis results of the target trained model is high. Since it is the original features that participate in the training process of the target trained model, rather than the training data in the original data set, sensitive data is avoided from being involved in the training process of the target trained model, the confidential information of the training data is hidden, and security is improved.
[0084] The data processing method of the embodiment of the present application is described below in conjunction with specific application scenarios.
[0085] In order to realize artificial intelligence processing, an original data set can be constructed, an original trained model can be trained based on the original data set, and artificial intelligence processing can be realized by using the original trained model. When the original trained model is applied to actual business, the target categories detected by the original trained model may change due to business changes. For example, when the battery vehicle entering the elevator business just started, the original trained model needed to detect two target categories of battery vehicles and bicycles. As the accuracy requirements increased, it was necessary to add head and shoulder targets, that is, it was necessary to detect three target categories such as battery vehicles, bicycles and head and shoulders. For another example, the original trained model 1 needs to detect three target categories such as human body, dog, and cat, and the original trained model 2 needs to detect two target categories such as dog and pig. A new model is needed to detect four target categories such as human body, cat, dog and pig at the same time.
[0086] When the above situation occurs, the training data in the original data set can be recalibrated, that is, the label category of each training image in the original data set is recalibrated. The workload of recalibration is very large.
[0087] The above problem can be abstracted as follows: there is at least one original data set with independent calibration, and there may or may not be overlap in target categories between the original data sets, and each original data set has an independent original trained model, which can detect the calibrated targets under the original data set with high performance. On this basis, it is necessary to train a new model, namely the target trained model, which can detect the union of target categories of all original data sets. The training process of this target trained model cannot recalibrate the original data set. On this basis, a joint training method without recalibrating the original data set is proposed in an embodiment of the present application, so as to realize the fusion of target data sets for joint training without recalibrating the label categories of the original data sets.
[0088] See also Figure 2 As shown, it is a schematic diagram of the training process of an embodiment of the present application, and N original data sets can be constructed. These N original data sets are recorded as original data sets A1-original data sets An, and the original data sets include training images and label categories corresponding to the training images. For the convenience of distinction, the training images in the original data sets are recorded as original training images, and the label categories in the original data sets are recorded as original label categories. The original data set A1 corresponds to the original category set A11, and the original category set A11 includes all the original label categories of the original data set A1. The original data set A2 corresponds to the original category set A21, and the original category set A21 includes all the original label categories of the original data set A2. By analogy, the original data set An corresponds to the original category set An1, and the original category set An1 includes all the original label categories of the original data set An.
[0089] The original trained model A12 can be obtained based on the training data in the original data set A1, the original trained model A22 can be obtained based on the training data in the original data set A2, and so on, the original trained model An2 can be obtained based on the training data in the original data set An.
[0090] On this basis, if a new model needs to be trained, a target dataset B can be constructed, which includes training images and label categories corresponding to the training images. For the convenience of distinction, the training images in the target dataset B can be recorded as target training images, and the label categories in the target dataset B can be recorded as target label categories. The target dataset B corresponds to the target category set B1, and the target category set B1 can include all target label categories of the target dataset B. Among them, for the target category set B1, the target label categories in the target category set B1 can be the union of the original label categories in all the original category sets, for example, B1 = A11∪A21∪A31∪…∪An1.
[0091] The target trained model B2 can be obtained by training based on the training data in the target data set B. When the target trained model B2 is obtained by training based on the training data in the target data set B, it is also necessary to refer to the data features output by the original trained model A12, the data features output by the original trained model A22, ..., the data features output by the original trained model An2. In other words, the target trained model B2 is obtained by training based on the training data in the target data set B and the data features output by each original trained model.
[0092] 1. Train the original model to be trained based on the original data set to obtain the original trained model.
[0093] Exemplarily, the original model that has not completed training can be recorded as the original model to be trained, and the original model that has completed training can be recorded as the original trained model. Based on this, an original model to be trained can be constructed, and the structure and function of the original model to be trained are not restricted, and a data set for training the original model to be trained is constructed, and the data set is called the original data set. On this basis, the original model to be trained can be trained based on the original data set to obtain the original trained model.
[0094] See also Figure 3As shown, the original data set may include multiple training data. For each training data, the training data may include an original training image and an original label category, and the original label category is used to indicate the category of the original training image. Of course, the training data may also include other types of calibration information, such as the position of a target box, etc., and there is no limitation on this training data. The position of the target box may be the coordinate information of a rectangular box in the original training image (such as the coordinates of the upper left corner of the rectangular box, the width and height of the rectangular box, etc.).
[0095] Multiple training data in the original data set can be input into the original model to be trained, so as to use these training data to train the parameters of the original model to be trained (such as convolution layer parameters, pooling layer parameters, excitation layer parameters, full connection layer parameters, etc.), and there is no restriction on this training process. After the training of the original model to be trained is completed, the original model to be trained that has completed the training is the original trained model.
[0096] In a possible implementation, the following steps may be used to train and obtain the original trained model:
[0097] Step S11, input the training data in the original data set to the original model to be trained to obtain the second original feature. For example, multiple training data in the original data set (such as original training images and original label categories) are input to the original model to be trained, and the original model to be trained processes the training data. There is no restriction on this processing process, and the data feature after being processed by the original model to be trained is the second original feature.
[0098] For example, the original model to be trained includes multiple network layers, such as network layer 1, network layer 2, ..., network layer M, network layer M+1, ... network layer K, K is greater than M, that is, there are K network layers in total, and network layer K is the last network layer of the original model to be trained. Based on this, the training data can be input to network layer 1 of the original model to be trained, and network layer 1 processes the training data to obtain data feature 1, and data feature 1 is input to network layer 2, and network layer 2 processes data feature 1 to obtain data feature 2, and data feature 2 is input to network layer 3, ..., network layer M processes data feature M-1 to obtain data feature M, and data feature M is input to network layer M+1, ..., network layer K processes data feature K-1 to obtain data feature K, and data feature K is the second original feature.
[0099] Step S12: Determine whether the original model to be trained has converged.
[0100] If not, execute step S13, and if so, execute step S14.
[0101] For example, after obtaining the second original feature, the loss value can be determined based on the second original feature. For example, a target loss function is pre-configured, the input of the target loss function is the data feature output by the network layer K, and the output of the target loss function is the loss value. There is no restriction on this target loss function as long as it can satisfy the above input-output relationship. Based on this, after obtaining the second original feature, the second original feature can be substituted into the target loss function to obtain the loss value corresponding to the second original feature.
[0102] After obtaining the loss value corresponding to the second original feature, it can be determined whether the original model to be trained has converged based on the loss value. For example, if the loss value is less than a preset threshold, it is determined that the original model to be trained has converged, and if the loss value is not less than the preset threshold, it is determined that the original model to be trained has not converged.
[0103] Of course, other methods can also be used to determine whether the original model to be trained has converged, and there is no limitation on this determination method. For example, if the number of iterations of the original model to be trained (a training process of the original model to be trained using all the training data in the original data set is called an iteration) reaches a preset number threshold, it can be determined that the original model to be trained has converged, otherwise, it can be determined that the original model to be trained has not converged. For another example, if the iteration duration of the original model to be trained reaches a preset duration threshold, it can be determined that the original model to be trained has converged, otherwise, it can be determined that the original model to be trained has not converged.
[0104] Step S13, adjusting the parameters of the original model to be trained based on the second original feature to obtain an adjusted model, and determining the adjusted model as the original model to be trained, returning to execute the operation of inputting the training data in the original data set to the original model to be trained, that is, returning to execute step S11.
[0105] For example, the second original feature can be substituted into the target loss function to obtain the loss value corresponding to the second original feature, and the parameters of the original model to be trained (i.e., network parameters) can be adjusted based on the loss value to obtain the adjusted model. For example, based on the loss value corresponding to the second original feature, the network parameters (i.e., network weights) of the original model to be trained can be updated by the back propagation algorithm to obtain the updated original model to be trained, i.e., the adjusted model, and there is no restriction on this updating process. An example of the back propagation algorithm can be the gradient descent method, i.e., updating the network parameters of the original model to be trained by the gradient descent method.
[0106] After obtaining the adjusted model, the adjusted model can be determined as the original model to be trained, and the process returns to step S11 to retrain the original model to be trained based on the training data in the original data set.
[0107] Step S14: Determine the converged original to-be-trained model as the original trained model.
[0108] After obtaining the original trained model (i.e., the trained model), data processing, i.e., artificial intelligence processing, can be implemented through the original trained model. For example, if the original trained model is used to implement target detection, after obtaining the data to be processed, the data to be processed can be input into the original trained model, and the original trained model performs detection processing on the data to be processed to obtain the detection result, and target detection is performed based on the detection result. Alternatively, if the original trained model is used to implement target classification, after obtaining the data to be processed, the data to be processed can be input into the original trained model, and the original trained model performs classification processing on the data to be processed to obtain the classification result, and target classification is performed based on the classification result.
[0109] In one possible implementation, the number of original models to be trained is N, the number of original data sets is N, and the number of original trained models is N, where N is a positive integer. Based on this, the N original trained models correspond one-to-one to the N original data sets, the N original trained models correspond one-to-one to the N original models to be trained, and the N original models to be trained correspond one-to-one to the N original data sets.
[0110] For example, the original data set A1 corresponds to the original category set A11, and the original model to be trained A13 can be trained based on the training data in the original data set A1 to obtain the original trained model A12. The original data set A2 corresponds to the original category set A21, and the original model to be trained A23 can be trained based on the training data in the original data set A2 to obtain the original trained model A22. Similarly, the original data set An corresponds to the original category set An1, and the original model to be trained An3 can be trained based on the training data in the original data set An to obtain the original trained model An2.
[0111] 2. Train the target model to be trained based on the target data set to obtain the target trained model.
[0112] Exemplarily, the target model that has not completed training can be recorded as the target model to be trained, and the target model that has completed training can be recorded as the target trained model. Based on this, a target model to be trained can be constructed, and the structure and function of the target model to be trained are not restricted, and a data set for training the target model to be trained is constructed, and the data set is called the target data set. On this basis, the target model to be trained can be trained based on the target data set to obtain a target trained model.
[0113] See also Figure 4As shown, the target data set B includes multiple training data. For each training data, the training data includes a target training image and a target label category. The target label category is used to represent the category of the target training image. Of course, the training data can also include other types of calibration information, such as the target box position, etc., which is not limited. The target data set B corresponds to the target category set B1. The target category set B1 includes all target label categories of the target data set B, and the target label categories in the target category set B1 can be the union of the original label categories in all original category sets. For example, B1 = A11∪A21∪A31∪…∪An1, so that the original label categories in all original category sets are fully calibrated.
[0114] See also Figure 4 As shown, multiple training data in the original data set A1 can be input into the original trained model A12 to obtain the first original feature A14, and the first original feature A14 is input into the target model to be trained. Multiple training data in the original data set A2 can be input into the original trained model A22 to obtain the first original feature A24, and the first original feature A24 is input into the target model to be trained, ..., multiple training data in the original data set An can be input into the original trained model An2 to obtain the first original feature An4, and the first original feature An4 is input into the target model to be trained. And, multiple training data in the target data set B can be input into the target model to be trained. In summary, the parameters of the target model to be trained (such as convolution layer parameters, pooling layer parameters, excitation layer parameters, fully connected layer parameters, etc.) can be trained based on the multiple training data and each first original feature in the target data set B. After the training of the target model to be trained is completed, the target model to be trained that has completed the training is the target trained model.
[0115] In a possible implementation, the target trained model can be obtained by training using the following steps:
[0116] Step S21: input the training data in the original data set to the first original sub-network of the original trained model to obtain the first original feature. For example, the first original sub-network processes the training data to obtain data features, and the data features processed by the first original sub-network are recorded as the first original features.
[0117] Exemplarily, assuming that the original trained model includes K network layers (the original model to be trained also includes K network layers, and the structures of the two are the same), the K network layers of the original trained model can be divided into a first original sub-network and a second original sub-network, the first original sub-network includes the first M (M is less than K) network layers, and the second original sub-network includes the remaining network layers except the first M network layers. For example, the original trained model includes network layer 1, network layer 2, ..., network layer M, network layer M+1, ... network layer K, and network layer K is the last network layer of the original trained model. Then, the first original sub-network includes network layer 1, network layer 2, ..., network layer M, that is, network layer M is the last network layer of the first original sub-network, and the second original sub-network includes network layer M+1, ... network layer K, that is, network layer M+1 is the first network layer of the second original sub-network, and network layer K is the last network layer of the second original sub-network.
[0118] Based on this, the training data in the original data set can be input into the network layer 1 of the original trained model. The network layer 1 processes the training data to obtain data feature 1, and inputs data feature 1 into the network layer 2. The network layer 2 processes data feature 1 to obtain data feature 2, and inputs data feature 2 into the network layer 3, ..., the network layer M processes the data feature M-1 to obtain data feature M. Data feature M is the first original feature, and there is no need to input data feature M into the network layer M+1.
[0119] In the above embodiment, the data features output by the first original sub-network (i.e., the data features output by the network layer M) are used as the first original features, rather than the data features output by the second original sub-network (i.e., the data features output by the network layer K). That is, in the training process of the target trained model, the second original sub-network does not participate in the training process, and only the first original sub-network is required to participate in the training process.
[0120] Exemplarily, assuming that there are N original trained models, the training data in the original data set A1 can be input into the first original subnetwork of the original trained model A12 to obtain the first original feature A14, the training data in the original data set A2 can be input into the first original subnetwork of the original trained model A22 to obtain the first original feature A24,…, the training data in the original data set An can be input into the first original subnetwork of the original trained model An2 to obtain the first original feature An4.
[0121] Step S22: input the training data in the target data set to the first target sub-network of the target model to be trained to obtain a first target feature. For example, the first target sub-network processes the training data to obtain a data feature, and the data feature processed by the first target sub-network is recorded as the first target feature.
[0122] Exemplarily, assuming that the target model to be trained includes L network layers (the target trained model also includes L network layers, and the structures of the two are the same), the L network layers of the target model to be trained can be divided into a first target sub-network and a second target sub-network, the first target sub-network includes the first M (M is less than L) network layers, and the second target sub-network includes the remaining network layers except the first M network layers. For example, the target model to be trained includes network layer 1, network layer 2, ..., network layer M, network layer M+1, ... network layer L, and network layer L is the last network layer of the target model to be trained, then the first target sub-network includes network layer 1, network layer 2, ..., network layer M, that is, network layer M is the last network layer of the first target sub-network, and the second target sub-network includes network layer M+1, ... network layer L, that is, network layer M+1 is the first network layer of the second target sub-network, and network layer L is the last network layer of the second target sub-network.
[0123] Exemplarily, the value of L and the value of K may be the same or different.
[0124] Based on this, the training data in the target data set can be input into the network layer 1 of the target model to be trained. The network layer 1 processes the training data to obtain data feature 1, and inputs data feature 1 into the network layer 2. The network layer 2 processes data feature 1 to obtain data feature 2, and inputs data feature 2 into the network layer 3, ..., the network layer M processes the data feature M-1 to obtain data feature M (the output of the last network layer of the first target subnetwork), and the data feature M is the first target feature.
[0125] Step S23: input the first original feature and the first target feature to the second target sub-network of the target model to be trained, and the second target sub-network outputs the second target feature. For example, the first original feature and the first target feature are input to the second target sub-network together, and the second target sub-network processes the first original feature and the first target feature to obtain a data feature, and the processed data feature is the second target feature.
[0126] Exemplarily, assuming that the target model to be trained includes L network layers, the second target sub-network of the target model to be trained includes network layers M+1, ... network layers L. Based on this, the first original feature (such as the first original feature A14, the first original feature A24, ..., the first original feature An4) and the first target feature can be input into network layer M+1 of the target model to be trained, and network layer M+1 processes the first original feature and the first target feature to obtain data feature M+1, and inputs data feature M+1 into network layer M+2, ..., network layer L processes data feature L-1 to obtain data feature L (i.e., the output of the last network layer of the second target sub-network), and data feature L is the second target feature.
[0127] From the above, it can be seen that since the number of original trained models is N, the first original features input to the second target sub-network are the N first original features corresponding to the N original trained models.
[0128] In one possible implementation, the network structures of the first M network layers in the N original trained models are the same. The network structure of the first M network layers in the original trained model is the same as the network structure of the first M network layers in the target model to be trained (i.e., the network structure of the first M network layers in the target trained model). For example, the network structure of the first M network layers of the original trained model A12 is the same as the network structure of the first M network layers of the original trained model A22, the network structure of the first M network layers of the original trained model A12 is the same as the network structure of the first M network layers of the original trained model A32, ..., the network structure of the first M network layers of the original trained model A12 is the same as the network structure of the first M network layers of the original trained model An2. The network structure of the first M network layers of the original trained model A12 is the same as the network structure of the first M network layers of the target model to be trained / target trained model.
[0129] The network structure of the first M network layers of the original trained model A12 is the same as the network structure of the first M network layers of the original trained model A22, which may mean: the network structure of the first network layer (i.e., network layer 1) of the original trained model A12 is the same as the network structure of the first network layer (i.e., network layer 1) of the original trained model A22, the network structure of the second network layer (i.e., network layer 2) of the original trained model A12 is the same as the network structure of the second network layer (i.e., network layer 2) of the original trained model A22,…, the network structure of the Mth network layer (i.e., network layer M) of the original trained model A12 is the same as the network structure of the Mth network layer (i.e., network layer M) of the original trained model A22.
[0130] The network structure of the first M network layers of the original trained model A12 is the same as the network structure of the first M network layers of the target model to be trained / target trained model, which may mean: the network structure of the first network layer of the original trained model A12 is the same as the network structure of the first network layer of the target model to be trained / target trained model,…, the network structure of the Mth network layer of the original trained model A12 is the same as the network structure of the Mth network layer of the target model to be trained / target trained model.
[0131] The network structure of the first network layer of the original trained model A12 (denoted as network layer A121) is the same as the network structure of the first network layer of the original trained model A22 (denoted as network layer A221), which may mean that: network layer A121 and network layer A221 are composed of the same structural units, and the connection relationship of the structural units is the same, but the parameters of the structural units may be the same or different. For example, network layer A121 and network layer A221 are both composed of 2 convolutional layers and 1 pooling layer, and the connection relationship between the 2 convolutional layers and 1 pooling layer in network layer A121 is the same as the connection relationship between the 2 convolutional layers and 1 pooling layer in network layer A221.
[0132] Step S24: Determine whether the first target sub-network of the target model to be trained has converged.
[0133] If not, step S25 may be executed, and if so, step S26 may be executed.
[0134] Step S25: adjust the parameters of the first target sub-network based on the first target feature to obtain an adjusted sub-network, and determine the adjusted sub-network as the first target sub-network, and return to execute the operation of inputting the training data in the target data set to the first target sub-network, that is, return to execute step S22.
[0135] Step S26: determine the first target sub-network as the first converged target sub-network.
[0136] Step S27: Determine whether the second target sub-network of the target model to be trained has converged.
[0137] If not, step S28 may be executed, and if so, step S29 may be executed.
[0138] Step S28: Adjust the parameters of the second target subnetwork based on the second target feature to obtain an adjusted subnetwork, and determine the adjusted subnetwork as the second target subnetwork, and return to execute the operation of inputting the first original feature and the first target feature into the second target subnetwork, that is, return to execute step S23.
[0139] Step S29: determine the second target sub-network as the second converged target sub-network.
[0140] Step S30: After obtaining the first converged target subnetwork and the second converged target subnetwork, a target trained model is obtained based on the first converged target subnetwork and the second converged target subnetwork, wherein the target trained model includes the first converged target subnetwork (the first target subnetwork that has been trained) and the second converged target subnetwork (the second target subnetwork that has been trained), the first converged target subnetwork includes the first M network layers, and the second converged target subnetwork includes the remaining network layers except the first M network layers.
[0141] For step S24 to step S30, it can be implemented in the following manner:
[0142] Method 1: If the first target subnetwork meets the convergence condition and the second target subnetwork meets the convergence condition, it is determined that the first target subnetwork has converged and the second target subnetwork has converged. The first target subnetwork is determined as the first converged target subnetwork, the second target subnetwork is determined as the second converged target subnetwork, and the target trained model is obtained based on the first converged target subnetwork and the second converged target subnetwork.
[0143] If the first target subnetwork does not meet the convergence condition, and / or the second target subnetwork does not meet the convergence condition, it is determined that the first target subnetwork has not converged, and it is determined that the second target subnetwork has not converged. Based on this, the parameters of the first target subnetwork are adjusted based on the first target feature to obtain an adjusted subnetwork, and the adjusted subnetwork is determined as the first target subnetwork, and the parameters of the second target subnetwork are adjusted based on the second target feature to obtain an adjusted subnetwork, and the adjusted subnetwork is determined as the second target subnetwork. Based on the adjusted first target subnetwork and second target subnetwork, return to step S22 and step S23.
[0144] Method 2: If the first target subnetwork meets the convergence condition and the second target subnetwork does not meet the convergence condition, it can be determined that the first target subnetwork has converged and the second target subnetwork has converged. That is, as long as the first target subnetwork meets the convergence condition, it can be determined that the first target subnetwork and the second target subnetwork have converged. Based on this, the first target subnetwork can be determined as the first converged target subnetwork, and the second target subnetwork can be determined as the second converged target subnetwork, and the target trained model can be obtained based on the first converged target subnetwork and the second converged target subnetwork.
[0145] If the first target subnetwork does not meet the convergence condition, regardless of whether the second target subnetwork meets the convergence condition, it is determined that the first target subnetwork has not converged, and the second target subnetwork has not converged. Based on this, the parameters of the first target subnetwork are adjusted based on the first target feature to obtain an adjusted subnetwork, and the adjusted subnetwork is determined as the first target subnetwork, and the parameters of the second target subnetwork are adjusted based on the second target feature to obtain an adjusted subnetwork, and the adjusted subnetwork is determined as the second target subnetwork. Based on the adjusted first target subnetwork and second target subnetwork, return to step S22 and step S23.
[0146] Method 3: If the first target subnetwork does not meet the convergence condition, and the second target subnetwork meets the convergence condition, it can be determined that the first target subnetwork has converged, and the second target subnetwork has converged, that is, as long as the second target subnetwork meets the convergence condition, it can be determined that the first target subnetwork and the second target subnetwork have converged. Based on this, the first target subnetwork can be determined as the first converged target subnetwork, and the second target subnetwork can be determined as the second converged target subnetwork, and the target trained model can be obtained based on the first converged target subnetwork and the second converged target subnetwork.
[0147] If the second target subnetwork does not meet the convergence condition, regardless of whether the first target subnetwork meets the convergence condition, it is determined that the first target subnetwork has not converged, and it is determined that the second target subnetwork has not converged. Based on this, the parameters of the first target subnetwork are adjusted based on the first target feature to obtain an adjusted subnetwork, and the adjusted subnetwork is determined as the first target subnetwork, and the parameters of the second target subnetwork are adjusted based on the second target feature to obtain an adjusted subnetwork, and the adjusted subnetwork is determined as the second target subnetwork. Based on the adjusted first target subnetwork and second target subnetwork, return to step S22 and step S23.
[0148] Mode 4: If the first target subnetwork meets the convergence condition, but the second target subnetwork does not meet the convergence condition, it is determined that the first target subnetwork has converged, but the second target subnetwork has not converged. Based on this, the first target subnetwork can be determined as the first converged target subnetwork, and in the subsequent process, the parameters of the first converged target subnetwork are no longer adjusted, and the first converged target subnetwork is retained.
[0149] And, since the second target subnetwork has not converged, the parameters of the second target subnetwork are adjusted based on the second target feature to obtain an adjusted subnetwork, and the adjusted subnetwork is determined as the second target subnetwork, and based on the adjusted second target subnetwork, the process returns to step S23. When adjusting the parameters of the second target subnetwork, it is not necessary to adjust the parameters of the first target subnetwork.
[0150] In the subsequent process, it is only necessary to adjust the parameters of the second target subnetwork until the second target subnetwork meets the convergence condition and determines that the second target subnetwork has converged, so that the second target subnetwork can be determined as the second converged target subnetwork. On this basis, the target trained model can be obtained based on the first converged target subnetwork and the second converged target subnetwork.
[0151] Mode 5: If the second target subnetwork meets the convergence condition, but the first target subnetwork does not meet the convergence condition, it is determined that the second target subnetwork has converged, but it is determined that the first target subnetwork has not converged. Based on this, the second target subnetwork can be determined as the second converged target subnetwork, and in the subsequent process, the parameters of the second converged target subnetwork are no longer adjusted, and the second converged target subnetwork is retained.
[0152] And, since the first target subnetwork has not converged, the parameters of the first target subnetwork are adjusted based on the first target feature to obtain an adjusted subnetwork, and the adjusted subnetwork is determined as the first target subnetwork, and based on the adjusted first target subnetwork, the process returns to step S22. When adjusting the parameters of the first target subnetwork, it is not necessary to adjust the parameters of the second target subnetwork.
[0153] In the subsequent process, it is only necessary to adjust the parameters of the first target subnetwork until the first target subnetwork meets the convergence condition and determines that the first target subnetwork has converged, so that the first target subnetwork can be determined as the first converged target subnetwork. On this basis, the target trained model can be obtained based on the first converged target subnetwork and the second converged target subnetwork.
[0154] In the above embodiment, whether the first target subnetwork meets the convergence condition may include: after obtaining the first target feature, determining the loss value based on the first target feature, for example, pre-configuring the loss function a, the input of the loss function a is the first target feature, and the output of the loss function a is the loss value, based on this, after obtaining the first target feature, substituting the first target feature into the loss function a, obtaining the loss value corresponding to the first target feature, and determining whether the first target subnetwork meets the convergence condition based on the loss value. For example, if the loss value is less than a preset threshold, it is determined that the first target subnetwork meets the convergence condition, and if the loss value is not less than the preset threshold, it is determined that the first target subnetwork does not meet the convergence condition. Of course, other methods can also be used to determine whether the first target subnetwork meets the convergence condition, and there is no limitation on this determination method. For example, whether the first target subnetwork meets the convergence condition can be determined based on the number of iterations or the iteration duration.
[0155] In the above embodiment, whether the second target subnetwork meets the convergence condition may include: after obtaining the second target feature, determining the loss value based on the second target feature, for example, pre-configuring the loss function b, the input of the loss function b is the second target feature, and the output of the loss function b is the loss value, based on this, after obtaining the second target feature, substituting the second target feature into the loss function b, obtaining the loss value corresponding to the second target feature, and determining whether the second target subnetwork meets the convergence condition based on the loss value. For example, if the loss value is less than a preset threshold, it is determined that the second target subnetwork meets the convergence condition, and if the loss value is not less than the preset threshold, it is determined that the second target subnetwork does not meet the convergence condition. Of course, other methods can also be used to determine whether the second target subnetwork meets the convergence condition, and there is no limitation on this determination method. For example, whether the second target subnetwork meets the convergence condition can be determined based on the number of iterations or the iteration duration.
[0156] In the above embodiment, adjusting the parameters of the first target sub-network based on the first target feature may include but is not limited to: substituting the first target feature into the loss function a to obtain the loss value corresponding to the first target feature, and adjusting the parameters of the first target sub-network (i.e., network parameters) based on the loss value. For example, based on the loss value corresponding to the first target feature, the network parameters (i.e., network weights) of the first target sub-network may be updated by a back propagation algorithm, and there is no restriction on this updating process. An example of a back propagation algorithm may be a gradient descent method, i.e., updating the network parameters of the first target sub-network by a gradient descent method.
[0157] In the above embodiment, adjusting the parameters of the second target sub-network based on the second target feature may include but is not limited to: substituting the second target feature into the loss function b to obtain the loss value corresponding to the second target feature, and adjusting the parameters of the second target sub-network (i.e., network parameters) based on the loss value. For example, based on the loss value corresponding to the second target feature, the network parameters (i.e., network weights) of the second target sub-network may be updated by a back propagation algorithm, and there is no restriction on this updating process. An example of a back propagation algorithm may be a gradient descent method, i.e., updating the network parameters of the second target sub-network by a gradient descent method.
[0158] In a possible implementation, after obtaining the target trained model, data processing can be implemented through the target trained model, that is, artificial intelligence processing can be implemented. For example, if the target trained model is used to implement target detection, then after obtaining the data to be processed, the data to be processed can be input into the target trained model, and the target trained model performs detection processing on the data to be processed to obtain the detection result, and target detection is performed based on the detection result. Alternatively, if the target trained model is used to implement target classification, then after obtaining the data to be processed, the data to be processed can be input into the target trained model, and the target trained model performs classification processing on the data to be processed to obtain the classification result, and target classification is performed based on the classification result.
[0159] In the above embodiment, the target model to be trained can be trained based on the training data in the target data set and the training data in the original data set, so as to realize the incremental joint training of the target data set and the original data set, cleverly avoid the recalibration of a large amount of training data in the original data set, and the target data set and the original data set can be used together to participate in the training of the new model without recalibrating the original data set. The target data set with complete calibration is integrated for joint training, and the knowledge of the target data set and the knowledge of the original data set can be learned. In each training process, there are both training data from the target data set and training data from the original data set. This method plays a role in data balancing. Since the target model to be trained has been trained based on a large amount of training data in the original data set, the training data in the target data set does not need to be too much, that is, it can be composed of a small amount of training data.
[0160] Since it is the original features (i.e., the features output by the Mth network layer of the original trained model, which can be a feature map) that participate in the training process of the target trained model, rather than the training data in the original data set, sensitive data is avoided from participating in the training process, confidential information of the training data is hidden, and security is improved. For example, considering that the training data in the original data set itself contains sensitive data and is not convenient to disclose, in this embodiment, the training data is first input to the original trained model to obtain the original features output by the Mth network layer of the original trained model. Obviously, the original features are a low-feature map (i.e., only low-level features of the training data are included, but not the training data itself). By introducing the original features to train the target model to be trained, that is, training with reference to a large amount of training data in the original data set, the performance of the trained model is better. Since the original features are low-feature maps and do not include sensitive data of the training data, the leakage of sensitive data can be avoided.
[0161] Based on the same application concept as the above method, a data processing device is proposed in the embodiment of the present application, see Figure 5FIG. 1 is a schematic diagram of the structure of the data processing device, which may include:
[0162] An acquisition module 51 is used to input the training data in the original data set to the first original sub-network of the original trained model to obtain the first original feature; wherein the original trained model also includes a second original sub-network, the first original sub-network includes the first M network layers in the original trained model, and the second original sub-network includes the remaining network layers in the original trained model except the first M network layers;
[0163] And, inputting the training data in the target data set into the first target sub-network of the target model to be trained to obtain the first target feature; wherein the target model to be trained also includes a second target sub-network, the first target sub-network includes the first M network layers in the target model to be trained, and the second target sub-network includes the remaining network layers in the target model to be trained except the first M network layers;
[0164] And, inputting the first original feature and the first target feature into a second target sub-network of the target model to be trained, and the second target sub-network outputs a second target feature;
[0165] A training module 52, configured to train the first target sub-network based on the first target feature, train the second target sub-network based on the second target feature, and acquire a target trained model based on the trained first target sub-network and the trained second target sub-network;
[0166] The target trained model is used to implement data processing.
[0167] In a possible implementation, the training module 52 is specifically used for:
[0168] Determine whether the first target subnetwork has converged; if not, adjust the parameters of the first target subnetwork based on the first target feature to obtain an adjusted subnetwork, determine the adjusted subnetwork as the first target subnetwork, and return to execute the operation of inputting the training data in the target data set to the first target subnetwork; if yes, determine the first target subnetwork as the first converged target subnetwork;
[0169] Determine whether the second target subnetwork has converged; if not, adjust the parameters of the second target subnetwork based on the second target feature to obtain an adjusted subnetwork, determine the adjusted subnetwork as the second target subnetwork, and return to perform the operation of inputting the first original feature and the first target feature into the second target subnetwork; if yes, determine the second target subnetwork as the second converged target subnetwork;
[0170] A target trained model is obtained based on the first converged target sub-network and the second converged target sub-network.
[0171] In a possible implementation manner, the training module 52 determines whether the first target sub-network has converged and determines whether the second target sub-network has converged, and is specifically used to:
[0172] If the first target sub-network meets the convergence condition and the second target sub-network meets the convergence condition, then it is determined that the first target sub-network has converged, and it is determined that the second target sub-network has converged; or,
[0173] If the first target subnetwork satisfies the convergence condition and the second target subnetwork does not satisfy the convergence condition, then it is determined that the first target subnetwork has converged, and it is determined that the second target subnetwork has converged; or,
[0174] If the first target subnetwork does not meet the convergence condition, and the second target subnetwork meets the convergence condition, it is determined that the first target subnetwork has converged, and it is determined that the second target subnetwork has converged.
[0175] Based on the same concept as the above method, an embodiment of the present application proposes a data processing device that may include: a processor and a machine-readable storage medium, the machine-readable storage medium storing machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the following steps:
[0176] Input the training data in the original data set to the first original sub-network of the original trained model to obtain the first original feature; wherein the original trained model also includes a second original sub-network, the first original sub-network includes the first M network layers in the original trained model, and the second original sub-network includes the remaining network layers in the original trained model except the first M network layers;
[0177] Input the training data in the target data set to the first target sub-network of the target model to be trained to obtain a first target feature; wherein the target model to be trained also includes a second target sub-network, the first target sub-network includes the first M network layers in the target model to be trained, and the second target sub-network includes the remaining network layers in the target model to be trained except the first M network layers;
[0178] Inputting the first original feature and the first target feature into a second target sub-network of the target to-be-trained model, and having the second target sub-network output a second target feature;
[0179] The first target sub-network is trained based on the first target feature, the second target sub-network is trained based on the second target feature, and a target trained model is obtained based on the trained first target sub-network and the trained second target sub-network; wherein the target trained model is used to implement data processing.
[0180] Based on the same application concept as the above method, an embodiment of the present application also provides a machine-readable storage medium, on which a number of computer instructions are stored. When the computer instructions are executed by a processor, the data processing method disclosed in the above example of the present application can be implemented.
[0181] The above-mentioned machine-readable storage medium may be any electronic, magnetic, optical or other physical storage device, which may contain or store information, such as executable instructions, data, etc. For example, the machine-readable storage medium may be: RAM (Radom Access Memory), volatile memory, non-volatile memory, flash memory, storage drive (such as hard disk drive), solid state drive, any type of storage disk (such as CD, DVD, etc.), or similar storage medium, or a combination thereof.
[0182] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which may be in the form of a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email transceiver, a game console, a tablet computer, a wearable device or a combination of any of these devices.
[0183] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.
[0184] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the embodiments of the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.
[0185] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0186] Moreover, these computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.
[0187] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0188] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.
Claims
1. A data processing method, characterized in that: The method comprises: Inputting the original training image in the original data set to the first original sub-network of the original trained model to obtain the first original feature corresponding to the original training image; wherein the original trained model also includes a second original sub-network, the first original sub-network includes the first M network layers in the original trained model, and the second original sub-network includes the remaining network layers in the original trained model except the first M network layers; Input the target training image in the target data set to the first target sub-network of the target model to be trained, obtain the first target feature corresponding to the target training image, determine whether the first target sub-network of the target model to be trained has converged, and if not, adjust the parameters of the first target sub-network based on the first target feature, and return to execute the step of inputting the target training image in the target data set to the first target sub-network of the target model to be trained; wherein the target model to be trained also includes a second target sub-network, the first target sub-network includes the first M network layers in the target model to be trained, and the second target sub-network includes the remaining network layers in the target model to be trained except the first M network layers, Inputting the first original feature corresponding to the original training image and the first target feature corresponding to the target training image into the second target sub-network of the target model to be trained, outputting the second target feature from the second target sub-network, determining whether the second target sub-network of the target model to be trained has converged, and if not, adjusting the parameters of the second target sub-network based on the second target feature, returning to the step of inputting the first original feature corresponding to the original training image and the first target feature corresponding to the target training image into the second target sub-network of the target model to be trained, Wherein, if the first target subnetwork satisfies the convergence condition and the second target subnetwork does not satisfy the convergence condition, it is determined that the first target subnetwork and the second target subnetwork have converged; or, if the first target subnetwork does not satisfy the convergence condition and the second target subnetwork satisfies the convergence condition, it is determined that the first target subnetwork and the second target subnetwork have converged; Acquire a target trained model based on the trained first target sub-network and the trained second target sub-network; wherein the target trained model is used to implement data processing; When the original trained model is applied to a first business, the original trained model is used to detect a first target category supported by the first business, and when the target trained model is applied to a second business after the business demand has changed, the target trained model is used to detect the first target category and the second target category supported by the second business; in, The original data set also includes the original label categories corresponding to the original training images, and the target data set also includes the target label categories corresponding to the target training images. The target category set is the union of the original label categories. The first original sub-network is different from the first target sub-network.
2. The method according to claim 1, characterized in that The number of original trained models is N, the N original trained models correspond one-to-one to the N original data sets, and the first original features input to the second target sub-network are the N first original features corresponding to the N original trained models; For each original trained model, the training process of the original trained model includes: The original training image in the original data set corresponding to the original trained model is input into the original model to be trained to obtain a second original feature; the original model to be trained is trained based on the second original feature to obtain the original trained model; wherein the original trained model is used to realize data processing.
3. The method according to claim 1, characterized in that: The target category set corresponding to the target data set is the union of N original category sets corresponding to N original data sets, and the N original data sets correspond to the N original category sets one by one; in, The target category set includes all target label categories in the target data set; Each original category set includes all original label categories in the original data set corresponding to the original category set.
4. The method according to claim 1, characterized in that: Whether the first target sub-network has converged further includes: If not, determining the adjusted first target subnetwork as the first target subnetwork; if so, determining the first target subnetwork as the first converged target subnetwork; Whether the second target sub-network has converged further includes: If not, the adjusted second target sub-network is determined as the second target sub-network; if so, the second target sub-network is determined as the second converged target sub-network.
5. The method according to claim 4, characterized in that Whether the first target sub-network has converged, and whether the second target sub-network has converged, includes: If the first target sub-network meets the convergence condition and the second target sub-network meets the convergence condition, it is determined that the first target sub-network has converged, and it is determined that the second target sub-network has converged.
6. The method according to claim 1, characterized in that After acquiring the target trained model based on the trained first target sub-network and the trained second target sub-network, the method further includes: If the target trained model is used to realize target detection, after obtaining the data to be processed, the data to be processed is input into the target trained model, and the target trained model performs detection processing on the data to be processed to obtain a detection result, and performs target detection based on the detection result; If the target trained model is used to achieve target classification, then after obtaining the data to be processed, the data to be processed is input into the target trained model, and the target trained model classifies the data to be processed to obtain a classification result, and the target classification is performed based on the classification result.
7. The method according to any one of claims 1 to 6, characterized in that: The first M network layers in the original trained model have the same network structure as the first M network layers in the target trained model, but different network parameters; If there are N original trained models, where N is a positive integer greater than 1, the network structures of the first M network layers in the N original trained models are the same.
8. A data processing device, characterized in that: The device comprises: An acquisition module is used to input the original training image in the original data set to the first original sub-network of the original trained model to obtain the first original feature corresponding to the original training image; wherein the original trained model also includes a second original sub-network, the first original sub-network includes the first M network layers in the original trained model, and the second original sub-network includes the remaining network layers in the original trained model except the first M network layers; A training module, used for inputting a target training image in a target data set to a first target sub-network of a target model to be trained, obtaining a first target feature corresponding to the target training image, determining whether the first target sub-network of the target model to be trained has converged, and if not, adjusting the parameters of the first target sub-network based on the first target feature, and returning to execute the step of inputting the target training image in the target data set to the first target sub-network of the target model to be trained; wherein the target model to be trained also includes a second target sub-network, the first target sub-network includes the first M network layers in the target model to be trained, and the second target sub-network includes the remaining network layers in the target model to be trained except the first M network layers; And, the first original feature corresponding to the original training image and the first target feature corresponding to the target training image are input to the second target subnetwork of the target model to be trained, and the second target subnetwork outputs the second target feature, and determines whether the second target subnetwork of the target model to be trained has converged. If not, adjust the parameters of the second target subnetwork based on the second target feature, and return to the step of inputting the first original feature corresponding to the original training image and the first target feature corresponding to the target training image to the second target subnetwork of the target model to be trained; if the first target subnetwork meets the convergence condition and the second target subnetwork does not meet the convergence condition, it is determined that the first target subnetwork and the second target subnetwork have converged; or, if the first target subnetwork does not meet the convergence condition and the second target subnetwork meets the convergence condition, it is determined that the first target subnetwork and the second target subnetwork have converged; and obtain the target trained model based on the trained first target subnetwork and the trained second target subnetwork; in, The target trained model is used to implement data processing; When the original trained model is applied to a first business, the original trained model is used to detect a first target category supported by the first business, and when the target trained model is applied to a second business after the business demand has changed, the target trained model is used to detect the first target category and the second target category supported by the second business; The original data set also includes the original label categories corresponding to the original training images, the target data set also includes the target label categories corresponding to the target training images, the target category set is the union of the original label categories, and the first original sub-network is different from the first target sub-network.
9. The device according to claim 8, characterized in that The training module is also used to: If the first target subnetwork has not converged, the adjusted first target subnetwork is determined as the first target subnetwork; if the first target subnetwork has converged, the first target subnetwork is determined as the first converged target subnetwork; If the second target subnetwork has not converged, determining the adjusted second target subnetwork as the second target subnetwork; If the second target sub-network has converged, the second target sub-network is determined as a second converged target sub-network.
10. A data processing device, characterized in that: include: a processor and a machine-readable storage medium storing machine-executable instructions executable by the processor; The processor is used to execute machine executable instructions to implement the following steps: Inputting the original training image in the original data set to the first original sub-network of the original trained model to obtain the first original feature corresponding to the original training image; wherein the original trained model also includes a second original sub-network, the first original sub-network includes the first M network layers in the original trained model, and the second original sub-network includes the remaining network layers in the original trained model except the first M network layers; Input the target training image in the target data set to the first target sub-network of the target model to be trained, obtain the first target feature corresponding to the target training image, determine whether the first target sub-network of the target model to be trained has converged, and if not, adjust the parameters of the first target sub-network based on the first target feature, and return to execute the step of inputting the target training image in the target data set to the first target sub-network of the target model to be trained; wherein the target model to be trained also includes a second target sub-network, the first target sub-network includes the first M network layers in the target model to be trained, and the second target sub-network includes the remaining network layers in the target model to be trained except the first M network layers, Inputting the first original feature corresponding to the original training image and the first target feature corresponding to the target training image into the second target sub-network of the target model to be trained, outputting the second target feature from the second target sub-network, determining whether the second target sub-network of the target model to be trained has converged, and if not, adjusting the parameters of the second target sub-network based on the second target feature, returning to the step of inputting the first original feature corresponding to the original training image and the first target feature corresponding to the target training image into the second target sub-network of the target model to be trained, If the first target subnetwork satisfies the convergence condition and the second target subnetwork does not satisfy the convergence condition, it is determined that the first target subnetwork and the second target subnetwork have converged; or, if the first target subnetwork does not satisfy the convergence condition and the second target subnetwork satisfies the convergence condition, it is determined that the first target subnetwork and the second target subnetwork have converged; Acquire a target trained model based on the trained first target sub-network and the trained second target sub-network; in, The target trained model is used to implement data processing; When the original trained model is applied to a first business, the original trained model is used to detect a first target category supported by the first business, and when the target trained model is applied to a second business after the business demand has changed, the target trained model is used to detect the first target category and the second target category supported by the second business; The original data set also includes the original label categories corresponding to the original training images, the target data set also includes the target label categories corresponding to the target training images, the target category set is the union of the original label categories, and the first original sub-network is different from the first target sub-network.
