Method, device and system for training model

Abstract

The invention relates to a method, device and system for training a model. Various embodiments related to model training are described. In one embodiment, a method for training a model includes: acquiring an approximate dataset composed of open world data similar to an actual dataset; and training a model using the approximate data set.

Description

Method, device and system for training model

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application is based on the Chinese application with application number 202210554969.8 and application date May 19, 2022, and claims its priority. The disclosed content of the Chinese application is hereby introduced as a whole into this application. Technical Field

[0003] The present disclosure relates generally to privacy protection, and in particular to privacy protection during model training. Background Art

[0004] With the continuous development of deep learning technology, neural network models have been widely deployed on various systems and devices, including edge computing devices, for real-time interaction with users.

[0005] However, because neural network models contain many parameters, they typically require large amounts of data and computational costs to train, and most commercial edge computing devices do not support model training with high computational complexity. To address this issue, one technical approach is to collect data on high-computing servers for training. However, data sharing can lead to unpredictable privacy leaks. In particular, existing privacy laws in many countries do not allow data to leave the local area. Another technical approach is to train the model locally using real data. Specifically, fine-tuning the model downloaded from the cloud to improve the model's performance on real data. However, model training consumes excessive resources and is therefore not suitable for low-power edge computing devices.

[0006] Currently, there are two approaches to privacy protection for model training. The first is to use differential privacy technology. However, this approach has a significant performance impact, requires high computing power, and slows training convergence, making it unsuitable for real-world applications. The other approach is to perform privacy-enhancing processing on local data before sharing it with the server for model training. Privacy-enhancing processing involves data encryption and privacy attribute decoupling. The former uses traditional numerical encryption methods to encrypt images or other data samples, ensuring data availability, but this also imposes a high computational burden and is not suitable for low-power devices. The latter requires pre-training a feature extraction model with decoupling capabilities and using supervised learning to remove private information from features during training. This relies on pre-defined privacy policies and may also remove information important for the learning task, reducing feature quality.

[0007] Therefore, it is necessary to effectively protect the privacy of local actual data while ensuring the performance of model training.

[0008] Summary of the Invention

[0009] One aspect of the present disclosure relates to a method for training a model. According to an embodiment of the present disclosure, the method includes: obtaining an approximate dataset consisting of open-world data that is similar to an actual dataset; and using the approximate dataset to train a model.

[0010] One aspect of the present disclosure relates to a data processing method. According to an embodiment of the present disclosure, the data processing method includes: obtaining an actual data set; receiving a representative data sample set of the open-world data set; performing feature matching on the representative data sample and the actual data; and returning a matching result between the representative data sample and the actual data.

[0011] One aspect of the present disclosure relates to a method for generating a model. According to an embodiment of the present disclosure, the method for generating a model includes: executing the steps of the method for training a model according to an embodiment of the present disclosure to generate a model.

[0012] One aspect of the present disclosure relates to a training device. According to an embodiment of the present disclosure, the device includes a training data acquisition module configured to acquire an approximate dataset composed of open-world data that is similar to an actual dataset; and a training module configured to train a model using the approximate dataset.

[0013] One aspect of the present disclosure relates to a data processing device. According to an embodiment of the present disclosure, the data processing device includes a data acquisition module configured to acquire an actual data set; an interaction module configured to receive a representative data sample set of an open-world data set; and a feature matching module configured to perform feature matching between the representative data sample and the actual data. The interaction module is further configured to return a matching result between the representative data sample and the actual data.

[0014] One aspect of the present disclosure relates to a system for training a model. According to an embodiment of the present disclosure, the system for training a model includes a training device according to an embodiment of the present disclosure; and a data processing device according to an embodiment of the present disclosure.

[0015] Yet another aspect of the present disclosure relates to a computer-readable storage medium storing one or more instructions. In some embodiments, the one or more instructions, when executed by a processor, may cause the processor to perform steps of various methods according to embodiments of the present disclosure.

[0016] Yet another aspect of the present disclosure relates to a computer program product comprising one or more instructions. In some embodiments, when the one or more instructions are executed by a processor, the processor may perform the steps of various methods according to embodiments of the present disclosure.

[0017] The above summary is provided to summarize some exemplary embodiments in order to provide a basic understanding of various aspects of the subject matter described herein. Therefore, the above features are merely examples and should not be construed as narrowing the scope or spirit of the subject matter described herein in any way. Other features, aspects, and advantages of the subject matter described herein will become apparent from the detailed description described below in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] A better understanding of the present disclosure may be obtained when the following detailed description of the embodiments is considered in conjunction with the accompanying drawings. The same or similar reference numerals are used in the various drawings to represent the same or similar components. The accompanying drawings, together with the following detailed description, are incorporated into and form a part of this specification and are used to illustrate the embodiments of the present disclosure and to explain the principles and advantages of the present disclosure. In particular:

[0019] FIG1A is a flowchart illustrating an example of steps of a method for training a model according to an embodiment of the present disclosure.

[0020] FIG1B is a flowchart illustrating an example of sub-steps of the step of obtaining an approximate dataset according to an embodiment of the present disclosure.

[0021] FIG1C is a flowchart illustrating an example of sub-steps of the step of screening representative data samples according to an embodiment of the present disclosure.

[0022] FIG. 2 is a flowchart illustrating an example of steps of a data processing method according to an embodiment of the present disclosure.

[0023] Figure 3 is a schematic diagram showing an example of the configuration of a system for training a model and a training device and a data processing device therein according to an embodiment of the present disclosure, in which the main functional modules and information interactions that constitute the device are illustrated.

[0024] 4A-4C are schematic diagrams illustrating multiple learning scenarios for a scheme for training a model according to an embodiment of the present disclosure.

[0025] FIG5 shows an example block diagram of a computer that can be implemented as a training device, a data processing device, or a system for training a model according to an embodiment of the present disclosure.

[0026] While the embodiments described in this disclosure may be susceptible to various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and are herein described in detail. However, it should be understood that the drawings and detailed description thereof are not intended to limit the embodiments to the particular forms disclosed, but on the contrary, the intent is to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the claims. DETAILED DESCRIPTION

[0027] The following describes representative applications of various aspects of the apparatus and method of the present disclosure. The description of these examples is only to add context and help understand the described embodiments. Therefore, it is clear to those skilled in the art that the embodiments described below can be implemented without some or all of the specific details. In other cases, well-known process steps are not described in detail to avoid unnecessarily obscuring the described embodiments. Other applications are also possible, and the solutions of the present disclosure are not limited to these examples.

[0028] The inventors recognize that, compared to general models, models trained using local actual data can be more accurately applied to specific local application scenarios. However, using actual data to train models directly locally is not suitable for low-power computing devices (local edge devices such as sensors), and collecting actual data to high-computing computing devices (such as cloud servers) for training may lead to unpredictable privacy leaks. In addition, models trained directly using actual data may leak privacy information in the actual data. For example, in some cases, it is possible to determine whether the target data is used to train the model by observing the output of the target data on the model, thereby obtaining the privacy information of the members of the target data. For example, more specifically, if the facial image data captured by the camera is directly used for model training, the trained model may be used maliciously, for example, to infer whether someone has entered the camera's observation range through the model output, such as in unmanned supermarkets.

[0029] The inventors also recognize that increasing the amount of training data is conducive to completing effective learning to obtain a high-precision model. However, existing privacy-preserving learning methods usually limit data sharing to a certain extent.

[0030] Therefore, it is necessary to obtain a large amount of valuable training data while protecting privacy information and ensuring the computing power required for training.

[0031] To this end, the applicant proposes in this disclosure to sample a large number of data samples from a vast and diverse set of open-world data to match the real-world data, and to train models on these non-privacy-sensitive data samples. On the one hand, the locally collected real-world data is only used for local data matching and is not utilized by other modules. On the other hand, local model training, which requires high computing power, is not required. As a result, models trained on large amounts of open-world data do not leak the privacy information of the real-world data, while also ensuring the practical application value of the model.

[0032] The solution of this disclosure is applicable to all scenarios requiring privacy protection of actual data for training models and requiring large amounts of data, such as smart cities and smart supermarkets. In some embodiments, the solution of this disclosure can incorporate time-series information from multiple frames of surveillance video to protect the privacy of, for example, user movement trajectories.

[0033] The solution disclosed in the present invention is also applicable to scenarios where high-energy-consuming learning tasks are transferred from low-computing-power edge devices to high-computing-power servers.

[0034] FIG1A illustrates a flowchart of an example of steps of a method for training a model according to an embodiment of the present disclosure. FIG1B-FIG1C illustrate a flowchart of an example of sub-steps of some steps of a method for training a model according to an embodiment of the present disclosure. The method for training a model according to an embodiment of the present disclosure can be performed by any device including a processing device, for example, can be performed by a high-computing-power server (such as a cloud server).

[0035] As shown in FIG1A , according to an embodiment of the present disclosure, a method 100 for training a model may mainly include the following steps:

[0036] At step 110 , an approximate dataset consisting of open-world data and similar to the actual dataset is obtained; and

[0037] At step 120 , the model is trained using the approximate dataset.

[0038] Here, open-world data can be understood as public data resources that can be legally obtained from any channel, including online images, public datasets, and so on. The inventors have recognized that while massive amounts of open-world data can provide rich and diverse data samples, directly using open-world data to train models is disadvantageous. On the one hand, the dispersed data distribution is not conducive to rapid model convergence. On the other hand, for a specific application scenario, large amounts of data with a distribution that differs significantly from the actual data can affect model accuracy.

[0039] Therefore, in an embodiment according to the present disclosure, an approximate dataset consisting of open-world data and similar to the actual dataset is obtained, and the model is trained using the approximate dataset.

[0040] Here, the "similarity" between the approximate dataset and the actual dataset can be understood as having a similar data distribution. A metric for this can be whether the loss obtained from training on the approximate dataset is similar to the loss obtained from training on the actual dataset. Therefore, using the approximate dataset to train a model can improve model performance on actual data.

[0041] Optionally, in some embodiments, the method 100 for training a model may further include obtaining a pre-trained model (step 130). Thus, using the approximate dataset to train the model (step 120) includes adjusting the pre-trained model using the approximate dataset.

[0042] In some embodiments, data can be collected from the open world, and the model can be pre-trained on the collected pre-training dataset to obtain a pre-trained model. For example, data from the common ImageNet can be used for pre-training. However, the present disclosure is not limited to this. For example, in some embodiments, the pre-trained model can be downloaded from the cloud. Alternatively, in some embodiments, the pre-trained model can be downloaded from an external storage device. It will be readily understood by those skilled in the art that the method for obtaining the pre-trained model is not limited to this and can be selected as needed.

[0043] The following detailed description of the sub-steps of the step of obtaining an approximate dataset (step 110) is provided in conjunction with the flowchart of FIG1B . Those skilled in the art will readily appreciate that the method for obtaining an approximate dataset shown in FIG1B is merely an example, and the present disclosure is not limited thereto. Those skilled in the art may also utilize various existing open-world data acquisition, sampling, collection, selection, selection, screening, and filtering methods in conjunction with the concepts disclosed herein to obtain an approximate dataset similar to the actual dataset.

[0044] As shown in FIG1B , in some embodiments, obtaining an approximate dataset may include performing abstract processing on an open world dataset D to obtain a representative data sample set S q (Step 112).

[0045] In some embodiments, the open-world dataset D can be collected based on the application scenario to which the actual data belongs. For example, if the application scenario is a smart city or a smart unmanned supermarket, data samples of a class that is adapted to the application scenario can be selected to form the open-world dataset D. Advantageously, this approach can more quickly and accurately match the distribution of sensor data and determine similar datasets. However, those skilled in the art will readily appreciate that the present disclosure is not limited to this. For example, data samples can be randomly collected from the open world to form the open-world dataset D.

[0046] In some examples, the open-world dataset D may have the same or similar data distribution as the pre-training dataset used to obtain the pre-trained model. Alternatively, the open-world dataset D may have a different data distribution than the pre-training dataset.

[0047] Here, “abstracting” the open-world dataset D may refer to various methods for extracting data from a massive open-world dataset to obtain representative data samples, including but not limited to unsupervised clustering methods.

[0048] The inventors have realized that the feature layer of a neural network, especially the penultimate layer, has inherent feature extraction and data clustering functions, and can be used to extract features from data and cluster the data.

[0049] Therefore, in some embodiments, step 112 of abstracting the open-world dataset includes performing feature extraction and clustering on the data in the open-world dataset using a pre-trained model.

[0050] For example, the feature extraction and clustering in step 112 may be performed using the pre-trained model obtained in step 130. However, the present disclosure is not limited thereto.

[0051] Assume that the pre-trained neural network is f θ By removing the last layer of the neural network and taking the remaining feature layer at the penultimate layer as the output, the neural network φ can be obtained. φ can be used as a feature extractor to extract the hidden layer features of the open world dataset D and cluster them to obtain C cluster centers of a total of C clusters. Each cluster center is regarded as a representative data sample of the cluster. All representative data samples can constitute the representative data sample set S mentioned above. q However, the present disclosure is not limited thereto, and other feature extractors may also be used to extract features of the open world dataset D and perform clustering.

[0052] It is easy for those skilled in the art to understand that the above process is only an example of abstract processing, and the present disclosure is not limited thereto. Those skilled in the art can also combine the ideas disclosed in the present disclosure with various existing abstract processing methods to obtain a representative data sample set S q .

[0053] It is worth noting that, since the open-world data can be considered to have no boundaries, the number of clusters C obtained in step 112 is not necessarily equal to the number of classes corresponding to all the open-world data.

[0054] As shown in FIG. 1B , in some embodiments, obtaining an approximate data set may include screening a representative data sample set for a representative data sample that matches the actual data in terms of features (step 114 ).

[0055] The inventors realized that by downloading a representative data sample set locally, comparing it with the actual data and returning the comparison results, it is possible to screen representative data samples with feature matching without sharing the local actual data, thereby advantageously avoiding the privacy leakage risks brought about by uploading local actual data.

[0056] Therefore, in some embodiments, step 114 of screening representative data samples that match the characteristics of the actual data may include exchanging information with a data processing device that has access to the actual data to determine the representative data samples that match the characteristics of the actual data, wherein the exchanged information does not include the actual data.

[0057] The following will illustrate in detail an example of screening representative data samples (step 114) with reference to the flowchart of FIG1C . Those skilled in the art will readily appreciate that the screening method shown in FIG1C is merely an example and the present disclosure is not limited thereto. Those skilled in the art may also utilize various existing screening methods in conjunction with the concepts disclosed herein to obtain representative data samples that match the characteristics of the actual data.

[0058] As shown in FIG. 1C , in some embodiments, step 114 of screening representative data samples that match the actual data in terms of features may include the following sub-steps.

[0059] At step 1142 , the representative data sample set is sent to a data processing device.

[0060] In some embodiments, the transmitted representative data samples may be represented by extracted features for one or more purposes of improving communication efficiency, reducing communication overhead, and reducing local storage burden. However, the present disclosure is not limited thereto. For example, the transmitted representative data samples may also be data samples in their original format.

[0061] At step 1144, the matching results of each representative data sample are received from the data processing device, wherein the matching result of each representative data sample is based on the statistical similarity in features between the representative data sample and each actual data.

[0062] In some embodiments, features of the actual data may also be extracted using a pre-trained model. Thus, the neural network φ may also be sent to a data processing device that has access to the actual data.

[0063] For example, the matching result of each representative data sample can be based on the statistical sum or average value of the similarity between the representative data sample and all actual data in terms of features. In some examples, the similarity between the representative data sample and any actual data can be represented by a numerical value, wherein the magnitude of the numerical value reflects the degree of matching with the actual data in terms of features. For example, the similarity between the representative data sample and any actual data can be represented by a discrete numerical value of 0 or 1, wherein for any actual data, the similarity of the representative data sample that best matches the actual data in terms of features is 1, and the similarity of other representative data samples is 0. In this case, the larger the statistical sum or average value (statistical similarity) of the similarity of the representative data samples, the more actual data the representative data sample is most matched, that is, the higher the degree of matching. In this example, the statistical similarity of the representative data sample c can be represented as follows:

[0064] [Formula 1]

[0065] Among them S p represents the actual data set, V c It represents the number of votes that the representative data sample c obtains that best matches the actual data, indicating its statistical similarity.

[0066] The inventors recognize that the statistical similarity in features between each representative data sample and the actual data may also contain some private information of the actual data.

[0067] Therefore, in some embodiments, the matching results of each representative data sample can be differential privacy processing A on the corresponding statistical similarity. ∈ (v c ) or other privacy protection processing. However, the present disclosure is not limited thereto. For example, the matching result can be directly represented by the corresponding statistical similarity.

[0068] A specific example of determining the matching result will be described in detail later in conjunction with FIG. 2 .

[0069] Those skilled in the art will readily understand that the definition of the matching result described above is only an example, and the present disclosure is not limited thereto. Those skilled in the art may also define the matching result in other ways in combination with the ideas disclosed in the present disclosure.

[0070] At step 1146 , representative data samples with relatively poor matching results are filtered out from the representative data sample set.

[0071] In some embodiments, the matching results of the representative data samples can be sorted, and the representative data samples with the lowest matching results can be filtered out from the representative data sample set. Alternatively, in some embodiments, representative data samples whose matching results are less than a preset threshold can be filtered out from the representative data sample set. For example, when the statistical similarity is obtained by the above formula 1, representative data samples whose matching results are 0 or close to 0 due to the addition of random perturbations can be filtered out.

[0072] Since the representative data samples selected in step 114 match the actual data in terms of features, it can be considered that the classes corresponding to the selected representative data samples are classes included in the actual data set. However, it is worth noting that since the number of classes in the actual data is unknown, the number of selected classes is not necessarily equal to the number of classes in the actual data.

[0073] Alternatively, in some embodiments, the received matching results are only related to the selected representative data samples, thus eliminating the need to perform step 1146.

[0074] The inventors recognize that, in addition to the selected representative data samples themselves, the approximate dataset may also include other open-world data that matches the selected representative data samples in features, to provide a large amount of data for model training, thereby ensuring the accuracy of the trained model.

[0075] Therefore, as shown in FIG1B , in some embodiments, obtaining the approximate dataset may further include supplementing and collecting relevant open-world data based on the screened representative data samples to expand the approximate dataset (step 116 ).

[0076] In some embodiments, the relevant open-world data is open-world data that matches the selected representative data samples in terms of features. Pre-trained models can also be used for clustering and feature extraction.

[0077] For example, for each selected representative data sample, data can be randomly sampled from the class of the representative data sample. Furthermore, data that is closer to the selected representative data sample can be preferentially selected based on the degree of feature matching.

[0078] In some embodiments, data can be collected from the open-world dataset D. Alternatively, data can be collected from another open-world dataset. For example, data can be collected from a larger open-world dataset to ensure data sufficiency and richness.

[0079] The inventors also recognized that the matching results of the selected representative data samples can generally reflect the distribution ratio of the actual data. Specifically, when the matching result value is positively correlated with the matching degree, the higher the matching result of the representative data sample, the higher the proportion of data in the class to which the representative data sample belongs in the actual data. Therefore, supplementing the collected data in accordance with the matching results is conducive to obtaining a similar dataset with a data distribution more similar to that of the actual dataset.

[0080] Therefore, in some embodiments, step 116 of expanding the approximate dataset includes collecting data that matches the characteristics of each selected representative data sample, in correlation with the matching result of the selected representative data sample. In cases where the value of the matching result is positively correlated with the degree of matching, "correspondingly" may include "proportionally" or "approximately proportionally." Thus, the closer the representative data sample matches the actual data, the more open-world data is collected for the class of the representative data sample.

[0081] Through step 116, an approximate dataset D consisting of a large amount of open world data and approximate to the actual dataset can be obtained. c .

[0082] As shown in FIG1A , in step 120 , an approximate dataset D is used. c to train the model.

[0083] In various embodiments, training the model in step 120 may include one or more of retraining the model, adjusting the model, updating the model, and the like.

[0084] Assume that the initial model is the pre-trained model f θ , where θ is the pre-trained model parameter and the training learning rate is r, then by using the approximate dataset D c , the model can be trained in the following ways:

[0085] [Formula 2]

[0086] Among them, B is from D c A batch of samples randomly drawn from .

[0087] By repeating the above fine-tuning process several times, the model can be trained.

[0088] In addition, in some embodiments, steps 110 - 120 may be repeatedly performed to adjust the trained model for updated actual data, thereby better adapting to actual conditions and reducing the workload required to retrain the model each time.

[0089] As shown in FIG1A , optionally, in some embodiments, the method 100 for training a model further includes performing model compression on the trained model (step 140 ).

[0090] Advantageously, using model compression technology to compress the model before model deployment can not only further reduce the model size, making the model more suitable for devices such as edge computing devices, but also further reduce model privacy leakage.

[0091] In some embodiments, the method 100 for training a model further includes distributing the trained model to relevant data processing devices (step 150 ).

[0092] Here, "correlated" with the actual dataset means that the data to be processed by the data processing device using the trained model has the same distribution as the actual dataset. For example, the visitor patterns of customers in different sections of a supermarket may vary, resulting in differences in the customer data collected by image sensors located in different sections. The data to be processed by the image sensors in a particular section has the same distribution as the actual customer dataset collected by the image sensors in that section. As a result, the trained model can perform better on the processed data.

[0093] In some embodiments, the method 100 for training a model further includes obtaining a trained model obtained by performing the above steps.

[0094] According to an embodiment of the present disclosure, the method for generating a model may include executing the steps of the method for training a model according to an embodiment of the present disclosure to generate a model.

[0095] Figure 2 illustrates a flowchart of an example of steps of a data processing method according to an embodiment of the present disclosure. The contents described above in conjunction with Figures 1A-1C can also be applied to corresponding features, and descriptions of some repeated contents will be omitted.

[0096] The data processing method according to the embodiment of the present disclosure can be executed by any apparatus including a processing device, for example, can be executed by a low-power computing device (such as a local edge device such as an image sensor).

[0097] As shown in FIG2A , according to an embodiment of the present disclosure, a data processing method 200 may mainly include the following steps:

[0098] In step 210, an actual data set is obtained;

[0099] At step 220 , a representative data sample set of an open-world dataset is received;

[0100] At step 230 , feature matching is performed on the representative data sample and the actual data; and

[0101] In step 240 , the matching result between the representative data sample and the actual data is returned.

[0102] In some embodiments, in step 210, the actual data set S can be obtained by directly collecting data. p For example, in some examples, real image data can be obtained by image capture. Alternatively, in some embodiments, the actual data set S can be obtained from an external device in a manner that is secure for privacy information. p .

[0103] In some embodiments, the data processing method 200 may further include extracting features from the acquired actual data set to facilitate feature matching.

[0104] In some embodiments, the above-mentioned method can be used by removing the pre-trained neural network f θ The neural network φ obtained by removing the last layer of the neural network φ and retaining the penultimate layer thereof is used as a feature extractor. Accordingly, the data processing method 200 may further include receiving the neural network φ. However, those skilled in the art will readily appreciate that the present disclosure is not limited thereto.

[0105] In some embodiments, in step 220, a representative data sample set S of the open world dataset D may be received from a high-computing computing device (such as a cloud server). q .

[0106] As mentioned above, in some embodiments, the representative data sample set S q It can be obtained by abstracting the open world dataset D. For example, the representative data sample set S q The cluster centers of a total of C clusters obtained by performing feature extraction and clustering using a feature extractor such as a neural network φ may be included, but the present disclosure is not limited thereto.

[0107] As described above, in some embodiments, the received representative data sample S q It may be a data sample represented by the extracted features, but the present disclosure is not limited thereto.

[0108] In some embodiments, performing feature matching (step 230 ) includes calculating statistical similarity in features between each representative data sample and each actual data.

[0109] For example, the statistical sum or average value of the similarity between each representative data sample and all actual data in terms of features can be calculated as statistical similarity. As described above, in some examples, the similarity between a representative data sample and any actual data can be represented by a numerical value, wherein the magnitude of the numerical value reflects the degree of matching with the actual data in terms of features. For example, the similarity between a representative data sample and any actual data can be represented by a discrete numerical value of 0 or 1, wherein for any actual data, the similarity of the representative data sample that best matches the actual data in terms of features is set to 1, and the similarity of other representative data samples is set to 0. In this case, the larger the statistical sum or average value (statistical similarity) of the similarity of the representative data sample, the more the representative data sample is matched with more actual data, i.e., the higher the degree of matching. In this example, the number of votes in the above formula 1 can be used to calculate the statistical similarity of the representative data sample. However, it will be readily understood by those skilled in the art that the present disclosure is not limited thereto.

[0110] In some embodiments, performing feature matching further includes performing differential privacy processing on statistical similarity.

[0111] The corresponding statistical similarity can be differentially privacy processed by A ∈ (v c ) to obtain matching results for each representative data sample. For example, a random Gaussian perturbation can be added to the statistical similarity of each representative data sample, and the corresponding differential privacy risk can be calculated. If the privacy loss is within an acceptable range, a matching result is obtained. However, the present disclosure is not limited to this. For example, other privacy protection technologies can be used for statistical similarity, or statistical similarity can be directly determined as the matching result.

[0112] Those skilled in the art will readily appreciate that the feature matching method described above is merely an example, and the present disclosure is not limited thereto. Those skilled in the art may also use various feature matching methods in combination with the concepts disclosed in the present disclosure.

[0113] In the case where the processed open-world data is unlabeled but the real data is labeled, in some embodiments, the data processing method 200 may further include labeling at least a portion of the representative data samples (step 250). For example, the representative data samples that match the real data in terms of features may be labeled. Therefore, in some embodiments, the data processing method 200 may further include receiving the representative data samples that match the real data in terms of features or information related thereto after returning the matching result between the representative data samples and the real data (step 240).

[0114] In some embodiments, labeling the representative data sample (step 250) may include respectively calculating the class similarity of the representative data sample with the actual data of each class in terms of features; and determining the class to which the representative data belongs based on the size sorting of the class similarities.

[0115] The class similarity between the representative data sample and the actual data of each class in terms of features can be represented by the number of actual data in the actual data of the class that matches the representative data sample in terms of features.

[0116] For example, a nearest neighbor pseudo-labeling method can be used to annotate representative data samples.

[0117] Assume that there are K types of actual data, for the representative data sample x, use represents the set of m nearest neighbors of x, where m is an optional hyperparameter, and can be annotated according to the following formula 3:

[0118] [Formula 3]

[0119] Among them, V k A represents the number of actual data of the kth class that meet the nearest neighbor requirements of the representative data sample x (number of votes), representing the class similarity of sample x. ∈ (v k ) performs noise privacy processing on the votes and calculates the privacy risk ∈. f(x) represents the class selected with the maximum number of votes, that is, the class to which the representative data x will be labeled.

[0120] The above labeling method can use local actual data labels to provide nearest neighbor pseudo labels for the filtered unlabeled open-world data, which effectively reduces the workload of the labeling task and improves the efficiency of model training.

[0121] Optionally, in some embodiments, the data processing method 200 may further include deploying a trained model obtained by executing the steps of the method for training a model according to an embodiment of the present disclosure to process data (step 260). Given that the similarity dataset used to obtain the trained model is similar to the actual dataset, and the data to be processed also has the same data distribution as the actual dataset, deploying the trained model will help improve the adaptability and accuracy of the model for actual application scenarios.

[0122] It is worth noting that the boundaries between the various steps in the method described above are merely illustrative. In actual operation, the various steps can be combined arbitrarily, or even combined into a single step. In addition, the order in which the various steps are performed is not limited by the order in which they are described, and some steps can be omitted. The operational steps of the various embodiments can also be combined with each other in any appropriate order to similarly implement more or fewer operations than described.

[0123] The following describes the system, training device, and data processing device for training a model according to an embodiment of the present disclosure in conjunction with Figure 3. For ease of understanding, Figure 3 also illustrates the main functional modules of each device and their information interaction.

[0124] According to an embodiment of the present disclosure, a system 300 for training a model may include a training device 310 and a data processing device 320. In some embodiments, the training device 310 is a high-power computing device (such as a cloud server), and the data processing device 320 is a low-power computing device (such as a local edge device such as an image sensor).

[0125] In some embodiments, the training device 310 can send a representative data sample set abstracted from the open world data set to the data processing device 320, generate a similar data set based on the returned matching results with the actual data, and use the similar data set to train the model, thereby improving the performance of the model in the actual data. Among them, the open world can advantageously provide a large number of data samples. Accordingly, the data processing device 320 only needs to perform feature matching processing on the actual data locally, without sharing the actual data or performing model training with high computing power requirements. The trained model can be deployed on the data processing device 320. As a result, the process of training the model will not leak the privacy information of the actual data, and can simultaneously ensure the accuracy and practical application value of the trained model.

[0126] In particular, in various embodiments, the training device 310 may be configured to perform the steps of the method for training a model according to an embodiment of the present disclosure. The contents described above in conjunction with FIG. 1A-FIG . 1C and FIG. 2 may also be applicable to the corresponding features, and descriptions of some repeated contents will be omitted.

[0127] In an embodiment of the present disclosure, as shown in FIG3 , a training device 310 may include:

[0128] A training data acquisition module 312 is configured to acquire an approximate dataset consisting of open-world data that is similar to the actual dataset; and

[0129] The training module 314 is configured to train the model using the approximate dataset.

[0130] In some embodiments, the training data acquisition module 312 may include an abstraction submodule 3122. The abstraction submodule 3122 may be configured to perform abstraction processing on the open-world dataset to obtain a representative data sample set. For example, the abstraction submodule 3122 may be configured to utilize a pre-trained model to perform feature extraction and clustering on the data in the open-world dataset.

[0131] In some embodiments, the training data acquisition module 312 may include a screening submodule 3124. The screening submodule 3124 may be configured to screen representative data samples that match the actual data in terms of features from the representative data sample set. For example, the screening submodule 3124 may be configured to interact with a data processing device that can access the actual data to determine representative data samples that match the actual data in terms of features, wherein the interacted information does not contain the actual data. That is, the processing of the actual data is only performed locally, and the actual data is not uploaded. It is worth noting that although the data processing device illustrated in FIG3 that interacts with the screening submodule 3124 for information is the data processing device 320 according to an embodiment of the present disclosure, the present disclosure is not limited thereto. It is easy for those skilled in the art to understand that the data processing device may be any data processing device that collects and / or stores actual data so as to be able to securely access the actual data without leaking privacy information.

[0132] In some examples, the screening submodule 3124 can be configured to send a representative data sample set to a data processing device and receive matching results for each representative data sample from the data processing device. The matching result for each representative data sample can be based on the statistical similarity between the representative data sample and the actual data in terms of features. The screening submodule 3124 can then be configured to filter out representative data samples with relatively poor matching results from the representative data sample set.

[0133] Alternatively, in some embodiments, the screening submodule 3124 may only receive matching results for representative data samples that have been screened out. That is, relatively poor matching results have been filtered out. Thus, the screening submodule 3124 may no longer perform the filtering operation.

[0134] In some embodiments, the training data acquisition module 312 may further include an expansion submodule 3126. The expansion submodule 3126 may be configured to supplement the collection of relevant open-world data based on the selected representative data samples, thereby expanding the approximate dataset. For example, the expansion submodule 3126 may be configured to collect data that matches the characteristics of each selected representative data sample in accordance with the matching results of the selected representative data sample.

[0135] In some embodiments, the training device 310 may further include a pre-training module 316. The pre-training module 316 may be configured to obtain a pre-trained model. Thus, the training module 314 may be configured to adjust the pre-trained model using the approximate dataset.

[0136] In some embodiments, training device 310 may further include a model distribution module 318. Model distribution module 318 is configured to distribute the trained model to devices associated with the actual dataset. Here, "associated with" the actual dataset means that the data to be processed by the device using the trained model has the same distribution as the actual dataset. For example, the associated device may be a data processing device that interacts with training device 310 to provide matching results, but the present disclosure is not limited to this.

[0137] Optionally, in some embodiments, the training device 310 may further include a model compression module (not shown). The model compression module is configured to compress the model using a model compression technology.

[0138] In particular, in various embodiments, the data processing device 320 may be configured to perform the steps of the data processing method according to the embodiments of the present disclosure. The contents described above in conjunction with FIG. 1A-FIG . 1C and FIG. 2 may also be applicable to the corresponding features, and the description of some repeated contents will be omitted.

[0139] In an embodiment of the present disclosure, as shown in FIG3 , the data processing device 320 may include:

[0140] The data acquisition module 322 is configured to obtain an actual data set;

[0141] An interaction module 324 configured to receive a representative data sample set of an open-world dataset; and

[0142] a feature matching module 326 configured to perform feature matching between the representative data sample and the actual data;

[0143] The interaction module 324 is further configured to return a matching result between the representative data sample and the actual data.

[0144] In some embodiments, the data acquisition module 322 may also be configured to perform feature extraction on the acquired actual data set.

[0145] In some embodiments, the data processing device 320 may further include a deployment module 328. The deployment module 328 may be configured to deploy the trained model obtained by executing the steps of the method for training a model according to an embodiment of the present disclosure, that is, the trained model obtained by the training device 310, to process data.

[0146] In some embodiments, feature matching module 326 can be configured to calculate the statistical similarity between each representative data sample and each actual data in terms of features, and determine a matching result based on the statistical similarity. In some embodiments, feature matching module 326 can also be configured to perform differential privacy processing on the statistical similarity. The relevant content has been discussed in detail above and will not be repeated here.

[0147] Then, the data processing device 320 can return the matching result through the interaction module 324. Therefore, the data processing device 320 only needs to perform feature matching processing on the actual data locally, without sharing the actual data or performing model training with high computing power requirements.

[0148] In some embodiments, the data processing device 320 may further include a labeling module (not shown). The labeling module may be configured to label at least part of the representative data samples.

[0149] In some embodiments, the annotation module can be configured to calculate the class similarity between the representative data sample and the actual data of each class in terms of features; and to determine the class to which the representative data belongs based on the order of the class similarities. In some embodiments, the annotation module can also be configured to perform differential privacy processing on the class similarities before sorting. This has been discussed in detail above and will not be repeated here.

[0150] Although the training device 310 and the data processing device 320 are depicted together in Figure 3, those skilled in the art will readily appreciate that the present disclosure is not limited thereto. In fact, in many cases, the training device 310 is set in the cloud, while the data processing device 320 is set locally.

[0151] In addition, those skilled in the art will appreciate that, although FIG3 illustrates only one data processing device 320, the number of data processing devices is not limited thereto. For example, in some embodiments, the system 300 may include multiple data processing devices 320. The training device 310 may utilize each of these data processing devices 320 to perform feature matching to determine similar data sets, and distribute and deploy the model trained on the similar data sets to the corresponding devices. Here, the "corresponding" device means that the distribution of the data to be processed by the device deploying the model is similar to the distribution of the similar data sets used to train the model.

[0152] The system 300 illustrated in FIG3 can be applied in various scenarios, especially business scenarios of various computer vision classification tasks.

[0153] Figures 4A-4C illustrate examples of learning scenarios for a solution for training a model according to an embodiment of the present disclosure. For ease of understanding, Figures 4A-4C simplify some functional modules and their interactions, but those skilled in the art will readily understand that the present disclosure is not limited thereto.

[0154] In the example shown in FIG4A , labels of local actual data are used to label unlabeled open-world data, for example, by providing nearest neighbor pseudo-labels. Thus, the model training scheme according to an embodiment of the present disclosure can perform semi-supervised learning to train the model. In this example, feature matching can be used to filter out cloud data with large distribution differences, reducing the amount of data that needs to be labeled and reducing the noise of the nearest neighbor pseudo-labeling algorithm caused by out-of-distribution samples, thereby improving the efficiency of semi-supervised learning.

[0155] In the example shown in Figure 4B, the processed open-world data is labeled. Thus, the model training scheme according to the embodiments of the present disclosure can first pre-train the model on open-world data (such as ImageNet) that is different from the actual data distribution, and then perform supervised adjustment training.

[0156] In the example shown in Figure 4C, both the open-world data and the real-world data being processed are unlabeled. Thus, the model training solution according to the disclosed embodiments can utilize outsourced services for manual labeling after screening, thereby enabling semi-supervised learning with a small number of labeled samples. Advantageously, by filtering out data that requires labeling, the number of public samples requiring labeling and the labor cost can be reduced.

[0157] Those skilled in the art should understand that the application of the scheme for training the model is not limited to the above examples.

[0158] The embodiments of the present disclosure also provide a computer-readable storage medium storing one or more instructions, which, when executed by a processor, can cause the processor to execute the steps of the method for training a model or the data processing method in the above embodiments.

[0159] The embodiments of the present disclosure also provide a computer program product comprising one or more instructions, which, when executed by a processor, can cause the processor to perform the steps of the method for training a model or the data processing method in the above embodiments.

[0160] It should be understood that the instructions in the computer-readable storage medium according to the embodiments of the present disclosure can be configured to perform operations corresponding to the above-mentioned system and method embodiments. When referring to the above-mentioned system and method embodiments, the embodiments of the computer-readable storage medium are clear to those skilled in the art and are therefore not described again. Computer-readable storage media for carrying or including the above-mentioned instructions also fall within the scope of the present disclosure. Such computer-readable storage media may include, but are not limited to, floppy disks, optical disks, magneto-optical disks, memory cards, memory sticks, and the like.

[0161] The embodiments of the present disclosure also provide various devices including components or units for executing the steps of the method for training a model or the data processing method in the above embodiments.

[0162] It should be noted that the above-mentioned various components or units are only logical modules divided according to the specific functions implemented by them, rather than being used to limit specific implementation methods, such as can be implemented in the form of software, hardware or a combination of software and hardware. In actual implementation, the above-mentioned various components or units can be implemented as independent physical entities, or can also be implemented by a single entity (for example, a processor (CPU or DSP, etc.), an integrated circuit, etc.). For example, a plurality of functions included in a unit in the above embodiments can be implemented by separate devices. Alternatively, a plurality of functions implemented by a plurality of units in the above embodiments can be implemented by separate devices respectively. In addition, one of the above functions can be implemented by a plurality of units.

[0163] Additionally, it should be understood that the aforementioned series of processes and devices can also be implemented via software and / or firmware. When implemented via software and / or firmware, the programs constituting the software are installed from a storage medium or network onto a computer with dedicated hardware, such as the general-purpose computer 500 shown in FIG5 . When various programs are installed, the computer can perform various functions, etc. FIG5 illustrates an example block diagram of a computer that can be implemented as a training device, application device, and system according to an embodiment of the present disclosure.

[0164] 5 , a central processing unit (CPU) 501 executes various processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage section 508 to a random access memory (RAM) 503. In the RAM 503, data required when the CPU 501 executes various processes and the like is also stored as needed.

[0165] The CPU 501, the ROM 502, and the RAM 503 are connected to one another via a bus 504. An input / output interface 505 is also connected to the bus 504.

[0166] The following components are connected to the input / output interface 505: an input section 506 including a keyboard, a mouse, etc.; an output section 507 including a display such as a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN card, a modem, etc. The communication section 509 performs communication processing via a network such as the Internet.

[0167] A drive 510 is also connected to the input / output interface 505 as needed. A removable medium 511 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is mounted on the drive 510 as needed so that a computer program read therefrom is installed into the storage section 508 as needed.

[0168] In the case of realizing the above-described series of processing by software, a program constituting the software is installed from a network such as the Internet or a storage medium such as the removable medium 511 .

[0169] Those skilled in the art will appreciate that such storage media are not limited to the removable medium 511 shown in FIG5 , which stores the program and is distributed separately from the device to provide the program to the user. Examples of the removable medium 511 include magnetic disks (including floppy disks (registered trademark)), optical disks (including compact disk read-only memories (CD-ROMs) and digital versatile disks (DVDs)), magneto-optical disks (including minidiscs (MDs) (registered trademark)), and semiconductor memories. Alternatively, the storage medium may be a ROM 502, a hard disk included in the storage section 508, or the like, in which the program is stored and distributed to the user together with the device containing the program.

[0170] The exemplary embodiments of the present disclosure are described above with reference to the accompanying drawings, but the present disclosure is certainly not limited to the above examples. Those skilled in the art may obtain various changes and modifications within the scope of the appended claims, and it should be understood that these changes and modifications will naturally fall within the technical scope of the present disclosure.

[0171] Although the present disclosure and its advantages have been described in detail, it should be understood that various changes, substitutions and transformations can be made without departing from the spirit and scope of the present disclosure as defined by the appended claims. Moreover, the terms "comprises," "comprising," or any other variations thereof in the embodiments of the present disclosure are intended to cover non-exclusive inclusions, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article, or device. In the absence of further restrictions, an element defined by the statement "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0172] Embodiments of the present disclosure also include:

[0173] 1. A method for training a model, comprising:

[0174] Obtain an approximate dataset consisting of open-world data that is similar to the actual dataset; and

[0175] Use an approximate dataset to train the model.

[0176] 2. The model training method according to item 1, wherein obtaining the approximate dataset comprises:

[0177] Abstract open-world datasets to obtain representative data samples; and

[0178] Select representative data samples from the representative data sample set that match the actual data in terms of characteristics.

[0179] 3. The method according to item 2, wherein obtaining the approximate dataset further comprises:

[0180] Based on the selected representative data samples, additional relevant open world data is collected to expand the approximate data set.

[0181] 4. The method of item 2, wherein abstracting the open world dataset comprises:

[0182] Leverage pre-trained models to perform feature extraction and clustering on data from open-world datasets.

[0183] 5. The method according to item 2, wherein screening a representative data sample that matches the actual data in terms of characteristics comprises:

[0184] Information is exchanged with a data processing device that can access actual data to determine a representative data sample that matches the actual data in terms of characteristics, wherein the exchanged information does not include the actual data.

[0185] 6. The method according to item 3, wherein expanding the approximate dataset comprises:

[0186] In association with the matching result of each selected representative data sample, data matching the representative data sample in terms of features is collected, wherein the matching result of each representative data sample is based on the statistical similarity in features between the representative data sample and respective actual data.

[0187] 7. The method according to item 1, further comprising obtaining a pre-trained model, and

[0188] The use of the approximate dataset to train the model includes adjusting the pre-trained model using the approximate dataset.

[0189] 8. A data processing method, comprising:

[0190] Get the actual dataset;

[0191] Receive a representative data sample set of an open-world dataset;

[0192] Matching features between representative data samples and actual data; and

[0193] Returns the matching results of the representative data sample and the actual data.

[0194] 9. The method according to item 8 further includes: deploying the trained model obtained by executing the method according to any one of items 1-7 to process data.

[0195] 10. The method according to item 8, wherein performing feature matching comprises:

[0196] Calculate the statistical similarity between each representative data sample and each actual data in terms of features, and determine the matching result based on the statistical similarity.

[0197] 11. The method according to item 10, wherein performing feature matching further comprises:

[0198] Perform differential privacy on statistical similarity.

[0199] 12. The method according to item 8, further comprising:

[0200] Annotate at least some representative data samples.

[0201] 13. The method according to item 12, wherein labeling the representative data comprises:

[0202] Calculate the class similarity between the representative data sample and the actual data of each class in terms of features; and

[0203] Based on the size sorting of class similarity, the class to which the representative data belongs is determined.

[0204] 14. The method according to item 13, wherein labeling the representative data further comprises:

[0205] Before ranking, differential privacy is performed on the class similarities.

[0206] 15. A training device comprising:

[0207] A training data acquisition module is configured to acquire an approximate dataset consisting of open-world data that is similar to the actual dataset; and

[0208] The training module is configured to train the model using the approximate dataset.

[0209] 16. A data processing device comprising:

[0210] a data acquisition module configured to obtain an actual data set;

[0211] an interaction module configured to receive a representative data sample set of an open-world dataset; and

[0212] a feature matching module configured to perform feature matching between the representative data sample and the actual data;

[0213] The interaction module is further configured to return a matching result between the representative data sample and the actual data.

[0214] 17. A system for training a model, comprising:

[0215] The training device according to item 15; and

[0216] The data processing device according to item 16.

[0217] 18. A computer-readable storage medium having one or more instructions stored thereon, which, when executed by a processor, causes the processor to perform the steps of the method according to any one of items 1-7 and / or the steps of the method according to any one of items 8-14.

[0218] 19. A computer program product comprising one or more instructions, which, when executed by a processor, causes the processor to perform the steps of the method according to any one of items 1 to 7 and / or the steps of the method according to any one of items 8 to 14.

[0219] 20. A method for generating a model, comprising: performing the steps of the method according to any one of items 1 to 7 to generate a model.

Claims

1. A method for training a model, comprising: Obtain an approximate dataset consisting of open-world data that is similar to the actual dataset; as well as Use an approximate dataset to train the model.

2. The method according to claim 1, wherein Obtaining an approximate dataset includes: Abstract open-world datasets to obtain representative data samples; and Select representative data samples from the representative data sample set that match the actual data in terms of characteristics.

3. The method according to claim 2, wherein: Obtaining an approximate dataset also includes: Based on the selected representative data samples, additional relevant open world data is collected to expand the approximate data set.

4. The method according to claim 2, wherein: Abstract processing of open-world datasets includes: Leverage pre-trained models to perform feature extraction and clustering on data from open-world datasets.

5. The method according to claim 2, wherein: The representative data samples that match the actual data in characteristics include: Information is exchanged with a data processing device that can access actual data to determine a representative data sample that matches the actual data in terms of characteristics, wherein the exchanged information does not include the actual data.

6. The method according to claim 3, wherein: The extended approximate dataset includes: In association with the matching result of each selected representative data sample, data matching the representative data sample in terms of features is collected, wherein the matching result of each representative data sample is based on the statistical similarity in features between the representative data sample and respective actual data.

7. The method of claim 1, further comprising obtaining a pre-trained model, and in, Training a model using an approximate dataset involves fine-tuning a pre-trained model using the approximate dataset.

8. A data processing method, comprising: Get the actual dataset; Receive a representative data sample set of an open-world dataset; Perform feature matching between representative data samples and actual data; as well as Returns the matching results of the representative data sample and the actual data.

9. The method of claim 8, further comprising: The trained model obtained by executing the method according to any one of claims 1 to 7 is deployed to process data.

10. The method of claim 8, wherein: Feature matching includes: Calculate the statistical similarity between each representative data sample and each actual data in terms of features, and determine the matching result based on the statistical similarity.

11. The method according to claim 10, wherein: Feature matching also includes: Perform differential privacy on statistical similarity.

12. The method of claim 8, further comprising: Annotate at least some representative data samples.

13. The method of claim 12, wherein: Annotating representative data includes: Calculate the class similarity between the representative data sample and the actual data of each class in terms of features; and Based on the size sorting of class similarity, the class to which the representative data belongs is determined.

14. The method of claim 13, wherein: Labeling representative data also includes: Before ranking, differential privacy is performed on the class similarities.

15. A training device comprising: a training data acquisition module, configured to acquire an approximate dataset consisting of open-world data that is similar to the actual dataset; as well as The training module is configured to train the model using the approximate dataset.

16. A data processing device comprising: a data acquisition module configured to obtain an actual data set; an interaction module, configured to receive a representative set of data samples of an open-world dataset; as well as a feature matching module configured to perform feature matching between the representative data sample and the actual data; The interaction module is further configured to return a matching result between the representative data sample and the actual data.

17. A system for training a model, comprising: The training device according to claim 15; as well as The data processing apparatus according to claim 16.

18. A computer-readable storage medium having one or more instructions stored thereon, which, when executed by a processor, causes the processor to perform the steps of the method according to any one of claims 1 to 7 and / or the steps of the method according to any one of claims 8 to 14.

19. A computer program product comprising one or more instructions, which, when executed by a processor, cause the processor to perform the steps of the method according to any one of claims 1 to 7 and / or the steps of the method according to any one of claims 8 to 14.

20. A method for generating a model, comprising: The steps of the method according to any one of claims 1 to 7 are performed to generate a model.

Images