Data training method and device for local training set, equipment and storage medium

Abstract

The invention relates to the technical field of machine learning, in particular to a data training method and device for a local training set, equipment and a storage medium. The method comprises the following steps: acquiring a first training set and a test set of local online learning, selecting one piece of test data in the test set, calculating the distance from at least one piece of training data in the first training set to a test point, and sorting all the calculated distances; and filtering a plurality of training data in the first training set to generate a second training set, predicting test data in the test set according to Gaussian process regression and at least one training data in the second training set, outputting a prediction result, and transmitting the prediction result to the server. And integrating the prediction results by the server to obtain a global machine learning model. According to the method, the correctness of an output prediction result of new test data is ensured, the calculation efficiency is improved, extreme training data of the prediction result is removed, and the robustness of a local prediction result on data poisoning attack is ensured.

Description

Technical Field

[0001] This application relates to the technical field of machine learning, and particularly to a data training method, device, equipment, and storage medium for a local training set. Background Art

[0002] The Internet of Things (IoT) generates a large amount of distributed data, and the training method for this data is to use this data to train a model on a server. However, during the training process, problems such as communication efficiency and computing efficiency will become prominent. For example, in the processing or computing process of a car's in-vehicle computer for several hours, hundreds of Gb of data can be generated, and these data will cause a huge burden during transmission to the server and the computing process.

[0003] Therefore, distributed machine learning emerged. Distributed machine learning uses a deep neural network as a machine learning model, and it has achieved unprecedented success in many applications, such as model classification and pattern recognition. However, deep learning is mainly limited to offline learning. To overcome this limitation, an online learning method was proposed. This method uses Gaussian process regression (GPR). However, Gaussian process regression is a non-hyperparameter statistical probability model. Therefore, during the process of using this model for prediction and inference, the amount of training data will continuously increase. When individual training data is subjected to a data poisoning attack, some data will deviate severely from the true value of the data. As a result, when the worker machine makes a local prediction, the local prediction expectation and prediction variance obtained will be inaccurate, and the robustness against data poisoning attacks cannot be achieved. Summary of the Invention

[0004] In view of this, this application provides a data training method, device, equipment, and storage medium for a local training set to solve the problem that when the worker machine makes a local prediction, the local prediction expectation and prediction variance obtained are inaccurate, and the robustness against data poisoning attacks cannot be achieved.

[0005] In a first aspect, this application provides a data training method for a local training set, and the method includes:

[0006] Obtain a first training set and a test set for local online learning. The first training set includes multiple training data, and the test set includes multiple test data;

[0007] Select a test data from the test set, calculate the distance between at least one training data in the first training set and the test point with the test data as the test point, and sort all the calculated distances;

[0008] Filter multiple training data in the first training set according to distance sorting to generate a second training set, where the second training set includes at least one training data;

[0009] Predict the test data in the test set according to Gaussian process regression and at least one training data in the second training set, and output a prediction result, where the prediction result includes the prediction expectation and prediction variance of at least one test data;

[0010] Transmit the prediction expectation and prediction variance to the server so that the server can integrate the global machine learning model according to the prediction result.

[0011] In a second aspect, the present application provides a data training device for a local training set, and the device includes:

[0012] An acquisition module for acquiring a first training set and a test set for local online learning, where the first training set includes multiple training data and the test set includes multiple test data;

[0013] A processing module for selecting a test data in the test set, calculating the distance between at least one training data in the first training set and the test point with the test data as the test point, and sorting all the calculated distances;

[0014] A filtering module for filtering multiple training data in the first training set according to distance sorting to generate a second training set, where the second training set includes at least one training data;

[0015] A prediction module for predicting the test data in the test set according to Gaussian process regression and at least one training data in the second training set, and outputting a prediction result, where the prediction result includes the prediction expectation and prediction variance of at least one test data;

[0016] A sending module for transmitting the prediction expectation and prediction variance to the server so that the server can integrate the global machine learning model according to the prediction result.

[0017] In a third aspect, the present application provides a computer device, including: a memory and a processor, which are communicatively connected to each other, the memory stores computer instructions, and the processor executes the data training method for the local training set described in the first aspect above by executing the computer instructions.

[0018] In a fourth aspect, the present application provides a computer-readable storage medium, on which computer instructions are stored, and the computer instructions are used to make a computer execute the data training method for the local training set described in the first aspect above.

[0019] In addition, the present application provides a computer program product, including computer instructions for causing a computer to execute the data training method of the local training set described in the first aspect above.

[0020] The data training method, device, equipment, and storage medium provided by the present application calculate the distances between at least one training data in the first training set and a test point, sort the calculated distances, filter a plurality of training data in the first training set according to the distance sorting to generate a feasible training data subset, that is, the second training set, and then calculate the low-dimensional covariance matrix of Gaussian process regression by using the correlation between the training data in the constructed second training set, predict the test data in the test set, and output the prediction result. This method not only ensures the correctness of the output prediction result for new test data, but also improves the calculation efficiency and saves calculation time.

[0021] In addition, by filtering the training data in the first training set through distance sorting, extreme training data that may affect the prediction result is removed, thereby ensuring the robustness of the local prediction result against data poisoning attacks. Description of the Drawings

[0022] In order to more clearly illustrate the specific embodiments of the present application or the technical solutions in the prior art, the following will briefly introduce the drawings required for use in the description of the specific embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0023] Figure 1 is a schematic diagram of Gaussian process regression and nearest neighbor Gaussian process regression according to an embodiment of the present application;

[0024] Figure 2 is a schematic flowchart of a data training method for a local training set according to an embodiment of the present application;

[0025] Figure 3 is a schematic flowchart of another data training method for a local training set according to an embodiment of the present application;

[0026] Figure 4 is a schematic diagram of filtering to generate a second training set according to an embodiment of the present application;

[0027] Figure 5 is a structural block diagram of a data training device according to an embodiment of the present application;

[0028] Figure 6 is a schematic hardware structure diagram of a computer device according to an embodiment of the present application. Detailed implementation manners

[0029] To make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are some, but not all, of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative efforts shall fall within the protection scope of the present application.

[0030] It should be noted that in the description of the present application, the terms "include", "comprise" or any other variation thereof are intended to cover a non-exclusive inclusion, such that a process, method, article or device including a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. The terms "first", "second", etc. in the present application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0031] To enable those skilled in the art of the present technology to better understand the solution of the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific implementation manners.

[0032] Describe the specific application environment architecture or specific hardware architecture on which the execution of the data training method for the local training set depends, and describe the specific application environment architecture or specific hardware architecture, as well as the resulting technical problems.

[0033] The technical solution of the present application can be applied to the field of machine learning, especially distributed machine learning. Generally, distributed machine learning uses a deep neural network as a machine learning model, but deep learning is mainly limited to offline learning. In practical applications, the worker machines will obtain data streams in actual applications. Therefore, online learning will become a means to solve this problem.

[0034] Gaussian process regression (GPR) is an effective means. In fact, the GPR model can be equivalent to a machine learning model in a certain sense, including: Bayesian linear model, multi-layer neural network. According to the central limit theorem, assuming that the weights in the neural network follow a Gaussian distribution, as the width of the neural network approaches infinity, such a neural network is equivalent to Gaussian process regression. However, Gaussian process regression is a non-hyperparameter statistical probability model. Unlike traditional learning models, such as linear regression, logistic regression, and neural networks, which need to solve an optimization problem to minimize the loss function to obtain the optimal model parameters, Gaussian process regression does not need to solve an optimization problem.

[0035] Based on the given training data and test input, the prediction of Gaussian process regression is divided into two steps: "inference" and "prediction". Among them, in the "inference" process, it is assumed that the function to be learned follows a Gaussian process, and the Gaussian prior probability distribution of the model is given. Then, using the observed values and Bayesian rules, the Gaussian posterior probability distribution of the model is obtained. However, when using Gaussian process regression for local training, the following problems generally occur:

[0036] First, if each working machine (denoted by the identifier i) uses all the data in the local data set for training, then as the amount of data increases and the data dimension increases, the computational efficiency will become low and it will not be timely when calculating the kernel function. Although Gaussian process regression can perform online prediction for test inputs, since each working machine needs to use all the training data in the data set for each test input and calculate the inverse of the covariance matrix during prediction, and the computational complexity of this operation is where n x is the dimension of the training data. As time goes by, the dimension n x of the training data will continuously increase, which makes it difficult to guarantee real-time performance.

[0037] Second, generally in the supervised learning framework, similar input data will correspond to similar target values. If the training data is very close to the test data, then the training data will provide more effective and useful information for this test point. In the context of Gaussian processes, this similarity is reflected in the covariance matrix. That is to say, the covariance matrix defines proximity or similarity.

[0038] To reduce the computational complexity, the nearest neighbor method is selected to handle the inference and prediction processes. Although the training time will be significantly reduced, for test points, only one corresponding projected training point is used, so there is no correlation between the training data. Therefore, using Gaussian process regression, especially for data points outside the training set, the prediction results of the test data are inaccurate. For example, as Figure 1 shown, for a trigonometric function curve, the training data is uniformly sampled in the interval [0, 1], and the test set is sampled in the interval [0, 1.2].[[]END]

[0039] In the Figure 1 schematic diagram, the black dashed line represents the prediction of the test output of the test set by Gaussian process regression GPR using all the training data, and the red solid line represents the estimation of the test output by the nearest neighbor Gaussian process regression NNGPR method. From Figure 1As can be seen, the prediction results of the nearest neighbor method are not accurate enough, especially outside the training set, such as in the range of [1, 1.2], because the training data is not sufficient to describe the correlation between data, and thus cannot provide accurate predictions.

[0040] Third, in an independently and identically distributed dataset, if individual training points in the training data are subject to data poisoning attacks, causing some data to deviate significantly from the true values of the data, then when the worker node makes local predictions, the obtained local prediction expectations and prediction variances will not be accurate, thus making it impossible to achieve robustness against data poisoning attacks.

[0041] To solve the above problems, in a distributed machine learning framework, this application proposes an extreme training data removal scheme based on the trimmed mean method to implement the local online learning algorithm of the worker node. The learning model uses Gaussian Process Regression (GPR) to process streaming data.

[0042] The embodiments of this application provide a data training method for a local training set. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0043] In this embodiment, a data training method for a local training set is provided, and this method can be executed by the worker node. Specifically, the worker node can be a programmable and general-purpose computer, designed to provide powerful computing and data processing capabilities.

[0044] Optionally, the worker node can be deployed locally. Figure 2 is a flowchart of the data training method for a local training set according to the embodiments of this application, as Figure 3 shown, this method includes:

[0045] Step S101: Obtain the first training set and test set for local online learning.

[0046] Among them, the first training set includes multiple training data, which are used to train the prediction model and make predictions on the test data. The test set includes multiple test data.

[0047] The worker node can obtain the first training set and test set for local online learning simultaneously or successively, and this embodiment does not limit this.

[0048] Step S102: Select a test data from the test set, calculate the distances between at least one training data in the first training set and the test point using the test data as the test point, and sort all the calculated distances.

[0049] Specifically, the principle for selecting test data can be random selection, or selecting one test data or a set of test data in sequence, and the selected set or one test value is used as a test point. In this embodiment, the purpose of using the test data as a test point is to locate the position of the test data and calculate the distance between the training points corresponding to the training data.

[0050] Each training data in the first training set can be used as a training point, and each training point corresponds to a coordinate. The distance between each training point and the test point is calculated through the position coordinates of each training point and the test point.

[0051] And at least one calculated distance is sorted. More specifically, the sorting order can be in descending order or ascending order, and this embodiment does not limit this.

[0052] Similarly, in the same way, distance calculations are performed on other test data in the test set, and all the calculated distances are sorted.

[0053] Step S103: Filter multiple training data in the first training set according to the distance sorting to generate a second training set.

[0054] Among them, the second training set includes at least one training data.

[0055] The filtering principle can be to filter out training data with a distance exceeding a certain threshold, or less than a certain threshold, or it can also be to filter out training data with a distance within a certain preset range. The filtered training data may be data that has suffered a poisoning attack or data with security risks.

[0056] Step S104: Predict the test data in the test set according to the Gaussian process regression and at least one training data in the second training set, and output the prediction result.

[0057] Among them, the prediction result includes the prediction expectation and prediction variance of at least one test data.

[0058] Specifically, the prediction principle is as follows: First, define the objective function as Among them, is the n x dimensional input space, and R represents real numbers. Optionally, assume that the output y is one-dimensional data, that is At time t, given The corresponding output prediction result is:

[0059] y = f(x) + ε (1)

[0060] Among them, ε represents a Gaussian noise that follows a Gaussian probability distribution with a mean of ε = 0 and a variance of , that is Define a training set in the following form Among them is the input data set, y = [y(1), y(2),..., y(n s )] Τ is the column vector that aggregates the outputs. The goal of Gaussian process regression is: using the training set to approximate the function f on the test data set .

[0061] Define a symmetric positive semi-definite kernel function, and the kernel function can be expressed as "k".

[0062]

[0063] That is, ∫k(x, x')f(x)f(x')dν(x)dν(x') ≥ 0 (3)

[0064] Among them, ν represents the measure. Let return a column vector such that its i-th element is equal to f(x(i)). Assume that the function f is a sample from the prior probability distribution of the Gaussian process, and the mean function of this prior distribution is μ, and the kernel function is k. The training output and the test output follow the joint probability distribution as:

[0065]

[0066] Among them, and return the vector composed of μ(x(i)) and μ(x * (i)), return a matrix such that the element in the i-th row and the j-th column is k(x(i), x * (j)).

[0067] Using the properties of the Gaussian process, Gaussian process regression uses the training set to predict the output of the test data set . This output still follows a normal distribution, that is At this time, the new prediction expectation and the prediction variance cov(f * ) are obtained. Specifically, it includes the following:

[0068]

[0069] It also includes: a distributed local model prediction algorithm based on random training points.

[0070] In distributed machine learning, consider that there are n worker nodes in a network. Define this set as At each moment t, each worker node utilizes local training data to predict the output of the function for a test input . Here, the locally predicted value trained by each worker node is

[0071]

[0072] Step S105: Transmit the prediction expectation and prediction variance to the server so that the server can integrate to obtain a global machine learning model based on the prediction results.

[0073] In the context of federated learning, each worker node will send the trained local prediction results, including the prediction expectation and the prediction variance cov i (f * ) to the server so that the server can integrate to obtain a global machine learning model based on the prediction results and the prediction results reported by other worker nodes.

[0074] The method provided in this embodiment calculates the distances between all training points and the test point in the training set for a test input on the client side locally and sorts them; then constructs a feasible local training point set, i.e., the second training set, by removing a certain proportion of the maximum distance values and a certain proportion of the minimum distance values, and calculates the inverse of the low-dimensional covariance matrix using the training data in the subset; finally, calculates the posterior prediction expectation and prediction variance to obtain the prediction results.

[0075] The local learning method provided in this embodiment first improves the correlation between training data, thereby improving the correctness of the test output; secondly, counteracts the data poisoning attack in the training data set, thus ensuring the tolerance of the local training results of each worker node to the poisoning attack.

[0076] The process of the above step S103 in this embodiment will be described in detail below. The method of step S103 in this embodiment can be called the "distributed training using the trimmed mean method", as Figure 3 shown. The above step S103: Filter multiple training data in the first training set according to the distance sorting to generate the second training set, specifically including:

[0077] Step S1031: Obtain the first ratio, where the first ratio is the proportion of the data attacked in the first training set to all the training data.

[0078] Step S1032: Calculate the number of training points to be filtered among the sorted distances according to the first ratio.

[0079] Among them, the number of training points to be filtered includes at least the sum of the training points with the maximum distance and the training points with the minimum distance.

[0080] Specifically, in one implementation, count the number of training points in the first training set whose distance from the test point is within the first distance range according to the sorted distances, and count the number of training points whose distance from the test point is within the second distance range, where the first distance range is greater than the second distance range;

[0081] Calculate the sum of the number of training points within the first distance range and the number of training points within the second distance range as the number of training points to be filtered.

[0082] Step S1033: Subtract the number of training points to be filtered from the number of training points in the first training set, and the remaining training points form the second training set.

[0083] Specifically, first, construct a training subset. Figure 4 This is a schematic diagram for filtering to generate the second training set provided in this embodiment. This diagram is also called the "schematic diagram of constructing a training subset by the trimmed mean method" and is used to ensure the effectiveness of the training results of each worker against poisoning attacks. More specifically, the method provided in this embodiment includes:

[0084] Based on the construction of the distance set of local training points, in this embodiment, the distance between two training data points x and x' is defined as d(x,x') = ||x - x'||.

[0085] The first step: Consider each worker and its local training data set For a test data x * (input), calculate the distance from the test data x * to each training point in the training set , and label it as

[0086]

[0087] At each moment t, in this embodiment, n i is defined as the number of training data of the i-th worker. Therefore, in this embodiment, all the calculated distances are sorted from smallest to largest. Among them, in this embodiment, assume that d 1 (x*,x 1 ) is the distance between the first training point x 1 and the test input x * , and this distance is the smallest; similarly, let be the nthi One training point from the test input x * such that the distance is the largest. That is

[0088] Step 2: Assume that αn i proportion of the training data of worker i is poisoned. In the set of distances calculated in the previous step, remove βn i the largest distances, i.e., the first distance range; remove βn i the smallest distances, i.e., the second distance range, and calculate the number of training points to be filtered as βn i +βn i = 2βn i After filtering out the training data with 2βn i extreme values, the remaining n i - 2βn i training data is saved to the set That is These data can be used to calculate the expected posterior probability and the variance of the posterior probability of the test input x * . Where α is the first proportion and β is the second proportion.

[0089] Step 3: For each worker According to the above steps, a new training set can be constructed

[0090] As Figure 4 shown, remove 5 training points with the smallest distances and remove 5 training points with the largest distances. After subtracting 10 from the total number n i , all the remaining training points form the second training set, and this second training set includes multiple feasible training data.

[0091] Furthermore, the above step S104: According to the Gaussian process regression and at least one training data in the second training set, predict the test data in the test set and output the prediction result, specifically including:

[0092] Define a kernel function, which is used to measure the similarity between two input data; according to the Gaussian process regression and at least one training data in the second training set, calculate the inverse of the covariance matrix; according to the kernel function and the inverse of the covariance matrix, predict the prediction expectation and prediction variance of the selected test data, and output the prediction result corresponding to the test data.

[0093] For each worker Use its training set to calculate the Gaussian posterior probability distribution, that is, the prediction expectation and prediction variance:

[0094]

[0095] Among them, represents the prediction expectation, cov i (f * ) represents the prediction variance, k represents the kernel function, represents the second training set, x * represents the test data as input, represents the output, represents the noise variance, I represents the identity matrix, and i is the working machine identifier.

[0096] In addition, the above "second step" defines the kernel function, specifically including: obtaining the kernel function, and for the kernel function selects the (squared exponential, SE) function, and the expression of the kernel function is obtained as follows:

[0097]

[0098] Among them, (x, x') represents a pair of input data, and a pair of input data comes from the first training set and / or the test set. k(x, x') represents the kernel function corresponding to a pair of input data (x, x'), represents the variance, and l represents a preset adjustment parameter.

[0099] Finally, send this local prediction and cov i (f * ) to the server. The server uses the aggregation algorithm to aggregate the local prediction values cov i (f * ) and gives the global prediction.

[0100] The data training method provided in this embodiment calculates the distance between at least one training data in the first training set and the test point, sorts the calculated distances, filters multiple training data in the first training set according to the distance sorting, generates a feasible training data subset, that is, the second training set, and uses the correlation between the training data in the constructed second training set to calculate the low-dimensional covariance matrix of the Gaussian process regression GPR, predicts the test data in the test set and outputs the prediction result. This method not only ensures the correctness of the output prediction result for new test data, but also improves the calculation efficiency and saves calculation time.

[0101] In addition, by filtering the training data in the first training set through distance sorting, the extreme training data that may affect the prediction result is removed, thereby ensuring the robustness of the local prediction result against data poisoning attacks.

[0102] Optionally, in another implementation of this embodiment, another implementation of generating the second training set in step S103 above is as follows: Determine a target distance among at least one distance according to distance sorting, where the target distance is the median among at least one distance; With the target distance as the center and a preset distance as the radius, form a circular area, and one or more training data covered by the circular area generate the second training set.

[0103] This embodiment uses the median method to filter training data, improving the accuracy of the predicted output for test inputs and ensuring the robustness of user prediction against poisoning attacks.

[0104] In this embodiment, a data training device for a local training set is also provided. This device is used to implement the above embodiments and preferred implementation manners, and those that have been described will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware is also possible and contemplated.

[0105] This embodiment also provides a data training device, which is used to implement the Figure 2 and Figure 3 method flow shown above. As Figure 5 shown, the data training device includes: an acquisition module 510, a processing module 520, a filtering module 530, a prediction module 540, and a sending module 550. In addition, the data training device may also include other more or fewer modules, which will not be elaborated here in this embodiment.

[0106] Among them, the acquisition module 510 is used to acquire the first training set and the test set for local online learning. The first training set includes multiple training data, and the test set includes multiple test data.

[0107] The processing module 520 is used to select a test data from the test set, calculate the distances between at least one training data in the first training set and the test point with the test data as the test point, and sort all the calculated distances.

[0108] The filtering module 530 is used to filter the multiple training data in the first training set according to the distance sorting to generate a second training set, and the second training set includes at least one training data.

[0109] The prediction module 540 is used to predict the test data in the test set according to Gaussian process regression and at least one training data in the second training set, and output a prediction result. The prediction result includes the prediction expectation and prediction variance of at least one test data.

[0110] A sending module 550, configured to transmit the prediction expectation and the prediction variance to a server, so that the server integrates to obtain a global machine learning model according to the prediction results.

[0111] Optionally, in some alternative embodiments, the filtering module 530 is specifically configured to obtain a first ratio, calculate the number of training points to be filtered in the sorted distances according to the first ratio, where the number of training points to be filtered includes at least the sum of the training points with the maximum distance and the minimum distance; subtract the number of training points to be filtered from the number of training points in the first training set, and the remaining training points form the second training set.

[0112] Optionally, in some other alternative embodiments, the filtering module 530 is specifically further configured to count the number of training points in the first training set whose distances from the test points are within a first distance range and the number of training points whose distances from the test points are within a second distance range according to the sorted distances, calculate the sum of the number of training points within the first distance range and the number of training points within the second distance range, and use it as the number of training points to be filtered.

[0113] Optionally, in some other alternative embodiments, the prediction module 540 is specifically configured to define a kernel function, calculate the inverse of the covariance matrix according to Gaussian process regression and at least one training data in the second training set; according to the kernel function and the inverse of the covariance matrix, predict the prediction expectation and the prediction variance of the selected test data, and output the prediction results corresponding to the test data.

[0114] Optionally, in some other alternative embodiments, the filtering module 530 is specifically configured to determine a target distance among at least one distance according to the distance sorting, form a circular area with the target distance as the center and a preset distance as the radius, and generate the second training set from one or more training data covered by the circular area.

[0115] The further function descriptions of the above-mentioned various modules and units are the same as those in the corresponding above-mentioned embodiments, and will not be elaborated here.

[0116] The data training device in this embodiment is presented in the form of functional units. Here, the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and a memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0117] This application embodiment also provides a computer device having the above-mentioned Figure 6 shown data training device.

[0118] Please refer to Figure 6, which is a schematic structural diagram of a computer device provided by an alternative embodiment of the present application. The computer device includes: one or more processors 10, a memory 20, and an interface for connecting various components, including a high-speed interface and a low-speed interface. Each component communicates with each other using different buses and can be installed on a common motherboard or installed in other ways as needed. The processor can process instructions executed within the computer device, including instructions stored in the memory or on the memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface).

[0119] In some alternative embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a set of blade servers, or a multi-processor system). Figure 6 Taking one processor 10 as an example.

[0120] The processor 10 can be a central processing unit, a network processor, or a combination thereof. Among them, the processor 10 can further include a hardware chip. The above-mentioned hardware chip can be an application-specific integrated circuit, a programmable logic device, or a combination thereof. The above-mentioned programmable logic device can be a complex programmable logic device, a field-programmable gate array, a general array logic, or any combination thereof.

[0121] Among them, the memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the data training method shown in the above embodiment.

[0122] The memory 20 can include a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function; the data storage area can store data created according to the use of the computer device. In addition, the memory 20 can include a high-speed random access memory, and can also include a non-transitory memory, such as at least one disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some alternative embodiments, the memory 20 can include a memory remotely set relative to the processor 10, and these remote memories can be connected to the computer device through a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0123] The memory 20 can include a volatile memory, such as a random access memory; the memory can also include a non-volatile memory, such as a flash memory, a hard disk, or a solid-state drive; the memory 20 can also include a combination of the above types of memories.

[0124] The computer device further includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30, and the output device 40 may be connected through a bus or other means. Figure 6 Taking the connection through the bus as an example. Among them, the input device 30 can receive input digital or character information, and generate key signal inputs related to the user settings and function controls of the computer device, such as a touch screen, a keypad, a mouse, a trackpad, a touchpad, etc. The output device 40 may include a display device, an auxiliary lighting device (e.g., an LED), and a tactile feedback device (e.g., a vibration motor), etc.

[0125] In addition, the computer device further includes a communication interface for the computer device to communicate with other devices or communication networks.

[0126] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored. Among them, the computer program is set to execute the steps in any of the above-described embodiments of the data training method for a local training set when running.

[0127] In an exemplary embodiment, the above computer-readable storage medium may include, but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disc, etc., various media that can store computer programs.

[0128] An embodiment of the present application further provides a computer program product. The above computer program product includes a computer program, and when the computer program is executed by a processor, it implements the steps in any of the above-described embodiments of the data training method for a local training set.

[0129] An embodiment of the present application further provides another computer program product, including a non-volatile computer-readable storage medium. The non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements the steps in any of the above-described embodiments of the data training method for a local training set.

[0130] Those skilled in the art can further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Skilled professionals can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0131] The above has introduced in detail a data training method for a local training set provided by this application. Specific examples are used in this article to elaborate on the principle and implementation manner of this application. The description of the above embodiments is only used to help understand the method and its core idea of this application. It should be noted that for those of ordinary skill in the art of this technology, without departing from the principle of this application, several improvements and modifications can still be made to this application, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A data training method for a local training set, characterized in that: The method comprises: Obtain a first training set and a test set for local online learning, wherein the first training set includes a plurality of training data, and the test set includes a plurality of test data; Selecting a test data in the test set, using the test data as a test point to calculate a distance between at least one training data in the first training set and the test point, and sorting all calculated distances; Filtering the plurality of training data in the first training set according to the distance sorting to generate a second training set, wherein the second training set includes at least one training data; Predicting the test data in the test set according to Gaussian process regression and at least one training data in the second training set, and outputting a prediction result, wherein the prediction result includes a prediction expectation and a prediction variance of at least one test data; The prediction expectation and the prediction variance are transmitted to the server so that the server integrates the prediction results to obtain a global machine learning model.

2. The method according to claim 1, characterized in that The filtering of the plurality of training data in the first training set according to the distance sorting to generate the second training set includes: Obtaining a first ratio, where the first ratio is the ratio of the data attacked in the first training set to all the training data; Calculating the number of training points to be filtered in the sorted distances according to the first ratio, wherein the number of training points to be filtered includes at least the sum of the training points with the maximum distance and the training points with the minimum distance; The number of training points to be filtered is subtracted from the number of training points in the first training set, and the remaining training points constitute the second training set.

3. The method according to claim 2, characterized in that Calculating the number of training points to be filtered in the sorted distances according to the first ratio includes: According to the sorted distances, counting the number of training points whose distances between the training data in the first training set and the test point are within a first distance range, and counting the number of training points whose distances between the training data and the test point are within a second distance range, wherein the first distance range is greater than the second distance range; The sum of the number of training points within the first distance range and the number of training points within the second distance range is calculated as the number of training points to be filtered.

4. The method according to any one of claims 1 to 3, characterized in that: The predicting the test data in the test set according to the Gaussian process regression and at least one training data in the second training set, and outputting the prediction result, comprises: Define a kernel function, which is used to measure the similarity between two input data; Calculating an inverse of a covariance matrix based on Gaussian process regression and at least one training data in the second training set; According to the kernel function and the inverse of the covariance matrix, the prediction expectation and the prediction variance of the selected test data are predicted, and the prediction result corresponding to the test data is output.

5. The method according to claim 4, characterized in that In the process of selecting the prediction expectation and prediction variance of the test data, the output prediction expectation and prediction variance are respectively expressed as: in, represents the predicted expectation, cov i (f * ) represents the prediction variance, k represents the kernel function, represents the second training set, x * represents the test data as input, Indicates the output, represents the noise variance, I represents the unit matrix, and i represents the working machine identifier.

6. The method according to claim 4, characterized in that The defining of the kernel function comprises: Get the kernel function, the expression of the kernel function is as follows: Wherein, (x, x') represents a pair of input data, the pair of input data comes from the first training set and / or the test set, k(x, x') represents the kernel function corresponding to the pair of input data (x, x'), represents the variance, and l represents the preset adjustment parameter.

7. The method according to claim 1, characterized in that The filtering of the plurality of training data in the first training set according to the distance sorting to generate the second training set includes: determining a target distance among at least one distance according to the distance ranking, wherein the target distance is a median among the at least one distance; A circular area is formed with the target distance as the center and the preset distance as the radius, and one or more training data covered by the circular area generate the second training set.

8. A data training device for a local training set, characterized in that: The device comprises: An acquisition module is used to acquire a first training set and a test set for local online learning, wherein the first training set includes a plurality of training data, and the test set includes a plurality of test data; a processing module, configured to select a test data in the test set, calculate a distance between at least one training data in the first training set and the test point by taking the test data as a test point, and sort all calculated distances; A filtering module, configured to filter the plurality of training data in the first training set according to the distance sorting to generate a second training set, wherein the second training set includes at least one training data; A prediction module, configured to predict the test data in the test set according to Gaussian process regression and at least one training data in the second training set, and output a prediction result, wherein the prediction result includes a prediction expectation and a prediction variance of at least one test data; A sending module is used to transmit the prediction expectation and the prediction variance to a server so that the server can integrate the prediction results to obtain a global machine learning model.

9. A computer device, characterized in that: comprising a memory and a processor, wherein the memory and the processor are connected; The memory stores computer instructions, and the processor executes the data training method for the local training set according to any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the data training method for a local training set according to any one of claims 1 to 7.

Images