A training set processing method and apparatus

Abstract

The application discloses an artificial intelligence field training set processing method for training a neural network by extracting principal components of a training set, which can use more robust features to train the neural network, thereby improving the robustness of the obtained neural network without reducing the output accuracy of the neural network. The method comprises the following steps: firstly, obtaining a base and a discrete sequence of a first training set, the discrete sequence comprising a discrete value corresponding to each base vector in the base, and the discrete sequence being used to represent the discrete degree of the base; then, obtaining a component value of each sample on each base vector to obtain a plurality of groups of first encoding data; subsequently, obtaining a principal component in each group of first encoding data to obtain a plurality of groups of second encoding data, and the discrete value corresponding to the principal component being higher than a preset discrete value; and performing mapping according to the plurality of groups of second encoding data to obtain new samples to form a second training set, the second training set being used to train the neural network.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence, and in particular to a training set processing method and device. BACKGROUND

[0002] In the field of artificial intelligence, neural networks have shown new possibilities in the intelligent era in recent years in a series of breakthroughs in computer vision, natural language processing, network security and other tasks. However, neural networks are generally vulnerable to adversarial samples that are difficult to explain, which can be deceived by very small but carefully designed noise added to their input data to obtain incorrect results. Such carefully constructed samples are called adversarial samples.

[0003] Facing the vulnerability of adversarial artificial intelligence systems, it is obviously impossible to use in fields with high security requirements such as medical treatment, intelligent driving and other fields. Therefore, it is crucial to analyze the factors that cause the vulnerability of artificial neural networks and obtain artificial neural networks with adversarial robustness. In the existing scheme, more adversarial samples can be added on the basis of the original samples, and the original samples and the adversarial samples are added to the training set for training, so that the neural network obtained by training can recognize the disturbance in the adversarial samples. However, the effect of adversarial training is greatly affected by the model, which may produce completely different effects for different models, and adversarial training needs to constantly generate adversarial samples during training, which greatly reduces the training efficiency and may cause the output accuracy of the neural network obtained by training to decrease, that is, the robustness of the neural network is reduced. Therefore, how to obtain a neural network with better robustness has become a problem to be solved. SUMMARY

[0004] The present application provides a training set processing method for training a neural network by extracting the principal components of the training set. The more robust features can be used for neural network training, thereby improving the robustness of the obtained neural network without reducing the output accuracy of the neural network.

[0005] In view of this, in a first aspect, this application provides a training set processing method, comprising: firstly, decomposing a first training set to obtain a basis of the first training set and a discrete sequence of the basis, wherein the first training set includes multiple samples, the basis of the first training set can be understood as a space composed of data in multiple directions, the basis includes at least one basis vector, and the discrete sequence includes discrete values ​​corresponding one-to-one with each basis vector in the basis, the discrete sequence can be used to represent the degree of discreteness of the basis; then, obtaining the component values ​​of each sample in the first training set on each basis vector to obtain multiple sets of first encoded data, wherein each set of first encoded data corresponds to one sample, and each sample has a component value on each basis vector. Each of the first encoded data has a component value, and each basis vector in the first encoded data corresponds to a component. The components in the first encoded data correspond one-to-one with at least one basis vector. The principal components in each of the multiple sets of first encoded data are obtained to obtain multiple sets of second encoded data. The discrete values ​​of the principal components are higher than a preset discrete value, or in other words, the components in the first encoded data whose discrete values ​​are higher than the preset value constitute the principal components. The multiple sets of second encoded data are mapped to the first training set to obtain new samples corresponding to the multiple sets of second encoded data. The samples corresponding to the multiple sets of second encoded data constitute the second training set. The second training set is used to train the neural network, thereby obtaining a neural network with better robustness.

[0006] Therefore, in this embodiment, after extracting the principal component from the first encoded data of each sample in the basis, a second encoded data with reduced interference is obtained. New samples can be obtained based on the second encoded data, and the new samples are used for neural network training. The new samples reduce interference from components other than the principal component, thus allowing the trained neural network to be trained using the principal component, improving the robustness of the neural network. Furthermore, the dataset processing method provided in this application processes the training set, and the processing method is only related to the distribution of the principal components of the training set itself. Therefore, even when training different models in different scenarios, the training set processing method provided in this application can be used to process the training set, exhibiting strong generalization ability and not relying on the general algorithm of the trained model, thus possessing high versatility.

[0007] In one possible implementation, obtaining the basis of the first training set and the discrete sequence of the basis based on the first training set may include: performing principal component analysis (PCA) on the first training set to obtain orthogonal basis and the discrete sequence of orthogonal basis.

[0008] There are usually several ways to decompose the first training set, such as PCA or sparse coding, to obtain the basis and discrete sequence of the first training set, and then obtain the component of each sample in the first training set.

[0009] In a possible implementation, the PCA processing on the first training set to obtain the orthogonal basis and the discrete sequence of the orthogonal basis can include: performing centering processing on the first training set to obtain a first training set after centering, the first training set after centering including data with a mean value of 0; and performing PCA processing on the first training set after centering to obtain the orthogonal basis and the discrete sequence of the orthogonal basis, the discrete sequence including a sequence of variances of the first training set after centering in the orthogonal basis. Generally, the variances included in the discrete sequence are arranged in descending order, and the variances included in the discrete sequence correspond to the basis vectors in the orthogonal basis in order one by one.

[0010] In the embodiments of the present application, the first training set can be subjected to centering processing to obtain a first training set after centering, and then the first training set after centering can be subjected to PCA processing, so as to complete the fast analysis of the first training set and obtain the orthogonal basis and the discrete sequence of the orthogonal basis.

[0011] In a possible implementation, the obtaining of the component value of each sample in the first training set in the basis can include: calculating, by a preset algorithm, the component value of each sample in the first training set after centering on each basis vector in the orthogonal basis to obtain a plurality of groups of first encoding data.

[0012] In the embodiments of the present application, if the first training set is subjected to PCA processing, the component value of each sample on each basis vector can be calculated based on the orthogonal basis after the PCA processing, so as to obtain the first encoding data.

[0013] In a possible implementation, the preset algorithm includes inner product operation or sparse coding operation. In the embodiments of the present application, the component value of each sample on each basis vector can be calculated by the inner product operation or the sparse coding operation, so as to realize the projection of each sample on each basis vector.

[0014] In a possible implementation, the obtaining of the principal component in each group of first encoding data to obtain a plurality of groups of second encoding data can include: retaining the principal component in each group of first encoding data, and replacing other components in each group of first encoding data except the principal component with a preset value to obtain a plurality of groups of second encoding data, wherein the principal component in each group of first encoding data constitutes at least one group of second encoding data. In the embodiments of the present application, the other components in each group of first encoding data except the principal component can be replaced with the preset value, so that the components in each group of first encoding data can constitute one group or a plurality of groups of second encoding data.

[0015] In a possible implementation, the preset value includes 0 or a preset noise vector, and the preset noise vector includes Gaussian noise or uniformly distributed noise.

[0016] Therefore, in the embodiments of the present application, the values other than the principal components in the first encoding data are replaced by 0 or a preset noise vector, so as to obtain one or more sets of second encoding data, so as to reduce the influence of the values other than the principal components in each sample in the first training set on the training of the neural network, and improve the robustness of the finally obtained neural network.

[0017] In a possible implementation, the preset noise vector is proportional to the discrete value of the replaced component. Therefore, in the embodiments of the present application, the replaced noise vector is similar to the original data in the first encoding data in terms of distribution, so as to reduce the influence of the values other than the principal components in each sample on the training of the neural network, reduce the interference on the training of the neural network, and obtain more accurate output of the neural network.

[0018] In a second aspect, the present application provides a training set processing apparatus, comprising:

[0019] a decomposition unit, configured to obtain a basis and a discrete sequence of the basis of the first training set according to the first training set, the first training set comprising a plurality of samples, the basis comprising at least one basis vector, the discrete sequence comprising a discrete value corresponding to each basis vector in the basis, and the discrete sequence being used to represent the discrete degree of the basis;

[0020] an obtaining unit, configured to obtain the component value of each sample in the first training set in the basis, so as to obtain a plurality of sets of first encoding data, each set of the first encoding data corresponding to one sample;

[0021] The obtaining unit is further configured to obtain the principal component in each set of the first encoding data in the plurality of sets of the first encoding data, so as to obtain a plurality of sets of second encoding data, and the discrete value of the principal component being higher than a preset discrete value;

[0022] a mapping unit, configured to map the plurality of sets of the second encoding data to the basis of the first training set, so as to obtain the samples corresponding to the plurality of sets of the second encoding data, the samples corresponding to the plurality of sets of the second encoding data forming a second training set, and the second training set being used to train the neural network.

[0023] The beneficial effects of the second aspect and any possible implementation of the second aspect can be referred to the description of the first aspect and any possible implementation of the first aspect.

[0024] In a possible implementation, the basis is an orthogonal basis, and the decomposition unit is specifically configured to perform principal component analysis (PCA) on the first training set, so as to obtain the orthogonal basis and the discrete sequence of the orthogonal basis.

[0025] In a possible implementation, the decomposing unit is specifically configured to: perform centering processing on the first training set to obtain a first centering training set, the first centering training set comprising data with a mean value of 0; and perform PCA processing on the first centering training set to obtain an orthogonal basis and a discrete sequence of the orthogonal basis, the discrete sequence comprising a sequence of variances of the first centering training set in the orthogonal basis.

[0026] In a possible implementation, the decomposing unit is specifically configured to calculate, by using a preset algorithm, a component value of each sample in the first centering training set on each basis vector in the orthogonal basis to obtain a plurality of groups of first encoding data.

[0027] In a possible implementation, the preset algorithm comprises inner product operation or sparse coding operation.

[0028] In a possible implementation, the obtaining unit is specifically configured to retain a principal component in each group of first encoding data, and replace other components in each group of first encoding data except the principal component with a preset value to obtain a plurality of groups of second encoding data, wherein the principal component in each group of first encoding data constitutes at least one group of second encoding data.

[0029] In a possible implementation, the preset value comprises 0 or a preset noise vector, and the preset noise vector comprises Gaussian noise or uniformly distributed noise.

[0030] In a possible implementation, the preset noise vector is proportional to a discrete value of the replaced component.

[0031] In a third aspect, an embodiment of the present application provides a training set processing apparatus, which has the function of implementing the training set processing method in the first aspect. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software comprises one or more modules corresponding to the above functions.

[0032] In a fourth aspect, an embodiment of the present application provides a training set processing apparatus, comprising a processor and a memory, wherein the processor and the memory are interconnected through a circuit, and the processor invokes program codes in the memory to execute functions related to processing in the training set processing method in any one of the first aspect. Optionally, the training set processing apparatus can be a chip.

[0033] In a fifth aspect, an embodiment of the present application provides a training set processing apparatus, which can also be referred to as a digital processing chip or a chip. The chip comprises a processing unit and a communication interface. The processing unit obtains program instructions through the communication interface. The program instructions are executed by the processing unit. The processing unit is configured to execute functions related to processing in the first aspect or any optional implementation of the first aspect.

[0034] In a sixth aspect, the embodiments of the present application provide a computer readable storage medium, including instructions, which, when executed on a computer, cause the computer to perform the method in the first aspect or any optional implementation of the first aspect.

[0035] In a seventh aspect, the embodiments of the present application provide a computer program product including instructions, which, when executed on a computer, cause the computer to perform the method in the first aspect or any optional implementation of the first aspect. BRIEF DESCRIPTION OF DRAWINGS

[0036] Figure 1 A schematic diagram of an artificial intelligence main body framework to which the present application is applied;

[0037] Figure 2 A schematic diagram of a system architecture provided by the present application;

[0038] Figure 3 A schematic diagram of a convolutional neural network structure provided by the embodiments of the present application;

[0039] Figure 4 A schematic diagram of another convolutional neural network structure provided by the embodiments of the present application;

[0040] Figure 5 Another schematic diagram of a system architecture provided by the present application;

[0041] Figure 6 A schematic diagram of a training set processing method provided by the embodiments of the present application;

[0042] Figure 7 A schematic diagram of an adversarial sample provided by the embodiments of the present application;

[0043] Figure 8 A schematic diagram of a training set processing device provided by the present application;

[0044] Figure 9 Another schematic diagram of a training set processing device provided by the present application;

[0045] Figure 10 A schematic diagram of a chip structure provided by the embodiments of the present application. DETAILED DESCRIPTION

[0046] The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0047] The training set processing method provided in the application can be applied in an artificial intelligence (AI) scene. AI is a theory, method, technology and application system for simulating, extending and expanding human intelligence by using a digital computer or a machine controlled by a digital computer, perceiving an environment, acquiring knowledge and using the knowledge to obtain optimal results. In other words, artificial intelligence is a branch of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is the design principle and implementation method of various intelligent machines, so that the machine has the functions of perception, reasoning and decision-making. The research in the field of artificial intelligence includes robots, natural language processing, computer vision, decision-making and reasoning, human-computer interaction, recommendation and search, AI basic theory, etc.

[0048] Figure 1 An artificial intelligence subject framework diagram is shown, which describes the overall workflow of an artificial intelligence system and is suitable for general artificial intelligence field requirements.

[0049] The above-mentioned artificial intelligence subject framework is described below from two dimensions of "intelligent information chain" (horizontal axis) and "IT value chain" (vertical axis).

[0050] The "intelligent information chain" reflects a series of processes from data acquisition to processing. For example, it can be a general process of intelligent information perception, intelligent information representation and formation, intelligent reasoning, intelligent decision-making, intelligent execution and output. In this process, data undergoes a condensation process of "data-information-knowledge-wisdom".

[0051] The "IT value chain" reflects the value brought by artificial intelligence to the information technology industry from the underlying infrastructure of human intelligence, information (provision and processing technology implementation) to the industrial ecological process of the system.

[0052] (1) Infrastructure:

[0053] Infrastructure provides computing power support for artificial intelligence systems, realizes communication with the external world, and realizes support through the underlying platform. Communication with the outside world through sensors; computing power is provided by intelligent chips such as central processing units (CPU), neural network processing units (NPU), graphics processing units (GPU), application specific integrated circuits (ASIC), or field programmable gate arrays (FPGA) and other hardware acceleration chips); The underlying platform includes distributed computing frameworks and related platform guarantees and support for networks, which can include cloud storage and computing, interconnection networks, etc. For example, sensors and external communication obtain data, which are provided to intelligent chips in the distributed computing system provided by the underlying platform for computing.

[0054] (2) Data

[0055] The data of the upper layer of the infrastructure is used to represent the data source in the field of artificial intelligence. The data relates to graphics, images, speech, video, text, and also relates to Internet of Things data of traditional devices, including business data of existing systems and sensing data such as force, displacement, liquid level, temperature, and humidity.

[0056] (3) Data processing

[0057] Data processing usually includes data training, machine learning, deep learning, search, reasoning, decision making, etc.

[0058] Among them, machine learning and deep learning can model, extract, preprocess, train, etc. symbolic and formalized intelligent information of data.

[0059] Reasoning refers to the process of simulating human intelligent reasoning methods in computers or intelligent systems, using formalized information to perform machine thinking and solve problems according to reasoning control strategies, and the typical function is search and matching.

[0060] Decision making refers to the process of decision making after intelligent information is reasoned, which usually provides functions such as classification, sorting, and prediction.

[0061] (4) General capabilities

[0062] After the data is processed as mentioned above, some general capabilities can be formed based on the results of the data processing, such as algorithms or a general system, for example, translation, text analysis, computer vision processing (such as image recognition, object detection, etc.), speech recognition, etc.

[0063] (5) Intelligent product and industry application

[0064] Intelligent product and industry application refers to the product and application of artificial intelligence system in various fields, which is the packaging of the overall solution of artificial intelligence, and realizes the application of intelligent information decision product. The application fields mainly include intelligent manufacturing, intelligent transportation, smart home, intelligent medical treatment, intelligent security, automatic driving, safe city, intelligent terminal, etc.

[0065] Referring to the accompanying drawings Figure 2 , the embodiment of the present application provides a system architecture 200. The system architecture includes a database 230, a client device 240. The data acquisition device 260 is used to collect data and store it in the database 230, and the construction module 202 generates the target model / rule 201 based on the data maintained in the database 230. The following will describe in more detail how the construction module 202 obtains the target model / rule 201 based on the data, and the target model / rule 201 is the neural network constructed in the following embodiments of the present application. For details, please refer to the related description of the following Figure 6 .

[0066] The computing module can include a construction module 202, and the target model / rule obtained by the construction module 202 can be applied in different systems or devices. In the accompanying Figure 2 , the execution device 210 is configured with a transceiver 212, which can be a wireless transceiver, an optical transceiver or a wired interface (such as an I / O interface) and the like, and interacts with external devices. The "user" can input data to the transceiver 212 through the client device 240, for example, in the following embodiments of the present application, the client device 240 can send a target task to the execution device 210, request the execution device to construct a neural network, and send a database for training to the execution device 210.

[0067] The execution device 210 can call data, code, etc. in the data storage system 250, or store data, instructions, etc. in the data storage system 250.

[0068] The calculation module 211 processes the input data using the target model / rule 201. Specifically, the calculation module 211 is used to: first, obtain a first training set, which includes multiple samples; then, obtain the basis and discrete sequence of the basis of the first training set based on the first training set, wherein the first training set includes multiple samples, the basis includes at least one basis vector, and the discrete sequence includes discrete values ​​corresponding one-to-one with each basis vector in the basis, and the discrete sequence can be used to represent the degree of discreteness of the basis; obtain the component values ​​of each sample in the first training set on each basis vector to obtain multiple sets of first encoded data, each set of first encoded data corresponding to one sample; obtain the principal components in each set of first encoded data to obtain multiple sets of second encoded data, wherein the discrete values ​​of the principal components are higher than preset discrete values; map the multiple sets of second encoded data to the basis of the first training set to obtain samples corresponding to the multiple sets of second encoded data, and the samples corresponding to the multiple sets of second encoded data form a second training set, which is used to train the neural network.

[0069] Finally, transceiver 212 returns the constructed neural network to client device 240 for deployment in client device 240 or other devices.

[0070] At a deeper level, the building module 202 can generate corresponding target models / rules 201 based on different data for different tasks, so as to provide users with better results.

[0071] In the appendix Figure 2 In the scenario shown, the data input to the execution device 210 can be determined based on the user's input data. For example, the user can operate on the interface provided by the transceiver 212. Alternatively, the client device 240 can automatically input data to the transceiver 212 and obtain results. If the client device 240 needs user authorization to automatically input data, the user can set appropriate permissions on the client device 240. The user can view the results output by the execution device 210 on the client device 240; the specific presentation format can be display, sound, animation, etc. The client device 240 can also act as a data acquisition terminal, storing the acquired data associated with the target task into the database 230.

[0072] It should be noted that the appendix Figure 2 This is merely an exemplary schematic diagram of a system architecture provided in this application embodiment, and the positional relationships between the devices, components, modules, etc., shown in the diagram do not constitute any limitation. For example, in the appendix... Figure 2 In this context, the data storage system 250 is an external memory relative to the execution device 210. In other scenarios, the data storage system 250 can also be placed within the execution device 210.

[0073] The training or updating process mentioned in the present application can be performed by the construction module 202. It can be understood that the training process of the neural network, i.e. learning the way of space transformation, more specifically, learning the weight matrix. The purpose of training the neural network is to make the output of the neural network as close to the expected value as possible, so the weight vector of each layer of the neural network can be updated by comparing the predicted value of the current network and the expected value, and according to the difference between the two. (Of course, the weight vector can be initialized before the first update, i.e. the parameters of each layer in the deep neural network are pre-configured.) For example, if the predicted value of the network is too high, the value of the weight in the weight matrix is adjusted to reduce the predicted value. After continuous adjustment, the value output by the neural network approaches or equals the expected value. Specifically, the difference between the predicted value and the expected value of the neural network can be measured by a loss function or an objective function. Taking the loss function as an example, the higher the output value (loss) of the loss function, the greater the difference. The training of the neural network can be understood as the process of minimizing the loss. The process of updating the weights of the starting network and training the serial network in the following embodiments of the present application can refer to this process, which will not be described below.

[0074] The neural network mentioned in the present application can include various types, such as a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), or a residual network or other neural network.

[0075] Exemplarily, the following takes CNN as an example.

[0076] A CNN (Convolutional Neural Network) is a deep neural network with a convolutional structure. CNN is a deep learning architecture, which refers to learning at multiple levels of abstraction using machine learning algorithms. As a deep learning architecture, a CNN is a feed-forward artificial neural network where neurons respond to overlapping regions in the input image. A convolutional neural network contains a feature extractor consisting of convolutional layers and subsampling layers. This feature extractor can be viewed as a filter, and the convolution process can be seen as using a trainable filter to convolve with an input image or a convolutional feature map. A convolutional layer is a layer of neurons in a convolutional neural network that performs convolution processing on the input signal. In a convolutional layer of a convolutional neural network, a neuron may only be connected to some of its neighboring neurons. A convolutional layer typically contains several feature planes, each composed of rectangularly arranged neural units. Neural units on the same feature plane share weights, which are the convolutional kernel. Shared weights can be understood as the way image information is extracted regardless of location. The underlying principle is that the statistical information of one part of an image is the same as that of other parts. This means that image information learned in one part can also be used in another part. Therefore, we can use the same learned image information for all locations in an image. Within the same convolutional layer, multiple convolutional kernels can be used to extract different image information. Generally, the more convolutional kernels there are, the richer the image information reflected by the convolution operation.

[0077] Convolutional kernels can be initialized as matrices of random size, and during the training of a convolutional neural network, they can learn appropriate weights. Furthermore, sharing weights directly reduces the number of connections between layers in the convolutional neural network, while also lowering the risk of overfitting.

[0078] Convolutional neural networks can employ backpropagation (BP) to correct the parameters in the initial super-resolution model during training, thereby reducing the reconstruction error loss. Specifically, forward propagation of the input signal to the output generates an error loss; this error loss information is then propagated back to update the parameters in the initial super-resolution model, leading to convergence of the error loss. The backpropagation algorithm is an error-loss-driven backpropagation process aimed at obtaining the optimal parameters of the super-resolution model, such as the weight matrix.

[0079] like Figure 3 As shown, the convolutional neural network (CNN) 100 may include an input layer 110, a convolutional / pooling layer 120, wherein the pooling layer is optional, and a neural network layer 130.

[0080] As Figure 3 shown, the convolutional / pooling layers 120 can include layers as examples 121-126, in one implementation, 121 is a convolutional layer, 122 is a pooling layer, 123 is a convolutional layer, 124 is a pooling layer, 125 is a convolutional layer, and 126 is a pooling layer; in another implementation, 121 and 122 are convolutional layers, 123 is a pooling layer, 124 and 125 are convolutional layers, and 126 is a pooling layer. That is, the output of a convolutional layer can be used as the input of a subsequent pooling layer, or as the input of another convolutional layer for further convolution operation.

[0081] Taking the convolutional layer 121 as an example, the convolutional layer 121 can include a plurality of convolutional operators, also known as kernels, which act as filters to extract specific information from the input image matrix in image processing. The convolutional operator can be essentially a weight matrix, which is usually predefined. In the process of convolution operation on the image, the weight matrix is usually processed on the input image along the horizontal direction one pixel after another (or two pixels after another, depending on the value of the stride), thereby completing the work of extracting specific features from the image. The size of the weight matrix should be related to the size of the image. It should be noted that the depth dimension of the weight matrix is the same as that of the input image, and the weight matrix extends to the entire depth of the input image during the convolution operation. Therefore, convolution with a single weight matrix will produce a single-depth convolution output, but most cases do not use a single weight matrix, but apply multiple weight matrices of the same dimension. The output of each weight matrix is stacked to form the depth dimension of the convolution image. Different weight matrices can be used to extract different features in the image, such as a weight matrix for extracting image edge information, another weight matrix for extracting specific colors of the image, and another weight matrix for blurring unwanted noise in the image, etc. The multiple weight matrices are of the same dimension, and the feature maps extracted by the multiple weight matrices of the same dimension are also of the same dimension. The extracted multiple feature maps of the same dimension are combined to form the output of the convolution operation.

[0082] Generally, the weight values in the weight matrix need to be obtained through a large amount of training in actual application. The weight matrices formed by the weight values obtained through training can extract information from the input image, thereby helping the convolutional neural network 100 to make correct predictions.

[0083] When a convolutional neural network 100 has multiple convolutional layers, the initial convolutional layers (e.g., 121) tend to extract more general features, which can also be called low-level features. As the depth of the convolutional neural network 100 increases, the features extracted by later convolutional layers (e.g., 126) become more and more complex, such as high-level semantic features. Features with higher semantic levels are more suitable for the problem to be solved.

[0084] Pooling layer:

[0085] Because it is often necessary to reduce the number of training parameters, pooling layers are often introduced periodically after convolutional layers, i.e., ... Figure 3 Layers 121-126 in example 120 can be a convolutional layer followed by a pooling layer, or multiple convolutional layers followed by one or more pooling layers. In image processing, the sole purpose of pooling layers is to reduce the spatial size of the image. Pooling layers can include average pooling and / or max pooling operators to sample the input image to obtain a smaller image size. Average pooling calculates the average value of pixel values ​​within a specific range. Max pooling takes the pixel with the largest value within a specific range as the result of max pooling. Furthermore, just as the size of the weight matrix in a convolutional layer should be related to the image size, the operators in a pooling layer should also be related to the image size. The size of the output image after pooling can be smaller than the size of the input image of the pooling layer. Each pixel in the output image represents the average or maximum value of the corresponding sub-region of the input image of the pooling layer.

[0086] Neural network layer 130:

[0087] After processing by the convolutional / pooling layers 120, the convolutional neural network 100 is still insufficient to output the required information. As mentioned earlier, the convolutional / pooling layers 120 only extract features and reduce the parameters introduced by the input image. However, to generate the final output information (the required class information or other relevant information), the convolutional neural network 100 needs to utilize neural network layers 130 to generate one or more outputs representing the required number of classes. Therefore, neural network layers 130 may include multiple hidden layers (such as...). Figure 3 The diagram shows layers 131, 132 to 13n) and an output layer 140. In this application, the convolutional neural network is obtained by performing at least one deformation on a selected starting network to obtain a serial network, and then obtaining the desired network based on the trained serial network. This convolutional neural network can be used for image recognition, image classification, image super-resolution reconstruction, etc.

[0088] After the multiple hidden layers in neural network layer 130, the final layer of the entire convolutional neural network 100 is the output layer 140. This output layer 140 has a loss function similar to classification cross-entropy, specifically used to calculate the prediction error. Once the entire convolutional neural network 100 has undergone forward propagation (e.g., ...), the loss function is applied. Figure 3 The propagation from 110 to 140 is completed (forward propagation), and the reverse propagation (such as...) Figure 3 The propagation from 140 to 110 (backpropagation) will begin to update the weight values ​​and biases of the layers mentioned above, in order to reduce the loss of the convolutional neural network 100 and the error between the output of the convolutional neural network 100 through the output layer and the ideal result.

[0089] It should be noted that, as Figure 3 The convolutional neural network 100 shown is merely an example of a convolutional neural network. In specific applications, convolutional neural networks can also exist in the form of other network models, such as... Figure 4 The multiple convolutional / pooling layers shown are run in parallel, and the extracted features are all input into the full neural network layer 130 for processing.

[0090] See appendix Figure 5 This application also provides a system architecture 300. The execution device 210 is implemented by one or more servers, optionally in conjunction with other computing devices, such as data storage, routers, load balancers, etc. The execution device 210 can be deployed on a single physical site or distributed across multiple physical sites. The execution device 210 can use data from the data storage system 250 or call program code from the data storage system 250 to implement the following aspects of this application. Figure 6 The steps of the corresponding training set processing method.

[0091] Users can interact with execution device 210 by operating their respective user devices (e.g., local device 301 and local device 302). Each local device can represent any computing device, such as a personal computer, computer workstation, smartphone, tablet, smart camera, smart car or other type of cellular phone, media consumption device, wearable device, set-top box, game console, etc.

[0092] The local device of each user can interact with the execution device 210 through a communication network of any communication mechanism / communication standard, which can be a wide area network, a local area network, a point-to-point connection, etc., or any combination thereof. Specifically, the communication network can include a wireless network, a wired network, or a combination of a wireless network and a wired network, etc. The wireless network includes, but is not limited to, any one or more of a combination of a 5th-Generation (5G) system, a long term evolution (LTE) system, a global system for mobile communication (GSM) or a code division multiple access (CDMA) network, a wideband code division multiple access (WCDMA) network, wireless fidelity (WiFi), Bluetooth, Zigbee, radio frequency identification (RFID), long range (Lora) wireless communication, near field communication (NFC), etc. The wired network can include an optical fiber communication network or a network composed of coaxial cables, etc.

[0093] In another implementation, one or more aspects of the execution device 210 can be implemented by each local device, for example, the local device 301 can provide local data or feedback calculation results for the execution device 210.

[0094] It should be noted that all functions of the execution device 210 can also be implemented by the local device. For example, the local device 301 implements the functions of the execution device 210 and provides services for its own user, or provides services for the user of the local device 302.

[0095] In some scenarios, more adversarial samples can be added to the original samples, and both the original samples and the adversarial samples are added to the training set for training, so that the neural network trained can recognize the perturbation in the adversarial sample. However, the effect of adversarial training is greatly affected by the model, which may produce completely different effects for different models, and adversarial training needs to constantly generate adversarial samples during training, which greatly reduces the training efficiency and may cause the output accuracy of the neural network trained to decrease, that is, the robustness of the neural network is reduced. Robustness can be understood as the ability of the neural network to remain unchanged in output value when facing slight changes in input. Or, in some other scenarios, the adversarial samples can be denoised, and a denoising network is added in front of the neural network, and the difference between the top-level representations generated by the pair of adversarial samples and original samples is reduced through gradient descent algorithm, so that the denoising network achieves the effect of adversarial robustness. However, the denoising network needs to be added purposefully, and different denoising networks may need to be set in different training scenarios, which has weak generalization ability and low training efficiency.

[0096] Generally, the vulnerability of the neural network is due to the fact that during the training of the neural network, more attention is paid to learning functions that change dramatically outside the data distribution. That is, the neural network mainly relies on those components that change slightly in the linear space but also contain information that can be used for classification to perform classification, that is, the components other than the main components mentioned above. For ease of understanding, it can be understood that the components other than the main components included in the samples of the training set have a greater impact on the training of the neural network, which are often more linearly separable and easier to learn and generalize by the neural network. The main reason for the influence of these components other than the main components on the robustness of the neural network is that the robustness of the neural network is reduced, which is referred to as the gradient leakage phenomenon below.

[0097] Therefore, the present application provides a training set processing method, which processes the training set before training to extract the main components of each sample, obtains new samples, and forms a second training set. The samples included in the second training set include the main components in the original samples, so that training the neural network by the second training set can improve the robustness of the neural network.

[0098] Referring to Figure 6 The flowchart of the training set processing method provided by the present application is as follows.

[0099] 601, obtain the base and the corresponding discrete sequence of the first training set.

[0100] The first training set can include all data, and a basis and a discrete sequence corresponding to the basis of the first training set can be obtained by decomposing the data. The first training set includes a plurality of samples, the basis can include one or more basis vectors, and the discrete sequence is used to represent the discrete degree of the basis. The discrete sequence can include a discrete value corresponding to each basis vector in the basis.

[0101] For ease of understanding, the aforementioned basis can be understood as a space, and the space can include one or more basis vectors. The discrete sequence is used to represent the discrete degree of the data included in the first training set in the basis.

[0102] It should be understood that the type of sample included in the first training set is related to the neural network to be trained. For example, if a classification network needs to be trained, a large number of classified images can be included in the first training set, if a face recognition network needs to be trained, a large number of face images can be included in the first training set, and the like.

[0103] In an optional embodiment, the aforementioned discrete value can be a variance, a standard deviation, or a range, or the like, which is used to represent the discrete degree. Hereinafter, the discrete degree of the basis in the embodiment of the present application is exemplarily described by taking the variance as an example. It should be understood that the variance mentioned below can also be replaced by a standard deviation or a range, or the like, which is used to represent the discrete degree. The following will not be described in detail.

[0104] Specifically, the first training set can be decomposed in various ways, such as a PCA or a sparse coding algorithm.

[0105] For example, the first training set can be decomposed by a sparse coding algorithm. For a data set X, the optimization problem s.t.||A i ||2≤1 wherein A i is the i-th column of A, S i , X i Similarly; ||.| p represents the L p norm, and λ is a regularization term; so that the sparse coding S i of X i and the dictionary matrix (i.e., the basis) A are obtained.

[0106] For example, the first training set is subjected to PCA processing to obtain an orthogonal basis and a discrete sequence corresponding to the orthogonal basis. PCA is an algorithm for dimension reduction by performing orthogonal spectral decomposition on the covariance matrix of input data to determine the linear direction of the main change between data points. For ease of understanding, the PCA processing can be understood as an algorithm for dimension reduction by performing orthogonal spectral decomposition on the covariance matrix of input data, i.e., the first data set, to determine the linear direction of the main change between data points. For ease of understanding, the subsequent steps are exemplarily described below by taking the PCA processing of the first training set as an example.

[0107] In a possible implementation, the aforementioned PCA processing of the first training set to obtain an orthogonal basis and a discrete sequence corresponding to the orthogonal basis can specifically include: performing centering processing on the first training set to obtain a first training set after centering processing. Then, the first training set after centering processing is subjected to PCA processing to obtain an orthogonal basis and a discrete sequence corresponding to the orthogonal basis. The discrete sequence is a sequence including the variance of the first training set after centering processing in the orthogonal basis. The variances included in the discrete sequence are arranged in descending order, and the variances included in the discrete sequence are sequentially and one-to-one corresponding to the basis vectors in the orthogonal basis.

[0108] The aforementioned centering processing can be understood as zero-meaning, that is, detecting the mean of all data included in the first training set, and translating the first training set in space, such as subtracting the mean from all data in the first training set, so that the mean of all data included in the first training set is 0, to obtain the first training set after centering processing.

[0109] In a possible implementation, after calculating the discrete value, i.e., variance, corresponding to each basis vector, the calculated variances can be sorted in descending order to obtain a discrete sequence. Then, the basis vectors included in the orthogonal basis are sorted, and the arrangement of the basis vectors in the orthogonal basis corresponds to the sorting manner in the discrete sequence. For example, the orthogonal basis can be represented as {v_1, v_2, …, v_N}, the discrete sequence can be represented as {λ_1, λ_2, …, λ_N}, the variance corresponding to the basis vector v_1 is λ_1, the variance corresponding to the basis vector v_2 is λ_2, and so on.

[0110] Of course, the discrete sequence can also be sorted in other manners, for example, arranged in ascending order or arranged according to other set rules, and correspondingly, the basis vectors in the orthogonal basis can also be arranged in the sorting manner corresponding to the discrete sequence.

[0111] 602、Obtain the component value of each sample in the first training set in the basis to obtain a plurality of groups of first encoding data.

[0112] Wherein, after decomposing the first training set to obtain the basis and the corresponding discrete sequence, the component value of each sample in the first training set in each basis vector in the basis is obtained, and a plurality of groups of first encoding data are obtained, and the component value of each sample in each basis vector constitutes a group of first encoding data.

[0113] For the convenience of understanding, the aforementioned basis can be understood as a space composed of the data included in the first training set, and the basis vector represents the direction in the space, and the step 602 obtains the component value of each sample projected to each direction, thereby obtaining the first encoding data of each sample.

[0114] Optionally, if the PCA processing is performed on the first training set, the orthogonal basis and the corresponding discrete sequence are obtained, and the component of each sample in the orthogonal basis is obtained, thereby obtaining the first encoding data corresponding to each sample.

[0115] Optionally, if the centering is performed on the first training set in the step 601, the first training set in the step 602 can be replaced by the first training set after the centering.

[0116] Optionally, the component value of each sample in the first training set in the basis can be calculated by a preset algorithm, thereby obtaining a plurality of groups of first encoding data. The preset algorithm can include inner product operation or sparse coding operation, etc. For example, taking the inner product operation as an example, the inner product operation is performed on each sample in the first training set and each basis vector in the basis, thereby obtaining the first encoding data of each sample.

[0117] 603, obtaining the principal component in each group of first encoding data in the plurality of groups of first encoding data, thereby obtaining a plurality of groups of second encoding data.

[0118] Wherein, after obtaining the first encoding data of each sample, the principal component in each group of first encoding data in the plurality of groups of first encoding data is extracted, thereby obtaining a plurality of groups of second encoding data. The principal component can be understood as the component with the corresponding discrete value greater than the preset discrete value, or in other words, the principal component can be understood as the preset number of components with the maximum corresponding discrete value in the first encoding data.

[0119] For example, the bases in the step 601 can be an orthogonal basis, which can be represented as {v_1, v_2, …, v_N}, the discrete sequence can be represented as {λ_1, λ_2, …, λ_N}, and the basis vectors in the orthogonal basis and the variances in the discrete sequence are one-to-one corresponding in order. The first encoding data can be represented as a: {a_1, a_2, …, a_N}, and the components included in the first encoding data are one-to-one corresponding to the variances in the discrete sequence in order. Therefore, the second encoding data can be obtained by determining the variance greater than the preset discrete value in the discrete sequence, and determining the component corresponding to the variance greater than the preset discrete value in the first encoding data as the main component. For example, when the discrete values are arranged in descending order, a’:{a_1, a_2, …, a_K} can be extracted from a: {a_1, a_2, …, a_N}, and a’ is a set of second encoding data, and K is less than or equal to N.

[0120] In a possible implementation, the main component in each set of first encoding data can be reserved, and the other components in each set of first encoding data except the main component can be replaced by a preset value to obtain a plurality of sets of second encoding data. Each set of first encoding data can correspond to one or more sets of second encoding data. For example, the other components in each set of first encoding data except the main component can be replaced by a plurality of different ways to obtain a plurality of sets of second encoding data, so as to obtain a plurality of sets of second encoding data corresponding to each set of first encoding data.

[0121] Optionally, the preset value can include 0 or a preset noise vector. The preset noise vector can include noise independent of the true value of the sample, or noise independent of the true value of the sample but similar to the data distribution in the first training set, or noise with a specific distribution, etc. For example, the preset noise vector can be Gaussian noise or uniform distribution characteristics, etc. Therefore, in the embodiments of the present application, by replacing the other components except the main component with 0 or the noise vector, the influence of the part of components on the training of the neural network is eliminated, so that the main component can be used as the main component in the subsequent training of the neural network, and a neural network with better robustness is obtained.

[0122] In one possible implementation, if all components in each set of first encoded data except the principal component are replaced with noise, the replaced noise can be proportional to the variance of the original component. For example, the noise can be 5 times, 10 times, or similarly, the variance of the original component. Therefore, in this embodiment, the distribution pattern of the replaced noise can be similar to the distribution pattern of the variance of the original component, thereby reducing the impact on subsequent training of the neural network. This allows for training the neural network using the principal component to obtain a more robust neural network while avoiding impact on the output accuracy, resulting in a neural network with a better balance between robustness and output accuracy. It can be understood that if PCA processing is used on the first training set in step 601, in this embodiment, a certain amount of noise can be added to the normal direction of the PCA subspace to suppress gradient leakage.

[0123] In one possible scenario, if the variance values ​​in the discrete sequence are arranged in descending order, the order of the components in the first encoded data corresponds to the order of the variances in the discrete sequence. The number of variances greater than a preset discrete value, d (where d is a positive integer), is determined. The first d components of the first encoded data are retained, and the (d+1)th to Dth components are replaced with independent Gaussian noise proportional to the variance of the corresponding component. Components after the Dth component are replaced with 0, resulting in the second encoded data. Therefore, in this scenario, all components in the first encoded data except the principal component can be replaced with 0 or noise. This allows the principal component to be used for subsequent neural network training, avoiding the impact of perturbations in adversarial examples on the neural network and improving its robustness.

[0124] 604. Map multiple sets of second-encoded data to the basis of the first training set to obtain samples corresponding to multiple sets of second-encoded data.

[0125] After obtaining multiple sets of second-encoded data, these sets are mapped to the basis of the first training set to obtain new samples corresponding to each set of second-encoded data. These samples constitute the second training set. For ease of understanding, the basis of the first training set can be understood as a space, and the second-encoded data includes component values ​​in one or more directions within this space. Mapping a set of second-encoded data to this space yields new samples.

[0126] Specifically, after obtaining the second encoding data, the bases in the aforementioned step 601 can be mapped according to the second encoding data to obtain new samples corresponding to each group of second encoding data. The plurality of new samples corresponding to the plurality of groups of second encoding data form a second training set. For ease of understanding, the bases mentioned in the aforementioned step 601 can be understood as a space, and the second encoding data includes the values of the projections of the new samples in the direction of each base vector. The components included in the second encoding data are mapped in the bases to obtain the new samples.

[0127] Optionally, after obtaining the second training set composed of new samples, the second training set can be used to train the neural network to obtain a neural network with better robustness.

[0128] Therefore, in the embodiments of the present application, after the main components of each sample in the first encoding data in the bases are extracted, the second encoding data after the interference is reduced is obtained, the new samples can be obtained according to the second encoding data, and the neural network is trained using the new samples. The interference of other components except the main components is reduced in the new samples, so that the neural network trained can be trained by the main components, and the robustness of the neural network is improved. Moreover, the training set processing method provided by the present application processes the training set, and the processing method is only related to the distribution of the main components of the training set itself. Therefore, even if different models are trained in different scenarios, the training set processing method provided by the present application can be used to process the training set, and the generalization ability is strong, and the method is not dependent on the general algorithm of the trained model, so that the method has high universality.

[0129] Generally, the neural network model is prone to misclassification for the adversarial samples. The adversarial samples include perturbation information, and the perturbation included in the adversarial samples will reduce the output accuracy of the neural network. The adversarial samples include perturbation information, and the perturbation included in the adversarial samples will affect the robustness of the neural network obtained by subsequent training and reduce the output accuracy of the neural network. Exemplarily, a schematic diagram of an adversarial sample can be as shown in Figure 7As shown, the image A1 is an image without including the disturbance, the disturbance information A2 is superimposed on the basis of the image A1, the coefficient of the added disturbance information is 0.007, so as to obtain the image A3 added with the disturbance information, that is, the adversarial sample. Generally, the adversarial sample includes the disturbance information, which is easy to cause the recognition error of the neural network, for example, the actual classification of the image A1 is panda, and the image 3 added with the disturbance information can be identified as being classified as a dog, therefore, if the adversarial sample is used for training, the robustness of the neural network can be poor. In the embodiment of the present application, before the training set is used for training the neural network, the training set is processed, and the disturbance part in each sample in the training set is reduced, so that the training set with reduced disturbance can be used for training in the subsequent neural network training, the influence of the disturbance in the sample on the neural network is avoided, and the robustness of the neural network is improved.

[0130] In the embodiment of the present application, the subsequent training of the neural network is mainly based on the principal component of the sample in the manner of eliminating interference, and a neural network with better robustness is trained. Moreover, the neural network can be trained mainly based on the principal component of the sample, the influence on the output accuracy of the neural network is reduced while the robustness of the neural network is improved, and a neural network with balanced robustness and output accuracy is obtained.

[0131] It can be understood that in the training set processing method provided by the present application, the principal component in the encoding data corresponding to the original sample is retained, and other components except the principal component are replaced by noise or 0, so as to obtain a new sample in which other components except the principal component are eliminated. When the neural network is trained by using the training set including the new sample, the training is mainly based on the principal component, and the disturbance included in the sample is reduced. In the process of training, the data manifold is introduced to defend against the adversarial sample, so as to improve the robustness of the neural network. The data manifold can be understood as a linear space embedded in the data real dimension in the hypothesis, but the dimension is much lower than the manifold of the space. The observed input data points are distributed near it. The data such as natural images which are good at classification of neural network basically meet the distribution assumption. Generally, the principal components of different samples are quite different, and the successful classification of the principal component can obtain a neural network with higher robustness. Therefore, the robustness of the neural network is improved without reducing the output accuracy of the neural network, and the problem of gradient leakage in the training process of the neural network is inhibited.

[0132] The foregoing flow of the training set processing method provided by the present application is described in detail, in order to facilitate understanding, the training set processing method provided by the present application is described in more detail in combination with a specific application scenario.

[0133] The sample in the first training set can be represented as a data point x iThe first training set can be an N*D dimensional dataset, which can be represented as X = [x1, ... x2]. N ] T After performing PCA projection on the training set, an orthogonal basis {v_1,v_2,…,v_N} and the corresponding discrete sequences {λ_1,λ_2,…,λ_N} are obtained, with the variances of the discrete sequences arranged in descending order. The component dimension d of the principal components, the added noise dimension m, and the noise magnitude c are determined.

[0134] Using the covariance matrix Perform spectral decomposition, where... get λ i ≥λ2≥…≥λ D .

[0135] calculate The components of the mean of a given orthogonal basis V, V = [v1, ... v2] D ],get a refers to the first encoded data mentioned above.

[0136] Project x onto the PCA subspace, and replace the i-th component of a with noise, where, And i = d+1, d+2...d+m, the components after d+m are replaced with 0, resulting in the updated a. To represent noise, ξ i They follow a standard normal (Gaussian) distribution that is independent and identically distributed.

[0137] Then, the data is reconstructed to obtain a new sample. Multiple new samples then form the second training set.

[0138] Then, the new second training set can be used to train the neural network.

[0139] Therefore, in this scenario, the other components of the principal component are replaced with noise proportional to the corresponding variance, thereby improving the robustness of the neural network without affecting its training process.

[0140] To facilitate understanding, some other specific scenarios will be illustrated with examples.

[0141] First, for the training set {(x i y i Centering the input data (denoted as row vectors) in the sequence i = 1, 2, ..., n, we obtain... Then calculate the covariance matrix: Let its orthogonal diagonalization result be U Tdiag(B)U, where B is a non-negative vector with the same dimension as the input data (i.e., the samples in the training set), indicating the magnitude of the change in the input data in these directions.

[0142] Then the formula of the sample can include: Ux old , where s is a scale vector, ∈ is a noise vector, and c is a constant greater than 1. Generally, s can be taken as a truncated vector {1, 1, …, 1, 1, 0, 0, …, 0}, and the number of 1s is a truncated hyperparameter, i.e., the aforementioned preset value d. In order to achieve the purpose of robustness, it is generally necessary to further suppress the components corresponding to the smaller scale, and ∈ can be taken as a standard normal distribution vector near the truncated value (i.e., the dimension near the truncated value is a standard normal distribution, and the other dimensions are 0).

[0143] Alternatively, the aforementioned scale vector can be replaced by a smoothing vector, such as a point in the smoothing vector can be represented as s i = sigmoid(-i+d), d represents a soft truncation threshold, and i represents the subscript of the dimension.

[0144] Alternatively, the aforementioned scale vector can also be replaced by a random vector, such as a point in the random vector can be represented as s i = Bernoulli(sigmoid(-i+d)).

[0145] Alternatively, the aforementioned noise vector can also be replaced by a non-Gaussian distribution, but a vector with other variance levels independent of the distribution of the sample itself.

[0146] Therefore, in the present scenario, a method for detecting and improving the non-robust part of the training set to cause the neural network to be vulnerable to adversarial examples is proposed. The method provided by the present application has universality and high efficiency, can be directly applied to most tasks and network architectures, has strong generalization ability, and can train a neural network with higher output accuracy on the basis of improving the robustness of the neural network, and obtain a neural network with balanced robustness and output accuracy.

[0147] Generally, taking a classification network as an example, if a defense method of projection in the test phase is adopted, that is, the perturbation in the adversarial sample is trained in the test phase, since there is an error in the manifold projected compared with the real data manifold, the data not subjected to projection is used for training during training, and the classification effect of the model on the data on the manifold is unknown. On the one hand, it will lead to a significant decline in classification accuracy, and on the other hand, since the classification network is not well trained on the manifold, it is easier to find adversarial samples on the manifold. However, the method provided in the present application can avoid the above problems by performing the projection operation in the training. In addition, the training set obtained by the method provided in the present application can not need to project the input data when the classification network is trained, which improves the training efficiency.

[0148] If the adversarial sample is trained, it often needs several times of time and higher computing resources to calculate the adversarial sample of the input sample. However, in the method provided in the present application, the relatively essential manifold outside the adversarial direction is separated to add noise and augment, which avoids the difficulty increase caused by the misclassification sample training in the manifold and the decline of the original test set accuracy during the model training. Therefore, the present application can have a better and more flexible trade-off between the robust accuracy and the original test set accuracy. Moreover, according to the application scene, the present application can adjust the parameter d and the specific distribution and amplitude of the added noise to flexibly adjust the final accuracy of the model training and balance the output accuracy and robustness of the neural network.

[0149] The foregoing has introduced the method provided in the present application in detail, and the device provided in the present application is introduced based on the foregoing method.

[0150] Referring to Figure 8 , a structural schematic diagram of a training set processing device provided in the present application is provided, and the training set processing device is used to execute the steps of the method in the foregoing Figure 6 .

[0151] The training set processing device can include:

[0152] A decomposition unit 801 is configured to decompose the first training set to obtain a basis and a discrete sequence corresponding to the basis, the first training set including a plurality of samples, the basis including at least one basis vector, and the discrete degree including a discrete value corresponding to each basis vector in the basis, and the discrete sequence can be used to represent the discrete degree of the basis;

[0153] An acquisition unit 802 is configured to acquire a component value of each sample in the first training set in the basis to obtain a plurality of groups of first encoding data, each group of first encoding data corresponding to a sample;

[0154] The acquisition unit 802 is further configured to acquire principal components in each of the plurality of groups of first encoded data, to obtain a plurality of groups of second encoded data, and the principal components correspond to discrete values higher than a preset discrete value.

[0155] The mapping unit 803 is configured to map the plurality of groups of second encoded data to bases of the first training set, to obtain samples corresponding to the plurality of groups of second encoded data, and the samples corresponding to the plurality of groups of second encoded data form a second training set, and the second training set is used for training a neural network.

[0156] In a possible implementation, the bases are orthogonal bases, and the decomposition unit 801 is specifically configured to perform principal component analysis (PCA) on the first training set, to obtain the orthogonal bases and discrete sequences corresponding to the orthogonal bases.

[0157] In a possible implementation, the decomposition unit 801 is specifically configured to: perform centering processing on the first training set, to obtain a centered first training set, and the centered first training set includes data with a mean value of 0; and perform PCA on the centered first training set, to obtain the orthogonal bases and a discrete sequence of the orthogonal bases, and the discrete sequence includes a sequence of variances of the centered first training set in the orthogonal bases.

[0158] In a possible implementation, the decomposition unit 801 is specifically configured to calculate, by using a preset algorithm, a component value of each sample in the centered first training set on each base vector in the orthogonal bases, to obtain the plurality of groups of first encoded data.

[0159] In a possible implementation, the preset algorithm includes inner product operation or sparse coding operation.

[0160] In a possible implementation, the acquisition unit 802 is specifically configured to retain the principal components in each of the plurality of groups of first encoded data, and replace other components in each of the plurality of groups of first encoded data except the principal components with preset values, to obtain the plurality of groups of second encoded data, and the principal components in each of the plurality of groups of first encoded data form at least one group of second encoded data.

[0161] In a possible implementation, the preset values include 0 or a preset noise vector, and the preset noise vector includes Gaussian noise or uniformly distributed noise.

[0162] In a possible implementation, the preset noise vector is proportional to a variance corresponding to the replaced components.

[0163] Please refer to Figure 9 Another structure diagram of the training set processing apparatus provided in the present application is provided as follows.

[0164] The training set processing apparatus can include a processor 901 and a memory 902. The processor 901 and the memory 902 are interconnected by a line. The memory 902 stores program instructions and data.

[0165] The memory 902 stores program instructions and data corresponding to the steps in the foregoing Figure 6 .

[0166] The processor 901 is configured to execute the method steps performed by the training set processing apparatus shown in any of the foregoing Figure 6 .

[0167] Optionally, the training set processing apparatus can further include a transceiver 903 for receiving or sending data.

[0168] The embodiments of the present application also provide a computer readable storage medium, which stores a program for generating vehicle driving speed, and when the program is executed on a computer, the computer is caused to perform the steps in the method described in the foregoing Figure 6 .

[0169] Optionally, the training set processing apparatus shown in the foregoing Figure 9 is a chip.

[0170] The embodiments of the present application also provide a training set processing apparatus, which can also be referred to as a digital processing chip or a chip. The chip includes a processing unit and a communication interface. The processing unit obtains program instructions through the communication interface. The program instructions are executed by the processing unit. The processing unit is configured to execute the method steps performed by the training set processing apparatus shown in the foregoing Figure 6 .

[0171] The embodiments of the present application also provide a digital processing chip. The digital processing chip integrates a circuit for implementing the processor 901 or the functions of the processor 901 and one or more interfaces. When the digital processing chip integrates a memory, the digital processing chip can complete the method steps of any one or more of the foregoing embodiments. When the digital processing chip does not integrate a memory, the digital processing chip can be connected with an external memory through a communication interface. The digital processing chip implements the actions of the training set processing apparatus in the foregoing embodiments according to the program code stored in the external memory.

[0172] The embodiments of the present application also provide a computer program product, which, when executed on a computer, causes the computer to perform the steps performed by the training set processing apparatus in the method described in the foregoing Figure 6 .

[0173] The training set processing apparatus provided by the embodiments of the present application can be a chip, which comprises a processing unit, for example, a processor, and a communication unit, for example, an input / output interface, a pin, a circuit, etc. The processing unit can execute computer execution instructions stored in a storage unit, so that the chip in the server executes the training set processing method described above. Figure 6 Optionally, the storage unit is a storage unit in the chip, such as a register, a cache, etc. The storage unit can also be a storage unit outside the chip in the wireless access device, such as a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM), etc.

[0174] Specifically, the aforementioned processing unit or processor can be a central processing unit (CPU), a neural-network processing unit (NPU), a graphics processing unit (GPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), or a field programmable gate array (FPGA) or other programmable logic devices, a discrete gate or transistor logic device, a discrete hardware component, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0175] For example, refer to Figure 10 , Figure 10 A structural schematic diagram of the chip provided by the embodiments of the present application is shown in FIG. 1. The chip can be a neural-network processing unit (NPU) 100, which is mounted on a host CPU (Host CPU) as a coprocessor and is assigned tasks by the Host CPU. The core part of the NPU is an operation circuit 100, which extracts matrix data in a memory and performs multiplication operation under the control of a controller 1004.

[0176] ​In some implementations, the arithmetic circuit 1003 includes a plurality of processing engines (PEs) inside. In some implementations, the arithmetic circuit 1003 is a two-dimensional systolic array. The arithmetic circuit 1003 can also be a one-dimensional systolic array or other electronic circuit capable of performing mathematical operations such as multiplication and addition. In some implementations, the arithmetic circuit 1003 is a general-purpose matrix processor.

[0177] For example, assume that there is an input matrix A, a weight matrix B, and an output matrix C. The arithmetic circuit takes the data of the matrix B corresponding from the weight memory 1002 and caches it on each PE of the arithmetic circuit. The arithmetic circuit takes the data of the matrix A from the input memory 1001 and performs matrix operations with the matrix B to obtain partial results or final results of the matrix, which are saved in the accumulators 1008.

[0178] The unified memory 1006 is used to store input data and output data. The weight data is transferred to the weight memory 1002 through the direct memory access controller (DMAC) 1005. The input data is also transferred to the unified memory 1006 through the DMAC.

[0179] The bus interface unit (BIU) 1010 is used for the interaction between the AXI bus and the DMAC and the instruction fetch buffer (IFB) 1009.

[0180] The bus interface unit 1010 (BIU) is used for the instruction fetch buffer 1009 to obtain instructions from the external memory and is also used for the direct memory access controller 1005 to obtain the original data of the input matrix A or the weight matrix B from the external memory.

[0181] The DMAC is mainly used to transfer the input data in the external memory DDR to the unified memory 1006 or to transfer the weight data to the weight memory 1002 or to transfer the input data to the input memory 1001.

[0182] The vector calculation unit 1007 includes a plurality of arithmetic processing units, which further process the output of the arithmetic circuit as needed, such as vector multiplication, vector addition, exponential operation, logarithmic operation, size comparison, etc. It is mainly used for non-convolutional / full connection layer network calculation in neural networks, such as batch normalization, pixel-level summation, upsampling of feature planes, etc.

[0183] In some implementations, the vector computation unit 1007 can store the processed output vector to the unified memory 1006. For example, the vector computation unit 1007 can apply a linear function and / or a non-linear function to the output of the arithmetic circuit 1003, such as linear interpolation on the feature planes extracted by a convolution layer, and / or accumulate the vector of values to generate activation values. In some implementations, the vector computation unit 1007 generates normalized values, pixel-wise summed values, or both. In some implementations, the processed output vector can be used as an activation input to the arithmetic circuit 1003, such as for use in a subsequent layer in a neural network.

[0184] The controller 1004 is connected to an instruction fetch buffer 1009 for storing instructions used by the controller 1004;

[0185] The unified memory 1006, the input memory 1001, the weight memory 1002, and the instruction fetch buffer 1009 are on-chip memories. Off-chip memories are private to the NPU hardware architecture.

[0186] In some implementations, the operations of the layers in the recurrent neural network can be performed by the arithmetic circuit 1003 or the vector computation unit 1007.

[0187] In some implementations, the processor mentioned in any of the above can be a general central processing unit, a microprocessor, an ASIC, or one or more integrated circuits for executing programs for controlling the above Figure 6 method.

[0188] It should be noted that the apparatus embodiments described above are merely illustrative, and the units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, i.e., they can be located in one place or distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the present embodiment. In addition, the connection relationship between the modules in the apparatus embodiment provided in the present application indicates that there is a communication connection between them, which can be implemented as one or more communication buses or signal lines.

[0189] Those skilled in the art can clearly understand that the application can be implemented by means of software plus necessary universal hardware, and of course can also be implemented by means of dedicated hardware including special-purpose integrated circuits, special-purpose CPUs, special-purpose memories, special-purpose components, etc. Generally, any function completed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure for implementing the same function can also be various, such as analog circuits, digital circuits, or special-purpose circuits, etc. However, for the present application, software program implementation is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a readable storage medium, such as a floppy disk, a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc., and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments of the present application.

[0190] In the above embodiments, all or part can be implemented by software, hardware, firmware, or any combination thereof. When implemented by software, all or part can be implemented in the form of a computer program product.

[0191] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another, for example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a server, data center, etc. integrated with one or more available media. The available media can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk (SSD)), etc.

[0192] The terms "first", "second", "third", "fourth" and the like in the description and in the claims of the present application, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the use of these terms herein is to be construed to cover a generalised use of these terms to refer to similar elements independently of any specific sequential or chronological order. It is to be understood that the data used herein is to be taken in a generic and descriptive sense and not necessarily in a limiting sense. Furthermore, the terms "comprising", "having", "including", and "containing" are to be construed as open-ended terms (i.e., meaning "including, but not limited to", "comprising, but not limited to", "having, but not limited to", or "including, but not limited to") unless otherwise noted. It is to be understood that such terms are merely to be construed as specifying the presence of what follows the term, but does not preclude the presence of additional, non-recited items.

[0193] Finally, it should be noted that the above-mentioned merely preferred embodiments of the present application are illustrative, and not restrictive, rather, the scope of the present application should be given by the appended claims along with full scope of equivalents to which such claims are entitled.

Claims

1. A training set processing method, characterized by, The method comprises the following steps: receiving a first training set sent by a client device, the first training set comprising images; performing decomposition on all images in the first training set to obtain a basis of the first training set and a discrete sequence of the basis, the first training set comprising a plurality of samples, the basis comprising at least one basis vector, and the discrete sequence comprising a discrete value corresponding to each basis vector in the basis; obtaining component values of each sample in the first training set on each basis vector to obtain a plurality of groups of first encoding data, each group of first encoding data corresponding to a sample; obtaining principal components in each group of first encoding data in the plurality of groups of first encoding data to obtain a plurality of groups of second encoding data, the discrete value of the principal component being higher than a preset discrete value; mapping the plurality of groups of second encoding data to the basis of the first training set to obtain samples corresponding to the plurality of groups of second encoding data, the samples corresponding to the plurality of groups of second encoding data forming a second training set, and the second training set being used for training a neural network; training the neural network using the second training set to obtain a trained neural network; deploying the trained neural network in the client device, and the trained neural network being used for processing images.

2. The method of claim 1, wherein, The basis of the first training set is an orthogonal basis, and the decomposition of all images in the first training set comprises the following steps: performing principal component analysis (PCA) processing on the first training set to obtain the orthogonal basis and a discrete sequence of the orthogonal basis.

3. The method of claim 2, wherein, The PCA processing on the first training set comprises the following steps: performing centering processing on the first training set to obtain a centered first training set, the centered first training set comprising data with a mean value of 0; performing the PCA processing on the centered first training set to obtain the orthogonal basis and a discrete sequence of the orthogonal basis, the discrete sequence comprising a sequence of variances of the centered first training set in the orthogonal basis.

4. The method of claim 3, wherein, The obtaining of the component values of each sample in the first training set comprises the following steps: calculating, by a preset algorithm, the component values of each sample in the centered first training set on each basis vector in the orthogonal basis to obtain the plurality of groups of first encoding data.

5. The method of claim 4, wherein, The preset algorithm comprises inner product operation or sparse coding operation.

6. The method according to any one of claims 1-5, characterized in that, The obtaining of the principal components in each group of first encoding data comprises the following steps: retaining the principal components in each group of first encoding data, replacing other components in each group of first encoding data except the principal components with a preset value to obtain the plurality of groups of second encoding data, wherein the principal components in each group of first encoding data form at least one group of second encoding data.

7. The method of claim 6, wherein, The preset value comprises 0 or a preset noise vector, and the preset noise vector comprises Gaussian noise or uniformly distributed noise.

8. The method of claim 7, wherein, The preset noise vector is proportional to the discrete value of the replaced component.

9. A training set processing apparatus characterized by comprising: The method comprises the following steps: The acquisition unit is configured to receive a first training set sent by the client device, the first training set comprising images; The decomposition unit is configured to decompose all images in the first training set to obtain a basis of the first training set and a discrete sequence of the basis, the first training set comprising a plurality of samples, the basis comprising at least one basis vector, and the discrete sequence comprising a discrete value corresponding to each basis vector in the basis; The acquisition unit is further configured to obtain a component value of each sample in the first training set on each basis vector to obtain a plurality of groups of first encoding data, each group of first encoding data corresponding to one sample; The acquisition unit is further configured to obtain a principal component in each group of first encoding data in the plurality of groups of first encoding data to obtain a plurality of groups of second encoding data, a discrete value of the principal component being higher than a preset discrete value, wherein the principal component of each group of first encoding data constitutes at least one group of second encoding data; The mapping unit is configured to map the plurality of groups of second encoding data to the basis of the first training set to obtain samples corresponding to the plurality of groups of second encoding data, the samples corresponding to the plurality of groups of second encoding data constituting a second training set, and the second training set being used to train a neural network; The mapping unit is further configured to train the neural network using the second training set to obtain a trained neural network; The mapping unit is further configured to deploy the trained neural network in the client device, and the trained neural network is used to process images.

10. The apparatus of claim 9, wherein, The basis is an orthogonal basis, The decomposition unit is specifically configured to perform principal component analysis (PCA) processing on the first training set to obtain the orthogonal basis and a discrete sequence of the orthogonal basis.

11. The apparatus of claim 10, wherein, The decomposition unit is specifically configured to: perform centering processing on the first training set to obtain a centered first training set, the centered first training set comprising data with a mean value of 0; perform the PCA processing on the centered first training set to obtain the orthogonal basis and a discrete sequence of the orthogonal basis, the discrete sequence comprising a sequence of variances of the centered first training set in the orthogonal basis.

12. The apparatus of claim 11, wherein The decomposition unit is specifically configured to calculate a component value of each sample in the centered first training set on each basis vector in the orthogonal basis by using a preset algorithm to obtain the plurality of groups of first encoding data.

13. The apparatus of claim 12, wherein, The preset algorithm comprises inner product operation or sparse coding operation.

14. The apparatus of any one of claims 9-13, wherein The acquisition unit is specifically configured to retain the principal component in each group of first encoding data and replace other components in the each group of first encoding data except the principal component with a preset value to obtain the plurality of groups of second encoding data.

15. The apparatus of claim 14, wherein, The preset value comprises 0 or a preset noise vector, and the preset noise vector comprises Gaussian noise or uniformly distributed noise.

16. The apparatus of claim 15, wherein, The preset noise vector is proportional to a discrete value of the replaced component.

17. A training set processing apparatus characterized by comprising: A processor coupled to a memory storing a program which, when executed by the processor, implements the method of any one of claims 1 to 8.

18. A computer readable storage medium comprising a program which, when executed by a processing unit, performs the method of any one of claims 1 to 8.

19. A training set processing apparatus characterized by comprising: A processing unit and a communication interface, the processing unit obtaining program instructions via the communication interface, the program instructions, when executed by the processing unit, implementing the method of any one of claims 1 to 8.

Images