Training data processing of a neural network model, device, electronic equipment and storage medium

Abstract

The application provides a kind of neural network model training data processing, device, equipment and storage medium, method includes: to the first training data set is rotated self-supervision processing, to form corresponding second training data set;Determine the initial training data set corresponding to second training data set;According to second training data set, initial training data set, by first training data processing network and second training data processing network, determine the gradient parameter matched with the initial training data set;Initial training data set is updated, determine the target training data set matched with target neural network model.Therefore, it can be under the premise of reducing the total amount of training data, the accuracy of small-scale training data to neural network model training is steadily improved, the generalization ability of target neural network model is enhanced, the trained target neural network model is deployed in mobile terminal, and the large-scale application of target neural network model is realized.

Description

Technical Field

[0001] This invention relates to machine learning technology, and more particularly to a training data processing device, electronic device, and storage medium for a neural network model. Background Technology

[0002] In artificial intelligence, deep neural networks have achieved excellent performance in many computer vision tasks. Generally, the larger the amount of training data and the greater the computational cost, the better the performance of the trained neural network model. However, deploying such large-scale training data on resource-constrained embedded systems is very difficult, and the performance of neural networks obtained directly from small-scale training data is far lower than that of larger-scale networks. Related technologies can compress training data through data distillation, but distillation cannot be performed on unlabeled data, making the training data processing complex. Furthermore, the excessive amount of training data leads to excessively high hardware costs for training and storing neural network models, hindering large-scale deployment. Summary of the Invention

[0003] In view of this, embodiments of the present invention provide a training data processing device, electronic device, and storage medium for a neural network model. This device can process large-scale unlabeled training data into smaller-scale training data through data distillation. This reduces the total amount of training data and eliminates the need for retraining, while steadily improving the accuracy of training the neural network model with smaller-scale training data. Furthermore, it is generally applicable to most neural network models and data. The smaller-scale training data also reduces overfitting of the target neural network model, enhances its generalization ability, and facilitates the deployment of the trained target neural network model on mobile terminals, enabling large-scale application of the target neural network model.

[0004] The technical solution of this invention is implemented as follows:

[0005] This invention provides a method for processing training data for a neural network model, the method comprising:

[0006] Obtain a first training data set, wherein the first training data set includes at least one set of image datasets;

[0007] The first training dataset is subjected to rotational self-supervised processing to form a corresponding second training dataset.

[0008] Based on the second training data set, determine the initial training data set corresponding to the second training data set;

[0009] The first training data processing network is trained using the second training data set, and the second training data processing network is trained using the initial training data set to determine gradient parameters that match the initial training data set.

[0010] Based on the gradient parameters that match the initial training data set, the initial training data set is updated to determine the target training data set that matches the target neural network model.

[0011] This invention also provides a training data processing device for a neural network model, the training device comprising:

[0012] The information transmission module is used to acquire a first training data set, wherein the first training data set includes at least one image dataset;

[0013] The information processing module is used to perform rotational self-supervised processing on the first training data set to form a corresponding second training data set;

[0014] The information processing module is used to determine an initial training data set corresponding to the second training data set based on the second training data set.

[0015] The information processing module is used to train the first training data processing network using the second training data set, and to train the second training data processing network using the initial training data set, so as to determine gradient parameters that match the initial training data set.

[0016] The information processing module is used to update the initial training data set based on gradient parameters that match the initial training data set, and to determine the target training data set that matches the target neural network model.

[0017] In the above scheme,

[0018] The information processing module is used to rotate the training images in the first training dataset at a fixed center angle.

[0019] The information processing module is used to randomly scale the training images in the first training dataset after being rotated at a fixed angle.

[0020] The information processing module is used to randomly shift the training images in the first training data set after random scaling to form a corresponding second training data set.

[0021] In the above scheme, the information processing module is used to insert noise into the training images in the first training data set after being rotated at a fixed angle, and the noise includes at least one of fixed noise and dynamic noise.

[0022] In the above scheme,

[0023] The information processing module is used to determine a dynamic noise threshold that matches the usage environment of the target neural network model.

[0024] The information processing module is configured to add noise to the first training data set according to the dynamic noise threshold, so as to form a second training data set that matches the dynamic noise threshold; or,

[0025] The information processing module is used to determine a fixed noise threshold corresponding to the target neural network model, and to add noise to the first training data set according to the fixed noise threshold to form a second training data set that matches the fixed noise threshold.

[0026] In the above scheme,

[0027] The information processing module is used to determine a sample acquisition interval parameter matching the target neural network model based on the usage environment of the target neural network model, and to acquire image data in the second training data set according to the sample acquisition interval parameter to form an initial training data set; or

[0028] The information processing module is used to randomly collect image data from the second training data set to form an initial training data set.

[0029] In the above scheme,

[0030] The information processing module is used to train the first training data processing network using the second training data set, and determine the first gradient parameters corresponding to the first training data processing network.

[0031] The second training data processing network is trained using the initial training data set to determine the second gradient parameters corresponding to the second training data processing network.

[0032] Based on the first gradient parameter and the second gradient parameter, gradient parameter matching processing is performed to determine the gradient parameters that match the initial training data set.

[0033] In the above scheme,

[0034] The information processing module is used to determine the gradient of the parameters of each layer of the neural network in the first training data processing network when backpropagating through the second training data set.

[0035] The information processing module is used to determine the first loss function corresponding to the first training data processing network based on the gradient of the parameters of each layer of the neural network in the first training data processing network.

[0036] The information processing module is used to determine the first gradient parameters corresponding to the first training data processing network based on the first loss function corresponding to the first training data processing network.

[0037] In the above scheme,

[0038] The information processing module is used to determine the gradient of the parameters of each layer of the neural network in the second training data processing network when backpropagating through the second training data processing network using the initial training data set.

[0039] The information processing module is used to determine the second loss function corresponding to the second training data processing network based on the gradient of the parameters of each layer of the neural network in the second training data processing network.

[0040] The information processing module is used to determine the second gradient parameters corresponding to the second training data processing network based on the second loss function corresponding to the second training data processing network.

[0041] In the above scheme,

[0042] The information processing module is used to iteratively update the initial training data set based on gradient parameters that match the initial training data set until the distance between the first gradient parameter and the second gradient parameter reaches a distance threshold.

[0043] The information processing module is used to determine that when the distance between the first gradient parameter and the second gradient parameter reaches a distance threshold, the updated initial training dataset is used as a target training dataset that matches the target neural network model.

[0044] In the above scheme,

[0045] The information processing module is used to adjust the types of the first training data processing network and the second training data processing network based on the usage environment that matches the target neural network model, so as to make the first training data processing network and the second training data processing network compatible with the usage environment that matches the target neural network model.

[0046] This invention also provides an electronic device, the electronic device comprising:

[0047] Memory, used to store executable instructions;

[0048] A processor, used to implement a training data processing method for a preceding neural network model when executing executable instructions stored in the memory.

[0049] This invention also provides a computer-readable storage medium storing executable instructions, which, when executed by a processor, implement a method for processing training data of a preceding neural network model.

[0050] This application also provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform various alternative implementations and combinations thereof of the training data processing method for the described neural network model.

[0051] The embodiments of the present invention have the following beneficial effects:

[0052] By acquiring a first training dataset, which includes at least one image dataset; performing rotational self-supervised processing on the first training dataset to form a corresponding second training dataset; determining an initial training dataset corresponding to the second training dataset based on the second training dataset; training the first training data processing network using the second training dataset, and training the second training data processing network using the initial training dataset to determine gradient parameters matching the initial training dataset; updating the initial training dataset based on the gradient parameters matching the initial training dataset to determine a target training dataset matching the target neural network model. Therefore, it is possible to stably improve the accuracy of training a neural network model with a smaller amount of training data without retraining, and it is generally applicable to most neural network models and data. Simultaneously, the smaller training data reduces overfitting of the target neural network model, enhances its generalization ability, and facilitates the deployment of the trained target neural network model on mobile terminals, enabling large-scale application of the target neural network model. Attached Figure Description

[0053] Figure 1 This is a schematic diagram illustrating the usage environment for training data processing of a neural network model according to an embodiment of the present invention;

[0054] Figure 2 A schematic diagram of the composition structure of the training data processing device for the neural network model provided in an embodiment of the present invention;

[0055] Figure 3 A schematic diagram of an optional process for processing training data of the neural network model provided in this embodiment of the invention;

[0056] Figure 4 This is a schematic diagram illustrating the rotational self-supervised processing of the first training data set in an embodiment of the present invention;

[0057] Figure 5 This is a schematic diagram of the training data processing process in an embodiment of the present invention;

[0058] Figure 6 A schematic diagram of an optional process for processing training data of the neural network model provided in this embodiment of the invention;

[0059] Figure 7 This is a schematic diagram of gradient parameter matching during the training data processing in an embodiment of the present invention;

[0060] Figure 8 This is an optional flowchart illustrating the training data processing method for the neural network model provided in this embodiment of the invention. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on the present invention. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0062] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0063] In the implementation of this application, the collection and processing of relevant data should strictly comply with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0064] Before providing a further detailed description of the embodiments of the present invention, the nouns and terms involved in the embodiments of the present invention will be explained, and the nouns and terms involved in the embodiments of the present invention shall be interpreted as follows.

[0065] 1) BERT: Short for Bidirectional Encoder Representations from Transformers, it is a method for training language models using massive amounts of text. This method is widely used in various natural language processing tasks, such as text classification, text matching, and machine reading comprehension.

[0066] 2) Artificial Neural Network: Abbreviated as Neural Network (NN), in the fields of machine learning and cognitive science, it is a mathematical or computational model that imitates the structure and function of biological neural networks and is used to estimate or approximate functions.

[0067] 3) Model parameters: These are quantities used to establish the relationship between functions and variables using general variables. In artificial neural networks, model parameters are typically real-valued matrices.

[0068] 4) Downsampling: For a sample sequence, samples are taken at intervals of several samples. The resulting new sequence is a downsampled version of the original sequence. For example, for an image I with size M*N, downsampling it by a factor of s yields a resolution image of size (M / s)*(N / s), where s should be a common divisor of M and N.

[0069] 5) Encoder-decoder structure: A commonly used network structure in machine translation technology. It consists of two parts: an encoder and a decoder. The encoder converts the input text into a series of context vectors that can express the features of the input text. The decoder receives the output of the encoder as its own input and outputs the corresponding text sequence in another language.

[0070] 6) Dataset distillation: The knowledge from tens of thousands of images is compressed into a few synthetic training images, which are called distilled images. (a) On the MNIST dataset, 10 distilled images can train a standard LENET with a specific fixed initialization setting to achieve 94% test accuracy (the model trained on the full training dataset achieves 99% performance).

[0071] Figure 1 This is a schematic diagram illustrating a usage scenario for training data processing of the neural network model provided in this embodiment of the invention. (Refer to...) Figure 1 The terminals (including terminals 10-1 and 10-2) are equipped with image processing software clients. Users can input corresponding images to be processed through the image processing software clients, and the image processing clients can also receive the corresponding image processing results and display the received image processing results to the users. The terminals are connected to the server 200 through network 300, which can be a wide area network, a local area network, or a combination of both, and data transmission is achieved using a wireless link.

[0072] As an example, server 200 is used to deploy the target neural network model and train the target neural network model to iteratively update the generator parameters and discriminator parameters of the target neural network model, so as to generate image processing results for the target image to be processed through the generator network in the target neural network model, and display the image processing results generated by the target neural network model corresponding to the image to be processed through the terminal (terminal 10-1 and / or terminal 10-2).

[0073] Of course, before processing the target image to be processed through the target neural network model to generate the corresponding image processing result, it is necessary to train the target neural network model. During training, it is necessary to first obtain training data that has undergone data distillation, specifically including: obtaining a first training data set, wherein the first training data set includes at least one set of image datasets; performing rotation self-supervised processing on the first training data set to form a corresponding second training data set; determining an initial training data set corresponding to the second training data set based on the second training data set; training the first training data processing network through the second training data set, and training the second training data processing network through the initial training data set to determine gradient parameters that match the initial training data set; updating the initial training data set based on the gradient parameters that match the initial training data set to determine a target training data set that matches the target neural network model.

[0074] The structure of the training data processing device for the neural network model according to an embodiment of the present invention will be described in detail below. The training data processing device for the neural network model can be implemented in various forms, such as a dedicated terminal with target neural network model training function, or a server with target neural network model training function, for example, the preceding... Figure 1 Server 200 in the middle. Figure 2 This is a schematic diagram of the composition of the training data processing device for the neural network model provided in an embodiment of the present invention. It can be understood that... Figure 2 This is merely an exemplary structure of the training data processing device for the neural network model, and not the entire structure; it can be implemented as needed. Figure 2 The structure shown may be part or all of the structure.

[0075] The neural network model training data processing apparatus provided in this embodiment of the invention includes: at least one processor 201, a memory 202, a user interface 203, and at least one network interface 204. The various components in the neural network model training data processing apparatus are coupled together via a bus system 205. It can be understood that the bus system 205 is used to implement communication between these components. In addition to a data bus, the bus system 205 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 2 The general labeled all buses as Bus System 205.

[0076] The user interface 203 may include a monitor, keyboard, mouse, trackball, click wheel, buttons, touchpad, or touch screen.

[0077] It is understood that memory 202 can be volatile memory or non-volatile memory, or both. In this embodiment of the invention, memory 202 is capable of storing data to support the operation of a terminal (such as 10-1). Examples of this data include any computer programs used to operate on the terminal (such as 10-1), such as operating systems and applications. The operating system includes various system programs, such as the framework layer, core library layer, driver layer, etc., used to implement various basic services and handle hardware-based tasks. Applications can include various applications.

[0078] In some embodiments, the training data processing device for the neural network model provided in this invention can be implemented using a combination of hardware and software. For example, the training data processing device for the neural network model provided in this invention can be a processor in the form of a hardware decoding processor, which is programmed to perform the training data processing of the neural network model provided in this invention. For instance, the processor in the form of a hardware decoding processor can employ one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0079] As an example of the combined hardware and software implementation of the training data processing device for the neural network model provided in this embodiment of the invention, the training data processing device for the neural network model provided in this embodiment of the invention can be directly embodied as a combination of software modules executed by the processor 201. The software modules can be located in a storage medium, which is located in the memory 202. The processor 201 reads the executable instructions included in the software modules in the memory 202 and combines them with necessary hardware (e.g., including the processor 201 and other components connected to the bus 205) to complete the training data processing of the neural network model provided in this embodiment of the invention.

[0080] As an example, processor 201 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., wherein the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0081] As an example of the hardware implementation of the training data processing device for the neural network model provided in this embodiment of the invention, the device provided in this embodiment of the invention can be directly executed by a processor 201 in the form of a hardware decoding processor. For example, it can be executed by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components to implement the training data processing of the neural network model provided in this embodiment of the invention.

[0082] In this embodiment of the invention, the memory 202 is used to store various types of data to support the operation of the training data processing device for the neural network model. Examples of such data include: any executable instructions for operation on the training data processing device for the neural network model, such as executable instructions, which may be included in the executable instructions to implement the training data processing of the neural network model according to this embodiment of the invention.

[0083] In other embodiments, the training data processing apparatus for the neural network model provided in this invention can be implemented in software. Figure 2A training data processing device for a neural network model stored in memory 202 is shown. This device can be software in the form of programs and plugins, and includes a series of modules. As an example of a program stored in memory 202, it may include a training data processing device for a neural network model. This device includes the following software modules: an information transmission module 2081 and an information processing module 2082. When the software modules in the training data processing device are read into RAM and executed by processor 201, the training data processing of the neural network model provided in this embodiment of the invention will be implemented. The functions of each software module in the training data processing device for the neural network model in this embodiment of the invention are described below.

[0084] Information transmission module 2081 is used to acquire a first training data set, wherein the first training data set includes at least one set of image datasets;

[0085] The information processing module 2082 is used to perform rotational self-supervised processing on the first training data set to form a corresponding second training data set;

[0086] The information processing module 2082 is used to determine an initial training data set corresponding to the second training data set based on the second training data set.

[0087] The information processing module 2082 is used to train the first training data processing network using the second training data set, and to train the second training data processing network using the initial training data set, so as to determine gradient parameters that match the initial training data set.

[0088] The information processing module 2082 is used to update the initial training data set based on gradient parameters that match the initial training data set, and to determine the target training data set that matches the target neural network model.

[0089] The technical solution provided by this invention utilizes artificial intelligence (AI) technology. AI is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0090] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0091] Machine learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instructional learning.

[0092] With the research and advancement of artificial intelligence (AI) technology, AI is being studied and applied in various fields, such as smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, autonomous driving, drones, robots, smart healthcare, and smart customer service. It is believed that with the development of technology, AI will be applied in more fields and play an increasingly important role.

[0093] Combination Figure 2The training sample generation device shown illustrates the training data processing method for the neural network model provided in this embodiment of the invention. Before introducing the training data processing method for the neural network model provided in this embodiment of the invention, it is first introduced that in the process of processing large-scale training data in related technologies, in order to make the target neural network corresponding to the training data adapt to practical application scenarios such as federated learning, continuous learning, and hyperparameter optimization, it is necessary to compress the large-scale training data. For example, 1) through the core set construction method, this type of method selects the most informative samples from the complete dataset, so that the model's training performance on these samples is as close as possible to the complete dataset; 2) through the active learning process, the most valuable samples are selected from the complete unlabeled dataset for labeling, so as to reduce the labeling cost as much as possible; 3) through the data set distillation method, the complete dataset is distilled into a small-scale synthetic dataset, so that it covers all the information of the original dataset as much as possible. However, in these schemes, the core set construction and active learning process select samples from the complete dataset. Therefore, the information contained in the unselected samples is directly discarded, which cannot guarantee the integrity of the training data. Dataset distillation cannot process training data that has not been labeled. At the same time, the processing time for training data with a large data scale is too long, which increases the computational load of hardware devices and increases hardware costs.

[0094] To address the shortcomings in this related technology, see [link to relevant documentation]. Figure 3 , Figure 3 This is an optional flowchart illustrating the training data processing of the neural network model provided in this embodiment of the invention. It can be understood that... Figure 3 The steps shown can be performed by various electronic devices of the training data processing device that runs the neural network model, such as a dedicated terminal with sample generation function, a server or server cluster with target neural network model training function. Terminal 1 and Terminal 2 can be located in different image processing environments; for example, Terminal 1 can be located in a facial expression image recognition environment, and Terminal 2 can be located in a facial age image recognition environment. The complete training sample can be 100GB of facial images, and the acquired target training dataset 1 and target training dataset 2 can have a data volume of 10GB (occupying only one-tenth of the hardware storage space). Through the training data processing method provided in this application, target training dataset 1 and target training dataset 2 can be obtained based on the complete training sample. However, the training data contained in the complete training sample is not completely consistent, enabling training of the target neural network in different application scenarios to achieve the same training effect as the complete training sample. The following section focuses on... Figure 3 The steps shown are explained.

[0095] Step 301: The training data processing device for the neural network model acquires a first training data set, wherein the first training data set includes at least one set of unlabeled image datasets.

[0096] Labeling image datasets increases the manual cost of training data generation. Furthermore, labeled image datasets are not always suitable for the target neural network model. Given the complex environments in which target neural network models are used, traditional data distillation typically involves manual data labeling by professionals. In many real-world scenarios, the same target may have multiple types of labeling information. The same labeling content cannot effectively handle this one-to-many problem, resulting in a poor user experience. Therefore, the neural network model training data processing method provided in this application processes general unlabeled datasets. The first training data set obtained includes at least one set of unlabeled image datasets to achieve data distillation processing of the unlabeled image datasets.

[0097] Step 302: The training data processing device of the neural network model performs rotational self-supervised processing on the first training data set to form a corresponding second training data set.

[0098] In some embodiments of the present invention, the first training data set is subjected to rotational self-supervised processing to form a corresponding second training data set, which can be achieved in the following ways:

[0099] The training images in the first training dataset are rotated at a fixed angle; the rotated images are then randomly scaled; and the scaled images are then randomly translated to form a second training dataset. (Referring to...) Figure 4 , Figure 4 This is a schematic diagram of rotation self-supervised processing of the first training data set in an embodiment of the present invention. For a given image data set x, each image used as a training sample can be rotated by 0 degrees, 90 degrees, 180 degrees and 270 degrees respectively and then input into the neural network model to construct a 4-classification problem to train the target neural network model.

[0100] Because target neural networks are used in a variety of environments, taking a target neural network as a medical image processing model as an example, in the process of processing a large amount of unlabeled medical image data, the image processing model processes a set of medical images to be segmented for the same target region, which can be a set of CT images or endoscopic images of the lesion of the same patient. To enable the medical image processing model encapsulated in medical equipment to process medical images and reduce the hardware storage space occupied by the training data, further self-supervised rotation processing can be performed on the medical images: by performing a fixed-angle rotation and random scaling, the directionality and information value at different sizes are increased. The type of input medical image can be a two-dimensional planar image or a three-dimensional image; for example, images generated by medical instruments such as CT computed tomography (CT), magnetic resonance imaging (MRI), ultrasound, X-ray, electrocardiogram (ECG), electroencephalogram (EEG), and optical photography. What needs to be segmented and output is the region of interest (ROI) in the medical image, such as the image of lesion tissue or disease area.

[0101] To further improve the training performance of complex data structure neural network models using data distillation, noise can be inserted into the training images in the first training dataset after fixed-angle rotation. This noise includes at least one of fixed noise and dynamic noise. In some embodiments of the invention, a dynamic noise threshold matching the usage environment of the target neural network model is determined. The first training dataset is then subjected to noise addition processing based on the dynamic noise threshold to form a second training dataset matching the dynamic noise threshold. For example, taking an image processing model as the target neural network model, the dynamic noise threshold matching the usage environment of the image processing model varies depending on the environment in which it is used. For instance, in the context of image processing in academic papers, the dynamic noise threshold matching the usage environment of the image processing model needs to be lower than the dynamic noise threshold in the context of a user browsing entertainment information.

[0102] In some embodiments of the present invention, a fixed noise threshold corresponding to the target neural network model can be determined, and noise is added to the first training data set according to the fixed noise threshold to form a second training data set that matches the fixed noise threshold. For example, taking an image processing model as the target neural network model, since the image processing model is used in different environments, when the image processing model is embedded in a corresponding hardware device, such as a medical terminal, and the environment is medical image segmentation and status image display, the noise is relatively simple. By fixing the fixed noise threshold corresponding to the image processing model, the training speed of the image processing model can be effectively improved, reducing the user's waiting time. This allows the user to obtain the same training effect as with large-scale training data when using the neural network model training data processing method provided in this application, while simultaneously compressing the training time of the neural network model.

[0103] Step 303: The training data processing device of the neural network model determines an initial training data set corresponding to the second training data set based on the second training data set.

[0104] In some embodiments of the present invention, determining an initial training data set corresponding to the second training data set based on the second training data set can be achieved in the following ways:

[0105] Based on the usage environment of the target neural network model, a sample acquisition interval parameter matching the target neural network model is determined, and image data is acquired from the second training data set according to the sample acquisition interval parameter to form an initial training data set; or image data is randomly acquired from the second training data set to form an initial training data set. The training data processing device for the neural network model can randomly acquire multiple sample data points from the image data source or acquire them at fixed frame intervals. To ensure that the dimensions of the obtained input sample feature vectors are the same, the training data processing device for the neural network model can acquire a preset number of sample data points. The specific value of the preset number can be pre-set by the training data processing device for the neural network model. Alternatively, the training data processing device for the neural network model obtains the sample feature vector of each sample data point in the data source, and randomly acquires a preset number of 3 sample feature vectors from the obtained multiple sample features. Furthermore, the acquisition method of the initial training data set can be flexibly configured according to actual application scenarios such as federated learning, continuous learning, and hyperparameter optimization.

[0106] Step 304: The training data processing device of the neural network model trains the first training data processing network using the second training data set, and trains the second training data processing network using the initial training data set, so as to determine the gradient parameters that match the initial training data set.

[0107] In some embodiments of the present invention, the first training data processing network is trained using the second training data set, and the second training data processing network is trained using the initial training data set to determine gradient parameters that match the initial training data set. This can be achieved in the following ways:

[0108] Using the second training data set, the first training data processing network is trained to determine the first gradient parameters corresponding to the first training data processing network; using the initial training data set, the second training data processing network is trained to determine the second gradient parameters corresponding to the second training data processing network; based on the first and second gradient parameters, gradient parameter matching processing is performed to determine gradient parameters that match the initial training data set. Wherein, reference... Figure 5 , Figure 5 This is a schematic diagram of the training data processing process in an embodiment of the present invention. For the obtained fixed complete dataset X, the training data processing method of a neural network model provided in this application uses dataset distillation to obtain a small-scale dataset. Referring to Formula 1, such that at X and The model obtained from training and They have similar performance on test set T.

[0109] Formula 1

[0110] Where L represents the loss function used when training the network, which can be cross-entropy or least squares loss function, etc. However, as shown in the optimization problem in Equation 1, the model parameters... In generating datasets The results obtained after multiple updates, and then utilized Computing the loss function on the complete dataset results in a very large computational graph, causing gradient backpropagation to be affected by the generated data. The updates are very inefficient. Applying this method to large-scale datasets would incur extremely high distillation costs. The neural network model training data processing method in this application employs gradient matching to handle large-scale unlabeled training data.

[0111] Step 305: The training data processing device of the neural network model updates the initial training data set based on the gradient parameters that match the initial training data set, and determines the target training data set that matches the target neural network model.

[0112] Continue to refer to Figure 6 , Figure 6 This is an optional flowchart illustrating the training data processing of the neural network model provided in this embodiment of the invention. It can be understood that... Figure 6 The steps shown can be performed by various electronic devices of the training data processing device that runs the neural network model. For example, it could be a dedicated terminal with target neural network model training capabilities, a server with target neural network model training capabilities, or a server cluster. This enables gradient parameter matching based on the first and second gradient parameters to determine gradient parameters that match the initial training dataset. Server A and terminal 2 can be located in different image processing environments. For example, server A can be located in a CT image recognition environment of a medical terminal, and server B can be located in an ultrasound image recognition environment. The complete training sample can be a medical image set with a total data volume of 100TB (e.g., a server in a hospital). The target training dataset A and target training dataset B, which are collections of medical images from various departments stored in the database, can have a data volume of 10GB (occupying only one-thousandth of the hardware storage space, facilitating training on portable devices). Using the training data processing method provided in this application, target training dataset A and target training dataset B can be obtained based on complete medical image training samples. However, the training data contained in the complete training samples is not entirely consistent. This allows for training of target neural networks in different usage scenarios (for the same medical equipment in different departments, or different medical equipment in the same department) to achieve the same training effect as the complete training samples, saving hardware storage resources. The following section discusses... Figure 6 The steps shown are explained.

[0113] Step 601: The training data processing device of the neural network model determines the gradient of the parameters of each layer of the neural network in the first training data processing network when backpropagating through the second training data set.

[0114] In some embodiments of the present invention, the first training data processing network can be a pre-trained neural network, and the first training data processing network can be a large-scale neural network, for example, the number of network parameters of the first training data processing network is greater than a certain value, but the embodiments of the present invention do not limit this. Optionally, the first training data processing network can be a convolutional neural network (CNN), a deep neural network (DNN), or a recurrent neural network (RNN), etc., and the embodiments of the present invention do not limit the type of the first training data processing network. The first training data processing network can be a neural network suitable for different computer vision tasks, such as: object recognition tasks, object classification tasks, object detection tasks, or pose estimation tasks, etc. The first training data processing network can also be a neural network suitable for different application scenarios, such as: security monitoring scenarios, face unlock scenarios, intelligent driving or remote sensing scenarios, etc., and the embodiments of the present invention do not limit the scope of application of the first training data processing network. Optionally, the network structure of the first training data processing network can be designed according to the computer vision task, or the network structure of the first training data processing network can adopt at least a part of the existing network structure, such as: deep residual network or visual geometry group network (VGG Net Visual Geometry Group Network), etc. The embodiments of the present invention do not limit the network structure of the first training data processing network.

[0115] Step 602: The training data processing device of the neural network model determines the first loss function corresponding to the first training data processing network based on the gradient of the parameters of each layer of the neural network in the first training data processing network.

[0116] Step 603: The training data processing device of the neural network model determines the first gradient parameters corresponding to the first training data processing network based on the first loss function corresponding to the first training data processing network.

[0117] Step 604: The training data processing device of the neural network model determines the gradient of the neural network parameters of each layer in the second training data processing network when backpropagating through the initial training data set.

[0118] Step 605: The training data processing device of the neural network model determines the second loss function corresponding to the second training data processing network based on the gradient of the parameters of each layer of the neural network in the second training data processing network.

[0119] Step 606: The training data processing device of the neural network model determines the second gradient parameters corresponding to the second training data processing network based on the second loss function corresponding to the second training data processing network.

[0120] In some embodiments of the present invention, the initial training dataset can be iteratively updated based on gradient parameters that match the initial training dataset until the distance between the first gradient parameter and the second gradient parameter reaches a distance threshold; when the distance between the first gradient parameter and the second gradient parameter reaches the distance threshold, the updated initial training dataset is considered a target training dataset that matches the target neural network model. (See reference...) Figure 7 , Figure 7 This is a schematic diagram of gradient parameter matching during the training data processing in an embodiment of the present invention. In the gradient parameter matching process, the complete data X and the generated data are first used respectively. Forward propagation is performed independently in two neural network models. Then, backpropagation is used to calculate the gradient parameters of the loss function at each layer. By adjusting the gradient parameters, the gradient parameters generated by the small-scale training data are made close to (ideally identical to) the gradient parameters generated by the complete training data (large-scale dataset), thus achieving the same training effect on the target neural network model. The loss function is calculated by the distance between the gradient parameters in the two different neural network models (optionally Euclidean distance or cosine similarity distance). Backpropagation is then applied to the generated data... Update and optimize.

[0121] The following describes the training data processing method of the neural network model provided by this invention, using the processing of medical images in the case of cerebral hemorrhage as an example. Various images forming medical images are constantly being generated, such as CT scans and MRI images, which are continuously captured and thus become a large amount of training data. This data urgently needs to be used for large-scale classification and recognition. Simultaneously, a medical image processing model with accurate recognition requires a large amount of training data. For portable intelligent medical devices, storing a large number of training samples increases the hardware cost. Portable intelligent medical devices can participate in federated learning, continuous learning, hyperparameter optimization, and other practical application scenarios during use and training to update the encapsulated medical image processing model in a timely manner and obtain more accurate auxiliary diagnostic information. Specifically, 1) data transmission between the client and server in federated learning incurs high communication costs; 2) in continuous learning, as the number of new tasks increases, the data storage cost for old tasks and the training cost for new models both increase; 3) in hyperparameter optimization, evaluating a set of hyperparameters using a complete dataset takes a long time, which is not conducive to the large-scale use of intelligent medical devices.

[0122] Continue to combine Figure 2 The illustrated neural network model training data processing apparatus illustrates the neural network model training data processing method provided in this embodiment of the invention. See also: Figure 8 , Figure 8 This is an optional flowchart illustrating the training data processing method for the neural network model provided in this embodiment of the invention. Before using the medical image processing model (ultrasound and CT images of different body parts), it needs to be trained to determine its parameters and initialize it. For example, in a single medical image processing environment (diagnosing a disease type only in a specific department), the dynamic noise threshold matching the environment of the medical image processing model needs to be lower than the dynamic noise threshold in an emergency department or community general practice clinic. Since the training samples come from different data sources, including data from various application scenarios as the corresponding training data sources, for example, the medical image processing model provided by this invention can be encapsulated as a software module in a mobile detection electronic device, or in different fixed medical examination devices (including but not limited to: handheld diagnostic instruments, ward central monitoring systems, bedside monitoring systems), or even embedded in the hardware of an intelligent robot. For different application scenarios of these medical image processing models, corresponding training samples can be used to train the medical image processing model specifically.

[0123] Understandably, Figure 8The steps shown can be performed by various electronic devices that process training data for running neural network models, such as dedicated terminals, servers, or server clusters with medical image processing capabilities. The following section addresses... Figure 8 The steps shown are explained.

[0124] Step 801: Obtain the unlabeled training data set and perform rotational self-supervised processing.

[0125] The first training dataset comprises a set of different medical images of the same target region of the target object. (Set Preorder) Figure 7 As shown, in the training data processing of the neural network model in this application, the dataset distillation based on gradient matching still requires that the labels of all data in the complete dataset X are known, that is, for both networks... and The training of both networks uses the classification loss function from supervised learning, which cannot handle data distillation of unlabeled samples. Therefore, the limitation of not being able to process large-scale unlabeled training data can be addressed by training both networks using a self-supervised task, then calculating the gradients and using gradient matching.

[0126] Furthermore, deep unsupervised clustering algorithms can be used. This method maps data from a high-dimensional space to a low-dimensional space using an autoencoder, then uses a clustering algorithm to cluster the low-dimensional vectors obtained from the mapping, and uses the new data distribution after clustering to update the neural network parameters and cluster centers in reverse, thus achieving self-supervised processing of large-scale unlabeled training data.

[0127] Step 802: Determine the initial training data set based on the medical image processing environment.

[0128] Step 803: Determine the model structure of the first training data processing network and the second training data processing network based on the medical image processing environment.

[0129] Step 804: Determine the gradient parameters that match the initial training dataset.

[0130] Step 805: Based on the gradient parameters that match the initial training dataset, iteratively update the initially generated training dataset through gradient parameter matching until the corresponding distance threshold is reached, and then determine the target training dataset that matches the medical image processing model.

[0131] Step 806: Train the medical image processing model using the target training dataset to determine the parameters of the image processing model.

[0132] The present invention has the following beneficial technical effects:

[0133] By acquiring a first training dataset, which includes at least one image dataset; performing rotational self-supervised processing on the first training dataset to form a corresponding second training dataset; determining an initial training dataset corresponding to the second training dataset based on the second training dataset; training the first training data processing network using the second training dataset, and training the second training data processing network using the initial training dataset to determine gradient parameters matching the initial training dataset; updating the initial training dataset based on the gradient parameters matching the initial training dataset to determine a target training dataset matching the target neural network model. Therefore, it is possible to stably improve the accuracy of training a neural network model with a smaller amount of training data without retraining, and it is generally applicable to most neural network models and data. Simultaneously, the smaller training data reduces overfitting of the target neural network model, enhances its generalization ability, and facilitates the deployment of the trained target neural network model on mobile terminals, enabling large-scale application of the target neural network model.

[0134] The above description is merely an embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for processing training data for a neural network model, characterized in that, The method includes: Obtain a first training data set, wherein the first training data set includes at least one set of unlabeled image datasets; The first training dataset is subjected to rotational self-supervised processing to form a corresponding second training dataset. Based on the usage environment of the target neural network model, a sample acquisition interval parameter matching the target neural network model is determined, and image data is acquired in the second training dataset according to the sample acquisition interval parameter to form an initial training dataset; or Image data is randomly collected from the second training dataset to form the initial training dataset. The first training data processing network is trained using the second training data set to determine the first gradient parameters corresponding to the first training data processing network. The second training data processing network is trained using the initial training data set to determine the second gradient parameters corresponding to the second training data processing network. Based on the first gradient parameter and the second gradient parameter, gradient parameter matching processing is performed to determine the gradient parameters that match the initial training data set; Based on the gradient parameters that match the initial training data set, the initial training data set is updated to determine the target training data set that matches the target neural network model.

2. The method according to claim 1, characterized in that, The step of performing rotational self-supervised processing on the first training data set to form a corresponding second training data set includes: Rotate the training images in the first training dataset by a fixed center angle. Randomly scale the training images in the first training dataset after rotating at a fixed angle. The training images in the first training dataset, after being randomly scaled, are randomly shifted to form a corresponding second training dataset.

3. The method according to claim 2, characterized in that, The method further includes: Noise is inserted into the training images in the first training dataset after a fixed-angle rotation, the noise including at least one of fixed noise and dynamic noise.

4. The method according to claim 2, characterized in that, The method further includes: Determine a dynamic noise threshold that matches the usage environment of the target neural network model; The first training dataset is subjected to noise addition processing based on the dynamic noise threshold to form a second training dataset that matches the dynamic noise threshold; or, A fixed noise threshold corresponding to the target neural network model is determined, and noise is added to the first training data set according to the fixed noise threshold to form a second training data set that matches the fixed noise threshold.

5. The method according to claim 1, characterized in that, The step of training the first training data processing network using the second training data set to determine the first gradient parameters corresponding to the first training data processing network includes: Using the second training data set, determine the gradient of the parameters of each layer of the neural network in the first training data processing network when backpropagating through the first training data processing network; Based on the gradient of the parameters of each layer of the neural network in the first training data processing network, the first loss function corresponding to the first training data processing network is determined. Based on the first loss function corresponding to the first training data processing network, the first gradient parameters corresponding to the first training data processing network are determined.

6. The method according to claim 1, characterized in that, The step of training the second training data processing network using the initial training data set to determine the second gradient parameters corresponding to the second training data processing network includes: Using the initial training data set, determine the gradient of the neural network parameters of each layer in the second training data processing network when performing backpropagation through the second training data processing network; Based on the gradient of the parameters of each layer of the neural network in the second training data processing network, the second loss function corresponding to the second training data processing network is determined. Based on the second loss function corresponding to the second training data processing network, the second gradient parameters corresponding to the second training data processing network are determined.

7. The method according to claim 1, characterized in that, The step of updating the initial training data set based on gradient parameters that match the initial training data set to determine the target training data set that matches the target neural network model includes: Based on the gradient parameters that match the initial training data set, the initial training data set is iteratively updated until the distance between the first gradient parameter and the second gradient parameter reaches a distance threshold. When the distance between the first gradient parameter and the second gradient parameter reaches a distance threshold, the updated initial training dataset is combined into a target training dataset that matches the target neural network model.

8. The method according to claim 1, characterized in that, The method further includes: Based on the usage environment that matches the target neural network model, the types of the first training data processing network and the second training data processing network are adjusted to adapt the first training data processing network and the second training data processing network to the usage environment that matches the target neural network model.

9. A training data processing device for a neural network model, characterized in that, The device includes: The information transmission module is used to acquire a first training data set, wherein the first training data set includes at least one image dataset; The information processing module is used to perform rotational self-supervised processing on the first training data set to form a corresponding second training data set; The information processing module is used to determine a sample acquisition interval parameter matching the target neural network model based on the usage environment of the target neural network model, and to acquire image data in the second training data set according to the sample acquisition interval parameter to form an initial training data set; or Image data is randomly collected from the second training dataset to form the initial training dataset. The information processing module is used to train the first training data processing network using the second training data set to determine the first gradient parameters corresponding to the first training data processing network; to train the second training data processing network using the initial training data set to determine the second gradient parameters corresponding to the second training data processing network; and to perform gradient parameter matching processing based on the first gradient parameters and the second gradient parameters to determine the gradient parameters that match the initial training data set. The information processing module is used to update the initial training data set based on gradient parameters that match the initial training data set, and to determine the target training data set that matches the target neural network model.

10. The apparatus according to claim 9, characterized in that, The information processing module is also used for: The training images in the first training dataset are rotated at a fixed angle; the training images in the first training dataset after the fixed angle rotation are randomly scaled; the training images in the first training dataset after the random scaling are randomly translated to form a corresponding second training dataset.

11. The apparatus according to claim 10, characterized in that, The information processing module is also used for: Noise is inserted into the training images in the first training dataset after a fixed-angle rotation, the noise including at least one of fixed noise and dynamic noise.

12. The apparatus according to claim 10, characterized in that, The information processing module is also used for: Determine a dynamic noise threshold that matches the usage environment of the target neural network model; add noise to the first training data set according to the dynamic noise threshold to form a second training data set that matches the dynamic noise threshold; or, determine a fixed noise threshold corresponding to the target neural network model, and add noise to the first training data set according to the fixed noise threshold to form a second training data set that matches the fixed noise threshold.

13. The apparatus according to claim 9, characterized in that, The information processing module is also used for: Using the second training data set, the gradient of the parameters of each layer of the neural network in the first training data processing network is determined when backpropagating through the first training data processing network; based on the gradient of the parameters of each layer of the neural network in the first training data processing network, the first loss function corresponding to the first training data processing network is determined; based on the first loss function corresponding to the first training data processing network, the first gradient parameter corresponding to the first training data processing network is determined.

14. The apparatus according to claim 9, characterized in that, The information processing module is also used for: Using the initial training data set, the gradient of the parameters of each layer of the second training data processing network is determined when backpropagating through the second training data processing network; based on the gradient of the parameters of each layer of the second training data processing network, the second loss function corresponding to the second training data processing network is determined; based on the second loss function corresponding to the second training data processing network, the second gradient parameter corresponding to the second training data processing network is determined.

15. The apparatus according to claim 9, characterized in that, The information processing module is also used for: Based on the gradient parameters that match the initial training dataset, the initial training dataset is iteratively updated until the distance between the first gradient parameter and the second gradient parameter reaches a distance threshold; when the distance between the first gradient parameter and the second gradient parameter reaches the distance threshold, the updated initial training dataset is considered as the target training dataset that matches the target neural network model.

16. The apparatus according to claim 9, characterized in that, The information processing module is also used for: Based on the usage environment that matches the target neural network model, the types of the first training data processing network and the second training data processing network are adjusted to adapt the first training data processing network and the second training data processing network to the usage environment that matches the target neural network model.

17. An electronic device, characterized in that, The electronic device includes: Memory, used to store executable instructions; A processor, when executing executable instructions stored in the memory, implements the training data processing method for the neural network model according to any one of claims 1 to 9.

18. A computer-readable storage medium storing executable instructions, characterized in that, When the executable instructions are executed by the processor, they implement the training data processing method of the neural network model according to any one of claims 1 to 9.

19. A computer program product comprising computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the training data processing method for the neural network model according to any one of claims 1 to 9.

Images