Neural network training method and apparatus, electronic device, and storage medium

Abstract

Embodiments of the present application disclose a neural network training method and device, electronic equipment and a storage medium. The method can be applied to the field of artificial intelligence. The neural network training method trains two neural networks in parallel, distinguishes noise contained in samples according to loss values of the two neural networks, respectively, sets corresponding confidence probabilities for the samples according to the distinguishing results, determines weights of the samples according to the confidence probabilities, and trains the neural networks according to the weighted samples, so that samples with less noise can play a more important guiding role in the training process of the neural networks, and samples with more noise have less influence on the training process of the neural networks, which can effectively overcome the interference of noise labels on the neural network training stage, and further obtain a more robust computer vision model and greatly improve the accuracy of the model prediction stage.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to neural network training methods, apparatus, electronic devices, and storage media. Background Technology

[0002] With the development of machine learning and artificial intelligence, computer vision technology has been increasingly widely applied in the cloud. Computer vision technology relies on the operation of a trained neural network, which can be understood as a computer vision model. The training of this neural network is carried out using image labels as ground truth information. However, the labels contain noise. When the neural network is trained on such noisy labels, it will fit to these noisy labels, thereby reducing the generalization performance of the resulting computer vision model. Summary of the Invention

[0003] To address at least one of the aforementioned technical problems, embodiments of this application provide a neural network training method, apparatus, electronic device, and storage medium.

[0004] On one hand, embodiments of this application provide a neural network training method, wherein the neural network includes a first neural network and a second neural network, and the method includes:

[0005] Obtain multiple sample images and the category label corresponding to each sample image;

[0006] Each of the sample images is input into the first neural network and the second neural network respectively to obtain the first loss and the second loss accordingly;

[0007] The plurality of sample images are classified according to the first loss to obtain a first clean sample set;

[0008] The multiple sample images are classified according to the second loss to obtain a second clean sample set;

[0009] For each sample image in the first clean sample set, based on the processing result obtained by processing the sample image using the second neural network, the first confidence probability corresponding to the sample image is determined, and the parameters of the first neural network are adjusted based on the first confidence probability and the first loss.

[0010] For each sample image in the second clean sample set, based on the processing result obtained by processing the sample image using the first neural network, a second confidence probability corresponding to the sample image is determined, and the parameters of the second neural network are adjusted based on the second confidence probability and the second loss.

[0011] On the other hand, embodiments of this application provide a neural network training device, wherein the neural network includes a first neural network and a second neural network, and the device includes:

[0012] The sample acquisition module is used to acquire multiple sample images and the category label corresponding to each sample image;

[0013] The joint training module is used to input each of the sample images into the first neural network and the second neural network respectively, and obtain the first loss and the second loss accordingly;

[0014] A classification module is configured to classify the plurality of sample images according to the first loss to obtain a first clean sample set; and to classify the plurality of sample images according to the second loss to obtain a second clean sample set.

[0015] The joint training module is further configured to, for each sample image in the first clean sample set, determine a first confidence probability corresponding to the sample image based on the processing result obtained by processing the sample image using the second neural network, and adjust the parameters of the first neural network based on the first confidence probability and the first loss; and, for each sample image in the second clean sample set, determine a second confidence probability corresponding to the sample image based on the processing result obtained by processing the sample image using the first neural network, and adjust the parameters of the second neural network based on the second confidence probability and the second loss.

[0016] On the other hand, embodiments of this application provide a computer-readable storage medium storing at least one instruction or at least one program, wherein the at least one instruction or at least one program is loaded and executed by a processor to implement the aforementioned neural network training method.

[0017] On the other hand, embodiments of this application provide an electronic device, including at least one processor and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the at least one processor implements the aforementioned neural network training method by executing the instructions stored in the memory.

[0018] On the other hand, embodiments of this application provide a computer program product, including a computer program or instructions, which, when executed by a processor, implement the aforementioned neural network training method.

[0019] This application provides a neural network training method, apparatus, electronic device, and storage medium. The neural network training method trains two neural networks in parallel, independently distinguishing the noise contained in the samples based on the loss values ​​of the two neural networks, assigning corresponding confidence probabilities to the samples based on the distinction results, determining the weights of the samples based on the confidence probabilities, and training the neural networks based on the weighted samples. This allows samples with less noise to play a more important guiding role in the training process of the neural networks, while samples with more noise have a smaller impact on the training process of the neural networks. It can effectively overcome the interference of noise labels on the training stage of the neural networks, thereby obtaining a more robust computer vision model and significantly improving the accuracy of the model prediction stage. Attached Figure Description

[0020] To more clearly illustrate the technical solutions and advantages in the embodiments of this application or related technologies, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a schematic diagram of a feasible implementation framework for the neural network training method provided in the embodiments of this application;

[0022] Figure 2 This is a flowchart illustrating the neural network training method provided in an embodiment of this application;

[0023] Figure 3 These are schematic diagrams of sample images related to the category of industrial defects provided in the embodiments of this application;

[0024] Figure 4 This is a schematic diagram of the implementation framework provided in the embodiments of this application;

[0025] Figure 5 This is a block diagram of the neural network training apparatus provided in the embodiments of this application;

[0026] Figure 6 This is a schematic diagram of the hardware structure of a device for implementing the method provided in the embodiments of this application. Detailed Implementation

[0027] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of the embodiments of this application.

[0028] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of the embodiments of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the present application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or devices.

[0029] To make the objectives, technical solutions, and advantages disclosed in the embodiments of this application clearer, the embodiments of this application will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely illustrative of the embodiments of this application and are not intended to limit the embodiments of this application.

[0030] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this embodiment, unless otherwise stated, "multiple" means two or more. To facilitate understanding of the above-described technical solutions and their resulting technical effects in the embodiments of this application, the embodiments of this application first explain the relevant technical terms:

[0031] Cloud technology refers to a hosting technology that unifies hardware, software, and network resources within a wide area network (WAN) or local area network (LAN) to achieve data computation, storage, processing, and sharing. It encompasses network technologies, information technologies, integration technologies, management platform technologies, and application technologies based on cloud computing business models. These technologies can form resource pools, allowing for flexible and convenient on-demand use. Cloud computing technology will become a crucial support. Backend services of technical network systems require substantial computing and storage resources, such as video websites, image websites, and many portal websites. With the rapid development and application of the internet industry, each resource may have its own identification mark in the future, requiring transmission to backend systems for logical processing. Data at different levels will be processed separately, and various industry data will require robust system support, which can only be achieved through cloud computing.

[0032] Intelligent Traffic Systems (ITS), also known as Intelligent Transportation Systems, effectively integrate advanced science and technology (information technology, computer technology, data communication technology, sensor technology, electronic control technology, automatic control theory, operations research, artificial intelligence, etc.) into transportation, service control, and vehicle manufacturing. This strengthens the connection between vehicles, roads, and users, thereby forming a comprehensive transportation system that ensures safety, improves efficiency, enhances the environment, and saves energy.

[0033] Intelligent Vehicle Infrastructure Cooperative Systems (IVICS) are a development direction of Intelligent Transportation Systems (ITS). IVICS utilizes advanced wireless communication and next-generation Internet technologies to implement comprehensive, real-time dynamic information exchange between vehicles and infrastructure. Based on the collection and fusion of dynamic traffic information across all times and spaces, it conducts active vehicle safety control and cooperative road management, fully realizing effective collaboration between people, vehicles, and roads. This ensures traffic safety, improves traffic efficiency, and ultimately forms a safe, efficient, and environmentally friendly road traffic system.

[0034] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0035] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / switching systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0036] Machine learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instructional learning.

[0037] Deep learning: The concept of deep learning originated from research on artificial neural networks. A multilayer perceptron with multiple hidden layers is a type of deep learning architecture. Deep learning discovers distributed feature representations of data by combining low-level features to form more abstract high-level representations of attribute categories or features.

[0038] Convolutional Neural Networks (CNNs) are a class of feedforward neural networks that incorporate convolutional computations and have a deep structure. They are one of the representative algorithms of deep learning. CNNs have representation learning capabilities and can classify input information in a translation-invariant manner according to their hierarchical structure; therefore, they are also known as "translation-invariant artificial neural networks."

[0039] With the development of machine learning and artificial intelligence, computer vision technology has been increasingly widely used in cloud applications. For example, computer vision technology can be used to improve work efficiency in fields such as industrial defect detection, security systems, intelligent transportation, and target detection.

[0040] Taking industrial defect detection as an example, industrial defect detection refers to the quality inspection of industrial products during the manufacturing process. Traditional industrial inspection is generally carried out by inspection workers through visual inspection. In recent years, with the rise of AI technology, AI inspection based on machine vision can significantly improve the accuracy of inspection and save labor costs. The input of this machine vision AI inspection is an image taken of the surface of the industrial product, and the output is the confidence level of the defect.

[0041] Traditional machine vision-based industrial defect detection algorithms extract handcrafted features from the input image, including gradient and texture features. Then, based on these features, a base classifier is trained. This base classifier can be a tree-based classifier, such as a random forest, to perform a binary classification to determine whether the input image is defective. This approach has two problems: first, the extracted handcrafted features have poor generalization ability, often extracting harmful features that confuse the classifier's performance with diverse video data; second, feature extraction and classifier training are performed independently, resulting in high training costs. To address these two issues, the recently developed CNN has provided a good solution. CNN-based approaches directly feed the input image into a CNN network structure for feature extraction, followed by classification using fully connected layers. Model training utilizes a loss function derived from logistic regression results, calculating the loss and updating model weights through gradient backpropagation—an end-to-end training process. This eliminates the need for manual feature design; the model automatically learns the most suitable features for the current classification task. Furthermore, the entire process is end-to-end, requiring no separate training for feature extraction and classification. However, this CNN-based whole-image classification still has some problems. Industrial defect detection is not a simple binary classification. Many defective images have mild defects, even classifying them as defect-free, but in reality, they are defective. Simple binary classification cannot accurately describe complex defects and their degree of detail. Simple binary labels may be highly subjective, leading to noisy manually labeled defect information. Training the model with these noisy labels causes it to fit these noisy labels, thus reducing its generalization performance.

[0042] In view of this, embodiments of this application provide a neural network training method. This method trains two neural networks in parallel, distinguishes the noise contained in the samples based on the loss values ​​of the two neural networks, assigns corresponding confidence probabilities to the samples based on the distinction results, determines the weights of the samples based on the confidence probabilities, and trains the neural networks based on the weighted samples. This allows samples with less noise to play a more important guiding role in the training process of the neural networks, while samples with more noise have a smaller impact on the training process of the neural networks. This effectively overcomes the interference of noise labels on the training stage of the neural networks, thereby obtaining a more robust computer vision model and significantly improving the accuracy of the model's prediction stage.

[0043] Please see Figure 1 , Figure 1 This is a schematic diagram illustrating a feasible implementation framework for the neural network training method provided in the embodiments of this specification, such as... Figure 1 As shown, this implementation framework may include at least a client 10 and a server 20. The client 10 and the server 20 communicate via a network 30. This implementation framework can also be considered a neural network training system. The server 20 may be located in a cloud environment, and the server 20 is a neural network training node in the cloud environment. This node can be any node in the cloud environment.

[0044] The server 20 can deploy neural networks, including a first neural network and a second neural network. The first and second neural networks can have the same or different structures. Both the first and second neural networks are used for image classification. The server 20 can acquire multiple sample images and the category label corresponding to each sample image; input each sample image into the first neural network and the second neural network respectively to obtain a first loss and a second loss; classify the multiple sample images according to the first loss to obtain a first clean sample set; classify the multiple sample images according to the second loss to obtain a second clean sample set; for each sample image in the first clean sample set, determine the first confidence probability corresponding to the sample image based on the processing result obtained by processing the sample image using the second neural network, and adjust the parameters of the first neural network based on the first confidence probability and the first loss; for each sample image in the second clean sample set, determine the second confidence probability corresponding to the sample image based on the processing result obtained by processing the sample image using the first neural network, and adjust the parameters of the second neural network based on the second confidence probability and the second loss.

[0045] This neural network training method trains two neural networks in parallel, distinguishes the noise contained in the samples based on the loss values ​​of the two neural networks, assigns corresponding confidence probabilities to the samples based on the discrimination results, determines the weights of the samples based on the confidence probabilities, and trains the neural network based on the weighted samples. This allows samples with less noise to play a more important guiding role in the training process of the neural network, while samples with more noise have a smaller impact on the training process of the neural network. It can effectively overcome the interference of noise labels on the neural network training stage, thereby obtaining a more robust computer vision model and significantly improving the accuracy of the model in the prediction stage.

[0046] Each neural network trained by the server 20 can be used as a computer vision model to provide computer vision services to the client 10. The training process of this computer vision model effectively reduces the influence of noise labels and learns as little information related to noise labels as possible, thereby enabling the computer vision model to have a good classification effect on images. The first and second neural networks mentioned above can be used as the aforementioned computer vision models because of their outstanding classification effect.

[0047] The framework described in this invention can provide the neural network training and classification capabilities required for applications in various scenarios, including but not limited to cloud technology, cloud gaming, cloud rendering, artificial intelligence, smart transportation, assisted driving, video media, smart communities, and instant messaging. Each component in this framework can be a terminal device or a server. Terminal devices include, but are not limited to, mobile phones, computers, intelligent voice exchange devices, smart home appliances, and in-vehicle terminals.

[0048] The following describes a neural network training method according to an embodiment of this application. Figure 2 This document illustrates a flowchart of a neural network training method provided in an embodiment of this application. This method can be executed based on the server 20 described above. The neural network includes a first feature neural network and a second feature neural network. This application provides the method operation steps as described in the embodiments or flowcharts, but based on conventional or non-inventive methods, more or fewer operation steps may be included. The order of steps listed in the embodiments is merely one possible execution order among many and does not represent the only possible execution order. In actual systems, terminal devices, or server products, the method can be executed sequentially according to the embodiments or drawings, or in parallel (e.g., in a parallel processor or multi-threaded processing environment). The method may include:

[0049] S201. Obtain multiple sample images and the category label corresponding to each of the above sample images.

[0050] This application does not limit the category label of the sample image. The category label can be a binary category label, representing whether the sample image contains a target of a certain category or not. However, binary category labels are difficult to carry degree information. Please refer to [reference needed]. Figure 3 It shows a schematic diagram of sample images related to the category of industrial defects. Figure 3 The left image is an image that does not include industrial defects. Figure 3 The middle image shows a scene containing minor industrial defects. Figure 3 The image on the right contains a serious industrial defect. Figure 3 Left image and Figure 3 The category labels in the right-hand image could be 0 and 1, representing the absence and presence of industrial defects, respectively. Figure 3 The category label corresponding to the middle image could be 0 or 1, but neither 0 nor 1 can adequately represent the situation of "minor industrial defects." Figure 3 The category labels corresponding to the images in the middle are likely to contain noise. The category labels in this application embodiment may contain noise. The neural network training method in this application embodiment can automatically make the neural network minimize the learning of knowledge from sample images corresponding to category labels that contain noise or a lot of noise, and focus as much as possible on learning knowledge from sample images corresponding to category labels that do not contain or contain little noise.

[0051] S202. Input each of the above sample images into the first neural network and the second neural network respectively to obtain the first loss and the second loss.

[0052] Both the first neural network and the second neural network described above are networks for classifying sample images. Their network structures can be the same or different. The first loss and the second loss are used to measure the classification loss of the first neural network and the second neural network, respectively. The calculation method for the classification loss can refer to relevant technologies, such as using cross-entropy loss to measure the classification loss. Taking the first loss as an example, it can be obtained by measuring the difference between the classification result of the first neural network and the corresponding category label of the sample image based on cross-entropy loss. The second loss is obtained based on the same inventive concept as the first loss and will not be elaborated here.

[0053] S203. Classify the plurality of sample images according to the first loss to obtain a first clean sample set; classify the plurality of sample images according to the second loss to obtain a second clean sample set.

[0054] A first noisy sample set is obtained based on the aforementioned multiple sample images and the aforementioned first clean sample set; a second noisy sample set is obtained based on the aforementioned multiple sample images and the aforementioned second clean sample set. The method of classifying the aforementioned multiple sample images according to the aforementioned second loss to obtain the second clean sample set is based on the same inventive concept as the method of classifying the aforementioned multiple sample images according to the aforementioned first loss to obtain the first clean sample set. This application embodiment uses the method of classifying the aforementioned multiple sample images according to the aforementioned first loss to obtain the first clean sample set as an example for explanation.

[0055] Specifically, D can be used to represent the entire training dataset consisting of multiple sample images and their labels, and (x,y) can be used to represent a sample label pair. The first loss is used as the classification criterion, that is, a hyperparameter τ is set, and samples whose first loss l(x,y) is less than this value are considered as clean samples X in the first clean sample set. c Conversely, it is considered to be noise sample X of the first noise sample set. u ;

[0056] That is, X c ={(x,y)|l(x,y)<τ,(x,y)∈D};

[0057] X u ={(x,y)|l(x,y)≥τ,(x,y)∈D};

[0058] Of course, τ can be set according to requirements.

[0059] As the first and second neural networks are trained, the losses generated by each neural network will also change accordingly. Each iteration may lead to changes in the data in the first clean sample set, the second clean sample set, the first noisy sample set, and the second noisy sample set.

[0060] In step S203, the coarse-grained classification of the samples is completed. In this embodiment of the application, the network training based on the confidence probability can be further performed on the samples that have been classified into the clean sample set, so as to perform confidence modeling from the fine-grained dimension, thereby curbing the influence of difficult and noisy samples and improving the training accuracy.

[0061] S204. For each sample image in the first clean sample set, based on the processing result obtained by processing the sample image using the second neural network, determine the first confidence probability corresponding to the sample image, and adjust the parameters of the first neural network based on the first confidence probability and the first loss.

[0062] S205. For each sample image in the second clean sample set, based on the processing result obtained by processing the sample image using the first neural network, determine the second confidence probability corresponding to the sample image, and adjust the parameters of the second neural network based on the second confidence probability and the second loss.

[0063] Steps S204-S205 constitute a fine-grained sample credibility modeling process, and steps S204 and S205 are based on the same inventive concept. Taking step S204 as an example, the temporal likelihood score information and temporal distribution variance information corresponding to the sample image can be obtained from the processing result sequence obtained by processing the sample image multiple times using the second neural network described above; based on the temporal likelihood score information and the temporal distribution variance information, the first credibility probability corresponding to the sample image is determined.

[0064] The above processing result sequence is as follows t represents the timing information, n represents the number of processing steps, and f represents the time sequence information. t =f(x; θ) t ), f t Let f represent the processing function of the second neural network on the input information x given the temporal information t. Specifically, if the second neural network performs feature extraction and classification on the input information x, then f t It at least represents the process of feature extraction and classification, f t This can be simply understood as the mapping relationship generated by the processing of the second neural network. The parameters of this second neural network, given time-series information t, are obtained through θ. t The input information x corresponds to the above sample image, and the temporal likelihood score information corresponding to the above sample image is the joint likelihood between the above processed result sequence and the label corresponding to the above sample image; the above temporal distribution variance information is the variance determined based on the above joint likelihood.

[0065] Specifically, the temporal likelihood score information corresponding to the sample images is as follows: y represents the label corresponding to the above sample image, where P() represents the conditional probability. The time-series distribution variance information is...

[0066] Accordingly, determining the first confidence probability corresponding to the sample image based on the temporal likelihood score information and the temporal distribution variance information includes: obtaining a target parameter based on the difference between a preset value and the variance; and using the product of the target parameter and the joint likelihood as the first confidence probability. In one embodiment, the preset value is 1, then the first confidence probability is...

[0067] The aforementioned adjustment of the parameters of the first neural network based on the first confidence probability and the first loss includes: adjusting the parameters of the first neural network based on the first loss, and performing weighted gradient backpropagation on the parameters of the first neural network during the adjustment process, wherein the weights used for weighting are the first confidence probability. Specifically, it can be based on... Weighted gradient backpropagation is performed, where μ is a hyperparameter that can be preset, and y is the label information corresponding to the input information x. This label information can be the initially carried label information or the updated label information, as long as the label corresponding to the input information x is given by the time sequence information t. The label update will be explained in detail below. By performing weighted gradient backpropagation on the parameters of the first neural network during the adjustment process, difficult noise samples that cannot be correctly identified in the coarse-grained stage can be further suppressed to reduce their impact on the first neural network and improve the efficiency of the first neural network. In order to maximize the utilization of the first noise sample set and the second noise sample set, the labels of the first noise sample set and the second noise sample set can be updated. That is, the label value is used as a trainable parameter and is updated collaboratively with the training process of the first neural network and the second neural network. The updated label value becomes a softened label distribution. After the labels of samples belonging to the first noise sample set and the second noise sample set are updated, during the training process of the first neural network and the second neural network, they may be reclassified into the first clean sample set and the second clean sample set. This can be used to adjust the parameters of the first neural network or the second neural network, so that the information of samples located in the first noise sample set and the second noise sample set can be maximized to optimize the first neural network or the second neural network.

[0068] In this embodiment, the target parameters can be updated based on a joint training objective. This joint training objective measures the consistency between the classification results output by the first neural network and the second neural network for the same sample image, and the difference between the classification results output by the first neural network and the corresponding labels. The target parameters include: the labels corresponding to the sample images in the first noisy sample set, the labels corresponding to the sample images in the second noisy sample set, the parameters of the first neural network, and the parameters of the second neural network. Based on the updated target parameters, the multiple sample images are reclassified, and the first clean sample set and the second clean sample set are updated based on the classification results. The updating of the first clean sample set and the second clean sample set allows the scheme to iterate. As the noise in the labels of samples belonging to the first and second noisy sample sets gradually decreases, the number of samples in the first and second clean sample sets gradually increases, and the generalization ability of the two neural networks improves.

[0069] The aforementioned joint training objective includes first-type distribution information and second-type distribution information. The first-type distribution information includes the divergence information between the label corresponding to the sample image and the output value corresponding to the first neural network, and the divergence information between the label corresponding to the sample image and the output value corresponding to the second neural network. The second-type distribution information includes the divergence information between the output value obtained by the first neural network processing the sample image and the output value obtained by the second neural network processing the sample image. Specifically, the joint training objective L = D JS (y||h(f1(x;θ)))+D JS (y||h(f2(x;θ))+D JS (h(f1(x; θ))||h(f2(x; θ))), D JS() represents the JS divergence. Of course, KL divergence can also be used in other embodiments. f1(x; θ) represents the processing function of the first neural network for the input information x under parameter θ, f2(x; θ) represents the processing function of the second neural network for the input information x under parameter θ, and h() represents the logistic regression function. Clearly, the joint training objective L comprehensively measures the difference between the output probabilities and labels of the two networks, as well as the consistency of the output probabilities of the two networks. KL divergence is also called relative entropy or KL distance. For the difference (or simply similarity) between two probability distributions P and Q, the more similar they are, the smaller the KL divergence. JS divergence is a variant of KL divergence. Similar to KL divergence, the more similar P and Q are, the smaller the JS divergence. Based on this joint training objective L, the parameters of the first and second neural networks, as well as the labels of the relevant samples, can be adjusted simultaneously. The parameter adjustment process can also be integrated with the gradient reversal mentioned above, which makes the convergence speed of the first and second neural networks faster and improves the accuracy of label updates. By quickly removing noise in the labels through label updates, the clean sample content is increased, and finally, the goal of obtaining a high-accuracy first and second neural network is achieved both well and quickly.

[0070] Figure 4 The diagram illustrates the implementation framework of an embodiment of this application. During the training phase, two neural network structures need to be initialized for parallel training. First, a coarse-grained partitioning is performed; that is, for each neural network, the training data can be divided into X based on the loss generated by that neural network. c and X u Let X represent the sets of clean samples (mostly clean) and noisy samples (mostly noise), respectively. Then, the fine-grained sample credibility modeling method proposed in this application is used to evaluate X. c Fine-grained credibility modeling is performed to generate a first credibility probability W1 and a second credibility probability W2. Then, W1 and W2 are used to evaluate X. c Perform sample weighting, that is, use W1 to weight the samples. Weighted, W2 to A weighted processing is performed, which is reflected in the weighted gradient inversion stage. This is the second clean sample set corresponding to the second neural network. This represents the first clean sample set corresponding to the first neural network. The purpose of this is to utilize the difference in the confidence probabilities of samples generated by the two neural networks to avoid "self-perception errors," where a single neural network becomes increasingly certain of its own judgments (even if these judgments are incorrect) as training progresses. Weighting the two neural networks can alleviate this problem. The weighted samples will contribute differently to the network training phase; samples with smaller weights will generate smaller gradients, thus contributing less to the network weight updates. Therefore, this achieves fine-grained modulation of difficult samples, further curbing the harm of these noisy samples to the model. Simultaneously, for the coarse-grained noisy samples X... u In contrast to the crude approach of directly discarding these samples, this application embodiment uses a local label update strategy to attempt to correct their labels. This can make the most of these noisy samples and further improve the model performance.

[0071] The embodiments of this application can be applied to any application scenario based on classification tasks. Taking industrial defect detection as an example, the embodiments of this application can accurately detect the degree of defects in the input image. By using coarse-grained and fine-grained sample credibility division, noise samples of different difficulty levels can be effectively distinguished and processed separately. This can effectively overcome the interference of noise labels on the training stage, thereby obtaining a more robust model and significantly improving the accuracy of the model prediction stage, thus providing reliable technical support for industrial AI defect quality detection.

[0072] Specifically, industrial data commonly contains label noise. However, the type of label noise differs between samples. For neural networks, during the training phase, a loss value can be used to coarsely distinguish noisy samples. Noisy samples have larger loss values ​​and variances than clean samples, meaning their loss curves fluctuate more significantly. Therefore, either the loss value or variance can be used for coarse-grained distinction. However, simply using a single loss value or variance threshold can only distinguish most noisy samples; a certain number will still be classified as clean samples. These samples are difficult to distinguish from clean samples and are therefore considered hard noise samples. For these samples, this application embodiment comprehensively considers the temporal information during training to perform fine-grained modeling of the noise's credibility, thereby further eliminating the impact of these hard noise samples on the model. Specifically, the neural network training method of this application embodiment trains two neural networks in parallel, distinguishes the noise contained in the samples based on the loss values ​​of the two neural networks, assigns corresponding credibility probabilities to the samples based on the distinction results, determines the weights of the samples based on the credibility probabilities, and trains the neural network based on the weighted samples, thereby further eliminating the impact of these hard noise samples on the model. This allows samples with less noise to play a more significant guiding role in the training process of the neural network, while samples with more noise have a smaller impact on the training process. This effectively overcomes the interference of noise labels on the training stage of the neural network, thereby obtaining a more robust computer vision model and significantly improving the accuracy of the model's prediction stage.

[0073] Please refer to Figure 5 This diagram illustrates a block diagram of a neural network training device in this embodiment. The neural network includes a first neural network and a second neural network. The device includes:

[0074] The sample acquisition module 101 is used to acquire multiple sample images and the category label corresponding to each of the above sample images;

[0075] The joint training module 102 is used to input each of the above sample images into the first neural network and the second neural network respectively, and obtain the first loss and the second loss accordingly.

[0076] The classification module 103 is used to classify the plurality of sample images according to the first loss to obtain a first clean sample set; and to classify the plurality of sample images according to the second loss to obtain a second clean sample set.

[0077] The aforementioned joint training module 102 is further configured to, for each sample image in the first clean sample set, determine the first confidence probability corresponding to the sample image based on the processing result obtained by processing the sample image using the second neural network, and adjust the parameters of the first neural network based on the first confidence probability and the first loss; and, for each sample image in the second clean sample set, determine the second confidence probability corresponding to the sample image based on the processing result obtained by processing the sample image using the first neural network, and adjust the parameters of the second neural network based on the second confidence probability and the second loss.

[0078] In one embodiment, the classification module 103 is configured to perform the following operations:

[0079] Based on the above multiple sample images and the above first clean sample set, a first noisy sample set is obtained;

[0080] Based on the above multiple sample images and the above second clean sample set, a second noisy sample set is obtained;

[0081] The joint training module 102 further performs the following operations: updating the target parameters based on the joint training objective, which is used to measure the consistency between the classification results output by the first neural network and the second neural network for the same sample image, and to measure the difference between the classification results output by the first neural network and the second neural network and their corresponding labels. The target parameters include: the labels corresponding to the sample images in the first noise sample set, the labels corresponding to the sample images in the second noise sample set, the parameters of the first neural network, and the parameters of the second neural network.

[0082] In one embodiment, the classification module 103 is configured to perform the following operations:

[0083] Based on the updated target parameters, the multiple sample images are reclassified, and the first clean sample set and the second clean sample set are updated based on the classification results.

[0084] In one embodiment, the joint training module 102 described above is used to perform the following operations:

[0085] Based on the processing result sequence obtained by processing the sample image multiple times using the second neural network, the temporal likelihood score information and temporal distribution variance information corresponding to the sample image are obtained.

[0086] Based on the aforementioned temporal likelihood score information and temporal distribution variance information, the first confidence probability corresponding to the aforementioned sample image is determined.

[0087] In one embodiment, the above processing result sequence is as follows: t represents the timing information, n represents the number of processing steps, and f represents the time sequence information. t =f(x; θ) t ), f t This represents the processing function of the second neural network for processing input information x given time-series information t. The parameters of the second neural network given time-series information t are expressed by θ. t The input information x corresponds to the sample image mentioned above.

[0088] The temporal likelihood score information corresponding to the above sample images is the joint likelihood between the above processed result sequence and the label corresponding to the above sample images;

[0089] The above time series distribution variance information is the variance determined based on the above joint likelihood.

[0090] In one embodiment, the joint training module 102 described above is used to perform the following operations:

[0091] The target parameter is obtained based on the difference between the preset value and the above variance;

[0092] The product of the above target parameters and the above joint likelihood is taken as the above first confidence probability.

[0093] In one embodiment, the joint training module 102 described above is used to perform the following operations:

[0094] The parameters of the first neural network are adjusted based on the first loss mentioned above. During the adjustment process, the parameters of the first neural network are backpropagated with weights, and the weights used for weighting are the first confidence probability mentioned above.

[0095] In one embodiment, the joint training objective includes first-class distribution information and second-class distribution information.

[0096] The aforementioned first type of distribution information includes the divergence information between the label corresponding to the sample image and the output value corresponding to the first neural network, as well as the divergence information between the label corresponding to the sample image and the output value corresponding to the second neural network.

[0097] The second type of distribution information mentioned above includes the divergence information between the output value obtained by the first neural network processing the sample image and the output value obtained by the second neural network processing the sample image.

[0098] The apparatus portion of the embodiments in this application is based on the same inventive concept as the method embodiment, and will not be described in detail here.

[0099] Furthermore, Figure 6A schematic diagram of a hardware structure for implementing the method provided in the embodiments of this application is shown. This device can participate in or include the apparatus or system provided in the embodiments of this application. Figure 6 As shown, device 10 may include one or more processors 102 (shown as 102a, 102b, ..., 102n in the figure) 102 (processor 102 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 6 The structure shown is for illustrative purposes only and does not limit the structure of the electronic device described above. For example, device 10 may also include a... Figure 6 The more or fewer components shown, or having the same Figure 6 The different configurations shown.

[0100] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the device 10 (or mobile device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0101] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the method described in the embodiments of this application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby implementing the aforementioned neural network training method or classification method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the device 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0102] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of device 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a radio frequency (RF) module used for wireless communication with the Internet.

[0103] The display may be, for example, a touchscreen liquid crystal display (LCD) that allows the user to interact with the user interface of device 10 (or mobile device).

[0104] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, the above description focuses on specific embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired result. Additionally, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some implementations, multitasking and parallel processing are also possible or may be advantageous.

[0105] The embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device and server embodiments are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0106] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0107] The instructions in the aforementioned storage medium can execute a neural network training method, wherein the neural network includes a first neural network and a second neural network, and the method includes:

[0108] Obtain multiple sample images and the category label corresponding to each of the above sample images;

[0109] Each of the above sample images is input into the first neural network and the second neural network respectively to obtain the first loss and the second loss accordingly;

[0110] Based on the first loss described above, the above multiple sample images are classified to obtain a first clean sample set;

[0111] The above multiple sample images are classified according to the second loss to obtain a second clean sample set;

[0112] For each sample image in the first clean sample set, based on the processing result obtained by processing the sample image using the second neural network, the first confidence probability corresponding to the sample image is determined, and the parameters of the first neural network are adjusted based on the first confidence probability and the first loss.

[0113] For each sample image in the second clean sample set, based on the processing result obtained by processing the sample image using the first neural network, the second confidence probability corresponding to the sample image is determined, and the parameters of the second neural network are adjusted based on the second confidence probability and the second loss.

[0114] In one embodiment, the above method further includes:

[0115] Based on the above multiple sample images and the above first clean sample set, a first noisy sample set is obtained;

[0116] Based on the above multiple sample images and the above second clean sample set, a second noisy sample set is obtained;

[0117] The target parameters are updated based on the joint training objective, which is used to measure the consistency between the classification results output by the first neural network and the second neural network for the same sample image, and to measure the difference between the classification results output by the first neural network and the second neural network and their corresponding labels. The target parameters include: the labels corresponding to the sample images in the first noise sample set, the labels corresponding to the sample images in the second noise sample set, the parameters of the first neural network, and the parameters of the second neural network.

[0118] In one embodiment, the above method further includes:

[0119] Based on the updated target parameters, the multiple sample images are reclassified, and the first clean sample set and the second clean sample set are updated based on the classification results.

[0120] In one embodiment, determining the first confidence probability corresponding to the sample image based on the processing result obtained from processing the sample image using the second neural network includes:

[0121] Based on the processing result sequence obtained by processing the sample image multiple times using the second neural network, the temporal likelihood score information and temporal distribution variance information corresponding to the sample image are obtained.

[0122] Based on the aforementioned temporal likelihood score information and temporal distribution variance information, the first confidence probability corresponding to the aforementioned sample image is determined.

[0123] In one embodiment, the above processing result sequence is as follows: t represents the timing information, n represents the number of processing steps, and f represents the time sequence information. t =f(x; θ) t ), f t This represents the processing function of the second neural network for processing input information x given time-series information t. The parameters of the second neural network given time-series information t are expressed by θ. t The input information x corresponds to the sample image mentioned above.

[0124] The temporal likelihood score information corresponding to the above sample images is the joint likelihood between the above processed result sequence and the label corresponding to the above sample images;

[0125] The above time series distribution variance information is the variance determined based on the above joint likelihood.

[0126] In one embodiment, determining the first confidence probability corresponding to the sample image based on the temporal likelihood score information and the temporal distribution variance information includes:

[0127] The target parameter is obtained based on the difference between the preset value and the above variance;

[0128] The product of the above target parameters and the above joint likelihood is taken as the above first confidence probability.

[0129] In one embodiment, adjusting the parameters of the first neural network based on the first confidence probability and the first loss includes:

[0130] The parameters of the first neural network are adjusted based on the first loss mentioned above. During the adjustment process, the parameters of the first neural network are backpropagated with weights, and the weights used for weighting are the first confidence probability mentioned above.

[0131] In one embodiment, the joint training objective includes first-class distribution information and second-class distribution information.

[0132] The aforementioned first type of distribution information includes the divergence information between the label corresponding to the sample image and the output value corresponding to the first neural network, as well as the divergence information between the label corresponding to the sample image and the output value corresponding to the second neural network.

[0133] The second type of distribution information mentioned above includes the divergence information between the output value obtained by the first neural network processing the sample image and the output value obtained by the second neural network processing the sample image.

[0134] The above are merely preferred embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present application should be included within the protection scope of the present application.

Claims

1. A neural network training method, characterized in that, The neural network includes a first neural network and a second neural network, and the method includes: Obtain multiple sample images and the category label corresponding to each sample image; Each of the sample images is input into the first neural network and the second neural network respectively to obtain the first loss and the second loss accordingly; The plurality of sample images are classified according to the first loss to obtain a first clean sample set; The multiple sample images are classified according to the second loss to obtain a second clean sample set; For each sample image in the first clean sample set, based on the processing result sequence obtained by the second neural network through multiple processing of the sample image, the temporal likelihood score information and temporal distribution variance information corresponding to the sample image are obtained; the temporal likelihood score information corresponding to the sample image is the joint likelihood between the processing result sequence and the label corresponding to the sample image; the temporal distribution variance information is the variance determined based on the joint likelihood. Based on the temporal likelihood score information and the temporal distribution variance information, the first confidence probability corresponding to the sample image is determined; The parameters of the first neural network are adjusted based on the first confidence probability and the first loss. For each sample image in the second clean sample set, based on the processing result obtained by processing the sample image using the first neural network, a second confidence probability corresponding to the sample image is determined, and the parameters of the second neural network are adjusted based on the second confidence probability and the second loss.

2. The method according to claim 1, characterized in that, The method further includes: Based on the plurality of sample images and the first clean sample set, a first noisy sample set is obtained; Based on the multiple sample images and the second clean sample set, a second noisy sample set is obtained; The target parameters are updated based on the joint training objective, which is used to measure the consistency between the classification results output by the first neural network and the second neural network for the same sample image, and to measure the difference between the classification results output by the first neural network and the second neural network and their corresponding labels. The target parameters include: the labels corresponding to the sample images in the first noise sample set, the labels corresponding to the sample images in the second noise sample set, the parameters of the first neural network, and the parameters of the second neural network.

3. The method according to claim 2, characterized in that, The method further includes: Based on the updated target parameters, the multiple sample images are reclassified, and the first clean sample set and the second clean sample set are updated based on the classification results.

4. The method according to claim 1, characterized in that, The processing result sequence is: t represents the timing information, and n represents the number of processing steps. middle This indicates that the second neural network, given the time information t, processes the input information. The processing function performs the processing, and the parameters of the second neural network are obtained under the condition of time information t. Expression, input information Corresponding to the sample image, The temporal likelihood score information corresponding to the sample image is the joint likelihood between the processed result sequence and the label corresponding to the sample image; The time-series distribution variance information is the variance determined based on the joint likelihood.

5. The method according to claim 4, characterized in that, The step of determining the first confidence probability corresponding to the sample image based on the temporal likelihood score information and the temporal distribution variance information includes: The target parameter is obtained based on the difference between the preset value and the variance; The product of the target parameter and the joint likelihood is used as the first confidence probability.

6. The method according to claim 1, characterized in that, The adjustment of the parameters of the first neural network based on the first confidence probability and the first loss includes: The parameters of the first neural network are adjusted based on the first loss. During the adjustment process, the parameters of the first neural network are backpropagated with weights, and the weights used for weighting are the first confidence probability.

7. The method according to claim 2, characterized in that, The joint training objective includes first-class distribution information and second-class distribution information. The first type of distribution information includes the divergence information between the label corresponding to the sample image and the output value corresponding to the first neural network, and the divergence information between the label corresponding to the sample image and the output value corresponding to the second neural network; The second type of distribution information includes the divergence information between the output value obtained by the first neural network processing the sample image and the output value obtained by the second neural network processing the sample image.

8. A neural network training device, characterized in that, The neural network includes a first neural network and a second neural network, and the device includes: The sample acquisition module is used to acquire multiple sample images and the category label corresponding to each sample image; The joint training module is used to input each of the sample images into the first neural network and the second neural network respectively, and obtain the first loss and the second loss accordingly; A classification module is configured to classify the plurality of sample images according to the first loss to obtain a first clean sample set; and to classify the plurality of sample images according to the second loss to obtain a second clean sample set. The joint training module is further configured to, for each sample image in the first clean sample set, obtain temporal likelihood score information and temporal distribution variance information corresponding to the sample image based on the processing result sequence obtained by the second neural network processing the sample image multiple times; the temporal likelihood score information corresponding to the sample image is the joint likelihood between the processing result sequence and the label corresponding to the sample image; the temporal distribution variance information is the variance determined based on the joint likelihood; determine a first confidence probability corresponding to the sample image based on the temporal likelihood score information and the temporal distribution variance information; adjust the parameters of the first neural network based on the first confidence probability and the first loss; and, for each sample image in the second clean sample set, determine a second confidence probability corresponding to the sample image based on the processing result obtained by the first neural network processing the sample image, and adjust the parameters of the second neural network based on the second confidence probability and the second loss.

9. The apparatus according to claim 8, characterized in that, The classification module is used to perform the following operations: Based on the plurality of sample images and the first clean sample set, a first noisy sample set is obtained; Based on the multiple sample images and the second clean sample set, a second noisy sample set is obtained; The joint training module further performs the following operations: updating the target parameters based on the joint training objective, which is used to measure the consistency between the classification results output by the first neural network and the second neural network for the same sample image, and to measure the difference between the classification results output by the first neural network and the second neural network and their corresponding labels. The target parameters include: the labels corresponding to the sample images in the first noise sample set, the labels corresponding to the sample images in the second noise sample set, the parameters of the first neural network, and the parameters of the second neural network.

10. The apparatus according to claim 9, characterized in that, The classification module is used to perform the following operations: Based on the updated target parameters, the multiple sample images are reclassified, and the first clean sample set and the second clean sample set are updated based on the classification results.

11. The apparatus according to claim 8, characterized in that, The joint training module is used to perform the following operations: The target parameter is obtained based on the difference between the preset value and the variance; The product of the target parameter and the joint likelihood is used as the first confidence probability.

12. The apparatus according to claim 8, characterized in that, The joint training module is used to perform the following operations: The parameters of the first neural network are adjusted based on the first loss. During the adjustment process, the parameters of the first neural network are backpropagated with weights, and the weights used for weighting are the first confidence probability.

13. The apparatus according to claim 9, characterized in that, The joint training objective includes first-class distribution information and second-class distribution information. The first type of distribution information includes the divergence information between the label corresponding to the sample image and the output value corresponding to the first neural network, and the divergence information between the label corresponding to the sample image and the output value corresponding to the second neural network; The second type of distribution information includes the divergence information between the output value obtained by the first neural network processing the sample image and the output value obtained by the second neural network processing the sample image.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one instruction or at least one program, which is loaded and executed by a processor to implement a neural network training method as described in any one of claims 1 to 7.

15. An electronic device, characterized in that, The system includes at least one processor and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the at least one processor implements a neural network training method as described in any one of claims 1 to 7 by executing the instructions stored in the memory.

16. A computer program product, characterized in that, It includes a computer program or instructions that, when executed by a processor, implement a neural network training method as described in any one of claims 1 to 7.

Images