A method and apparatus for training a neural network model, an electronic device, and a medium

Abstract

The application discloses a neural network model training method and device, an electronic device, and a storage medium. The method comprises the following steps: dividing a plurality of pictures collected in advance into N types; taking one type in the N types as a current reference type; extracting one picture in the pictures of the current reference type as a current reference picture; constructing a current positive sample pair and a current negative sample pair based on the current reference picture, the pictures in the current reference type, and the pictures in the types other than the current reference type; and training a neural network model to be trained based on the current positive sample pair and the current negative sample pair. The embodiment of the application can adaptively compare the loss, keep different degrees of separation between samples with different similarities, improve the recognition ability of the model in the recognition scene of glaucoma screening, and thus achieve better training effect.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method, apparatus, electronic device, and medium for training a neural network model. Background Technology

[0002] Currently, the problem of glaucoma among drivers cannot be ignored. Glaucoma is a serious eye disease, causing anything from decreased vision to blindness. Drivers often have difficulty recognizing the disease in its early stages, easily leading to dangerous driving. Furthermore, vision loss or blindness caused by glaucoma is irreversible; neither medication nor surgery can restore vision. In addition, glaucoma has many other harmful effects and can easily trigger a series of complications.

[0003] Currently, deep learning is widely used for fundus image disease identification. In the field of metric learning within deep learning, constructing a suitable loss function is crucial for improving learning performance. In existing neural networks, contrastive loss and triplet loss aim to shorten the distance between similar samples and increase the distance between intermediate samples. Building upon this, there are ranking-driven structural losses, such as boosting structural loss and proxy-NCA loss. Contrastive loss predicts that all similar samples will overlap at a single point, and all different samples will approach a fixed boundary. Furthermore, in some neural network-based works, researchers fine-tune pre-trained portions of the network to meet practical requirements; for example, polar coordinate transformations can be performed on the extracted optic disc region as a parallel information flow in glaucoma prediction. End-to-end binary classification prediction can also be used on a large-scale glaucoma dataset. However, existing neural networks cannot adapt to contrastive loss and cannot maintain varying degrees of separation between samples with different similarities. Summary of the Invention

[0004] This application provides a training method, apparatus, electronic device, and storage medium for a neural network model, which can adaptively compare the loss and maintain different degrees of separation between samples with different similarities. In the glaucoma screening identification scenario, it can improve the model's recognition ability, thereby achieving better training results.

[0005] In a first aspect, embodiments of this application provide a method for training a neural network model, the method comprising:

[0006] If the neural network model to be trained does not meet the predetermined convergence condition, the pre-collected images will be divided into N types; where N is a natural number greater than 1.

[0007] Choose one of the N types as the current baseline type; and extract one image from the images of the current baseline type as the current baseline image;

[0008] Based on the current baseline image, images in the current baseline type, and images in types other than the current baseline type, construct a current positive sample pair and a current negative sample pair;

[0009] Based on the current positive sample pair and the current negative sample pair, the neural network model to be trained is trained, and the above operation is repeated until the neural network model to be trained satisfies the convergence condition.

[0010] Secondly, embodiments of this application also provide a training apparatus for a neural network model, the apparatus comprising: a partitioning module, an extraction module, a construction module, and a training module; wherein,

[0011] The partitioning module is used to partition multiple pre-collected images into N types if the neural network model to be trained does not meet the predetermined convergence conditions; where N is a natural number greater than 1.

[0012] The extraction module is used to select one of the N types as the current base type and extract an image from the image of the current base type as the current base image.

[0013] The construction module is used to construct a current positive sample pair and a current negative sample pair based on the current benchmark image, images in the current benchmark type, and images in types other than the current benchmark type.

[0014] The training module is used to train the neural network model to be trained based on the current positive sample pair and the current negative sample pair, and repeat the above operation until the neural network model to be trained satisfies the convergence condition.

[0015] Thirdly, embodiments of this application provide an electronic device, including:

[0016] One or more processors;

[0017] Memory, used to store one or more programs.

[0018] When the one or more programs are executed by the one or more processors, the one or more processors implement the training method of the neural network model described in any embodiment of this application.

[0019] Fourthly, embodiments of this application provide a storage medium storing a computer program thereon, which, when executed by a processor, implements the training method of the neural network model described in any embodiment of this application.

[0020] This application proposes a training method, apparatus, electronic device, and medium for a neural network model. If the neural network model to be trained does not meet a predetermined convergence condition, multiple pre-collected images are first divided into N types; then, one of the N types is used as the current baseline type; and an image is extracted from the images of the current baseline type as the current baseline image; then, based on the current baseline image, the images in the current baseline type, and the images in other types, a current positive sample pair and a current negative sample pair are constructed; then, based on the current positive sample pair and the current negative sample pair, the neural network model to be trained is trained, and the above operations are repeated until the neural network model to be trained meets the convergence condition. In other words, in the technical solution of this application, current positive sample pairs and current negative sample pairs can be constructed by classifying images, and the neural network model to be trained is trained based on the current positive sample pairs and the current negative sample pairs. After training, the neural network model will perform glaucoma disease analysis on the fundus images transmitted from the vehicle-mounted fundus camera, and the detection results will be transmitted to the driver as a basis for health data monitoring. However, existing neural networks cannot adaptively adjust contrast loss and cannot maintain different degrees of separation between samples with different similarities. Therefore, compared with the prior art, the training method, device, electronic device and medium of the neural network model proposed in this application can adaptively compare the loss and maintain different degrees of separation between samples with different similarities. In the identification scenario of glaucoma screening, it can improve the recognition ability of the model and thus achieve better training results. Moreover, the technical solution of this application is simple and convenient to implement, easy to popularize and has a wider range of applications. Attached Figure Description

[0021] Figure 1 A schematic diagram of the first process of training a neural network model provided in an embodiment of this application;

[0022] Figure 2 A schematic diagram of the second process of the training method for the neural network model provided in the embodiments of this application;

[0023] Figure 3 This is a schematic diagram of the structure of the neural network model provided in the embodiments of this application;

[0024] Figure 4 A schematic diagram of the third process of the training method for the neural network model provided in the embodiments of this application;

[0025] Figure 5 A flowchart illustrating the prediction method of the neural network model provided in the embodiments of this application;

[0026] Figure 6A schematic diagram of the structure of a training device for a neural network model provided in an embodiment of this application;

[0027] Figure 7 This is a schematic diagram of the structure of the building module provided in the embodiments of this application;

[0028] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0029] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the application and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present application, not the entire structure.

[0030] Example 1

[0031] Figure 1 This is a first flowchart illustrating a training method for a neural network model provided in an embodiment of this application. This method can be executed by a neural network model training device or electronic device, which can be implemented in software and / or hardware, and can be integrated into any smart device with network communication capabilities. Figure 1 As shown, the training method for a neural network model may include the following steps:

[0032] S101. If the neural network model to be trained does not meet the predetermined convergence condition, then the multiple images collected in advance will be divided into N types; where N is a natural number greater than 1.

[0033] In this step, if the neural network model to be trained does not meet the predetermined convergence condition, the pre-collected multiple images are divided into N types; where N is a natural number greater than 1. In this application, the pre-collected multiple images can be a set of images from the training dataset. These images can be acquired using a fundus image acquisition device, which can be a vehicle-mounted fundus camera. In this step, the electronic device can classify the multiple images according to the characteristics of the target objects in the images. For example, assuming 10 images are pre-collected, labeled images 1 to 10, this step can divide these 10 images into three categories: the first category includes images 1 to 3; the second category includes images 4 to 6; and the third category includes images 7 to 10.

[0034] S102. Select one of the N types as the current base type; and extract one image from the image of the current base type as the current base image.

[0035] In this step, the electronic device can choose one of N types as the current reference type and extract one image from the images of the current reference type as the current reference image. For example, suppose 10 images have been pre-collected, labeled images 1 to 10; and suppose these 10 images are divided into three categories: the first category includes images 1 to 3; the second category includes images 4 to 6; and the third category includes images 7 to 10. In this step, the first category can be used as the current reference type, and image 1 can be extracted from the images included in the first category as the current reference image.

[0036] S103. Based on the current baseline image, images in the current baseline type, and images in types other than the current baseline type, construct a current positive sample pair and a current negative sample pair.

[0037] In this step, the electronic device can construct a current positive sample pair and a current negative sample pair based on the current baseline image, images in the current baseline type, and images in types other than the current baseline type. Specifically, the electronic device can first extract one image from the images in the current baseline type and one image from the images in types other than the current baseline type as the current positive image and the current negative image, respectively; then, it combines the current baseline image and the current positive image into a current positive sample pair; and it combines the current baseline image and the current negative image into a current negative sample pair. For example, suppose 10 images are pre-collected, namely images 1 to 10; and suppose these 10 images are divided into three categories, where the first category includes images 1 to 3; the second category includes images 4 to 6; and the third category includes images 7 to 10; and suppose the first category is used as the current baseline type, and image 1 is used as the current baseline image. In this step, image 2 can be extracted from the images included in the first category as the current positive image; image 4 can be extracted from the images included in the second category as the current negative image; at this time, images 1 and 2 can be combined into a current positive sample pair; images 1 and 4 can be combined into a current negative sample pair.

[0038] S104. Based on the current positive sample pair and the current negative sample pair, train the neural network model to be trained. Repeat the above operation until the neural network model to be trained meets the convergence condition.

[0039] In this step, the electronic device can train the neural network model to be trained based on the current positive sample pair and the current negative sample pair, repeating the above operation until the neural network model to be trained meets the convergence condition. Specifically, the electronic device can first construct the loss function corresponding to the convolutional neural network to be trained in the current training based on the current positive sample pair and the positive sample pairs before the current training, and the current negative sample pair and the negative sample pairs before the current training; then, it can train the convolutional neural network to be trained based on the loss function corresponding to the current training; repeating the above operation until the convolutional neural network to be trained meets the corresponding convergence condition. Then, based on the current positive sample pair and the positive sample pairs before the current training, and the current negative sample pair and the negative sample pairs before the current training, it can construct the loss function corresponding to the classifier to be trained in the current training; then, it can train the classifier to be trained based on the loss function corresponding to the classifier to be trained in the current training; repeating the above operation until the classifier to be trained meets the corresponding convergence condition.

[0040] The neural network model training method proposed in this application involves classifying multiple pre-collected images into N types if the neural network model to be trained does not meet the predetermined convergence conditions. Then, one of the N types is used as the current baseline type. An image is extracted from the images of the current baseline type and used as the current baseline image. Next, a current positive sample pair and a current negative sample pair are constructed based on the current baseline image, images from the current baseline type, and images from other types. The neural network model to be trained is then trained based on the current positive sample pair and the current negative sample pair. This process is repeated until the neural network model meets the convergence conditions. In other words, in this technical solution, current positive sample pairs and current negative sample pairs can be constructed by classifying images. The neural network model to be trained is then trained based on these pairs. The trained neural network model will perform glaucoma disease analysis on fundus images transmitted from a vehicle-mounted fundus camera. The results will be transmitted to the driver as a basis for health data monitoring. However, existing neural networks cannot adaptively adapt to contrast loss and cannot maintain different degrees of separation between samples with different similarities. Therefore, compared with the prior art, the training method of the neural network model proposed in this application can adaptively compare the loss and maintain different degrees of separation between samples with different similarities. In the glaucoma screening identification scenario, it can improve the model's recognition ability and thus achieve better training results. Furthermore, the technical solution of this application is simple and convenient to implement, easy to popularize, and has a wider range of applications.

[0041] Example 2

[0042] Figure 2 This is a second flowchart illustrating the training method for the neural network model provided in this application embodiment. Further optimizations and extensions based on the above technical solution are possible, and it can be combined with the various optional implementation methods described above. For example... Figure 2 As shown, the training method for a neural network model may include the following steps:

[0043] S201. If the neural network model to be trained does not meet the predetermined convergence condition, then the multiple images collected in advance will be divided into N types; where N is a natural number greater than 1.

[0044] S202. Select one of the N types as the current base type; and extract one image from the image of the current base type as the current base image.

[0045] S203. Based on the current baseline image, images in the current baseline type, and images in types other than the current baseline type, construct a current positive sample pair and a current negative sample pair.

[0046] S204. Based on the current positive sample pair and the positive sample pairs before the current training, as well as the current negative sample pair and the negative sample pairs before the current training, construct the loss function corresponding to the convolutional neural network to be trained in the current training.

[0047] In this step, the electronic device can construct the loss function corresponding to the convolutional neural network to be trained in the current training iteration based on the current positive sample pair and the positive sample pairs before the current training iteration, as well as the current negative sample pair and the negative sample pairs before the current training iteration. Specifically, when constructing the loss function corresponding to the convolutional neural network to be trained in the current training iteration, the electronic device can first calculate the first loss term in the loss function corresponding to the convolutional neural network to be trained in the current training iteration based on the current positive sample pair and the positive sample pairs before the current training iteration; then, based on the current negative sample pair and the negative sample pairs before the current training iteration, it can calculate the second loss term in the loss function corresponding to the convolutional neural network to be trained in the current training iteration; finally, it sums the first loss term and the second loss term in the loss function corresponding to the convolutional neural network to be trained in the current training iteration to obtain the loss function corresponding to the convolutional neural network to be trained in the current training iteration. Specifically, the first loss term in the loss function corresponding to the convolutional neural network to be trained in the current training iteration can be expressed as: Where i represents the current baseline image; j represents the current positive image; P represents the set of positive sample pairs; s Euc(i,j) represents the feature similarity between the current baseline image and the current positive image, expressed using Euclidean distance; the second loss term in the loss function of the convolutional neural network to be trained in the current training iteration can be expressed as: Where i represents the current baseline image; k represents the current negative image; N represents the set of negative sample pairs; s Euc (i,k) represents the feature similarity between the current baseline image and the current negative image, expressed using Euclidean distance; m represents the pre-selected margin parameter; [·] represents the positive part of the function. Therefore, the loss function corresponding to the convolutional neural network to be trained in the current training iteration can be expressed as:

[0048] It should be noted that if similar samples have completely identical expressions in the feature space, the result will be unreliable. Therefore, the network should allow for differences in relative position to cause differences in content within the feature space. The more misclassified samples there are, the higher the percentage of loss value and the faster the optimization speed; conversely, those genes with relatively small differences should be tolerated. Therefore, this application suppresses the tendency for them to completely overlap, and one way to widen the gap between them is through a Gaussian distribution. Starting from this, this application proposes an adaptive contrastive loss function, as shown in the following equation, to guide the network during the training phase. Specifically, the first loss term in the loss function corresponding to the convolutional neural network to be trained in the current training iteration can also be expressed as: in, i represents the current baseline image; j represents the current positive image; P represents the set of positive sample pairs; s cos (i,j) represents the cosine similarity value between the current baseline image and the current positive image; w ij λ represents the adaptive weight of positive sample pairs. p The parameter values ​​represent the positive sample pairs. The second loss term in the loss function corresponding to the convolutional neural network to be trained in the current training iteration can also be expressed as: in, i represents the current baseline image; k represents the current negative image; N represents the set of negative sample pairs; s cos (i,j) represents the cosine similarity value between the current baseline image and the current negative image; w ik λ represents the adaptive weights of negative sample pairs. N The parameter values ​​represent the negative sample pairs. Therefore, the loss function corresponding to the convolutional neural network to be trained in the current training iteration can also be expressed as:

[0049] S205. Train the convolutional neural network to be trained based on the loss function corresponding to the convolutional neural network to be trained in the current training; repeat the above operation until the convolutional neural network to be trained satisfies the corresponding convergence condition.

[0050] In this step, the electronic device can train the convolutional neural network to be trained based on the loss function corresponding to the current training. If the convolutional neural network to be trained does not meet the corresponding convergence condition, the operation of dividing the pre-collected multiple images into N types is repeated until the convolutional neural network to be trained meets the corresponding convergence condition.

[0051] S206. Construct the loss function corresponding to the classifier to be trained in the current training based on the current benchmark image.

[0052] In this step, the electronic device can construct the loss function corresponding to the classifier to be trained in the current training iteration based on the current benchmark image. Specifically, in the classification task, convolutional blocks act as feature extractors, while fully connected layers function as classifiers. This application can treat fully connected layers as a class to perform predictions. A softmax operation is performed to ensure that the prediction results are ultimately mapped to the [0, 1] interval. In the classifier training phase, this application can select a weighted cross-entropy loss function. Specifically, the loss function corresponding to the classifier to be trained in the current training iteration can be expressed as... Where i represents the current baseline image; n represents the number of baseline images; y i =1 indicates that the sample has glaucoma; y i =0 indicates that the sample has no disease; p i w represents the predicted value of the classifier. p and w n These represent the weight values ​​for samples with glaucoma and samples without the disease, respectively; w p and w n The value range of w is (0, 1), and w satisfies p +w n =1.

[0053] S207. Train the classifier to be trained based on the loss function corresponding to the classifier to be trained in the current training; repeat the above operation until the classifier to be trained satisfies the corresponding convergence condition.

[0054] Figure 3 This is a schematic diagram of the structure of the neural network model provided in an embodiment of this application. Figure 3As shown, the neural network model may include: an input module, a skip connection module, a data imputation module, a convolutional neural network, a normalization module, and a classifier. The input module may include three input units: a first input unit, a second input unit, and a third input unit. The first input unit inputs image i into the neural network model; the second input unit inputs image j into the neural network model; and the third input unit inputs image k into the neural network model. The skip connection module includes an encoding unit and a decoding unit. The skip connection module obtains the regions of interest for each input image, designated as Region of Interest 1, Region of Interest 2, and Region of Interest 3. The data imputation module performs operations such as inversion on these regions of interest. The augmented data is then input into the convolutional neural network. The convolutional neural network extracts features from each region of interest, designated as Feature 1, Feature 2, and Feature 3. These features are then input into the normalization module. Finally, the feature vector output by the normalization module is input into the classifier.

[0055] The neural network model training method proposed in this application involves classifying multiple pre-collected images into N types if the neural network model to be trained does not meet the predetermined convergence conditions. Then, one of the N types is used as the current baseline type. An image is extracted from the images of the current baseline type and used as the current baseline image. Next, a current positive sample pair and a current negative sample pair are constructed based on the current baseline image, images from the current baseline type, and images from other types. The neural network model to be trained is then trained based on the current positive sample pair and the current negative sample pair. This process is repeated until the neural network model meets the convergence conditions. In other words, in this technical solution, current positive sample pairs and current negative sample pairs can be constructed by classifying images. The neural network model to be trained is then trained based on these pairs. The trained neural network model will perform glaucoma disease analysis on fundus images transmitted from a vehicle-mounted fundus camera. The results will be transmitted to the driver as a basis for health data monitoring. However, existing neural networks cannot adaptively adapt to contrast loss and cannot maintain different degrees of separation between samples with different similarities. Therefore, compared with the prior art, the training method of the neural network model proposed in this application can adaptively compare the loss and maintain different degrees of separation between samples with different similarities. In the glaucoma screening identification scenario, it can improve the model's recognition ability and thus achieve better training results. Furthermore, the technical solution of this application is simple and convenient to implement, easy to popularize, and has a wider range of applications.

[0056] Example 3

[0057] Figure 4 This is a schematic diagram of the third process of the training method for the neural network model provided in this application embodiment. Further optimizations and extensions can be made based on the above technical solution, and it can be combined with the various optional implementation methods described above. For example... Figure 4 As shown, the training method for a neural network model may include the following steps:

[0058] S401. If the neural network model to be trained does not meet the predetermined convergence condition, then the multiple images collected in advance will be divided into N types; where N is a natural number greater than 1.

[0059] S402. Select one of the N types as the current base type; and extract one image from the image of the current base type as the current base image.

[0060] S403. Based on the current baseline image, images in the current baseline type, and images in types other than the current baseline type, construct a current positive sample pair and a current negative sample pair.

[0061] S404. Based on the current positive sample pairs and the current negative sample pairs, construct the loss function corresponding to the convolutional neural network to be trained in the current training iteration.

[0062] S405. Train the convolutional neural network to be trained based on the loss function corresponding to the convolutional neural network to be trained in the current training; repeat the above operation until the convolutional neural network to be trained satisfies the corresponding convergence condition.

[0063] S406. Construct the loss function corresponding to the classifier to be trained in the current training based on the current benchmark image.

[0064] S407. Train the classifier to be trained based on the loss function corresponding to the classifier to be trained in the current training; repeat the above operation until the classifier to be trained satisfies the corresponding convergence condition.

[0065] S408. Input the current image into the trained neural network model, and obtain the feature vector of the current image through the trained neural network model.

[0066] In this step, the electronic device can use any image from the training dataset as the current image and input it into the trained neural network model to obtain the feature vector of the current image. This application utilizes the BN-Inception network, which first trains the network using pre-trained parameters, and then trains it using similarity-preserving training and feature inductive prediction methods. The ORIGA dataset was used in the experiment, with half of the images used for training and the other half for testing. The evaluation metric was the area under the operating characteristic curve (ROC), and the AUC score of the prediction result was 0.825. After training, the network will perform glaucoma disease analysis on fundus images transmitted from an in-vehicle fundus camera, and the results will be transmitted to the driver as a basis for health data monitoring.

[0067] S409. Input the feature vector of the current image into a pre-trained classifier, and use the trained classifier to recognize the current image to obtain the recognition result corresponding to the current image.

[0068] Figure 5 This is a flowchart illustrating the prediction method of the neural network model provided in an embodiment of this application. Figure 5 As shown, the neural network model may include: an input module, a skip connection module, a convolutional neural network, a normalization module, and a classifier; wherein, the input module is used to input the current image into the neural network model; the skip connection module may include: an encoding unit and a decoding unit; the skip connection module can obtain the region of interest of the current image; then the convolutional neural network can extract the features of the region of interest; then the above features are input into the normalization module; and then the feature vector output by the normalization module is input into the classifier.

[0069] The neural network model training method proposed in this application involves classifying multiple pre-collected images into N types if the neural network model to be trained does not meet the predetermined convergence conditions. Then, one of the N types is used as the current baseline type. An image is extracted from the images of the current baseline type and used as the current baseline image. Next, a current positive sample pair and a current negative sample pair are constructed based on the current baseline image, images from the current baseline type, and images from other types. The neural network model to be trained is then trained based on the current positive sample pair and the current negative sample pair. This process is repeated until the neural network model meets the convergence conditions. In other words, in this technical solution, current positive sample pairs and current negative sample pairs can be constructed by classifying images. The neural network model to be trained is then trained based on these pairs. The trained neural network model will perform glaucoma disease analysis on fundus images transmitted from a vehicle-mounted fundus camera. The results will be transmitted to the driver as a basis for health data monitoring. However, existing neural networks cannot adaptively adapt to contrast loss and cannot maintain different degrees of separation between samples with different similarities. Therefore, compared with the prior art, the training method of the neural network model proposed in this application can adaptively compare the loss and maintain different degrees of separation between samples with different similarities. In the glaucoma screening identification scenario, it can improve the model's recognition ability and thus achieve better training results. Furthermore, the technical solution of this application is simple and convenient to implement, easy to popularize, and has a wider range of applications.

[0070] Example 4

[0071] Figure 6 This is a schematic diagram of the structure of a training device for a neural network model provided in an embodiment of this application. Figure 6 As shown, the training device for the neural network model includes: a partitioning module 601, an extraction module 602, a construction module 603, and a training module 604; wherein,

[0072] The partitioning module 601 is used to partition multiple pre-collected images into N types if the neural network model to be trained does not meet the predetermined convergence conditions; where N is a natural number greater than 1.

[0073] The extraction module 602 is used to take one of the N types as the current base type; and extract an image from the image of the current base type as the current base image;

[0074] The construction module 603 is used to construct a current positive sample pair and a current negative sample pair based on the current benchmark image, images in the current benchmark type, and images in types other than the current benchmark type.

[0075] The training module 604 is used to train the neural network model to be trained based on the current positive sample pair and the current negative sample pair, and repeat the above operation until the neural network model to be trained satisfies the convergence condition.

[0076] Figure 7 This is a schematic diagram of the building module provided in an embodiment of this application. Figure 7 As shown, the construction module 603 includes: an extraction submodule 6031 and a combination submodule 6032; wherein,

[0077] The extraction submodule 6031 is used to extract one image from the image in the current benchmark type and one image from the image in a type other than the current benchmark type, respectively, as the current positive image and the current negative image.

[0078] The combination submodule 6032 is used to combine the current baseline image and the current positive image into the current positive sample pair; and to combine the current baseline image and the current negative image into the current negative sample pair.

[0079] Furthermore, the training module 604 is specifically configured to: construct a loss function for the convolutional neural network to be trained in the current training iteration based on the current positive sample pair and previous positive sample pairs, and the current negative sample pair and previous negative sample pairs; train the convolutional neural network to be trained based on the loss function for the current training iteration; repeat the above operations until the convolutional neural network to be trained satisfies the corresponding convergence condition; construct a loss function for the classifier to be trained in the current training iteration based on the current benchmark image; train the classifier to be trained based on the loss function for the classifier to be trained in the current training iteration; repeat the above operations until the classifier to be trained satisfies the corresponding convergence condition.

[0080] The training apparatus for the aforementioned neural network model can execute the methods provided in any embodiment of this application, and has the corresponding functional modules and beneficial effects for executing the methods. Technical details not described in detail in this embodiment can be found in the training methods for the neural network models provided in any embodiment of this application.

[0081] Example 5

[0082] Figure 8This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Figure 8 A block diagram of an exemplary electronic device suitable for implementing embodiments of the present application is shown. Figure 8 The electronic device 12 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0083] like Figure 8 As shown, the electronic device 12 is represented in the form of a general-purpose computing device. The components of the electronic device 12 may include, but are not limited to: one or more processors or processing units 16, system memory 28, and bus 18 connecting different system components (including system memory 28 and processing unit 16).

[0084] Bus 18 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0085] Electronic device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by electronic device 12, including volatile and non-volatile media, removable and non-removable media.

[0086] System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. Electronic device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media (… Figure 8 Not shown; usually referred to as a "hard drive"). Although Figure 8 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. Memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of this application.

[0087] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 42 typically perform the functions and / or methods described in the embodiments of this application.

[0088] Electronic device 12 can also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), and with one or more devices that enable a user to interact with electronic device 12, and / or with any device that enables electronic device 12 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed via input / output (I / O) interface 22. Furthermore, electronic device 12 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 20. As shown, network adapter 20 communicates with other modules of electronic device 12 via bus 18. It should be understood that, although... Figure 8 As not shown, other hardware and / or software modules may be used in conjunction with electronic device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0089] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing the training method of the neural network model provided in the embodiments of this application.

[0090] Example 6

[0091] This application provides a computer storage medium.

[0092] The computer-readable storage medium of this application embodiment can be any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. For example, a computer-readable storage medium can be—but is not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0093] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0094] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including—but not limited to—wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0095] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0096] Note that the above are merely preferred embodiments and the technical principles employed in this application. Those skilled in the art will understand that this application is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of this application. Therefore, although this application has been described in detail through the above embodiments, this application is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of this application, the scope of which is determined by the scope of the appended claims.

Claims

1. A method for training a neural network model, the method comprising: The method comprises: If the neural network model to be trained does not satisfy a predetermined convergence condition, a plurality of pictures previously collected are divided into N types; wherein N is a natural number greater than 1; One of the N types is taken as a current reference type; and one picture in the pictures of the current reference type is extracted as a current reference picture; Based on the current reference picture, the pictures in the current reference type and the pictures in the types other than the current reference type, a current positive sample pair and a current negative sample pair are constructed; The neural network model to be trained is trained based on the current positive sample pair and the current negative sample pair, and the above operation is repeatedly performed until the neural network model to be trained satisfies the convergence condition; The training of the neural network model to be trained based on the current positive sample pair and the current negative sample pair comprises: Based on the current positive sample pair and the positive sample pairs before the current training, and the current negative sample pair and the negative sample pairs before the current training, a loss function corresponding to the convolutional neural network to be trained in the current training is constructed, and the specific formula is: wherein, represents the current reference picture, represents an adaptive weight of the positive sample pair, represents a current positive picture, represents a set of positive sample pairs, represents a cosine similarity value between the current reference picture and the current positive picture, represents a parameter value of the positive sample pair, represents a current negative picture, represents a set of negative sample pairs; represents a cosine similarity value between the current reference picture and the current negative picture; represents an adaptive weight of the negative sample pair, represents a parameter value of the negative sample pair; The convolutional neural network to be trained is trained based on the loss function corresponding to the convolutional neural network to be trained in the current training; and the above operation is repeatedly performed until the convolutional neural network to be trained satisfies a corresponding convergence condition; Based on the current reference picture, a loss function corresponding to the classifier to be trained in the current training is constructed; The classifier to be trained is trained based on the loss function corresponding to the classifier to be trained in the current training; and the above operation is repeatedly performed until the classifier to be trained satisfies a corresponding convergence condition.

2. The method of claim 1, wherein, The construction of a current positive sample pair and a current negative sample pair based on the current reference picture and the pictures in the current reference type and the pictures in the types other than the current reference type comprises: In the pictures in the current reference type and the pictures in the types other than the current reference type, one picture is extracted as the current positive picture and the current negative picture respectively; The current reference picture and the current positive picture are combined as the current positive sample pair; and the current reference picture and the current negative picture are combined as the current negative sample pair.

3. The method of claim 1, wherein, The construction of a loss function corresponding to the convolutional neural network to be trained in the current training based on the current positive sample pair and the positive sample pairs before the current training, and the current negative sample pair and the negative sample pairs before the current training comprises: Based on the current positive sample pair and the positive sample pairs before the current training, a first loss term in the loss function corresponding to the convolutional neural network to be trained in the current training is calculated; Based on the current negative sample pair and the negative sample pairs before the current training, a second loss term in the loss function corresponding to the convolutional neural network to be trained in the current training is calculated; and The first loss term in the loss function corresponding to the to-be-trained convolutional neural network in the current training and the second loss term in the loss function corresponding to the to-be-trained convolutional neural network in the current training are summed to obtain the loss function corresponding to the to-be-trained convolutional neural network in the current training.

4. The method of claim 1, wherein, The method further comprises: inputting the current picture into the trained neural network model to obtain a feature vector of the current picture through the trained neural network model; inputting the feature vector of the current picture into the pre-trained classifier to identify the current picture through the trained classifier to obtain an identification result corresponding to the current picture.

5. A device for training a neural network model, characterized by, The device comprises a division module, an extraction module, a construction module and a training module; wherein, the division module is configured to divide a plurality of pictures pre-acquired into N types if the to-be-trained neural network model does not satisfy a pre-determined convergence condition; wherein N is a natural number greater than 1; the extraction module is configured to take one of the N types as a current reference type and extract one picture in the pictures of the current reference type as a current reference picture; the construction module is configured to construct one current positive sample pair and one current negative sample pair based on the current reference picture and the pictures in the current reference type and the pictures in the types other than the current reference type; the training module is configured to train the to-be-trained neural network model based on the current positive sample pair and the current negative sample pair, and repeatedly execute the above operation until the to-be-trained neural network model satisfies the convergence condition; the training module is specifically configured to construct a loss function corresponding to the to-be-trained convolutional neural network in the current training based on the current positive sample pair and the positive sample pairs before the current training and the current negative sample pair and the negative sample pairs before the current training, and the specific formula is: wherein, denotes the current reference picture, denotes the adaptive weight of the positive sample pair, denotes the current positive picture, denotes the set of positive sample pairs, denotes the cosine similarity value between the current reference picture and the current positive picture, denotes the parameter value of the positive sample pair, denotes the current negative picture, denotes the set of negative sample pairs; denotes the cosine similarity value between the current reference picture and the current negative picture; denotes the adaptive weight of the negative sample pair, denotes the parameter value of the negative sample pair; training the to-be-trained convolutional neural network based on the loss function corresponding to the to-be-trained convolutional neural network in the current training; repeatedly performing the above operations until the to-be-trained convolutional neural network meets the corresponding convergence condition; constructing the loss function corresponding to the to-be-trained classifier in the current training based on the current reference picture; training the to-be-trained classifier based on the loss function corresponding to the to-be-trained classifier in the current training; repeatedly performing the above operations until the to-be-trained classifier meets the corresponding convergence condition.

6. The apparatus of claim 5, wherein, the construction module comprises an extraction submodule and a combination submodule; wherein, the extraction submodule is configured to extract one picture in the pictures in the current reference type and the pictures in the types other than the current reference type as a current positive picture and a current negative picture respectively; the combination submodule is configured to combine the current reference picture and the current positive picture as the current positive sample pair and combine the current reference picture and the current negative picture as the current negative sample pair.

7. An electronic device, comprising: comprise: one or more processors; a memory for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the training method of the neural network model as claimed in any one of claims 1 to 4.

8. A storage medium having stored thereon a computer program, characterized in that The program is executed by the processor to implement the training method of the neural network model as claimed in any one of claims 1 to 4.

Images