A training method and device for a neural network model
By introducing a dual loss function mechanism into the neural network model, combining student neural networks and teacher neural networks, the problems of insufficient model features and slow learning speed in the existing technology are solved, and the recognition accuracy and generalization ability of the model are improved.
Patent Information
- Application Number
- CN202011602983.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-12-29
- Publication Date
- 2024-12-13
- Estimated Expiration
- 2040-12-29
AI Technical Summary
During the training process of neural network models in the prior art, the use of single loss functions leads to insufficient learning features of the model, slow learning speed, and low generalization ability.
The dual loss function mechanism is adopted, through the combination of student neural network and teacher neural network, the student neural network is used to extract features and update them based on the feedback of teacher neural network, to improve the training convergence speed and feature learning ability of the model.
It effectively improves the training convergence status of neural network models, improves its recognition accuracy and generalization ability, and ensures that the model can better learn features.
Smart Images
Figure CN112668716B_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of machine learning technology, and in particular, relates to a training method and device for a neural network model. Background Art
[0002] With the development of artificial intelligence, deep learning is increasingly showing its irreplaceable role. During the training process, a loss function is pre-set, and the training data is passed to the preset neural network for forward propagation to extract the data features, and then the extracted features are substituted into the loss function to calculate the loss value. Finally, the preset parameters of the preset neural network model are updated using the back propagation algorithm based on the loss value.
[0003] However, the loss function in the prior art generally adopts a single loss function, which may result in the trained model learning insufficient features or a slow learning speed and low generalization ability. Summary of the invention
[0004] The embodiments of the present application provide a method and device for training a neural network model, which can solve the problem that the features learned by the current trained model are not sufficient or the learning speed is slow and the generalization ability is not high.
[0005] In a first aspect, an embodiment of the present application provides a method for training a neural network model, comprising:
[0006] Obtain a training sample set;
[0007] Inputting the training sample set into a preset student neural network model for processing to obtain first feature information, and calculating a first loss value according to the first feature information and a first loss function;
[0008] Input the training sample set into a preset teacher neural network model for processing to obtain fourth feature information, and calculate a second loss value according to the first feature information, the fourth feature information and the second loss function;
[0009] Determine a target loss value according to the first loss value and the second loss value;
[0010] If the target loss value does not meet the preset termination condition, the preset student neural network is updated according to the target loss value, and the training sample set is input into the preset student neural network model for processing;
[0011] If the target loss value meets the preset termination condition, the trained student neural network model is output.
[0012] Further, the preset student neural network model includes a first feature extraction structure, a second feature extraction structure and a third feature extraction structure;
[0013] The step of inputting the training sample set into a preset student neural network model for processing to obtain first feature information, and calculating a first loss value according to the first feature information and a first loss function includes:
[0014] Inputting the training sample set into the first feature extraction structure for processing to obtain second feature information, and calculating a first auxiliary loss value according to the second feature information;
[0015] Inputting the second feature information into the second feature extraction structure for processing to obtain third feature information, and calculating a second auxiliary loss value according to the third feature information;
[0016] The third feature information is input into the third feature extraction structure for processing to obtain the first feature information, and a third auxiliary loss value is calculated based on the first feature information.
[0017] Further, the first feature extraction structure includes a convolutional layer;
[0018] The step of inputting the training sample set into the first feature extraction structure for processing to obtain second feature information, and calculating a first auxiliary loss value according to the second feature information includes:
[0019] Inputting the training sample set into the convolution layer to extract features to obtain second feature information;
[0020] A first auxiliary loss value is calculated according to the second characteristic information and preset ideal data.
[0021] Furthermore, the first feature extraction structure further includes a fully connected layer and an output layer; the activation function of the output layer is a soft-max function;
[0022] The calculating the first auxiliary loss value according to the second characteristic information and preset ideal data includes:
[0023] Inputting the second feature information into the fully connected layer for processing to obtain a feature vector;
[0024] Inputting the feature vector into the output layer, and determining, according to preset ideal data, the probability that the category of the training sample in the training sample set belongs to the target category corresponding to the preset ideal data;
[0025] A first auxiliary loss value is calculated according to the probability and a preset cross entropy loss function.
[0026] Furthermore, the preset teacher neural network model includes a Resnet152 network.
[0027] Further, the calculating the second loss value according to the first feature information, the fourth feature information and the second loss function includes:
[0028] Calculate the square difference between the first feature information and the fourth feature information to obtain a second loss value;
[0029] or,
[0030] Calculate the mean square error of the first feature information and the fourth feature information to obtain a second loss value;
[0031] or,
[0032] The first feature information and the fourth feature information are used to calculate absolute values to obtain a second loss value.
[0033] Further, determining a target loss value according to the first loss value and the second loss value includes:
[0034] The first auxiliary loss value, the second auxiliary loss value, the third auxiliary loss value and the second loss value are weightedly summed to obtain a target loss value.
[0035] In a second aspect, an embodiment of the present application provides an object recognition method, including:
[0036] Get the object image;
[0037] The object image is input into a preset object recognition model for processing to obtain an object recognition result corresponding to the object image; wherein the object recognition model is obtained by the training method of the neural network model described in the first aspect above.
[0038] In a third aspect, an embodiment of the present application provides a training device for a neural network model, comprising:
[0039] A first acquisition unit, used to acquire a training sample set;
[0040] A first processing unit, configured to input the training sample set into a preset student neural network model for processing, obtain first feature information, and calculate a first loss value according to the first feature information and a first loss function;
[0041] A second processing unit is used to input the training sample set into a preset teacher neural network model for processing to obtain fourth feature information, and calculate a second loss value according to the first feature information, the fourth feature information and the second loss function;
[0042] A first determining unit, configured to determine a target loss value according to the first loss value and the second loss value;
[0043] A third processing unit is used for updating the preset student neural network according to the target loss value if the target loss value does not meet the preset termination condition, and returning to execute inputting the training sample set into the preset student neural network model for processing;
[0044] The fourth processing unit is used to output the trained student neural network model if the target loss value meets the preset termination condition.
[0045] Further, the preset student neural network model includes a first feature extraction structure, a second feature extraction structure and a third feature extraction structure;
[0046] The first processing unit is specifically configured to:
[0047] Inputting the training sample set into the first feature extraction structure for processing to obtain second feature information, and calculating a first auxiliary loss value according to the second feature information;
[0048] Inputting the second feature information into the second feature extraction structure for processing to obtain third feature information, and calculating a second auxiliary loss value according to the third feature information;
[0049] The third feature information is input into the third feature extraction structure for processing to obtain the first feature information, and a third auxiliary loss value is calculated based on the first feature information.
[0050] Further, the first feature extraction structure includes a convolutional layer;
[0051] The first processing unit is further configured to:
[0052] Inputting the training sample set into the convolution layer to extract features to obtain second feature information;
[0053] A first auxiliary loss value is calculated according to the second characteristic information and preset ideal data.
[0054] Furthermore, the first feature extraction structure further includes a fully connected layer and an output layer; the activation function of the output layer is a soft-max function;
[0055] The first processing unit is further configured to:
[0056] Inputting the second feature information into the fully connected layer for processing to obtain a feature vector;
[0057] Inputting the feature vector into the output layer, and determining, according to preset ideal data, the probability that the category of the training sample in the training sample set belongs to the target category corresponding to the preset ideal data;
[0058] A first auxiliary loss value is calculated according to the probability and a preset cross entropy loss function.
[0059] Furthermore, the preset teacher neural network model includes a Resnet152 network.
[0060] Furthermore, the second processing unit is specifically configured to:
[0061] Calculate the square difference between the first feature information and the fourth feature information to obtain a second loss value;
[0062] or,
[0063] Calculate the mean square error of the first feature information and the fourth feature information to obtain a second loss value;
[0064] or,
[0065] The first feature information and the fourth feature information are used to calculate absolute values to obtain a second loss value.
[0066] Furthermore, the first determining unit is specifically configured to:
[0067] The first auxiliary loss value, the second auxiliary loss value, the third auxiliary loss value and the second loss value are weightedly summed to obtain a target loss value.
[0068] In a fourth aspect, an embodiment of the present application provides an object recognition device, including:
[0069] An acquisition unit, used for acquiring an object image;
[0070] A processing unit is used to input the object image into a preset object recognition model for processing to obtain an object recognition result corresponding to the object image; wherein the object recognition model is obtained by the training method of the neural network model described in the first aspect above.
[0071] In a fifth aspect, an embodiment of the present application provides a training device for a neural network model, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the training method for the neural network model as described in the first aspect above is implemented.
[0072] In a sixth aspect, an embodiment of the present application provides an object recognition device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the object recognition method as described in the second aspect above is implemented.
[0073] In a seventh aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the training method of the neural network model as described in the first aspect above is implemented.
[0074] In an eighth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the object recognition method as described in the second aspect above is implemented.
[0075] On the one hand, in an embodiment of the present application, a training sample set is obtained; the training sample set is input into a preset student neural network model for processing to obtain first feature information, and a first loss value is calculated based on the first feature information and a first loss function; the training sample set is input into a preset teacher neural network model for processing to obtain fourth feature information, and a second loss value is calculated based on the first feature information, the fourth feature information and a second loss function; a target loss value is determined based on the first loss value and the second loss value; if the target loss value does not meet the preset termination condition, the preset student neural network is updated according to the target loss value, and the training sample set is input into the preset student neural network model for processing; if the target loss value meets the preset termination condition, the trained student neural network model is output. In the present application, the target loss value is determined by the first loss value obtained by the preset student neural network model and the second loss value obtained by the preset teacher neural network model, which effectively improves the training convergence of the neural network model, enables the neural network to better perform feature learning, and improves its recognition accuracy and generalization ability.
[0076] On the other hand, the object recognition method provided in the present application obtains an object image; the object image is input into a preset object recognition model for processing to obtain an object recognition result corresponding to the object image; wherein the object recognition model is trained by a training method of a neural network model. The object recognition model in the present application is trained by a training method of a neural network model, so the object recognition method in the present application can better extract the features of the object image, thereby improving the accuracy of object recognition. BRIEF DESCRIPTION OF THE DRAWINGS
[0077] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0078] Figure 1 is a schematic flow chart of a training method for a neural network model provided in the first embodiment of the present application;
[0079] Figure 2 It is a schematic flow chart of the refinement of S102 in a training method for a neural network model provided in the first embodiment of the present application;
[0080] Figure 3 is a schematic flow chart of an object recognition method provided in the second embodiment of the present application;
[0081] Figure 4 is a schematic diagram of a training device for a neural network model provided in the third embodiment of the present application;
[0082] Figure 5 is a schematic diagram of an object recognition device provided in a fourth embodiment of the present application;
[0083] Figure 6 is a schematic diagram of a training device for a neural network model provided in a fifth embodiment of the present application;
[0084] Figure 7 It is a schematic diagram of an object recognition device provided in the sixth embodiment of the present application. DETAILED DESCRIPTION
[0085] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.
[0086] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.
[0087] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.
[0088] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.
[0089] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.
[0090] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.
[0091] See also Figure 1 , Figure 1 1 is a schematic flow chart of a training method for a neural network model provided in the first embodiment of the present application. The execution subject of the training method for a neural network model in this embodiment is a device with a training function for a neural network model, such as a desktop computer, a server, etc. Figure 1 The training method of the neural network model shown may include:
[0092] S101: Obtain a training sample set.
[0093] The device obtains a training sample set, where the training sample set includes training samples and their corresponding sample labels, and each training sample corresponds to a sample label.
[0094] In one embodiment, the training sample may include an RGB image, an IR image, a depth image, or one of other images. The training sample may be used to train a neural network model for image processing, which is not limited here.
[0095] In one embodiment, in order to accelerate the calculation, based on computer principles, the number of training samples in the training sample set may be a multiple of 32.
[0096] S102: Input the training sample set into a preset student neural network model for processing to obtain first feature information, and calculate a first loss value based on the first feature information and a first loss function.
[0097] The device pre-stores a preset student neural network model, which is used to train the final required neural network model. The preset student neural network model has a simple structure and average performance. The preset student neural network model learns the behavior of the preset teacher neural network model.
[0098] The device inputs the training sample set into a preset student neural network model for processing to obtain first feature information, and calculates a first loss value based on the first feature information and a first loss function.
[0099] The first loss function may be a cross entropy loss function.
[0100] In one embodiment, in order to more accurately learn the behavior of the preset teacher neural network model, the preset student neural network model includes a first feature extraction structure, a second feature extraction structure and a third feature extraction structure; the first feature extraction structure, the second feature extraction structure and the third feature extraction structure are connected in series to form the preset student neural network model. S102 may include S1021~S1023, such as Figure 2 As shown, S1021~S1023 are as follows:
[0101] S1021: Input the training sample set into the first feature extraction structure for processing to obtain second feature information, and calculate the first auxiliary loss value based on the second feature information.
[0102] The device inputs the training sample set into the first feature extraction structure for processing, obtains second feature information, and calculates the first auxiliary loss value based on the second feature information.
[0103] Specifically, the first feature extraction structure includes a convolution layer; the first feature extraction structure may include multiple convolution layers, and the convolution layer may be 1 layer or 2 layers. In order to extract global features, this embodiment preferably has three layers, which is not limited here.
[0104] The convolution layer can be used to extract the second feature information of the training samples in the training sample set, and can also be used to reduce the dimension in the process of extracting features from the training samples to reduce the amount of calculation.
[0105] The device inputs the training sample set into the convolution layer for feature extraction to obtain the second feature information. In one embodiment, the first feature extraction structure includes at least 3 convolution layers, and the activation layer is connected to the convolution layer. Preferably, the activation layer is a Relu activation function layer. The activation layer performs nonlinear activation on the feature layer extracted by the linear calculation of the convolution layer, and introduces nonlinear features to improve the nonlinear expression ability of the network to be trained.
[0106] It can be understood that before inputting the first feature extraction structure, a batch normalization layer can be set to normalize the training samples to prevent the data distribution from changing during the training process, thereby causing the problem of gradient disappearance or gradient explosion.
[0107] After obtaining the second characteristic information, the device calculates the first auxiliary loss value according to the second characteristic information and the preset ideal data. The device calculates the first auxiliary loss value according to the second characteristic information, the preset ideal data and the preset loss function.
[0108] In one embodiment, the first feature extraction structure also includes a fully connected layer and an output layer; the activation function of the output layer is a soft-max function; the second feature information is input into the fully connected layer for processing to obtain a feature vector; wherein the fully connected layer is composed of multiple neurons and is fully connected with the last convolutional layer, so after the second feature information passes through the fully connected layer, a feature vector with the same dimension as the number of neurons can be obtained.
[0109] The activation function of the output layer is a soft-max function; the device inputs the feature vector into the output layer, and determines the probability that the category of the training sample in the training sample set belongs to the target category corresponding to the preset ideal data based on the preset ideal data. The output layer is connected to the fully connected layer, which includes neurons that match the number of training samples. The feature vector of the fully connected layer is combined with different weight matrices and biases and assigned to each neuron. Each neuron corresponds to the probability that the category of the training sample belongs to a category in the preset ideal data.
[0110] It can be understood that the preset ideal data contains the same categories and quantity as the training samples, and the sum of the probabilities corresponding to each neuron is 1.
[0111] The device finally calculates the first auxiliary loss value based on the probability and the preset cross entropy loss function.
[0112] In one embodiment, the classification result takes the maximum probability, assuming that the first auxiliary loss value is ,but
[0113]
[0114] Where N is the number of categories. Characterizes the indicator variable. If the category of the training sample is the same as the category in the preset ideal data, it is 1, otherwise it is 0; It indicates the predicted probability that the category belongs to a category K in the preset ideal data.
[0115] It can be understood that the fact that the category of the training sample is the same as the category of the preset ideal data indicates that when passing through the output layer, the category of the training sample has the highest probability of belonging to a category of the preset ideal data; and the calculation of the loss function is not limited to this, and can also be a log loss function, which is not limited here.
[0116] S1022: Input the second feature information into the second feature extraction structure for processing to obtain third feature information, and calculate the second auxiliary loss value based on the third feature information.
[0117] In this embodiment, the device inputs the second feature information into the second feature extraction structure for processing to obtain the third feature information, and calculates the second auxiliary loss value based on the third feature information. The specific details can be referred to in S1021, in which the training sample set is input into the first feature extraction structure for processing to obtain the second feature information, and the first auxiliary loss value is calculated based on the second feature information. The relevant description will not be repeated here.
[0118] It should be noted that the number of convolutional layers of the second feature extraction structure and the first feature extraction structure may be the same or different.
[0119] Preferably, the number of convolutional layers of the second feature extraction structure is greater than the number of convolutional layers of the first feature extraction structure, so as to extract more global features.
[0120] S1023: Input the third feature information into the third feature extraction structure for processing to obtain the first feature information, and calculate the third auxiliary loss value based on the first feature information.
[0121] In this embodiment, the third feature information is input into the third feature extraction structure for processing to obtain the first feature information, and the specific details of calculating the third auxiliary loss value based on the first feature information can refer to the relevant description of inputting the training sample set into the first feature extraction structure for processing to obtain the second feature information, and calculating the first auxiliary loss value based on the second feature information in S1021, which will not be repeated here.
[0122] It should be noted that the number of convolutional layers in the third feature extraction structure and the first feature extraction structure may be the same or different.
[0123] S103: Input the training sample set into the preset teacher neural network model for processing to obtain fourth feature information, and calculate the second loss value according to the first feature information, the fourth feature information and the second loss function.
[0124] The device pre-stores a preset teacher neural network model, and the functions implemented by the teacher network model should be the same as the functions implemented by the preset student neural network model. For example, both the teacher network model and the preset student neural network model can be used for computer image processing and output the processed image data.
[0125] In one embodiment, the preset teacher neural network model includes a Resnet152 network. Resnet152 is a highly complex super-large network model, including 152 convolutional layers. The more convolutional layers it includes, the richer the features that can be extracted, and the deeper the network, the higher the completeness of the features extracted. However, Resnet152 has a slow training speed and requires high-performance computing. By using the teacher network model to teach the student network model to learn its accurate behavior, a smaller and faster student network model can be obtained, which is a method that can match the output results of the teacher network model, namely knowledge distillation.
[0126] The device calculates a second loss value based on the first feature information, the fourth feature information and the second loss function.
[0127] In one embodiment, the square difference between the first feature information and the fourth feature information is calculated to obtain the second loss value. The training sample set is passed to the preset teacher network model to obtain the fourth feature information, and the square difference between the fourth feature information and the first feature information of the student network model is calculated, that is, the square loss function:
[0128]
[0129] Wherein, Y is a matrix indicating the fourth feature information, It is also a matrix indicating the first feature information, and n represents the number of samples contained in the training sample.
[0130] It should be noted that the squared difference calculated based on the squared loss function is a matrix. Based on the matrix, each element contained in it is added to obtain the second loss value.
[0131] In one embodiment, the device may also perform mean square error calculation on the first feature information and the fourth feature information to obtain the second loss value; or, perform absolute value calculation on the first feature information and the fourth feature information to obtain the second loss value, which is not limited here.
[0132] S104: Determine a target loss value according to the first loss value and the second loss value.
[0133] The device determines a target loss value based on the first loss value and the second loss value. Specifically, the device can calculate the target loss value by setting a weight coefficient of the first loss value and the second loss value; or the device can calculate the target loss value based on the first loss value and the second loss value according to a preset calculation rule.
[0134] In one embodiment, the device may perform weighted summation of the first auxiliary loss value, the second auxiliary loss value, the third auxiliary loss value, and the second loss value to obtain a target loss value.
[0135] The target loss value L can be:
[0136]
[0137] Among them, L1 is the first auxiliary loss value, L2 is the second auxiliary loss value, L3 is the third auxiliary loss value, and L4 is the second loss value.
[0138] It is understandable that the expression of the final loss value is not limited to this, and different loss values can also be reasonably configured using preset weights to obtain the target loss value, which is not limited here.
[0139] S105: If the target loss value does not meet the preset termination condition, the preset student neural network is updated according to the target loss value, and the process returns to execute inputting the training sample set into the preset student neural network model for processing.
[0140] The device stores preset termination conditions. If the target loss value does not meet the preset termination conditions, the preset student neural network is updated according to the target loss value, and the batch gradient descent method is used for back propagation to update the learning parameters of the model, and the training sample set is input into the preset student neural network model for processing.
[0141] S106: If the target loss value meets the preset termination condition, the trained student neural network model is output.
[0142] If the target loss value meets the preset termination condition, the trained student neural network model is output.
[0143] In an embodiment of the present application, a training sample set is obtained; the training sample set is input into a preset student neural network model for processing to obtain first feature information, and a first loss value is calculated based on the first feature information and a first loss function; the training sample set is input into a preset teacher neural network model for processing to obtain fourth feature information, and a second loss value is calculated based on the first feature information, the fourth feature information, and the second loss function; a target loss value is determined based on the first loss value and the second loss value; if the target loss value does not meet the preset termination condition, the preset student neural network is updated according to the target loss value, and the training sample set is input into the preset student neural network model for processing; if the target loss value meets the preset termination condition, the trained student neural network model is output. In the present application, the target loss value is determined by the first loss value obtained by the preset student neural network model and the second loss value obtained by the preset teacher neural network model, which effectively improves the training convergence of the neural network model, enables the neural network to learn better features, and improves its recognition accuracy and generalization ability.
[0144] See also Figure 3 , Figure 3 is a schematic flow chart of an object recognition method provided in the second embodiment of the present application. In this embodiment, the execution subject of an object recognition method is a device with object recognition function, such as a desktop computer, a server, a robot, etc. Figure 1 The object recognition method shown may include:
[0145] S201: Obtain an object image.
[0146] When the device detects the image recognition instruction, it obtains the object image. The way in which the device obtains the object image is not limited here. It can be taken by a device with a camera function and then sent to the device, or the device itself has a shooting function to obtain the object image. For example, a robot equipped with a camera module can directly obtain the object image through its own camera function.
[0147] S202: Input the object image into a preset object recognition model for processing to obtain an object recognition result corresponding to the object image; wherein the object recognition model is obtained by the aforementioned training method of the neural network model.
[0148] The object recognition method provided in the present application obtains an object image; the object image is input into a preset object recognition model for processing to obtain an object recognition result corresponding to the object image; wherein the object recognition model is trained by a training method of a neural network model. The object recognition model in the present application is trained by a training method of a neural network model, so the object recognition method in the present application can better extract the features of the object image, thereby improving the accuracy of object recognition.
[0149] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.
[0150] See also Figure 4 , Figure 4 Schematic diagram of a training device for a neural network model provided in the third embodiment of the present application. The units included are used to execute Figure 1~2 For details, please refer to the steps in the corresponding embodiment. Figure 1~2 For the convenience of explanation, only the parts related to this embodiment are shown. Figure 4 , the training device 4 of the neural network model includes:
[0151] A first acquisition unit 410 is used to acquire a training sample set;
[0152] A first processing unit 420 is used to input the training sample set into a preset student neural network model for processing, obtain first feature information, and calculate a first loss value according to the first feature information and a first loss function;
[0153] A second processing unit 430 is used to input the training sample set into a preset teacher neural network model for processing to obtain fourth feature information, and calculate a second loss value according to the first feature information, the fourth feature information and the second loss function;
[0154] A first determining unit 440, configured to determine a target loss value according to the first loss value and the second loss value;
[0155] The third processing unit 450 is used to update the preset student neural network according to the target loss value if the target loss value does not meet the preset termination condition, and return to execute inputting the training sample set into the preset student neural network model for processing;
[0156] The fourth processing unit 460 is used to output the trained student neural network model if the target loss value meets the preset termination condition.
[0157] Further, the preset student neural network model includes a first feature extraction structure, a second feature extraction structure and a third feature extraction structure;
[0158] The first processing unit 420 is specifically configured to:
[0159] Input the training sample set into the first feature extraction structure for processing to obtain second feature information, and calculate the first auxiliary loss value according to the second feature information;
[0160] Inputting the second feature information into a second feature extraction structure for processing to obtain third feature information, and calculating a second auxiliary loss value based on the third feature information;
[0161] The third feature information is input into the third feature extraction structure for processing to obtain the first feature information, and the third auxiliary loss value is calculated based on the first feature information.
[0162] Further, the first feature extraction structure includes a convolutional layer;
[0163] The first processing unit 420 is further configured to:
[0164] Input the training sample set into the convolution layer for feature extraction to obtain the second feature information;
[0165] The first auxiliary loss value is calculated according to the second characteristic information and preset ideal data.
[0166] Furthermore, the first feature extraction structure also includes a fully connected layer and an output layer; the activation function of the output layer is a soft-max function;
[0167] The first processing unit 420 is further configured to:
[0168] Input the second feature information into the fully connected layer for processing to obtain a feature vector;
[0169] The feature vector is input into the output layer, and the probability that the category of the training sample in the training sample set belongs to the target category corresponding to the preset ideal data is determined according to the preset ideal data;
[0170] The first auxiliary loss value is calculated based on the probability and the preset cross entropy loss function.
[0171] Furthermore, the preset teacher neural network model includes a Resnet152 network.
[0172] Further, the second processing unit 430 is specifically configured to:
[0173] Calculate the square difference between the first feature information and the fourth feature information to obtain a second loss value;
[0174] or,
[0175] Calculate the mean square error of the first feature information and the fourth feature information to obtain a second loss value;
[0176] or,
[0177] The absolute values of the first characteristic information and the fourth characteristic information are calculated to obtain a second loss value.
[0178] Further, the first determining unit 440 is specifically configured to:
[0179] The first auxiliary loss value, the second auxiliary loss value, the third auxiliary loss value and the second loss value are weightedly summed to obtain a target loss value.
[0180] See also Figure 5 , Figure 5 is a schematic diagram of an object recognition device provided in the fourth embodiment of the present application. The units included are used to execute Figure 3 For details, please refer to the steps in the corresponding embodiment. Figure 3 For the convenience of explanation, only the parts related to this embodiment are shown. Figure 5 , the object recognition device 5 comprises:
[0181] An acquisition unit 510 is used to acquire an object image;
[0182] The processing unit 520 is used to input the object image into a preset object recognition model for processing to obtain an object recognition result corresponding to the object image; wherein the object recognition model is obtained by the training method of the aforementioned neural network model.
[0183] Figure 6 Schematic diagram of a training device for a neural network model provided in the fifth embodiment of the present application. Figure 6 As shown, the training device 6 of the neural network model of this embodiment includes: a processor 60, a memory 61, and a computer program 62 stored in the memory 61 and executable on the processor 60, such as a training program for the neural network model. When the processor 60 executes the computer program 62, the steps in the training method embodiments of the neural network model described above are implemented, such as Figure 1 Alternatively, when the processor 60 executes the computer program 62, the functions of each module / unit in the above-mentioned device embodiments are realized, for example, Figure 4 Functions of modules 410 to 460 are shown.
[0184] Exemplarily, the computer program 62 may be divided into one or more modules / units, which are stored in the memory 61 and executed by the processor 60 to complete the present application. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program 62 in the training device 6 of the neural network model. For example, the computer program 62 may be divided into a first acquisition unit, a first processing unit, a second processing unit, a first determination unit, a third processing unit, and a fourth processing unit, and the specific functions of each unit are as follows:
[0185] A first acquisition unit, used to acquire a training sample set;
[0186] A first processing unit is used to input the training sample set into a preset student neural network model for processing, obtain first feature information, and calculate a first loss value according to the first feature information and a first loss function;
[0187] A second processing unit is used to input the training sample set into a preset teacher neural network model for processing to obtain fourth feature information, and calculate a second loss value according to the first feature information, the fourth feature information and the second loss function;
[0188] A first determining unit, configured to determine a target loss value according to the first loss value and the second loss value;
[0189] A third processing unit is used for updating the preset student neural network according to the target loss value if the target loss value does not meet the preset termination condition, and returning to execute inputting the training sample set into the preset student neural network model for processing;
[0190] The fourth processing unit is used to output the trained student neural network model if the target loss value meets the preset termination condition.
[0191] The training device of the neural network model may include, but is not limited to, a processor 60 and a memory 61. Those skilled in the art will appreciate that Figure 6 It is only an example of the training device 6 for the neural network model and does not constitute a limitation of the training device 6 for the neural network model. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the training device for the neural network model may also include input and output devices, network access devices, buses, etc.
[0192] The processor 60 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.
[0193] The memory 61 may be an internal storage unit of the training device 6 of the neural network model, such as a hard disk or memory of the training device 6 of the neural network model. The memory 61 may also be an external storage device of the training device 6 of the neural network model, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the training device 6 of the neural network model. Furthermore, the training device 6 of the neural network model may also include both an internal storage unit and an external storage device of the training device 6 of the neural network model. The memory 61 is used to store the computer program and other programs and data required by the training device of the neural network model. The memory 61 may also be used to temporarily store data that has been output or is to be output.
[0194] Figure 7 Schematic diagram of an object recognition device provided in the sixth embodiment of the present application. Figure 7 As shown, the object recognition device 7 of this embodiment includes: a processor 70, a memory 71, and a computer program 72 stored in the memory 71 and executable on the processor 70, such as an object recognition program. When the processor 70 executes the computer program 72, the steps in the above-mentioned various fault handling method embodiments are implemented, such as Figure 3 Alternatively, when the processor 70 executes the computer program 72, the functions of each module / unit in the above-mentioned device embodiments are realized, for example Figure 5 Functions of modules 510 to 520 are shown.
[0195] Exemplarily, the computer program 72 may be divided into one or more modules / units, which are stored in the memory 71 and executed by the processor 70 to complete the present application. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program 72 in the object recognition device 7. For example, the computer program 72 may be divided into an acquisition unit and a processing unit, and the specific functions of each unit are as follows:
[0196] An acquisition unit, used for acquiring an object image;
[0197] The processing unit is used to input the object image into a preset object recognition model for processing to obtain an object recognition result corresponding to the object image; wherein the object recognition model is obtained by the training method of the aforementioned neural network model.
[0198] The object recognition device may include, but is not limited to, a processor 70 and a memory 71. Those skilled in the art will appreciate that Figure 7 It is only an example of the object recognition device 7 and does not constitute a limitation of the object recognition device 7. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the object recognition device may also include input and output devices, network access devices, buses, etc.
[0199] The processor 70 may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.
[0200] The memory 71 may be an internal storage unit of the object recognition device 7, such as a hard disk or memory of the object recognition device 7. The memory 71 may also be an external storage device of the object recognition device 7, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the object recognition device 7. Furthermore, the object recognition device 7 may also include both an internal storage unit and an external storage device of the object recognition device 7. The memory 71 is used to store the computer program and other programs and data required by the object recognition device. The memory 71 may also be used to temporarily store data that has been output or is to be output.
[0201] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.
[0202] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.
[0203] An embodiment of the present application also provides a timing device of a virtual timer, which includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, wherein the processor implements the steps of any of the above-mentioned method embodiments when executing the computer program.
[0204] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.
[0205] An embodiment of the present application provides a computer program product. When the computer program product runs on a mobile terminal, the mobile terminal can implement the steps in the above-mentioned method embodiments when executing the computer program product.
[0206] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium can at least include: any entity or device that can carry the computer program code to the camera / terminal device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.
[0207] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0208] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0209] In the embodiments provided in the present application, it should be understood that the disclosed devices / equipment and methods can be implemented in other ways. For example, the device / equipment embodiments described above are merely schematic, for example, the division of the modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0210] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0211] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.
Claims
1. A training method for a neural network model, characterized in that: include: Acquire a training sample set, wherein the training sample includes an image; Inputting the training sample set into a preset student neural network model for processing to obtain first feature information, and calculating a first loss value according to the first feature information and a first loss function; The training sample set is input into a preset teacher neural network model for processing to obtain fourth feature information, and a second loss value is calculated according to the first feature information, the fourth feature information and the second loss function; the preset student neural network model learns the behavior of the preset teacher neural network model; Determine a target loss value according to the first loss value and the second loss value; If the target loss value does not meet the preset termination condition, the preset student neural network is updated according to the target loss value, and the training sample set is input into the preset student neural network model for processing; If the target loss value meets the preset termination condition, the trained student neural network model is output; Wherein, the preset student neural network model is composed of a first feature extraction structure, a second feature extraction structure and a third feature extraction structure connected in series; The step of inputting the training sample set into a preset student neural network model for processing to obtain first feature information, and calculating a first loss value according to the first feature information and a first loss function includes: Inputting the training sample set into the first feature extraction structure for processing to obtain second feature information, and calculating a first auxiliary loss value according to the second feature information; Inputting the second feature information into the second feature extraction structure for processing to obtain third feature information, and calculating a second auxiliary loss value according to the third feature information; Inputting the third feature information into the third feature extraction structure for processing to obtain the first feature information, and calculating the third auxiliary loss value according to the first feature information; The first feature extraction structure includes a convolutional layer; the number of convolutional layers of the second feature extraction structure is greater than the number of convolutional layers of the first feature extraction structure; The step of inputting the training sample set into the first feature extraction structure for processing to obtain second feature information, and calculating a first auxiliary loss value according to the second feature information includes: Inputting the training sample set into the convolution layer to extract features to obtain second feature information; Calculating a first auxiliary loss value according to the second characteristic information and preset ideal data; The first feature extraction structure also includes a fully connected layer and an output layer; the activation function of the output layer is a soft-max function; The calculating the first auxiliary loss value according to the second characteristic information and preset ideal data includes: Inputting the second feature information into the fully connected layer for processing to obtain a feature vector; Inputting the feature vector into the output layer, and determining, according to preset ideal data, the probability that the category of the training sample in the training sample set belongs to the target category corresponding to the preset ideal data; The first auxiliary loss value is calculated according to the probability and the preset cross entropy loss function. The specific formula is as follows: Where N is the number of categories, y ik Characterizes the indicator variable. If the category of the training sample is the same as the category in the preset ideal data, it is 1, otherwise it is 0; p ik It indicates the predicted probability that the category belongs to a category K in the preset ideal data.
2. The training method of the neural network model according to claim 1, characterized in that: The preset teacher neural network model includes a Resnet152 network.
3. The training method of the neural network model according to claim 1, characterized in that: The calculating the second loss value according to the first feature information, the fourth feature information and the second loss function includes: Calculate the square difference between the first feature information and the fourth feature information to obtain a second loss value; or, Calculate the mean square error of the first feature information and the fourth feature information to obtain a second loss value; or, The first feature information and the fourth feature information are used to calculate absolute values to obtain a second loss value.
4. The training method of the neural network model according to claim 1, characterized in that: The determining a target loss value according to the first loss value and the second loss value comprises: The first auxiliary loss value, the second auxiliary loss value, the third auxiliary loss value and the second loss value are weightedly summed to obtain a target loss value.
5. An object recognition method, characterized in that: include: Get the object image; The object image is input into a preset object recognition model for processing to obtain an object recognition result corresponding to the object image; wherein the object recognition model is obtained by the training method of the neural network model described in any one of claims 1 to 4.
6. A training device for a neural network model, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the training method of the neural network model as described in any one of claims 1 to 4 is implemented.
7. A device for object recognition, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method for object recognition according to claim 5 is implemented.