Image processing model training method and device, electronic equipment and computer readable storage medium
By smoothing the eigenvalue and updating the loss function, the dead bit problem in model training is solved, and efficient training of the image processing model is achieved.
Patent Information
- Application Number
- CN202411667633.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-19
- Publication Date
- 2025-03-18
AI Technical Summary
During the model training process, the eigenvalue is in the saturated area of the activation function, resulting in the gradient value being 0 or close to 0, which makes the model difficult to train or converge, resulting in dead bit problems.
By smoothly quantizing the eigenvalues, a class hash code is obtained, and the loss function is determined based on the class hash code and eigenvalue. The loss function is derived to obtain the gradient value. When the positive and negative signs of the eigenvalue and the gradient value are inconsistent, the gradient value is updated and the image processing model is trained based on the updated gradient value.
By updating the gradient value to keep it away from 0, avoiding the dead bit problem, and ensuring the training efficiency and training effect of the image processing model.
Smart Images
Figure CN119649163A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the field of image processing, and specifically to an image processing model training method, device, electronic device and computer-readable storage medium. Background Art
[0002] The dead bit problem refers to the situation in which, during the model training process, the eigenvalue is in the saturation region of the activation function and the gradient value corresponding to the eigenvalue is 0 or close to 0, which causes the eigenvalue to fail to be optimized during the back-propagation training of the model, making the model difficult to train or even unable to converge. Summary of the invention
[0003] The embodiments of the present application provide an image processing model training method, device, electronic device and computer-readable storage medium, which can solve the dead bit problem in the model training process.
[0004] In a first aspect, an embodiment of the present application provides an image processing model training method, comprising:
[0005] Inputting the image into the image processing model to obtain the feature value of the image;
[0006] Smoothing and quantizing the eigenvalues to obtain a quasi-hash code of the image;
[0007] Determining a loss function based on the class hash code and the feature value of the image;
[0008] Derivative the loss function to obtain a gradient value;
[0009] When the signs of the characteristic value and the gradient value are inconsistent, updating the gradient value; and
[0010] The image processing model is trained based on the updated gradient value to obtain a trained image processing model.
[0011] In one embodiment, the method further comprises:
[0012] When the eigenvalue and the gradient value have the same sign, the image processing model is trained based on the gradient value to obtain a trained image processing model.
[0013] In one embodiment, updating the gradient value includes:
[0014] The characteristic value is used as an independent variable of a quadratic function, and the function value of the quadratic function is determined as an updated gradient value; wherein the quadratic function is a monotonically decreasing function in the first quadrant and a monotonically increasing function in the second quadrant.
[0015] In one embodiment, the step of smoothing and quantizing the characteristic value to obtain a quasi-hash code of the image includes:
[0016] The eigenvalues are smoothly quantized using a smoothing quantization function to obtain a quasi-hash code of the image; the smoothing quantization function includes but is not limited to: a sigmoid activation function or a tanh activation function.
[0017] In one embodiment, determining the loss function based on the class hash code and the feature value of the image includes:
[0018] Obtaining a true hash code for said characteristic value;
[0019] Determining a quantization loss function based on a difference between a true hash code and a class hash code of the feature value;
[0020] Get the feature values of other images;
[0021] Determining a similarity loss function according to the distance between the feature value of the image and the feature value of the other image;
[0022] The sum of the quantization loss function and the similarity loss function is determined as the loss function.
[0023] In one of the embodiments, the feature value of the image is obtained by the image processing model after convolution and pooling the image through a fully connected layer.
[0024] In one embodiment, the trained image processing model is used to perform image search based on a hash code of the image.
[0025] In a second aspect, an embodiment of the present application provides an image processing model training device, comprising:
[0026] An input module, used for inputting an image into an image processing model to obtain a feature value of the image;
[0027] A quantization module, used for performing smooth quantization on the eigenvalue to obtain a quasi-hash code of the image;
[0028] A determination module, configured to determine a loss function based on a class hash code and a feature value of the image;
[0029] A derivation module, used to derive the loss function to obtain a gradient value;
[0030] An updating module, used for updating the gradient value when the signs of the eigenvalue and the gradient value are inconsistent;
[0031] The training module is used to train the image processing model based on the updated gradient value to obtain a trained image processing model.
[0032] In one embodiment, the device further comprises:
[0033] The model training module is used to train the image processing model based on the gradient value when the positive and negative signs of the characteristic value and the gradient value are consistent, so as to obtain a trained image processing model.
[0034] In one embodiment, the update module includes:
[0035] An updating unit is used to use the characteristic value as an independent variable of a quadratic function and determine the function value of the quadratic function as an updated gradient value; wherein the quadratic function is a monotonically decreasing function in the first quadrant and a monotonically increasing function in the second quadrant.
[0036] In one embodiment, the quantization module includes:
[0037] The quantization unit is used to use a smooth quantization function to smoothly quantize the eigenvalue to obtain a quasi-hash code of the image; the smooth quantization function includes but is not limited to: a sigmoid activation function or a tanh activation function.
[0038] In one embodiment, the determining module includes:
[0039] A real hash code obtaining unit, used to obtain the real hash code of the feature value;
[0040] A first function determination unit, configured to determine a quantization loss function according to a difference between a true hash code and a class hash code of the feature value;
[0041] A feature value acquisition unit, used for acquiring feature values of other images;
[0042] A second function determination unit, configured to determine a similarity loss function according to a distance between a feature value of the image and a feature value of the other image;
[0043] The third function determination unit is used to determine the sum of the quantization loss function and the similarity loss function as the loss function.
[0044] In one of the embodiments, the feature value of the image is obtained by the image processing model after convolution and pooling the image through a fully connected layer.
[0045] In one embodiment, the trained image processing model is used to perform image search based on a hash code of the image.
[0046] In a third aspect, an embodiment of the present application further provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and which implements the steps in the above-mentioned image processing model training method when the computer program is executed by the processor.
[0047] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps in the above-mentioned image processing model training method are implemented.
[0048] In a fifth aspect, the embodiments of the present application further provide a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the method provided in various optional implementations described in the embodiments of the present application.
[0049] In summary, in the embodiment of the present application, the image processing model can obtain the eigenvalue of the image, and the image class hash code can be obtained by smoothing and quantizing the eigenvalue of the image. The loss function can be determined according to the class hash code and the eigenvalue of the image, and the gradient value can be obtained by deriving the loss function. When the positive and negative signs of the eigenvalue and the gradient value are inconsistent, the characterization gradient is optimized in the opposite direction, and there may be a dead bit problem. Therefore, when the positive and negative signs of the eigenvalue and the gradient value are inconsistent, the gradient value is updated, and the image processing model is trained based on the updated gradient value to obtain a trained image processing model. The updated gradient value can make the gradient value far away from 0, thereby avoiding the dead bit problem and ensuring the training efficiency and training effect of the image processing model. BRIEF DESCRIPTION OF THE DRAWINGS
[0050] In order to more clearly illustrate the technical solutions in the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.
[0051] Figure 1 is a schematic diagram of the workflow of an image search model provided in one embodiment of the present application;
[0052] Figure 2 is a schematic diagram of the gradient value corresponding to the sgn step function provided in an embodiment of the present application;
[0053] Figure 3 It is a schematic diagram of the gradient value corresponding to the tanh activation function provided in an embodiment of the present application;
[0054] Figure 4 It is a schematic diagram of the steps of the image processing model training method provided in one embodiment of the present application;
[0055] Figure 5 It is a gradient value curve diagram of two optimization directions provided in an embodiment of the present application;
[0056] Figure 6 It is a schematic diagram of the steps of a method for determining a loss function provided in an embodiment of the present application;
[0057] Figure 7 It is a schematic diagram of the workflow of the image processing model training method provided in one embodiment of the present application;
[0058] Figure 8 is a derivative curve graph of the loss function provided in an embodiment of the present application;
[0059] Fig. 9 It is a structural schematic diagram of an image processing model training device provided in one embodiment of the present application;
[0060] Fig.10 It is a schematic diagram of the structure of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION
[0061] The following will be combined with the drawings in this application to clearly and completely describe the technical solutions in this application. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments in the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.
[0062] The deep hash image processing model is an image processing model that processes an image according to the hash code of the image. The image processing model can be an image search model, an image clustering model, or an image deduplication model, etc. The image processing model can obtain the eigenvalue of the image, and then quantize the eigenvalue of the image using the sgn step function (sign function) to obtain the binary hash code of the image.
[0063] Figure 1It is a schematic diagram of the workflow of the image search model shown in an embodiment of the present application. The image search model is a deep hash model. The image search model performs convolution and pooling on the query image, and then passes through the fully connected layer to obtain the eigenvalue of the query image; the eigenvalue of the query image is quantized using the sgn step function to obtain the hash code of the query image. Based on a similar process, the hash code of each image in the database can be obtained. The hash codes of each image constitute a hash database. The Hamming distance between the hash code of the query image and each hash code in the hash database is calculated, so that the image corresponding to the hash code whose Hamming distance is less than the distance threshold can be determined as the query result image.
[0064] However, if Figure 2 As shown in the figure, the gradient value corresponding to the SGN step function is 0, which makes the image processing model unable to be back-propagated and optimized during the training process. Therefore, in order to approximate the eigenvalue quantization function of the SGN step function, a smooth quantization function is used to replace the SGN step function to quantize the eigenvalues during the training of the image processing model, and the image hash code is obtained.
[0065] The smoothing quantization function can be a sigmoid activation function or a tanh activation function. The sigmoid activation function and the tanh activation function are similar in nature, and the following uses the tanh activation function as an example for explanation. Figure 3 The gradient value obtained by quantizing the eigenvalue by the tanh activation function is shown. Figure 3 It can be seen that when the eigenvalue is in the saturation region of the tanh activation function, the function gradient value corresponding to the eigenvalue is close to 0, which may cause the eigenvalue to fail to be optimized during the back-propagation training of the image processing model, and eventually lead to the dead bit problem.
[0066] In order to solve the above-mentioned dead bit problem, an embodiment of the present application proposes an image processing model training method, which modifies the gradient value corresponding to the feature value that may cause the dead bit problem, thereby ensuring the smooth training of the image processing model.
[0067] In one embodiment, the image processing model training method can be applied to a terminal, wherein the terminal may include but is not limited to one or more of a smart phone, a tablet computer, a portable computer, a desktop computer, and a vehicle-mounted computer.
[0068] In another embodiment, the image processing model training method can be applied to a server, wherein the server can be a physical server or a cloud server that provides various cloud services. It is worth noting that the present application does not limit the number of terminals or servers. According to the implementation requirements, there can be any number of terminals or servers. For example, the server can be a single server or a server cluster composed of multiple servers.
[0069] It should be noted that the order of description of the following embodiments is not intended to limit the priority order of the embodiments.
[0070] according to Figure 4 The image processing model training method shown in the figure includes at least steps S110 to S160, which are described in detail as follows:
[0071] In step S110, the image is input into an image processing model to obtain a feature value of the image.
[0072] In step S120, the eigenvalues are smoothed and quantized to obtain a class hash code of the image.
[0073] The feature value of the image is used to characterize the features of the image. The feature value of the image may be obtained by the image processing model after convolution and pooling the image through a fully connected layer.
[0074] The image processing model may include a feature value extraction module and a quantization module. The feature value extraction module may include a convolution layer, a pooling layer, and a fully connected layer. The image is input into the image processing model. The feature value extraction module may extract the feature value of the image. The quantization module quantizes the feature value of the image using an activation function to obtain a hash code of the image.
[0075] The hash code of an image is a string of binary codes that represent the image; the hash code of the image can be obtained by extracting the eigenvalues of the image and mapping the eigenvalues of the image. The hash code of the image has the advantages of small data volume and fast retrieval and comparison.
[0076] The hash code of an image is obtained by discretely quantizing the eigenvalues of the image using the sgn step function, and the quasi-hash code of an image is obtained by smoothing and quantizing the eigenvalues of the image using a smoothing quantization function, and then mapping the smoothing quantization result to a discrete value.
[0077] In one embodiment, step S120 may include: using a smooth quantization function to smoothly quantize the feature value to obtain a quasi-hash code of the image; the smooth quantization function includes but is not limited to: a sigmoid activation function or a tanh activation function.
[0078] By using the sigmoid activation function or the tanh activation function to quantize the eigenvalues, the eigenvalues can be quantized to the range of (-1, +1), and then the class hash code corresponding to the eigenvalues in the (-1, 0) region can be determined as 0, and the class hash code corresponding to the eigenvalues in the (0, +1) region can be determined as 1. In this way, the class hash code of the image can be obtained by smoothing the quantization function.
[0079] In step S130 , a loss function is determined based on the class hash code and feature value of the image.
[0080] According to the image class hash code and the image real hash code, the quantization loss function can be determined. The quantization loss function is used to characterize the difference between quantizing the eigenvalues using the sgn step function and quantizing the eigenvalues using the smooth quantization function.
[0081] According to the distance between the feature values of each image, a similarity loss function can be determined. The similarity loss function is used to characterize the similarity or dissimilarity between two input samples.
[0082] The sum of the quantization loss function and the similarity loss function is determined as the loss function.
[0083] In step S140, the loss function is differentiated to obtain a gradient value.
[0084] The gradient is a vector that points to the direction in which the loss function increases or decreases fastest. The gradient value is the derivative of the loss function with respect to the model parameters, so by differentiating the loss function, we can get the gradient value.
[0085] When training an image processing model using the gradient descent algorithm, the loss function can be minimized by iteratively updating the model parameters of the image processing model. During the model training process, the model is trained along the gradient direction in the direction where the loss function is smaller. Through the gradient value, the weights and biases of the image processing model can be updated to reduce the loss and improve the performance of the image processing model.
[0086] In step S150, when the signs of the characteristic value and the gradient value are inconsistent, the gradient value is updated.
[0087] The inconsistency between the positive and negative signs of the eigenvalue and the gradient value indicates that the gradient is optimized in the reverse direction, and there may be a dead bit problem at this time. Therefore, when the positive and negative signs of the eigenvalue and the gradient value are inconsistent, the gradient value is updated so that the updated gradient value is far away from 0 in the saturation region of the smooth quantization function.
[0088] As an embodiment, updating the gradient value may include: taking the eigenvalue as an independent variable of a quadratic function, and determining the function value of the quadratic function as the updated gradient value; wherein the quadratic function is a monotonically decreasing function in the first quadrant and a monotonically increasing function in the second quadrant.
[0089] When the eigenvalue and the gradient value have different signs, the eigenvalue before smoothing and quantization can be used as the independent variable of the quadratic function, and the function value of the quadratic function after substituting the eigenvalue can be determined as the updated gradient value.
[0090] In one example, the quadratic function used may be G = -(1 - f(x i ,θ) 2 )+2, where G represents the updated gradient value, f(x i ,θ) represents the eigenvalue. This quadratic function is used to update the gradient value. The derivative of the quadratic function and the tanh activation function has an intersection at the coordinate (0,1), which can distinguish the gradient values of different strengths for the eigenvalues that need to be optimized in the positive direction and the reverse direction.
[0091] In this way, the gradient value is updated through a quadratic function. When the distance between the eigenvalue and the origin is greater, the updated gradient value corresponding to the eigenvalue is further away from 0, so that the eigenvalue with the opposite sign to the gradient can be quickly optimized to the correct direction.
[0092] In step S160, the image processing model is trained based on the updated gradient value to obtain a trained image processing model.
[0093] The image processing model is trained based on the updated gradient value. Since the updated gradient value is far away from 0 in the saturation region of the smooth quantization function, the eigenvalue can be optimized when the updated gradient value is back-propagated.
[0094] By adopting the technical solution of the embodiment of the present application, the image processing model can obtain the eigenvalue of the image, and the image class hash code can be obtained by smoothing and quantizing the eigenvalue of the image. The loss function can be determined according to the class hash code and the eigenvalue of the image, and the gradient value can be obtained by deriving the loss function. When the positive and negative signs of the eigenvalue and the gradient value are inconsistent, the characterization gradient is optimized in the opposite direction, and there may be a dead bit problem. Therefore, when the positive and negative signs of the eigenvalue and the gradient value are inconsistent, the gradient value is updated, and the image processing model is trained based on the updated gradient value to obtain a trained image processing model. The updated gradient value can make the gradient value far away from 0, thereby avoiding the dead bit problem and ensuring the training efficiency and training effect of the image processing model.
[0095] Based on the above technical solution, as an embodiment, when the positive and negative signs of the characteristic value and the gradient value are consistent, the image processing model is trained based on the gradient value to obtain a trained image processing model.
[0096] The consistent positive and negative signs of the eigenvalue and the gradient value indicate that the gradient is optimized in the positive direction, and there is no dead bit problem. Therefore, there is no need to update the gradient value. Instead, the gradient value can be directly used to train the image processing model to obtain a trained image processing model.
[0097] The optimization direction where the positive and negative signs of the eigenvalue and the gradient value are consistent is determined as the positive optimization direction; the optimization direction where the positive and negative signs of the eigenvalue and the gradient value are inconsistent is determined as the negative optimization direction. It can be understood that the loss function is the sum of the quantization loss function and the similarity loss function, and the dead bit problem is generally caused by the quantization loss function. Figure 5 The gradient value curves of two optimization directions are shown, which are the gradient value curves corresponding to the quantization loss function. In the positive optimization direction, the gradient value corresponding to the quantization loss function can be directly the derivative of the smooth quantization function, and in the reverse optimization direction, the gradient value corresponding to the quantization loss function is a quadratic function value.
[0098] When the eigenvalue itself is in the positive optimization direction, using the quadratic function as the gradient value may lead to optimization in the wrong direction. Therefore, in the embodiment of the present application, the eigenvalue in the positive optimization direction still maintains the original correct gradient value. When the eigenvalue is in the reverse optimization direction, the quadratic function value is used as the updated gradient value, and the updated gradient value is used for model training, so that the eigenvalue in the reverse optimization direction can be quickly optimized to the correct area.
[0099] By adopting the technical solution of the embodiment of the present application, the optimization direction of the eigenvalue is distinguished, and different gradient values are selected for the eigenvalues in different optimization directions based on different optimization strengths. The original correct gradient value is maintained for the eigenvalue in the positive optimization direction, and the quadratic function value is selected as the updated gradient value for the eigenvalue in the reverse optimization direction, so that the eigenvalue with the correct direction is gradually optimized to a more correct area, while the eigenvalue with the wrong direction is quickly optimized to the correct area.
[0100] Based on the above technical solution, as an embodiment, Figure 6 As shown, step S130 may include steps S131 to S135.
[0101] In step S131, the real hash code of the feature value is obtained.
[0102] The real hash code of the eigenvalue is the hash code obtained by quantizing the eigenvalue using the sgn step function. The eigenvalue is obtained by quantizing the eigenvalue using the hash code-like smoothing quantization function.
[0103] In step S132, a quantization loss function is determined according to the difference between the true hash code and the class hash code of the feature value.
[0104] The difference between the true hash code of the eigenvalue and the quasi-hash code of the eigenvalue represents the difference between quantization by the sgn step function and quantization by the smooth quantization function. According to the difference between the true hash code of the eigenvalue and the quasi-hash code of the eigenvalue, the quantization loss function can be determined.
[0105] In step S133, feature values of other images are obtained.
[0106] In the training process of the image processing model, a large number of images are usually input into the image processing model, and the image processing model can obtain the feature values of each image. Therefore, the feature values of other images for each image can be obtained.
[0107] In step S134, a similarity loss function is determined according to the distance between the feature value of the image and the feature value of the other image.
[0108] The similarity loss function can measure the similarity between the feature values of the same or different classes of images. The similarity between features can be characterized by the distance between the feature values. The similarity loss function can be determined by the distance between the feature values of an image and the feature values of other images.
[0109] In step S135 , the sum of the quantization loss function and the similarity loss function is determined as the loss function.
[0110] The sum of the quantization loss function and the similarity loss function is determined as the loss function. The image processing model is trained based on the loss function. The trained image processing model obtains similar feature values for images of the same category, dissimilar feature values for images of different categories, and can correctly quantize the feature values of the image to obtain an accurate hash code.
[0111] On the basis of the above technical solution, as an embodiment, the trained image processing model is an image search model, and the trained image processing model is used for: performing image search based on a hash code of the image.
[0112] Because hash codes can be calculated quickly, image search based on image hash codes has the advantages of being fast and efficient. And because hash codes are robust to local changes in images, images that are slightly different from the query image can be searched based on hash codes.
[0113] Based on the above technical solution, as an embodiment, Figure 7 As shown, the training method of the image processing model may include the following steps.
[0114] Input multiple images into the image processing model to obtain the image feature values. Assume Represents a set of n images, n is a positive integer, x i The pixel matrix representing the i-th image. The mapping of the image processing model is defined as f(x i ,θ).
[0115] Input image x i into the image processing model f, and obtain a low-dimensional feature vector through the tanh function Where k represents The kth eigenvalue. It is a continuous eigenvalue, which needs to be converted into a binary hash code through the sgn step function, that is, b i =h(u i )=sgn(u i ), where b i The hash code representing the i-th image, u i represents the eigenvalue of the i-th image, h represents the hash function, and sgn represents the sgn step function. Finally, the hash code set In addition, the similarity set of paired images is represented by S = {s ij} means that when s ij = 1, it means that the i-th image and the j-th image have the same category, that is, the i-th image and the j-th image are similar, otherwise s ij = 0, they are not similar. The inner product of the image pair is given by Represented as, where T represents the matrix device, and the inner product of image pairs is used to measure the similarity of paired images.
[0116] For simplicity, the formula for defining the loss function L is:
[0117]
[0118] Among them, L similar Characterize the similarity loss function, L quantification Characterize the quantization loss function, K is the hash code length, λ is the hyperparameter, u j Characterize the feature value of the jth image. The meaning of the remaining characters can be found in the previous text.
[0119] By derivatizing the loss function, we can obtain:
[0120]
[0121] in, Represents the derivative, and the meaning of the other characters can be referred to in the previous text.
[0122] Assumptions ij =1,λ=4, When s ij =1, at this time Need to be optimized In order to obtain the correct hash code during quantization. The derivative curve of the loss function L is as follows Figure 8 As shown, it can be found that when the eigenvalue output by the image processing model When the gradient is close to -1, it is close to 0. At this time, the back propagation of the model may fail, resulting in errors. Unable to The direction of the value is optimized, resulting in the problem of dead bits that cannot be optimized.
[0123] In order to solve the dead bit problem, the eigenvalues that need to be optimized in the opposite direction are counted, and the positive and negative signs of the eigenvalues and gradient values can be compared to determine whether they are consistent. The following formula can be used to determine:
[0124]
[0125] in, Characterization The corresponding gradient value. The positive and negative signs of the eigenvalue and the gradient value are consistent. Characterizes that the eigenvalue and the gradient value have inconsistent signs. When the eigenvalue and the gradient value have the same sign, the eigenvalue is still optimized in the direction of the current gradient; when the eigenvalue and the gradient value have inconsistent signs, the eigenvalue needs to be optimized in the opposite direction of the gradient.
[0126] When the eigenvalue is still optimized in the direction of the current gradient, the gradient value originally corresponding to the eigenvalue is still used as the gradient value of the back propagation. When the eigenvalue needs to be optimized in the opposite direction of the gradient and is in the saturation region of the smooth quantization function, the eigenvalue will be difficult to optimize in the gradient back propagation. Therefore, in one embodiment of the present application, the quadratic function value is used as the gradient value of the eigenvalue optimized in the reverse direction, so that the gradient value does not tend to 0 during back propagation. The gradient value of back propagation can be expressed by the following formula:
[0127]
[0128] The characteristic value is optimized in the current direction, and the original gradient value is used directly. As the gradient value of back propagation; The characteristic value is optimized in the opposite direction of the gradient. At this time, the gradient value is updated and -(1-f(x i ,θ) 2 )+2 is used as the gradient value for back propagation.
[0129] In the reverse optimization process, when f(x i When the value of ,θ) is smaller or larger, the quadratic function can exponentially enhance the gradient value of the eigenvalue, so that the eigenvalue can be quickly optimized to the correct hash code after back propagation.
[0130] By adopting the technical solution of the embodiment of the present application, different quantization gradients are given to the eigenvalues optimized in the positive direction of the gradient and the eigenvalues optimized in the reverse direction, so as to avoid the phenomenon that the dead bit gradient tends to 0 and cannot be optimized. At the same time, a quadratic function is used to provide an exponentially weighted gradient value for the eigenvalues that need to be optimized in the reverse direction, which can avoid the trouble of manual parameter adjustment.
[0131] Through experiments, it is found that the average retrieval accuracy of the model trained by the image processing model training method proposed in the embodiment of the present application is significantly higher than that of other models of the related art. The model trained by the image processing model training method proposed in the embodiment of the present application significantly reduces the number of dead bit problems in the first 20 iterations, and the number is always less than 100. When the image processing model training method proposed in the embodiment of the present application is not adopted, as the model converges, the number of dead bit problems gradually increases, resulting in the model being unable to further optimize these "dead bits", thereby reducing the accuracy of the image processing model. Moreover, after about 100 back-propagation optimizations, the number of eigenvalues that need to be optimized in the reverse direction generated by the image processing model training method proposed in the embodiment of the present application is lower than the number of image processing model training methods proposed in the embodiment of the present application. In summary, the image processing model training method proposed in the embodiment of the present application can well avoid the dead bit problem and reduce the number of eigenvalues that need to be optimized in reverse during model optimization, thereby improving the performance and accuracy of the image processing model.
[0132] In order to better implement the image processing model training method of the present application, the present application also provides an image processing model training device based on the above image processing model training method. The meanings of the terms are the same as those in the above image processing model training method, and the specific implementation details can refer to the description in the method embodiment.
[0133] See also Fig. 9 , Fig. 9 : is a structural diagram of an image processing model training device provided in an embodiment of the present application, and the image processing model training device comprises:
[0134] An input module 901 is used to input an image into an image processing model to obtain a feature value of the image;
[0135] A quantization module 902, configured to perform smoothing quantization on the eigenvalues to obtain a quasi-hash code of the image;
[0136] A determination module 903, configured to determine a loss function based on the class hash code and the feature value of the image;
[0137] A derivation module 904 is used to derive the loss function to obtain a gradient value;
[0138] An updating module 905, configured to update the gradient value when the signs of the eigenvalue and the gradient value are inconsistent;
[0139] The training module 906 is used to train the image processing model based on the updated gradient value to obtain a trained image processing model.
[0140] In one embodiment, the device further comprises:
[0141] The model training module is used to train the image processing model based on the gradient value when the positive and negative signs of the characteristic value and the gradient value are consistent, so as to obtain a trained image processing model.
[0142] In one embodiment, the update module 905 includes:
[0143] An updating unit is used to use the characteristic value as an independent variable of a quadratic function and determine the function value of the quadratic function as an updated gradient value; wherein the quadratic function is a monotonically decreasing function in the first quadrant and a monotonically increasing function in the second quadrant.
[0144] In one embodiment, the quantization module 902 includes:
[0145] The quantization unit is used to use a smooth quantization function to smoothly quantize the eigenvalue to obtain a quasi-hash code of the image; the smooth quantization function includes but is not limited to: a sigmoid activation function or a tanh activation function.
[0146] In one embodiment, the determining module 903 includes:
[0147] A real hash code obtaining unit, used to obtain the real hash code of the feature value;
[0148] A first function determination unit, configured to determine a quantization loss function according to a difference between a true hash code and a class hash code of the feature value;
[0149] A feature value acquisition unit, used for acquiring feature values of other images;
[0150] A second function determination unit, configured to determine a similarity loss function according to a distance between a feature value of the image and a feature value of the other image;
[0151] The third function determination unit is used to determine the sum of the quantization loss function and the similarity loss function as the loss function.
[0152] In one of the embodiments, the feature value of the image is obtained by the image processing model after convolution and pooling the image through a fully connected layer.
[0153] In one embodiment, the trained image processing model is used to perform image search based on a hash code of the image.
[0154] By adopting the technical solution of the embodiment of the present application, the image processing model can obtain the eigenvalue of the image, and the image class hash code can be obtained by smoothing and quantizing the eigenvalue of the image. The loss function can be determined according to the class hash code and the eigenvalue of the image, and the gradient value can be obtained by deriving the loss function. When the positive and negative signs of the eigenvalue and the gradient value are inconsistent, the characterization gradient is optimized in the opposite direction, and there may be a dead bit problem. Therefore, when the positive and negative signs of the eigenvalue and the gradient value are inconsistent, the gradient value is updated, and the image processing model is trained based on the updated gradient value to obtain a trained image processing model. The updated gradient value can make the gradient value far away from 0, thereby avoiding the dead bit problem and ensuring the training efficiency and training effect of the image processing model.
[0155] For the specific definition of the image processing model training device, please refer to the definition of the image processing model training method above, which will not be repeated here. Each module in the above-mentioned image processing model training device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.
[0156] In addition, the present application also provides an electronic device, such as Fig.10 As shown, it shows a schematic diagram of the structure of the electronic device involved in this application, specifically:
[0157] The electronic device may include one or more processors 101 of processing cores and one or more computer-readable storage media memories 102 and other components. Those skilled in the art will appreciate that Fig.10 The electronic device structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.
[0158] The processor 101 is the control center of the electronic device. It uses various interfaces and lines to connect various parts of the entire electronic device. By running or executing software programs and / or modules stored in the memory 102, and calling data stored in the memory 102, it performs various functions of the electronic device and processes data, thereby monitoring the electronic device as a whole. Optionally, the processor 101 may include one or more processing cores; preferably, the processor 101 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 101.
[0159] The memory 102 can be used to store software programs and modules. The processor 101 executes various functional applications and data processing by running the software programs and modules stored in the memory 102. The memory 102 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 102 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 102 may also include a memory controller to provide the processor 101 with access to the memory 102.
[0160] In one embodiment, the electronic device further includes a power supply 103 for supplying power to various components. Preferably, the power supply 103 can be logically connected to the processor 101 through a power management system, so as to manage charging, discharging, power consumption and other functions through the power management system. The power supply 103 can also include any components such as one or more DC or AC power supplies, recharging systems, power supply device debugging circuits, power converters or inverters, and power status indicators.
[0161] In one embodiment, the electronic device may further include an input unit 104, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.
[0162] Although not shown, the electronic device may also include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 101 in the electronic device will load the executable file corresponding to the process of one or more application programs into the memory 102 according to the following instructions, and the processor 101 will run the application program stored in the memory 102, thereby implementing the steps in any one of the image processing model training methods provided in the embodiments of the present application.
[0163] Those skilled in the art will understand that Fig.10 The structure shown in the figure is merely a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the electronic device to which the scheme of the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.
[0164] In one embodiment, an electronic device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the method described in any embodiment of the present application is implemented.
[0165] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method described in any embodiment of the present application is implemented.
[0166] In some embodiments, a computer program product is also proposed, including a computer program or instructions, which implement the method described in any embodiment of the present application when executed by a processor.
[0167] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.
[0168] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.
[0169] To this end, the present application provides a computer-readable storage medium, on which a computer program is stored. The computer program can be loaded by a processor to execute the steps in any image processing model training method provided in the present application.
[0170] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.
[0171] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.
[0172] Since the instructions stored in the computer-readable storage medium can execute the steps in any image processing model training method provided in the present application, the beneficial effects that can be achieved by any image processing model training method provided in the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.
[0173] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.
[0174] The above is a detailed introduction to an image processing model training method, device, electronic device and computer-readable storage medium provided in the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for technical personnel in this field, according to the idea of the present invention, there will be changes in the specific implementation method and scope of application. In summary, the content of this specification should not be understood as a limitation on the present invention.
Claims
1. A method for training an image processing model, characterized in that: include: Inputting the image into the image processing model to obtain the feature value of the image; Smoothing and quantizing the eigenvalues to obtain a quasi-hash code of the image; Determining a loss function based on the class hash code and the feature value of the image; Derivative the loss function to obtain a gradient value; When the signs of the characteristic value and the gradient value are inconsistent, updating the gradient value; and The image processing model is trained based on the updated gradient value to obtain a trained image processing model.
2. The method according to claim 1, characterized in that The method further comprises: When the eigenvalue and the gradient value have the same sign, the image processing model is trained based on the gradient value to obtain a trained image processing model.
3. The method according to claim 1, characterized in that The updating of the gradient value comprises: The characteristic value is used as an independent variable of a quadratic function, and the function value of the quadratic function is determined as an updated gradient value; wherein the quadratic function is a monotonically decreasing function in the first quadrant and a monotonically increasing function in the second quadrant.
4. The method according to claim 1, characterized in that: The step of smoothing and quantizing the characteristic value to obtain a quasi-hash code of the image includes: The eigenvalues are smoothly quantized using a smoothing quantization function to obtain a quasi-hash code of the image; the smoothing quantization function includes but is not limited to: a sigmoid activation function or a tanh activation function.
5. The method according to claim 1, characterized in that: The determining of the loss function based on the class hash code and the feature value of the image comprises: Obtaining a true hash code for said characteristic value; Determining a quantization loss function based on a difference between a true hash code and a class hash code of the feature value; Get the feature values of other images; Determining a similarity loss function according to the distance between the feature value of the image and the feature value of the other image; The sum of the quantization loss function and the similarity loss function is determined as the loss function.
6. The method according to claim 1, characterized in that The characteristic value of the image is obtained by the image processing model after convolution and pooling the image through a fully connected layer.
7. The method according to any one of claims 1 to 6, characterized in that: The trained image processing model is used to perform image search based on the hash code of the image.
8. An image processing model training device, characterized in that: include: An input module, used for inputting an image into an image processing model to obtain a feature value of the image; A quantization module, used for performing smooth quantization on the eigenvalue to obtain a quasi-hash code of the image; A determination module, configured to determine a loss function based on a class hash code and a feature value of the image; A derivation module, used to derive the loss function to obtain a gradient value; An updating module, used for updating the gradient value when the signs of the eigenvalue and the gradient value are inconsistent; The training module is used to train the image processing model based on the updated gradient value to obtain a trained image processing model.
9. An electronic device, characterized in that: It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, it implements the steps in the image processing model training method as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the steps in the image processing model training method as described in any one of claims 1-7 are implemented.