Model training method, image feature extraction method, system, device and medium

Abstract

The present invention discloses a model training method, an image feature extraction method, a system, a device and a medium. The model training method comprises: obtaining a historical training sample data set, the historical training sample data set comprising historical image classification label data, the historical image classification label data corresponding to the hash features of the historical images; training a neural network model based on the historical training sample data set to obtain an image feature extraction model. The present invention obtains a historical training sample data set of historical image classification label data corresponding to the hash features of the historical images, and trains a neural network model based on the historical training sample data set to obtain an image feature extraction model, so as to extract the hash features of the images using the image feature extraction model, thereby saving the space occupied by the image features and accelerating the speed at which the image feature extraction model extracts the hash features of the images.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a model training method, an image feature extraction method, a system, a device and a medium. Background Art

[0002] In the prior art, the image depth features extracted by traditional neural networks are high-dimensional floating-point features. During the extraction process, the interference of difficult samples with noise labels on model training and class center updating is not considered; and high-dimensional floating-point features have many disadvantages in practical applications, that is, high-dimensional floating-point features not only consume a large amount of memory and are not conducive to storage, but also have a slower calculation speed during model inference. Summary of the invention

[0003] The technical problem to be solved by the present invention is to overcome the defects in the prior art that high-dimensional floating-point features of images extracted by traditional neural networks not only consume a large amount of memory and are not conducive to storage, but also have a slower calculation speed during model reasoning, and to provide a model training method, image feature extraction method, system, device and medium.

[0004] The present invention solves the above technical problems through the following technical solutions:

[0005] A first aspect of the present invention provides a model training method, the model training method comprising:

[0006] Acquire a historical training sample data set, wherein the historical training sample data set includes historical image classification label data, and the historical image classification label data corresponds to the hash feature of the historical image;

[0007] A neural network model is trained based on the historical training sample data set to obtain an image feature extraction model.

[0008] Preferably, the model training method further comprises:

[0009] Obtain the hash class center triplet loss function, cross entropy loss function, and quantization loss function;

[0010] A multi-loss joint optimization function is calculated according to the hash class center triplet loss function, the cross entropy loss function and the quantization loss function;

[0011] The image feature extraction model is optimized based on the multi-loss joint optimization function.

[0012] Preferably, the step of obtaining the hash class center triplet loss function, the cross entropy loss function and the quantization loss function comprises:

[0013] Extract image deep embedding features and image class features from historical image classification label data;

[0014] Mapping the image depth embedding feature into a continuous feature vector of a preset dimension;

[0015] The cross entropy loss function and the hash feature of the historical image are calculated according to the preset dimension continuous feature vector;

[0016] A hash class center triplet loss function is calculated based on the preset dimension continuous feature vector and the image class feature;

[0017] A quantization loss function is calculated based on the continuous feature vector of the preset dimension and the hash feature of the historical image.

[0018] A second aspect of the present invention provides a model training system, the model training system comprising:

[0019] A first acquisition module is used to acquire a historical training sample data set, wherein the historical training sample data set includes historical image classification label data, and the historical image classification label data corresponds to a hash feature of the historical image;

[0020] The training module is used to train the neural network model based on the historical training sample data set to obtain an image feature extraction model.

[0021] Preferably, the model training system further includes:

[0022] The second acquisition module is used to obtain the hash class center triplet loss function, the cross entropy loss function and the quantization loss function;

[0023] A first calculation module, used for calculating a multi-loss joint optimization function according to the hash class center triplet loss function, the cross entropy loss function and the quantization loss function;

[0024] An optimization module is used to optimize the image feature extraction model based on the multi-loss joint optimization function.

[0025] Preferably, the second acquisition module includes:

[0026] An extraction unit, used to extract image deep embedding features and image class features from historical image classification label data;

[0027] A mapping unit, used for mapping the image depth embedding feature into a continuous feature vector of a preset dimension;

[0028] A first calculation unit, configured to calculate the cross entropy loss function and the hash feature of the historical image according to the continuous feature vector of the preset dimension;

[0029] A second calculation unit, used for calculating a hash class center triplet loss function based on the preset dimension continuous feature vector and the image class feature;

[0030] The third calculation unit is used to calculate a quantization loss function based on the continuous feature vector of the preset dimension and the hash feature of the historical image.

[0031] A third aspect of the present invention provides an image feature extraction method, the image feature extraction method comprising:

[0032] Get the target image;

[0033] Inputting the target image into an image feature extraction model to obtain a hash feature of the target image;

[0034] Wherein, the image feature extraction model is trained using the model training method described in the first aspect above.

[0035] A fourth aspect of the present invention provides an image feature extraction system, the image feature extraction system comprising:

[0036] A target image acquisition module, used for acquiring a target image;

[0037] An input module, used for inputting the target image into an image feature extraction model to obtain a hash feature of the target image;

[0038] Among them, the image feature extraction model is trained using the model training system described in the second aspect above.

[0039] The fifth aspect of the present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the processor implements the model training method as described in the first aspect, or executes the image feature extraction method as described in the third aspect.

[0040] The sixth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the model training method as described in the first aspect is implemented, or the image feature extraction method as described in the third aspect is executed.

[0041] The positive and progressive effects of the present invention are:

[0042] The present invention obtains a historical training sample data set of historical image classification label data corresponding to the hash features of historical images, and trains a neural network model based on the historical training sample data set to obtain an image feature extraction model, so that the hash features of the image can be extracted using the image feature extraction model, thereby saving the space occupied by the image features and speeding up the speed at which the image feature extraction model extracts the image hash features. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 This is the first flow chart of the model training method of Example 1 of the present invention.

[0044] Figure 2 This is the second flow chart of the model training method of embodiment 1 of the present invention.

[0045] Figure 3 This is a flowchart of step 103 of the model training method of embodiment 1 of the present invention.

[0046] Figure 4 This is a module diagram of the model training system of Example 2 of the present invention.

[0047] Figure 5 This is a schematic diagram of the structure of an electronic device according to Embodiment 3 of the present invention.

[0048] Figure 6 This is a flow chart of the image feature extraction method of Example 5 of the present invention.

[0049] Figure 7 This is a module diagram of the image feature extraction system of Example 6 of the present invention. DETAILED DESCRIPTION

[0050] The present invention is further described below by way of examples, but the present invention is not limited to the scope of the examples.

[0051] Example 1

[0052] This embodiment provides a model training method, such as Figure 1 As shown, the model training method includes:

[0053] Step 101: Obtain a historical training sample data set, where the historical training sample data set includes historical image classification label data, where the historical image classification label data corresponds to a hash feature of the historical image;

[0054] In this embodiment, the historical image classification label data corresponds to the semantics described by the hash features of the historical image.

[0055] Step 102: Train a neural network model based on a historical training sample data set to obtain an image feature extraction model.

[0056] In this embodiment, there are multiple categories of historical image classification label data. During the iterative training of the model, each round of iteration batch input randomly samples P categories of historical image classification label data, and K samples are sampled for each category (that is, K pieces of historical image classification label data are sampled for each category), and the batch size N = P*K.

[0057] In one feasible solution, Figure 2 As shown, the model training method also includes:

[0058] Step 103, obtaining a hash class center triplet loss function, a cross entropy loss function, and a quantization loss function;

[0059] Step 104: Calculate a multi-loss joint optimization function according to the hash class center triplet loss function, the cross entropy loss function, and the quantization loss function;

[0060] In this embodiment, in order to jointly optimize the above three losses (i.e., jointly optimize the Hash Center-Triplet loss function, the cross entropy loss function, and the quantization loss function), it is necessary to coordinate the contributions of the three in the iterative update of the model. From experience, it can be seen that in the early stage of model training, the update of the Hash Center is unstable, and the hash quantization error is also large. In addition, considering the influence of noise labels, the early stage mainly relies on the penalty information of the cross entropy loss function, and the Hash Center triplet loss function and the quantization loss function should not impose too much penalty on the update of model parameters; and when the Hash Center tends to be stable in the middle and late stages of model training, the error of the quantization loss function gradually decreases. At this time, the supervision information of the Hash Center triplet loss function and the quantization loss function is more effective and reliable, so the punishment of the Hash Center triplet loss function and the quantization loss function on the model should be increased. In this regard, the annealing algorithm is used to introduce time-varying weight parameters α and β into the Hash Center triplet loss function and the quantization loss function respectively:

[0061]

[0062] α,β increase linearly with the increase of model iteration round t, T 1 ,T 2 is the starting and stopping point of the linear change iteration round, α f ,β f They represent the stabilized weight values ​​respectively. The optimization objective of the final image feature extraction model is the multi-loss joint optimization function (i.e., minimizing the total loss function) L:

[0063] L=L c +αL h +βL q

[0064] Among them, L represents the multi-loss joint optimization function, L c represents the cross entropy loss function, L h represents the hash class center triplet loss function, L qRepresents the quantization loss function.

[0065] Step 105: Optimize the image feature extraction model based on a multi-loss joint optimization function.

[0066] In this embodiment, a multi-loss joint optimization function is used to assist in optimizing feature distribution based on the Hash Center that is stably updated in a momentum manner, thereby achieving effective clustering and increasing the feature distance of heterogeneous samples; minimizing the information loss when converting image deep embedding features into hash codes; and optimizing the quality of hash codes;

[0067] Specifically, when the image feature extraction model is first trained, the Hash Center C of each class is randomly initialized. i (i=1,2,…,n; n is the number of categories), where C i is the image class feature in the historical image classification label data. In each iteration of training, only the HashCenter of the same category as the current round of historical training sample data set (i.e., historical image classification label data) is updated. i Unreliable, so when training a complete historical training sample data set (epoch) for the first time, C i Based on the mean features of the samples of the corresponding class in each round of iteration, the sliding average method is used to update:

[0068]

[0069] T 3 is the number of iterations required to iterate a complete historical training sample data set. Afterwards, in order to alleviate the interference of noise label sample features, C i Instead of updating based on the mean of similar samples, the features of the closest similar samples in the batch are mined. Momentum update:

[0070]

[0071] It should be noted that momentum 1 ,momentum 2 are momentum parameters of two different values. In the initial stage, C i Fast update, momentum 1 The value is small, which can be less than 0.5 by default, or it can be set to other values ​​that meet the conditions according to the actual situation; the latter momentum 2 This is to ensure the stable update of the Hash Center, so the update is slow and the value is often large, such as momentum 2The value of can be 0.9 or 0.999, etc., or other values.

[0072] This embodiment encodes high-dimensional floating-point features into binary hash features for representation, and optimizes the hash feature distribution and the information loss problem of converting floating-point features to hash features based on a multi-loss joint optimization function, ultimately optimizing the expression and reasoning acceleration of image feature encoding.

[0073] In one feasible solution, Figure 3 As shown, step 103 includes:

[0074] Step 1031, extracting image deep embedding features and image class features from historical image classification label data;

[0075] Step 1032: Map the image depth embedding feature into a continuous feature vector of a preset dimension;

[0076] Step 1033: Calculate the cross entropy loss function and the hash features of the historical image according to the continuous feature vector of the preset dimension;

[0077] In this embodiment, after using the neural network model to obtain the image deep embedding features in the historical image classification label data, it is mapped into a continuous feature vector of a preset dimension through a hash layer and a tanh activation function, and then can be further converted into a hash feature of a binary historical image for representation through a sign function;

[0078] For example, taking the commonly used ResNet50 as an example, the backbone of ResNet50 is used to extract the image deep embedding features (feature map before the global average pooling layer), and then the 2048-dimensional image deep embedding feature vector f is extracted through Global Generalized-MeanPooling. 1 ; f 1 Input the defined fully connected hash layer ([2048, D], default D = 512), and use the tanh function for nonlinear transformation to obtain a D-dimensional continuous feature vector f 2 (For example, the preset dimension is D dimension); after implementing binary encoding through the Sign function, the hash feature f of the historical image is obtained 3 (f 3 =sgn(f 2 )); then relevant reasoning tasks can be performed, such as calculating similarity, etc.

[0079] It should be noted that the ResNet50 backbone is followed by a fully connected hash layer using the tanh activation function, and then a Softmax layer for multi-label classification prediction.2 Perform joint optimization of multiple losses;

[0080] In addition, considering that binary hash coding is difficult to calculate gradients and update parameters during the model back propagation process, the optimization target in the training phase is the D-dimensional continuous feature vector f 2 , for each type of sample in the training sample data set, a D-dimensional hash continuous image class feature vector C is defined i (i=1,2,…,n; n is the number of categories) to establish a Hash Center queue; C in the queue i The momentum will be updated as the model is iterated.

[0081] In this embodiment, the D-dimensional continuous feature vector f 2 (i.e., a continuous feature vector of preset dimension) is input into the softmax classification layer for multi-label prediction to calculate the cross entropy loss function.

[0082] Step 1034, calculating a hash class center triplet loss function based on the preset dimension continuous feature vector and the image class feature;

[0083] In this embodiment, for each category of P samples selected in each batch, the corresponding Hash Center image class feature vector C is selected from the Center queue. i Participate in the training and update of this round of iteration; for each selected C i , calculate the distance with the features of the N samples in the current batch respectively, and use the hard mining method to mine the most difficult positives (the N samples with C i Belong to the same category and distance C i The farthest top-p positive samples) and the most difficult negatives (the N samples that are consistent with C i Different categories and distance C i The most recent top-n negative samples), the Top-K difficult mining method can effectively alleviate the interference of noise labels and avoid mining simple samples. The features of the top-p positives and top-n negatives are averaged. and negative Construct a triplet, pull each mean feature closer to its corresponding Hash Center, optimize the clustering effect, and make the distribution of samples of different classes as different as possible. The formula for calculating the loss function of the hash class center triplet is as follows:

[0084]

[0085] Among them, L hrepresents the hash class center triplet loss function, p represents the number of image categories, Represents the image class feature vector C i With the mean feature positive The distance between Represents the image class feature vector C i and mean feature negative The distance between them, m represents the hyperparameter.

[0086] It should be noted that since binary hash coding often uses hamming distance to calculate differences, but this distance calculation method is not differentiable and cannot be used for back propagation calculation optimization, the cosine distance associated with hamming distance is selected as the distance calculation between features. The formula for calculating the distance between features is as follows:

[0087]

[0088] Among them, k represents the number of samples of each category in the historical training sample dataset.

[0089] This embodiment learns and maintains a momentum-updated Hash Center feature for each type of sample through hash center-triplet loss, and optimizes feature clustering, thereby strengthening the stable and discriminative feature representation of network learning.

[0090] Step 1035: Calculate a quantization loss function based on the continuous feature vector of a preset dimension and the hash features of the historical image.

[0091] In this embodiment, in order to control the D-dimensional continuous feature vector f 2 The quantization error converted to binary hash code is calculated using the following formula:

[0092]

[0093] This embodiment obtains a historical training sample data set of historical image classification label data corresponding to the hash features of historical images, and trains a neural network model based on the historical training sample data set to obtain an image feature extraction model, so that the image feature extraction model can be used to extract the hash features of the image, thereby saving the space occupied by the image features and speeding up the speed at which the image feature extraction model extracts the image hash features.

[0094] Example 2

[0095] This embodiment provides a model training system, such as Figure 4 As shown, the model training system includes:

[0096] A first acquisition module 1 is used to acquire a historical training sample data set, where the historical training sample data set includes historical image classification label data, where the historical image classification label data corresponds to a hash feature of the historical image;

[0097] In this embodiment, the historical image classification label data corresponds to the semantics described by the hash features of the historical image.

[0098] The training module 2 is used to train the neural network model based on the historical training sample data set to obtain the image feature extraction model.

[0099] In this embodiment, there are multiple categories of historical image classification label data. During the iterative training of the model, each round of iterative batch input randomly samples P categories of historical image classification label data, and K samples are sampled for each category (that is, K pieces of historical image classification label data are sampled for each category), and the batch size N = P*K.

[0100] In one feasible solution, Figure 4 As shown, the model training system also includes:

[0101] The second acquisition module 3 is used to obtain the hash class center triplet loss function, the cross entropy loss function and the quantization loss function;

[0102] A first calculation module 4 is used to calculate a multi-loss joint optimization function according to a hash class center triplet loss function, a cross entropy loss function and a quantization loss function;

[0103] In this embodiment, in order to jointly optimize the above three losses (i.e., jointly optimize the Hash Center-Triplet loss function, the cross entropy loss function, and the quantization loss function), it is necessary to coordinate the contributions of the three in the iterative update of the model. From experience, it can be seen that in the early stage of model training, the update of the Hash Center is unstable, and the hash quantization error is also large. In addition, considering the influence of noise labels, the early stage mainly relies on the penalty information of the cross entropy loss function, and the Hash Center triplet loss function and the quantization loss function should not impose too much penalty on the update of model parameters; and when the Hash Center tends to be stable in the middle and late stages of model training, the error of the quantization loss function gradually decreases. At this time, the supervision information of the Hash Center triplet loss function and the quantization loss function is more effective and reliable, so the punishment of the Hash Center triplet loss function and the quantization loss function on the model should be increased. In this regard, the annealing algorithm is used to introduce time-varying weight parameters α and β into the Hash Center triplet loss function and the quantization loss function respectively:

[0104]

[0105] α,β increase linearly with the increase of model iteration round t, T 1 ,T 2 is the starting and stopping point of the linear change iteration round, α f ,β f They represent the stabilized weight values ​​respectively. The optimization objective of the final image feature extraction model is the multi-loss joint optimization function (i.e., minimizing the total loss function) L:

[0106] L=L c +αL h +βL q

[0107] Among them, L represents the multi-loss joint optimization function, L c represents the cross entropy loss function, L h represents the hash class center triplet loss function, L q Represents the quantization loss function.

[0108] The optimization module 5 is used to optimize the image feature extraction model based on a multi-loss joint optimization function.

[0109] In this embodiment, a multi-loss joint optimization function is used to assist in optimizing feature distribution based on the Hash Center that is stably updated in a momentum manner, thereby achieving effective clustering and increasing the feature distance of heterogeneous samples; minimizing the information loss when converting image deep embedding features into hash codes; and optimizing the quality of hash codes;

[0110] Specifically, when the image feature extraction model is first trained, the Hash Center C of each class is randomly initialized. i (i=1,2,…,n; n is the number of categories), where C i is the image class feature in the historical image classification label data. In each iteration of training, only the HashCenter of the same category as the current round of historical training sample data set (i.e., historical image classification label data) is updated. i Unreliable, so when training a complete historical training sample data set (epoch) for the first time, C i Based on the mean features of the samples of the corresponding class in each round of iteration, the sliding average method is used to update:

[0111]

[0112] T 3 is the number of iterations required to iterate a complete historical training sample data set. Afterwards, in order to alleviate the interference of noise label sample features, Ci Instead of updating based on the mean of similar samples, the features of the closest similar samples in the batch are mined. Momentum update:

[0113]

[0114] It should be noted that momentum 1 ,momentum 2 are momentum parameters of two different values. In the initial stage, C i Fast update, momentum 1 The value is small, which can be less than 0.5 by default, or it can be set to other values ​​that meet the conditions according to the actual situation; the latter momentum 2 This is to ensure the stable update of the Hash Center, so the update is slow and the value is often large, such as momentum 2 The value of can be 0.9 or 0.999, etc., or other values.

[0115] This embodiment encodes high-dimensional floating-point features into binary hash features for representation, and optimizes the hash feature distribution and the information loss problem of converting floating-point features to hash features based on a multi-loss joint optimization function, ultimately optimizing the expression and reasoning acceleration of image feature encoding.

[0116] In one feasible solution, Figure 4 As shown, the second acquisition module 3 includes:

[0117] An extraction unit 311 is used to extract image deep embedding features and image class features from historical image classification label data;

[0118] A mapping unit 312, configured to map the image depth embedding feature into a continuous feature vector of a preset dimension;

[0119] A first calculation unit 313 is used to calculate a cross entropy loss function and a hash feature of a historical image according to a continuous feature vector of a preset dimension;

[0120] In this embodiment, after using the neural network model to obtain the image deep embedding features in the historical image classification label data, it is mapped into a continuous feature vector of a preset dimension through a hash layer and a tanh activation function, and then can be further converted into a hash feature of a binary historical image for representation through a sign function;

[0121] For example, taking the commonly used ResNet50 as an example, the backbone of ResNet50 is used to extract the image deep embedding features (feature map before the global average pooling layer), and then the 2048-dimensional image deep embedding feature vector f is extracted through Global Generalized-MeanPooling. 1 ; f 1 Input the defined fully connected hash layer ([2048, D], default D = 512), and use the tanh function for nonlinear transformation to obtain a D-dimensional continuous feature vector f 2 (For example, the preset dimension is D dimension); after implementing binary encoding through the Sign function, the hash feature f of the historical image is obtained 3 (f 3 =sgn(f 2 )); then relevant reasoning tasks can be performed, such as calculating similarity, etc.

[0122] It should be noted that the ResNet50 backbone is followed by a fully connected hash layer using the tanh activation function, and then a Softmax layer for multi-label classification prediction. 2 Perform joint optimization of multiple losses;

[0123] In addition, considering that binary hash coding is difficult to calculate gradients and update parameters during the model back propagation process, the optimization target in the training phase is the D-dimensional continuous feature vector f 2 , for each type of sample in the training sample data set, a D-dimensional hash continuous image class feature vector C is defined i (i=1,2,…,n; n is the number of categories) to establish a Hash Center queue; C in the queue i The momentum will be updated as the model is iterated.

[0124] In this embodiment, the D-dimensional continuous feature vector f 2 (i.e., a continuous feature vector of preset dimension) is input into the softmax classification layer for multi-label prediction to calculate the cross entropy loss function.

[0125] The second calculation unit 314 is used to calculate the hash class center triplet loss function based on the preset dimension continuous feature vector and the image class feature;

[0126] In this embodiment, for each category of P samples selected in each batch, the corresponding Hash Center image class feature vector C is selected from the Center queue. i Participate in the training and update of this round of iteration; for each selected C i, calculate the distance with the features of the N samples in the current batch respectively, and use the hard mining method to mine the most difficult positives (the N samples with C i Belong to the same category and distance C i The farthest top-p positive samples) and the most difficult negatives (the N samples that are consistent with C i Different categories and distance C i The most recent top-n negative samples), the Top-K difficult mining method can effectively alleviate the interference of noise labels and avoid mining simple samples. The features of the top-p positives and top-n negatives are averaged. and negative Construct a triplet, pull each mean feature closer to its corresponding Hash Center, optimize the clustering effect, and make the distribution of samples of different classes as different as possible. The formula for calculating the loss function of the hash class center triplet is as follows:

[0127]

[0128] Among them, L h represents the hash class center triplet loss function, p represents the number of image categories, Represents the image class feature vector C i With the mean feature positive The distance between Represents the image class feature vector C i and mean feature negative The distance between them, m represents the hyperparameter.

[0129] It should be noted that since binary hash coding often uses hamming distance to calculate differences, but this distance calculation method is not differentiable and cannot be used for back propagation calculation optimization, the cosine distance associated with hamming distance is selected as the distance calculation between features. The formula for calculating the distance between features is as follows:

[0130]

[0131] Among them, k represents the number of samples of each category in the historical training sample dataset.

[0132] This embodiment learns and maintains a momentum-updated Hash Center feature for each type of sample through hash center-triplet loss, and optimizes feature clustering, thereby strengthening the stable and discriminative feature representation of network learning.

[0133] The third calculation unit 315 is used to calculate and obtain a quantization loss function based on the continuous feature vector of a preset dimension and the hash feature of the historical image.

[0134] In this embodiment, in order to control the D-dimensional continuous feature vector f 2 The quantization error converted to binary hash code is calculated using the following formula:

[0135]

[0136] This embodiment obtains a historical training sample data set of historical image classification label data corresponding to the hash features of historical images, and trains a neural network model based on the historical training sample data set to obtain an image feature extraction model, so that the image feature extraction model can be used to extract the hash features of the image, thereby saving the space occupied by the image features and speeding up the speed at which the image feature extraction model extracts the image hash features.

[0137] Example 3

[0138] Figure 5 This is a schematic diagram of the structure of an electronic device provided in Example 3 of the present invention. The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the model training method of Example 1 is implemented when the processor executes the program. Figure 5 The electronic device 30 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0139] like Figure 5 As shown, the electronic device 30 may be in the form of a general-purpose computing device, for example, it may be a server device. The components of the electronic device 30 may include, but are not limited to: at least one processor 31, at least one memory 32, and a bus 33 connecting different system components (including the memory 32 and the processor 31).

[0140] The bus 33 includes a data bus, an address bus, and a control bus.

[0141] The memory 32 may include a volatile memory, such as a random access memory (RAM) 321 and / or a cache memory 322 , and may further include a read-only memory (ROM) 323 .

[0142] The memory 32 may also include a program / utility 325 having a set (at least one) of program modules 324, such program modules 324 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0143] The processor 31 executes various functional applications and data processing by running the computer program stored in the memory 32, such as the model training method of Example 1 of the present invention.

[0144] The electronic device 30 may also communicate with one or more external devices 34 (e.g., keyboards, pointing devices, etc.). Such communication may be performed via an input / output (I / O) interface 35. Furthermore, the model generating device 30 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 36. Figure 5 As shown, the network adapter 36 communicates with other modules of the model-generated device 30 via the bus 33. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the model-generated device 30, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems, etc.

[0145] It should be noted that although several units / modules or sub-units / modules of the electronic device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to an embodiment of the present invention, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided into multiple units / modules to be embodied.

[0146] Example 4

[0147] This embodiment provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the model training method provided in Embodiment 1 is implemented.

[0148] The readable storage medium may include but is not limited to: a portable disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical storage device, a magnetic storage device or any suitable combination of the above.

[0149] In a possible implementation manner, the present invention can also be implemented in the form of a program product, which includes a program code. When the program product runs on a terminal device, the program code is used to enable the terminal device to execute the model training method described in Example 1.

[0150] Among them, the program code for executing the present invention can be written in any combination of one or more programming languages, and the program code can be executed completely on the user device, partially on the user device, as an independent software package, partially on the user device and partially on a remote device, or completely on the remote device.

[0151] Although the specific embodiments of the present invention are described above, it should be understood by those skilled in the art that this is only for illustration and the protection scope of the present invention is defined by the appended claims. Those skilled in the art may make various changes or modifications to these embodiments without departing from the principles and essence of the present invention, but these changes and modifications all fall within the protection scope of the present invention.

[0152] Example 5

[0153] This embodiment provides an image feature extraction method, such as Figure 6 As shown, the image feature extraction method includes:

[0154] Step 201, obtaining a target image;

[0155] Step 202: input the target image into an image feature extraction model to obtain a hash feature of the target image;

[0156] The image feature extraction model is trained using the model training method of the first embodiment.

[0157] This embodiment optimizes the feature distribution of hash codes by introducing a Hash Center based on momentum update, and ultimately obtains high-quality hash features to replace high-dimensional floating-point deep features. It effectively reduces the consumption of storage memory while ensuring that performance loss is not large, and can efficiently accelerate similarity reasoning in practical applications, greatly improving application efficiency.

[0158] Example 6

[0159] This embodiment provides an image feature extraction system. Figure 7 As shown, the image feature extraction system includes:

[0160] A target image acquisition module 71, used to acquire a target image;

[0161] An input module 72, used for inputting a target image into an image feature extraction model to obtain a hash feature of the target image;

[0162] The image feature extraction model is trained using the model training system of the second embodiment.

[0163] This embodiment optimizes the feature distribution of hash codes by introducing a Hash Center based on momentum update, and ultimately obtains high-quality hash features to replace high-dimensional floating-point deep features. It effectively reduces the consumption of storage memory while ensuring that performance loss is not large, and can efficiently accelerate similarity reasoning in practical applications, greatly improving application efficiency.

[0164] Example 7

[0165] A schematic diagram of the structure of an electronic device provided in Embodiment 7 of the present invention. The schematic diagram of the structure of the electronic device in this embodiment is similar to Figure 5 The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the image feature extraction method of embodiment 5 is implemented. Figure 5 The electronic device 30 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0166] like Figure 5 As shown, the electronic device 30 may be in the form of a general-purpose computing device, for example, it may be a server device. The components of the electronic device 30 may include, but are not limited to: at least one processor 31, at least one memory 32, and a bus 33 connecting different system components (including the memory 32 and the processor 31).

[0167] The bus 33 includes a data bus, an address bus, and a control bus.

[0168] The memory 32 may include a volatile memory, such as a random access memory (RAM) 321 and / or a cache memory 322 , and may further include a read-only memory (ROM) 323 .

[0169] The memory 32 may also include a program / utility 325 having a set (at least one) of program modules 324, such program modules 324 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0170] The processor 31 executes various functional applications and data processing by running the computer programs stored in the memory 32, such as the image feature extraction method of embodiment 5 of the present invention.

[0171] The electronic device 30 may also communicate with one or more external devices 34 (e.g., keyboards, pointing devices, etc.). Such communication may be performed via an input / output (I / O) interface 35. Furthermore, the model generating device 30 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 36. Figure 5 As shown, the network adapter 36 communicates with other modules of the model-generated device 30 via the bus 33. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the model-generated device 30, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems, etc.

[0172] It should be noted that although several units / modules or sub-units / modules of the electronic device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to an embodiment of the present invention, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided into multiple units / modules to be embodied.

[0173] Example 8

[0174] This embodiment provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the image feature extraction method provided in Embodiment 5 is implemented.

[0175] The readable storage medium may include but is not limited to: a portable disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical storage device, a magnetic storage device or any suitable combination of the above.

[0176] In a possible implementation manner, the present invention can also be implemented in the form of a program product, which includes a program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the image feature extraction method described in Example 5.

[0177] Among them, the program code for executing the present invention can be written in any combination of one or more programming languages, and the program code can be executed completely on the user device, partially on the user device, as an independent software package, partially on the user device and partially on a remote device, or completely on the remote device.

[0178] Although the specific embodiments of the present invention are described above, it should be understood by those skilled in the art that this is only for illustration and the protection scope of the present invention is defined by the appended claims. Those skilled in the art may make various changes or modifications to these embodiments without departing from the principles and essence of the present invention, but these changes and modifications all fall within the protection scope of the present invention.

Claims

1. A model training method, characterized in that: The model training method comprises: Acquire a historical training sample data set, wherein the historical training sample data set includes historical image classification label data, and the historical image classification label data corresponds to the hash feature of the historical image; Training a neural network model based on the historical training sample data set to obtain an image feature extraction model; The model training method also includes: Obtain the hash class center triplet loss function, cross entropy loss function, and quantization loss function; A multi-loss joint optimization function is calculated according to the hash class center triplet loss function, the cross entropy loss function and the quantization loss function; Optimizing the image feature extraction model based on the multi-loss joint optimization function; The steps of obtaining the hash class center triplet loss function, the cross entropy loss function and the quantization loss function include: Extract image deep embedding features and image class features from historical image classification label data; Mapping the image depth embedding feature into a continuous feature vector of a preset dimension; The cross entropy loss function and the hash feature of the historical image are calculated according to the preset dimension continuous feature vector; A hash class center triplet loss function is calculated based on the preset dimension continuous feature vector and the image class feature; A quantization loss function is calculated based on the continuous feature vector of the preset dimension and the hash feature of the historical image.

2. A model training system, characterized in that: The model training system comprises: A first acquisition module is used to acquire a historical training sample data set, wherein the historical training sample data set includes historical image classification label data, and the historical image classification label data corresponds to a hash feature of the historical image; A training module, used for training a neural network model based on the historical training sample data set to obtain an image feature extraction model; The model training system also includes: The second acquisition module is used to obtain the hash class center triplet loss function, the cross entropy loss function and the quantization loss function; A first calculation module, used for calculating a multi-loss joint optimization function according to the hash class center triplet loss function, the cross entropy loss function and the quantization loss function; An optimization module, used for optimizing the image feature extraction model based on the multi-loss joint optimization function; The second acquisition module includes: An extraction unit, used to extract image deep embedding features and image class features from historical image classification label data; A mapping unit, used for mapping the image depth embedding feature into a continuous feature vector of a preset dimension; A first calculation unit, configured to calculate the cross entropy loss function and the hash feature of the historical image according to the preset dimension continuous feature vector; A second calculation unit, configured to calculate a hash class center triplet loss function based on the preset dimension continuous feature vector and the image class feature; The third calculation unit is used to calculate a quantization loss function based on the continuous feature vector of the preset dimension and the hash feature of the historical image.

3. A method for extracting image features, characterized in that: The image feature extraction method comprises: Get the target image; Inputting the target image into an image feature extraction model to obtain a hash feature of the target image; Wherein, the image feature extraction model is trained using the model training method described in claim 1 above.

4. An image feature extraction system, characterized in that: The image feature extraction system comprises: A target image acquisition module, used for acquiring a target image; An input module, used for inputting the target image into an image feature extraction model to obtain a hash feature of the target image; Wherein, the image feature extraction model is trained using the model training system described in claim 2 above.

5. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, it implements the model training method as described in claim 1, or executes the image feature extraction method as described in claim 3.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the model training method as described in claim 1, or executes the image feature extraction method as described in claim 3.

Images